Mouse Events are several frames behind

[shrewd hearth]

I want to add an icon next to the mouse to remind you what tool you're using. it obviously should stick to the mouse like glue or else it would look weird.

I wrote this system:

fn mouse_icon(
    window: Query<&Window, With<bevy::window::PrimaryWindow>>,
    mut icon: Query<(&mut Style, &mut Visibility), With<MouseIconMarker>>,
) {
    let mut icon = icon.single_mut();
    let window = window.single();
    let Some(position) = window.cursor_position() else {
        return;
    };
    icon.0.left = Val::Px(position.x);
    icon.0.bottom = Val::Px(window.height() - position.y);
    // *icon.1 = Visibility::Visible;
    info!("mouse icon at {:?} {:?}", icon.0.left, icon.0.top);
}

it works as intended, but the icon lags one frame behind the mouse. Is there some alternate way of reading the mouse, or an alternate way of drawing the icon, or what?

[shrewd hearth]

Mouse Events are several frames behind

[agile mango]

This is just input latency + buffered frames.

See: https://github.com/aevyrie/bevy_framepace

For a pointer icon, you should be using the winit commands to change the actual cursor

However the current api doesn't support custom sprites, just predefined OS cursors

[shrewd hearth]

(a) i need custom icons and (b) i don't want to mess with the user's cursor choice

[agile mango]

This is technically the correct way of doing things. I'm not sure what you mean by messing with the user's cursor choice. It requests a specific OS cursor, like the 4-way panning icon you show here. If a user has a custom icon pack installed, it will use that one

https://docs.rs/bevy/latest/bevy/window/enum.CursorIcon.html

This is a field on the bevy Window

But yeah, does not support custom icons afaik.

The only option right now is using framepacing in addition to what you have right now.

[shrewd hearth]

4-way panning is just the first one i thought of; i need totally custom ones

[agile mango]

@rugged relic FYI. This is something people will need, but is blocked on winit. The solution shown here has been recommended in the past, but it is really a workaround for proper support in winit.

https://github.com/rust-windowing/winit/issues/3005

I've seen (I think @little forge?, sorry if I'm isremembering ❤️ ) say that you can just draw a sprite, but I'm like 99.999% sure this is not correct. You need to use OS APIs to get the best possible latency. The reason I'm confident this is the case is because I've seen custom game cursors when using game streaming before the rendered output, and it is also able to leave the bounds of the window. This is only possible if the game was using an OS cursor API.

[shrewd hearth]

framepace didn't help much but PresentMode::Mailbox helped a lot

[agile mango]

If it doesn't help, that's a bug. Though it has limitations due to a lack of wgpu renderer timing information. 😦

It should look as good as the video I posted above.

[shrewd hearth]

actually it's probably more because mailbox runs at 1300 fps

[agile mango]

Yeah, that and vsync off are the best options short of proper support, but requires you are able to run your game at insane framerates.

@rugged relic found this existing issue and added some detail: https://github.com/bevyengine/bevy/issues/9557

Oooh, it looks like there is a draft PR that is trying to revive this, and it's only 9 hours old: https://github.com/rust-windowing/winit/pull/3218

[shrewd hearth]

rip mailbox doesn't work on web

[little forge]

Nope. Wasn’t me. I’m pretty sure there was hardware acceleration for mouse pointer rendering a long time ago. In any case I too would be confident it is an OS API to use a custom cursor in a particular application

[agile mango]

Sorry Rob, my bad.

[little forge]

No worries 🙂

[shrewd hearth]

is vsync just hard-enabled on the web? is it a web restriction or just has nobody bothered to get around to it?

[little forge]

Web uses an in-browser loop for when to update graphics in order to avoid background tabs using resources

[shrewd hearth]

my fps counter says 60 despite AutoNoVsync and Limiter::from_framerate(144.)

[little forge]

I don’t remember what it’s called right now. JavaScript API

[shrewd hearth]

are you allowed to change the behaviour at all?

[little forge]

It may ignore you. I’m not sure. But it’s also good to use for timing. I think it’s set up to get smooth rendering as time is needed to execute JavaScript and to rerender the page and there are budgets involved.

It’s been a while since I looked at it

[shrewd hearth]

i'll just leave it ig. these icons are only meant to show short-term while you're holding the relevant key.

[little forge]

https://developer.mozilla.org/en-US/docs/Web/API/window/requestAnimationFrame

[agile mango]

the plugin does nothing on web for the reason Rob mentioned

[shrewd hearth]

....and AutoNoVsync also does nothing on the web

[misty mist]

High framerate just masks the underlying behavior, it's not going to be a viable fix if you intend this behavior to work on less powerful PCs or browsers.

Ultimately the cursor, when rendered in game, will always be one frame behind, and increasing the framerate is just minimizing that frame time

Browser API tends to be super restrictive, they like to be the one in control of the fine-grained user experience