#bevy_mod_picking

1 messages · Page 2 of 1

prisma karma
#

Thanks, I'll give the physics layers a try!

novel hamlet
#

Render layers are automatically considered

#

As well as camera order

#

A camera rendering on top of another should receive the picks. This is shown in one of the examples

#

Render layers example I think

prisma karma
#

That is strange — I already have them rendering in a higher layer with a second camera. Is there anything else I could be missing? I did take a look at the two-pass example — I assumed the reason mine didn't work the same way is because my cameras are positioned and oriented exactly the same

#

(The second camera is a child of the first)

#

But the second one spawns in with a delay, not in a Startup system

novel hamlet
#

I didn't even know camera parenting was supported. Not sure if that would break anything.

#

What picking backend(s) are you using?

prisma karma
#

I think the defaults + Rapier:

"backend_bevy_ui",
"backend_rapier",
"backend_raycast",
"backend_sprite",
novel hamlet
#

You should probably disable raycast if you're using rapier

#

Can you try enabling only one camera at a time to make sure they work in isolation?

prisma karma
#

Gotcha, I'll give that a try and also try unparenting the second camera

novel hamlet
#

It's hard to say, simply because this should be supported and I have a working example.

prisma karma
#

And yeah, I'll try turning off the main camera (I know that one works on its own because I added the second one after the first was working)

#

No worries, I'll fiddle and report back if I'm still having issues 🙂 Appreciate your help!

novel hamlet
#

Sounds good!

prisma karma
#

So after some more testing, it does seem that if mesh A is in front of mesh B from all cameras' perspectives, mesh A will always get picked even if mesh B is rendering in a higher render layer with a higher camera order. I forked the two_passes example to demonstrate the issue: https://github.com/aevyrie/bevy_mod_picking/compare/main...dannymcgee:bevy_mod_picking:minimal-repro

In that setup, both cameras have the same transform (but no funny business with parenting). Camera 2 renders into layer 1. There's a sphere at the origin also rendering on layer 1, bisected by a plane rendering on layer 0. In the view, the whole sphere is visible because of the render layers, but it can only be picked when the pointer is over the upper hemisphere (i.e. the portion that's in front of the plane). It's possible I'm misunderstanding the intention, but let me know if that's not what you expected and I'll go ahead and file an issue. (And no rush, obviously!)

GitHub
Comparing aevyrie:main...dannymcgee:minimal-repro · aevyrie/bevy_mo...

Picking and pointer events for Bevy. Contribute to aevyrie/bevy_mod_picking development by creating an account on GitHub.

novel hamlet
#

Great, I'll take a look.

#

Thanks for doing this work, very helpful.

prisma karma
#

No problem!

peak field
#

@novel hamlet I'm getting an error from Highlight<StandardMaterial> that StandardMaterial does not implement Asset after updating to 0.12. Is this something on your end or on mine? 😅

novel hamlet
#

The plugin hasn't been updated to 0.12 yet.

tawny adder
novel hamlet
#

I'm working through updating my crates to 0.12. Hopefully will get to this tonight.

#

Only potential blocker is I'm not sure if bevy_rapier has been updated yet

opaque garden
#

Yea Rapier hasn't updated 😢

tawny adder
#

It hasn't updated, but the PR into it seems to work. I'm just running on PR's right now, which seems to be working okay. Of course it'll be nicer once everything is merged and I don't have to use forks of different things and do patching

peak field
#

setting my target to the PR fork for 0.12 gets me errors about it not finding bevy_mod_raycast 0.16^ which is fair enough I suppose since it's not on crates.io yet

novel hamlet
#

Rapier was just updated.

#

I need to update and publish mod_raycast, then update mod_picking and publish.

peak field
novel hamlet
#

If I set the other camera and plane to layer 2, everything behaves as expected

#

Also, if I set it to default, which is 0, it also works

#

It only breaks when the other camera and plane do not specify a render layer

#

okay

#

I think the plugin is working correctly

#

In your example, the other camera has no renderlayer, so it should be able to pick any entity, including the plane, so it should be able to pick the plane

#

hm, maybe I have this backward

#

camera 1, can only pick the sphere, rendering on top, should take priority
other camera, can pick plane and sphere, but is below

novel hamlet
#

Alright, found and fixed the bug

#

Wasn't properly accounting for all renderlayer cases in the rapier and raycast backends

novel hamlet
#

Closed out all but 3 issues on the repo, gonna get a release out now.

prisma karma
#

Thank you so much!

copper wolf
#

does anyone else get this jittering with the latest version of the plugin? I'm using the default features + egui, and I have no idea what's going on, cause the spheres are getting highlighted/selected in rows(?)

copper wolf
#

There we go

#

Selection seems to work as expected, so the problem seems to be in the highlighting code

novel hamlet
#

Do you have a minimal repro?

#

I haven't seen this when testing the examples.

copper wolf
#

I got it

# 
use bevy::{
    pbr::wireframe::{Wireframe, WireframeConfig, WireframePlugin},
    prelude::*,
};
use bevy_mod_picking::prelude::*;

fn main() {
    App::new()
        .add_plugins(DefaultPlugins)
        .add_plugins(DefaultPickingPlugins)
        .add_plugins(WireframePlugin)
        .add_systems(Startup, setup)
        .init_resource::<WireframeConfig>()
        .run();
}

fn setup(
    mut commands: Commands,
    mut meshes: ResMut<Assets<Mesh>>,
    mut materials: ResMut<Assets<StandardMaterial>>,
) {
    let mesh = meshes.add(shape::Cube { size: 0.5 }.into());
    let material = materials.add(Color::WHITE.into());
    (-1..=1).for_each(|x| {
        commands.spawn((
            PbrBundle {
                mesh: mesh.clone(),
                material: material.clone(),
                transform: Transform::from_xyz(x as f32, 0.0, 0.0),
                ..Default::default()
            },
            PickableBundle::default(),
            Wireframe,
        ));
    });

    commands.spawn(Camera3dBundle {
        transform: Transform::from_xyz(0.0, 5.0, 5.0).looking_at(Vec3::ZERO, Vec3::Y),
        ..Default::default()
    });
}
#

The problem seems to be with the wireframe plugin

native vessel
#

RaycastPickCamera and RaycastPickTarget are missing in 0.17

peak field
#

Just remove them from your entity, everything will work like before c:

native vessel
#

Thanks

novel hamlet
copper wolf
#

Something I noticed just now

#

Selecting through ui elements gets blocked by the egui backend, but dragging doesn't

steel imp
#

Does bevy mod picking support something like layered selection of overlapping sprites? I.e.: select the first, and if clicked again, select the next down, etc, until we go back to selecting the top again?

#

Ah, maybe I can just get this behavior by replacing the sprite picking backend.

novel hamlet
#

It's possible for backends to report all hits under a pointer, not just the first, though I don't think the sprite backend has this feature.

fiery night
#

@novel hamlet I'm having some difficulties sending custom events. I have two entities, one which sends a Clicked (not Click) event. Sometimes when I run the program, the first click is sent but never received by the listener, but subsequent clicks are. The second entity emits a SplitterDrag event, which is never received at all. My question is, what's the best way to debug this?

My listener code:

e.insert((
    On::<SplitterDragged>::run(|ev: Res<ListenerInput<SplitterDragged>>| {
        println!("Dragged {}", ev.id);
    }),
    On::<Clicked>::run(|ev: Res<ListenerInput<Clicked>>| {
        println!("Clicked button {}", ev.id);
    }),
));

I have verified that both types of events and event listeners have been registered in the app.

novel hamlet
#

Is this a bevy_eventlisteners question? i.e. this isn't debugging a picking issue?

#

The events are all normal bevy events, so I would start by adding a system that prints the events you are interested in.

#

And then start tracing that to answer "did the listener see this event?"

#

SplitterDrag
Was this a typo, did you mean SplitterDragged?

fiery night
novel hamlet
#

If you get stuck from there, it would help to see if you can develop a minimal repro?

#

The crate doesn't really do much, it listens for bevy events and traverses the entity hierarchy. If the callback isn't being executed, it might be that the event stops bubbling?

fiery night
novel hamlet
#

Next I would store the id of the entity of interest (the one you are storing the listeners on), and compare that with the entities in those printed events.

#

My line of inquiry would revolve around learning if the events being emitted are in the correct entity hierarchy.

fiery night
novel hamlet
#

Additionally, it's possible the callback is being overwritten if you are adding a listener of the same type to the entity.

novel hamlet
fiery night
#

In other words, the first click on any button gets eaten, but after that first event all subsequent events get received normally.

novel hamlet
#

That could be ordering?

#

Maybe the event listener system is running before the button is clicked?

#

Though I would still expect it to trigger

#

Alternatively, is there any reactivity that is touching the hierarchy?

#

e.g. broken -> click -> magically fixed

fiery night
#

For button clicks, it's just printing to the console.

novel hamlet
#

Does this happen on the picking examples? I'm surprised I don't see the issue there

#

I think the bevy ui example has a listener on a clicked button iirc

fiery night
#

I should clarify that these "buttons" are not Bevy buttons, they are Bevy UiNodes. However, that shouldn't matter, I know the event is getting emitted.

novel hamlet
#

Yep, I'm pretty sure that example is also just a node with text on it.

#

I have to get back to job-work for a while. 🙂

fiery night
#

OK np

fiery night
novel hamlet
#

That should be fine. The hierarchy is traversed and mapped out at least one time each update, so it should be fine to add it whenever.

fiery night
#

I've added some debugging statements to the example to illustrate the problem better:

Adding event handlers
2023-11-17T18:06:57.641321Z  INFO bevy_winit::system: Creating new window "App" (0v0)
Sending Clicked id='save' target=16v0
Reading global clicked: id='save' target=16v0

Sending Clicked id='save' target=16v0
Received Clicked Button id='save' target=16v0
Reading global clicked: id='save' target=16v0

The first line says that it's inserting the event handlers (the On components) into the world.
The second line says that it's writing the event to the event writer.
The third line says that the event is successfully received by a global event reader running as a separate system.
The On listener does not receive the event.

The next several lines show the same thing, only this time there's an additional line showing that the event was received by the On listener.

#

Also, the problem is intermittent - sometimes when I run the example the problem does not appear.

brazen geyser
#

do any of the picking backends not need rendering, so I can use it for unit testing?

novel hamlet
#

They all hit test on the CPU.

#

The picking shader doesn't exist yet.

brazen geyser
#

ok. I saw that they needed bevy_render. I guess I can try to just add whatever pieces they need

novel hamlet
fiery night
#

@novel hamlet Been doing more experiments, I've encountered other case of "first event sometimes getting eaten". (The other problem, with SplitterDrag, I figured out, handler was on the wrong entity). However, I added a "SplitterDragStart" event, and like the Clicked event it gets received intermittently.

novel hamlet
fiery night
#

Another question: I need my event handler to both (a) modify a component on the target entity, and (b) send a custom event. I assume the only way to do this is run() - the target_component_mut doesn't appear to allow additional injection params. My question is, do I need to inject a query and look up the target entity, or is there a way to inject the target entity component directly?

novel hamlet
#

The target entity is supplied in the run function, in the listener resource

#

Take a look at the source for target_component_mut. You could easily define your own variant if you want to, that would look something like my_special_callback(EventFoo(3.6), ComponentBar("boo"))

tranquil quest
#

I really like this crate and would like to ask how to implement drag-and-drop in 3D space, allowing objects to follow the cursor. Using 2D methods in the example make the movement of the object too fast or too slow, unable to completely match the cursor's movement.

novel hamlet
#

That's really a math problem, not a picking one. You would need to compute the new position of the object where the cursor intersects a plane perpendicular to the camera at the depth when dragging started.

#

This is implemented in bevy_transform_gizmo. You could use that as reference.

crude talon
#

Any pointers on creating a backend for sprites?

novel hamlet
crude talon
#

Awesome, but how is it used? The example isn't very clear. Adding the same on::pointer components as I do on meshes doesn't do anything, and I can't find a SpritePickable or anything like that. Apparently it isn't required? The sprite example doesn't seem that clear to me

novel hamlet
#

Does the example not work for you?

#

Most of the backends don't require extra components to work

#

On works for anything that has a pickking backend.

crude talon
#

Something's wrong in my code, can't figure out what

crude talon
#

Are there any requirements for the sprite picking to work that may not be obvious?

novel hamlet
#

Everything required is shown in the example

#

You just need to add the default picking plugins, and the sprite backend feature must be enabled.

#

However, the feature is enabled by default.

#

I made an absolutely minimal example, and this works fine on the latest version:

use bevy::prelude::*;
use bevy_mod_picking::prelude::*;

fn main() {
    App::new()
        .add_plugins((DefaultPlugins, DefaultPickingPlugins))
        .add_systems(Startup, setup)
        .run();
}

fn setup(mut commands: Commands, asset_server: Res<AssetServer>) {
    commands.spawn(Camera2dBundle::default());
    commands.spawn((
        SpriteBundle {
            texture: asset_server.load("images/boovy.png"),
            ..default()
        },
        On::<Pointer<Click>>::run(|| info!("clicked!")),
    ));
}
crude talon
#

Something weird is def happening, because a child entity's pointer events also stop working when I make the parent a sprite

crude talon
# 
        SpriteBundle {
            texture: image,
            sprite: Sprite {
                anchor: bevy::sprite::Anchor::Center,
                ..default()
            },
            transform: Transform {
                translation: Vec3::new(
                    ev.position.x + rng.gen_range(-10.0..10.0),
                    ev.position.y + rng.gen_range(-10.0..10.0),
                    view_data.top_z,
                ),
                ..default()
            },
            ..default()
        },
# 
        MaterialMesh2dBundle {
            mesh: meshes.add(shape::Circle::new(radius).into()).into(),
            material: materials.add(ColorMaterial::from(Color::rgb(0.3, 0.0, 0.0))),
            transform: Transform::from_translation(Vec3::new(
                ev.position.x + rng.gen_range(-10.0..10.0),
                ev.position.y + rng.gen_range(-10.0..10.0),
                view_data.top_z,
            )),
            ..default()
        },
#

This is the only difference between the two types of entities I have

# 
        commands.entity(ev.entity).insert((
            GraphViewNode,
            Velocity2D::default(),
            //PickableBundle::default(),
            
            On::<Pointer<Drag>>::send_event::<MoveNodesEvent>(),
            On::<Pointer<Click>>::send_event::<NodeClickEvent>(),
            On::<Pointer<Down>>::send_event::<NodePressedEvent>(),
            On::<Pointer<Over>>::send_event::<NodeHoverEvent>(),
            
            On::<Pointer<DragStart>>::target_insert(Selected),
            On::<Pointer<DragEnd>>::target_remove::<Selected>(),
            On::<Pointer<Deselect>>::target_remove::<Selected>(),
        ));

Both types share these pointer components

novel hamlet
#

It would really help if you could make a minimal reproduction of the issue, so I can debug, if there is a bug.

crude talon
#

I'll try to reproduce it

#

Quickly though: changing the spritebundle to a spatialbundle unblocks the child entity

novel hamlet
#

Just guessing, but probably because the sprite is literally blocking the child entity, because the sprite is pickable?

#

You could add Pickable::IGNORE to the sprite parent to see if that changes anything

crude talon
#

I made a minimal repo, but couldn't reproduce the issue

#

It's very odd

novel hamlet
#

Then it's probably not the picking plugin. If the spatialbundle is changing behavior, it might just be a core bevy thing. Hard to say.

crude talon
#

Yeah

fiery night
crude talon
#

Still trying to look at what exactly in the system causes this

#

I think I found the line:

if img == &bevy::prelude::Handle::Weak(id.clone()) {
#

If this line runs, picking stops working

#

I must be doing something weird with the handles, I don't fully understand how they work

crude talon
#

Finally solved it

#

I shouldn't have done the scaling of the sprites with the transform

crude talon
#

Scaling instead with the sprite's custom size fixed the picking issue

#

I'll try to reproduce this later

fiery night
#

Q: My library requires a number of bevy_mod_picking plugins to be installed: CorePlugin, InputPlugin, InteractionPlugin, BevyUiBackend. Currently I require the app to install the plugins that I need, but it would be convenient if my library's plugin did this automatically so the user doesn't have to. However, I want to avoid a conflict in case the user is using other bevy_mod_picking backends, or is using a more recent version of those plugins. What's the standard practice for this sort of thing?

novel hamlet
fiery night
novel hamlet
#

So I could pull in your lib that uses mod picking with QuillUi backend, then turn on the rapier feature, and mod picking will work with my physics stuff too.

fiery night
#

@novel hamlet On another topic: this is somewhat related to the issue about events getting eaten, but not exactly the same: since I first integrated mod_picking into my demo app, I've seen various kinds of erratic behavior when dragging. The window splitter widget listens to a number of events, but the most important ones are DragStart and Drag.

Because the splitter position is clamped (you don't want to drag it off-screen), it can't use just delta positions, it needs to know the starting position of the split bar in order to do the clamping calculation correctly. It does this by listening for DragStart and storing the initial position in a local state var. If this variable is incorrect, then what you'd see would be that the position of the splitter is offset from where the mouse is.

Given all that background, the behavior that I see is very odd. When I run the app, there are four different behaviors that I observe, three of which are wrong and only one is correct:

  • Correct: you can drag the splitter bar with the mouse.
  • Incorrect: the splitter bar doesn't move at all.
  • Incorrect: the splitter bar, when dragged, immediately snaps to the left side of the window and stays there.
  • Incorrect: the splitter bar moves to the left for one frame and then drags normally.

Each of these scenarios is explainable in terms of some Pointer event not being received, either the initial DragStart or the Drag. For example, scenario #2 is what would happen if Drag events were not received, whereas scenario #3 is what would happen if there was no DragStart (the offset would be wrong).

Each scenario seems to occur with equal probability: each time I run the demo, I get one of the four behaviors, and the behavior persists until I kill the app and restart.

novel hamlet
#

Have you tried running the debug overlay? The egui one will help you see what interactions are being detected. The strange thing is I haven't seen this behavior and wasn't able to replicate it on the issue you opened.

#

My only thought is there is something funky going on with the evenlistener events, but again, I haven't been able to replicate events being swallowed.

fiery night
novel hamlet
fiery night
novel hamlet
#

What the heck

#

I'm testing again on apple silicon. When I click the top bottom, sometimes it is correct, and sometimes it is just delayed:

--> Sending Clicked id='One' target=4v0
? Reading global clicked: id='One' target=4v0
<-- Received Clicked Button id='One' target=4v0
fiery night
#

I went to stackblitz to see if I could spin up a runtime environment to test it on a machine other than mine, but stackblitz doesn't support rust. I wonder if there are any other "docker as a service" sites that let you run Rust programs...

#

Probably wouldn't work anyway since Bevy requires a GPU.

novel hamlet
#

We could use just eventlistener which does not require GPU, but rust playground wouldn't have it available.

novel hamlet
#

I'll push something real quick, and you can give it a try.

fiery night
#

I can do branches

#

I did AssetV2 for months as a branch

novel hamlet
#

There you go.

# 
git clone https://github.com/aevyrie/event_gobbler_mre
cd event_gobbler_mre
cargo run
#

I commited the lockfile and everything, so there should be no room for dependency shennanigans

#

oh! I was able to replicate

fiery night
# novel hamlet There you go.

No change, same results as before.

    Finished dev [unoptimized + debuginfo] target(s) in 0.11s
     Running `target/debug/event_gobbler_mre`
2023-11-26T00:37:11.366115Z  INFO bevy_render::renderer: AdapterInfo { name: "Apple M1 Pro", vendor: 0, device: 0, device_type: IntegratedGpu, driver: "", driver_info: "", backend: Metal }
2023-11-26T00:37:11.716191Z  INFO bevy_diagnostic::system_information_diagnostics_plugin::internal: SystemInfo { os: "MacOS 12.7.1 ", kernel: "21.6.0", cpu: "Apple M1 Pro", core_count: "10", memory: "32.0 GiB" }
2023-11-26T00:37:11.764776Z  INFO bevy_winit::system: Creating new window "App" (0v0)
--> Sending Clicked id='One' target=3v0
? Reading global clicked: id='One' target=3v0
2023-11-26T00:37:13.365465Z  INFO bevy_window::system: No windows are open, exiting
2023-11-26T00:37:13.366656Z  INFO bevy_winit::system: Closing window 0v0
novel hamlet
# 
--> Sending Clicked id='One' target=4v0
? Reading global clicked: id='One' target=4v0
--> Sending Clicked id='One' target=4v0
<-- Received Clicked Button id='One' target=4v0
? Reading global clicked: id='One' target=4v0
#

Well, I have a replication now. I will go dig in.

fiery night
novel hamlet
#

Thanks for the help, always love a minimal repro. 🙏

#

oooooh I found a hint

#

Clicking the second button allowed the missing event from the first button to be received. However the second button even is missing now.

#

Clicking the third button though, allowed the second button event to be recieved, but now the third button event is missin

#

Subsequent clicks work though.

#

Okay, so it is related to graph construction (?) but other events are able to give it a "kick"

#

hmmmmm

#

these are typed, why would eventlistener plugins of different type affect each other?

novel hamlet
fiery night
novel hamlet
#

Also, I might have a way to loop through events to completion in a single frame.

#

so if A -> B -> C, that won't take up to 3 frames to process. Currently the delay is based on system ordering, which is ambiguous between events of different type.

novel hamlet
fiery night
fiery night
# novel hamlet It should be available on crates.io now. `cargo update -p bevy_eventlistener`

OK so most of the issues I was seeing are fixed. I am now receiving all events consistently. The one remaining problem is drag events out of order: that is, when I drag an item, I sometimes receive a Drag event before I receive a DragStart. This is a problem because DragStart is used to set up the state needed for a drag; if I get the Drag event first, then the state won't be properly initialized.

#

In fact, in some cases I see 2-3 Drag events before the DragStart, which is weird.

novel hamlet
#

This is a more fundamental known issue with this approach. I think the only way to get a proper order would be if all pointer events were combined into a single event stream. They are currently parallel event streams with no ordering within a frame.

#

That would also mean that you could only have a single event listener per entity though, unless I also made it possible to have multiple of the same type.

fiery night
fiery night
#

@novel hamlet New problem: I realized that there's a bug in my :hover style selector. In order to make it behave the same as CSS it needs to be "true if pointer is hovering over the element or any descendant"

#

Currently I use the HoverMap, which tells me that the pointer is hovering over an element.

#

However, when I hover over, say, a button label, the button no longer registers as hovering.

#

Now, walking the tree of descendants is going to be way too expensive.

#

So instead, I think what I need to do is iterate through the entries in the HoverMap and walk upwards until I find my element.

#

Note that I don't use Enter/Leave for this because hover styles shouldn't require registering event handlers.

#

Anyway, wondering if there's a more efficient way to do this.

#

One concern is that walking up the tree of ancestors is hard to do from, say, a text node, because there's no specific component on the text node that you can query on - you can query on (&Text, &Parent), but then you get a query containing every text node in the whole app.

fiery night
#

Requiring UI developers to disable pointer events on button text labels seems a non-starter.

novel hamlet
#

You can stick a listener on a root and you can listen for any descendants in the hierarchy being hovered

fiery night
hasty sail
#

I'm rendering my sprites to an off-screen render target with a smaller resolution than my actual window (later scaled up). Is there any way to have the plugin adjust to this context? For example, with the sprite backend the click area of my sprite is way too small.

novel hamlet
#

It might just be the backend or the input system is not accounting to the scaling.

#

Where is that scaling set?

hasty sail
#

I'll send you a minimal reproduction. I don't think this is something that bevy_mod_picking can account automatically. The camera is rendered to a small image, which is then put into a sprite bundle that is scaled up. I was hoping that there's some way to configure the "scale" of the cursor coordinates though, but I can't figure it out..

hasty sail
novel hamlet
#

My guess is it's probably only a couple line change.

#

That's why backends are so small though, to make stuff like this easy to accomodate.

hasty sail
#

What I did was trying to mutate the PointerLocation, but it had some weird effects.

novel hamlet
#

Yeah, I wouldn't recommend that.

#

This is more of an implementation detail of how you are rendering stuff to the target.

hasty sail
#

Hmm, but I'm wishing that there's a backend-agnostic way of doing this. Basically all I want is to "scale" the read coordinates of the cursor location so that it aligns with the offscreen render target.

hasty sail
#

Basically my ideal result:

image.png
novel hamlet
#

How is the scale being applied?

#

Or, how does the sprite backend even know which sprites are visible to the scaled camera?

#

Unless you have the renderlayers set up, my guess is the nonscaled sprites are also rendering to this camera, so you're seeing the sprites being picked behind the texture, which the plugin is completely unaware of.

#

I think you might be able to get around this by adding a shim of sorts, where you create new scaled pointer locations that target the camera that is rendering the texture.

#

Because that seems like the correct solution.

hasty sail
#

The sprite isn't scaled at all. I'm just having the camera render to a small image and then scaling that image up. So the sprite is still, say 8x8 instead of 16x16. The plugin is being accurate here, but I just wanted to manipulate the cursor position.

novel hamlet
#

And then make it so the sprites are only rendering to that camera.

#

Yes, I understand.

#

Are you taking the camera rendertarget, and then piping that to scale up and fit the window?

#

My question is what do you actually mean by scaling it up? Are you rendering to a quad and filling the screen? Are you doing some render graph plumbing?

hasty sail
#

I'm simply using Transform to scale up the SpriteBundle.

#

By that I mean the SpriteBundle containing the render target.

novel hamlet
#

So you are scaling the texture you are rendering to? How does that make it to the screen though?

hasty sail
#

I have a second camera that simply looks at the SpriteBundle.

novel hamlet
#

That's what I meant by by rendering the texture to a quad then pointing a camera at that quad

#

Yeah, exactly

#

Okay, that's what I needed to know.

#

Yeah, I think my guess was right then.

hasty sail
#

Btw, I do have render layers setup, so the character sprites are not visible to that second camera.

novel hamlet
#

Oh, interesting.

#

So, the picking should not be picking with the second camera, because there is nothing to hit

#

I'm not sure how it would be picking at all then, because you would need the pointer locations on the first camera.

#

Okay, yeah, you want a shim then.

#

Basically, you just need a system that takes the pointerlocations (which will be targeting the second camera), and transform them to what the first camera should be recieving.

#

Note the location is both a coordinate, and the rendertarget the coordinate is relative to.

#

This is how I planned to get diagetic UI working, but slightly more generalized

#

Basically, it would raycast against any quads that have a render texture, then do that piping, to use the hit location as a pointer on the quad's render target.

hasty sail
#

When should the system be scheduled? I've tried to mess around with the pointer location but it always ended up spazzing out.

novel hamlet
#

You can skip the raycasting part because you don't need this to work for arbitrary positions of the secong camera.

novel hamlet
hasty sail
#

This is what I tried: .add_systems(PreUpdate, scale_cursor.before(PickSet::Backend)) but it didn't work so well.

#

The debug popup thing is basically flickering like crazy between several positions.

novel hamlet
#

If you enable the egui feature it should work correctly, though I haven't tried with what you're doing

hasty sail
#

I don't think it's a egui limitation hmm. I think the cursor position just can't be overriden easily at this moment. Basically cursor moves => physical position updated => popup updates => my system tries to mutate it => popup updates. Or something.

#

Also, it looks like that once the cursor stops moving, the value just keeps on getting recursively mutated (because it's not updated). In this case it eventually reaches 0.

fn scale_cursor(mut query: Query<&mut PointerLocation>) {
    for mut pointer in &mut query {
        if let Some(location) = pointer.location.as_mut() {
            println!("{:?}", location.position);
            location.position.x *= 0.5;
        }
    }
}
hasty sail
#

Okay, I think I'm starting to get it to work 🤔

pub fn scale_cursor(
    mut events: EventReader<InputMove>,
    mut pointers: Query<(&PointerId, &mut PointerLocation)>,
) {
    for event_pointer in events.read() {
        pointers.for_each_mut(|(id, mut pointer)| {
            if *id == event_pointer.pointer_id {
                if let Some(location) = pointer.location.as_mut() {
                    location.position.x = event_pointer.location.position.x * 0.5;
                    println!("{:?}", location.position);
                }
            }
        })
    }
}

I'll fiddle around with it more later

hasty sail
#

It works now! But drag still doesn't work properly...
It seems that the drag events doesn't rely on PointerLocation (it reads the events directly), but at the same time the callbacks doesn't give enough information for me to calculate the offset (it only gives delta/distance)...

topaz rivet
#

is there an easy way to get the click position in On::<Pointer<Click>>::run?

#

i want to send an event that also contains the world position of the click

novel hamlet
#

event.hit.position

topaz rivet
novel hamlet
#

You can use the Listener system param, as shown in the examples.

topaz rivet
#

i see. for some reason event.hit.position is None

novel hamlet
#

Depends on the backend. UI for example wouldn't be able to return a world position

topaz rivet
#

if i specify multiple backends, does bevy_mod_picking just use the "first one"?

#

curious since multiple backends are specified by default

novel hamlet
#

It uses all of them

barren tree
#

I want only the part where I can generate "input pressed, hovered, etc" events for bevy UI elements, without the automatic event bubbling. I should be able to do that by using everything except the event part of this crate right?

novel hamlet
#

Yup

fiery night
#

@novel hamlet Mouse wheel events: In the browser, "wheel" events are bubbled up the hierarchy just like pointer move events - that is, the wheel event is sent to the current hover element first. Is there a way to get this behavior in mod_picking? Do I need to declare my own Wheel/PointerScroll event type?

hasty sail
#

Is there a way to get the last active PointerId?

novel hamlet
fiery night
#

OK, I found another weird non-determinism when clicking on child nodes in mod_picking. I have a parent element, and then an absolutely positioned child element. No z-index on either. I have a pointer down handler on the parent element which prints the entity id of the target. Basically I want to distinguish between clicks on the parent and clicks on the child.

What I see is that when it prints the entity id, it sometimes prints the parent id and sometimes the child. This is surprising to me, because I'm used to browsers where children always are "in front" of their parent.

novel hamlet
#

Picking is based on depth. If they have the same depth, either can win.

#

It's hard to say though, because the ui backend should behave the way you are describing.

#

I would need a minimal repro to debug/understand.

fiery night
#

However, I wanted to nitpick on one thing: For 2D picking, I think that the correct behavior is not based on depth but on render order, or rather, visible order, which is not always the same thing as z-depth. It's intuitive to the user that if a polygon is covering another polygon, then the one that's visible should be the one picked. However, "depth" only approximately determines rendering order and visibility. In browsers, this is completely deterministic, there are no ties. Two children with the same parent at the same z-level will still have a priority based on their order within the child list.

fiery night
novel hamlet
#

Nice!

novel hamlet
brazen geyser
#

I'm having an issue where the cyan rectagles are blocking picking the items that are on top. Any idea why that would be?

image.png
novel hamlet
#

It's effectively the same bevy_ui hit code that exists in repo, just tweaked to work as a picking backend.

novel hamlet
brazen geyser
#

they're both sprites

#

the cyan are at z 0.1 and the books are at 1.0

novel hamlet
#

If you disable picking on the cyan blocks, do the books work?

brazen geyser
#

yeah

novel hamlet
#

Okay. Hm. Maybe this is a simple z sorting bug. Surprised I haven't seen this.

#

Lemme see if I can replicate in the repo.

#

Is this on the bevy 0.12 version?

brazen geyser
#

yeah

#

picking 0.17

novel hamlet
#

Hm, the sprite example depth is the same direction as your case, and works correctly

#

Are they being rendered to the same target, camera, etc? Anything else different between them?

brazen geyser
#

hmm, I'll take a look at the sprite example and see if anything looks different

novel hamlet
#

Are you using event handlers by chance?

brazen geyser
#

yeah

novel hamlet
#

are these in a hierarchy?

#

are books children to the cyan rect?

brazen geyser
#

no

#

hmm, I think I might be setting the transform wrong on the books somewhere. The debug pointer is saying they're at the same depth.

novel hamlet
#

oh, that's odd

brazen geyser
#

it was a me issue. I was setting the x to zero when I was placing them.

#

not sure why it was still drawing on top

#

but it works now. Thanks for the rubberducky

novel hamlet
#

Cool, glad it's working. Happy to rubberduck

fiery night
#

@novel hamlet Found another potential issue: I'm able to hover over 2d nodes that are clipped due to overflow::hidden.

#

Also, you know what would be super useful? Instead of having separate convenience methods (run, listener_component_mut and so on, if the various components were injectable, for example cmp: ListenerComponentMut. This would let me access various components on both the listener and target.

barren tree
#

@novel hamlet What plugins do I need for just bevy_ui events? I also don't want to use the callback components. How do I get a resource of all click events this frame, and the entity they were clicked on?

novel hamlet
#

What do you mean just bevy ui events? If I understand you, you can disable all the non bevy_ui backends, either by disabling the features, or setting DefaultPickingPlugins to disable them.

#

You don't have to use the callback components at all, they are optional.

#

The plugin emits normal bevy events that you can listen to.

#

Instead of On<Pointer<Click>> you would use Eventlistener<Pointer<Click>>

barren tree
#

I see, ty. I've never used Bevy events lol, I didn't know it used vanilla event listners.

#

If I read the events through that, will it have automatically done bubbling? I don't want that to happe

novel hamlet
#

Event bubbling is provided by bevy_eventlistener, it sits on top of the pointer events. If you don't use the callbacks anywhere, you don't pay a runtime price for it.

#

Restated, if you aren't using On<Event> components you have nothing to worry about.

barren tree
#

Ty!

novel hamlet
barren tree
#

Do you mean EventReader?

novel hamlet
#

Sorry, mistyped, yes

#

I had eventlisteners on the brain I guess. ferrissweatmegaspin

barren tree
#

Np lol

midnight mulch
#

Hey everyone!
Kind of a noob question in regards to best practice here:
I copied drag example (https://github.com/aevyrie/bevy_mod_picking/blob/main/examples/drag_and_drop.rs) to play with and wanted to do two things:
Applied snap to grid and make the target pickable both on DragEnd
I assume I can't achieve it with this, because there's no way I can add the bundle here.

On::<Pointer<DragEnd>>::target_component_mut::<Transform>(|drag, transform| {
    // Pickable::default();
}

I assume the easiest way of doing that is to do it with commands like so:

On::<Pointer<DragEnd>>::commands_mut(|event, commands| {
    commands.entity(event.target).insert(Pickable::default());
    commands.entity(event.target).insert(Snap);
}),

but since this operation runs once should I opt-in for an event here?

GitHub
bevy_mod_picking/examples/drag_and_drop.rs at main · aevyrie/bevy_m...

Picking and pointer events for Bevy. Contribute to aevyrie/bevy_mod_picking development by creating an account on GitHub.

novel hamlet
#

Please read through the event listener example or the docs.

#

target_component_mut and commands_mut are small helpers that only do one tiny thing, like mutating a specific component.

#

But they are all just bevy systems, and you can write your own.

barren tree
midnight mulch
#

Thanks!
I figured On<E>::run would be an overkill for my case, but yeah, def nice to know!

Also, I noticed from the spin in the file since we are not removing the bundle in the end it will trigger forever (and multiply the rotation till the end of time), which I assume it's undesired so I did it like hack like so:

fn spin(mut square: Query<(Entity, &mut Spin, &mut Transform)>, mut commands: Commands) {
    for (entity, mut spin, mut transform) in square.iter_mut() {
        transform.rotation = Quat::from_rotation_z(spin.0);
        let delta = -spin.0.clamp(-1.0, 1.0) * 0.05;
        spin.0 += delta;
        if spin.0 <= 0.01 {
            transform.rotation = Quat::default();
            commands.entity(entity).remove::<Spin>();
        }
    }
}

Is this ok, or am I overthinking it?

novel hamlet
midnight mulch
novel hamlet
#

Sure!

novel hamlet
midnight mulch
novel hamlet
#

Also, I don;t think you want to reset the rotation. The idea is that you are rotating it, you don't want it to snap back to its original orientation.

midnight mulch
novel hamlet
#

Oh, yeah, the spinning is setting the rotation, you're right, you would want to set it to the default.

novel hamlet
#

... do you think it would help to just make the spinning logic into its own little plugin in the example?

#

Then we could properly handle Spin cleanup, but keep it separate from the rest of the example.

midnight mulch
novel hamlet
#

To pre-nitpick, I think the plugin needs two little systems: one to do the rotation, and one to clean up Spin components near zero. That should make it clear what the plugin does, without needing to add docs.

barren tree
#

@novel hamlet what's the reccomended way to have a library depend on bevy_mod_picking (I only need the bevy ui backend), but also let the user use bevy_mod_picking if they want it themselves?

#

RN I'm doing

bevy_mod_picking = { version = "0.17", default-features = false, features = [
    "backend_bevy_ui",
] }

And then re-exporting bevy_mod_picking.

novel hamlet
#

You could just depend on bevy_picking_ui, instead of the entire mod picking workspace?

#

Then users need to add bevy_mod_picking as a dep and add DefaultPickingPlugin to their app, if they haven't already.

#

That, or you add those to your own plugin, but give users the ability to disable them, so they have a way of configuring it themselves.

barren tree
#

Thanks

novel hamlet
#

The latter is probably the most idiomatic. Similar to bevy's DefaultPlugins.

wide ginkgo
#

coming back to bevy after not using it for a year, and wow this crate's api is much easier to use! thanks for all the updates <3

midnight mulch
crude talon
#

Question about the selection implementation. Is there a reason there's a PickSelection component with a bool inside as opposed to adding a removing a PickSelected component on entities? Is the trade-off between adding/removing components vs. potentially iterating over many more entities worth it?

novel hamlet
#

If you really want to be able to query for Selected, you can add a system that looks at Changed<PickSelection> ands adds/removes your own selection component.

crude talon
crude talon
fiery night
#

@novel hamlet Is it possible that the listener and target entities could be injectable? Thing that I would like to be able to do:

  • Access the listener entity, but also call .stop_propagation().
  • Access various components on both the listener and the target entity.
    The first can be done using .run(), but not by the other methods such as .target_commands_mut() because the listener argument is immutable.
    The second, I'm not sure how to do at all.

What would be ideal is if access to the listener and target could be injectable params, c1: TargetComponent<SomeComponent>, c2: ListenerComponent<OtherComponent> so that we can mix and match according to our needs. However, I am not sure whether this is possible since the injection would require context.

novel hamlet
#

Why can't you use run for both of these?

#

The reason the helpers don't have that is to stay minimal. They are really only useful for simple cases. Expanding their scope defeats the purpose.

#

If you have specific helpers in mind, you can add your own with an extension trait, and it will look and behave as if they were part of the picking library.

#

You can access components with queries, as you would with any other system. E.g. my_query.get(listener) or my_query.get(target)

#

Although I'm not sure why ListenerInput is immutable, off the top of my head.

fiery night
# novel hamlet Why can't you use `run` for both of these?

I can use run. However, I think it might be a nicer API if, instead of having specialized helper methods that each injected one special thing, allow a more a la carte strategy where you ask for the things you need in the function signature.

novel hamlet
#

allow a more a la carte strategy where you ask for the things you need in the function signature.
Not to be obtuse, but isn't that exactly what a bevy system is?

#

I think you want something between the flexibility of a bevy system, and the specialization of the existing helpers?

fiery night
#

What we need is something like a "contextual SystemParam". Right now you can inject things, but the specification of what you inject is globally scoped: you can get a resource, or a query, or an event, but all of those things start from World, not from anything more specific. But for an event handler, there's a context, which consists of the event, target and listener. The current specialized methods on bevy_eventlistener give you convenient access to various aspects (components, entity muts, commands) of those specific contextual things. A contextual SystemParam would let you define SystemParams that take some context as a starting point.

#

Such a thing would have broader uses than just bevy_eventlistener I think. However, I'm not sure it's possible.

novel hamlet
#

This sounds kinda similar to EntityCommand

barren tree
#

@novel hamlet it looks like the Over/Out events can have entities blocked by other entities on top? Is there a different event that dosen't do that?

barren tree
#

I think I want Enter/Leave events...

fiery night
#

Specifically: "An element also matches :hover if one of its descendants...matches the above conditions."

#

And: "Since the :hover state can apply to an element because its child is designated by a pointing device, it is possible for :hover to apply to an element that is not underneath the pointing device."

#

My understanding is that this is not how bevy_mod_picking works; which means that if you want something like the CSS behavior you are going to have to emulate it.

#

I don't handle this correctly either BTW - for now, I've worked around the issue by disabling pointer events on the child elements, but that is not a good long-term solution.

#

Basically I think the hover behavior you want can be described by the following algorithm: To determine whether entity E should be styled in a hovering state:

  • look up the current hover entity in the HoverMap.
  • follow its ancestor chain until you find E.
  • If E is present, then it's hovering, if you reach the end of the ancestor chain then it is not hovering.
barren tree
#

I ended up making my own mouse enter/leave event based on RelativeCursorPosition::mouse_over()

#

Yeah dosen't match the CSS spec either, but w/e, not sure that matters...

fiery night
#

Well, matching CSS behavior exactly isn't the goal; the goal is "no surprises" for people using the library, which means getting a sense of what their expectations will be.

barren tree
#

Mhm. I honestly think the CSS spec is more surprising 😛

#

You could also always bubble the hover events if you wanted (I don't)

fiery night
#

Well, early on I ran into "surprising" cases where a button would highlight when you hovered over it, but if you hovered over the text caption it would no longer highlight.

barren tree
#

What kind of caption?

fiery night
#

A Text entity

barren tree
#

Positioned where relative to the button?

#

Also the way I have it setup, I don't use CSS selectors for changing styles. Styles are just properties you can set. So if you wanted to, it's easy to make a use_state() for the hover, and then tie that to multiple entities to group the hovers together/

fiery night
#

Bubbling the hover events won't work because the in and out events may get received out of order - so when you move the pointer from child A to child B, you may get the "in" for B before the "out" for A, which will confuse your state and make you think that it's "out".

#

With Bevy events, only events of the same type are guaranteed to be received in the order sent. There's no such guarantee for events of different types because they are received by different event readers.

barren tree
#

Mhmm true

fiery night
#

This is why I ended up using mod_picking's HoverMap, which is always up to date and consistent.

crude talon
#

How are backends for specific entity types selected?

#

I made a custom backend for one type because I didn't want to use the mesh backend for it, but because the mesh still exists, sometimes the mesh backend will be active for it

#

Seems like a coin toss whether the custom backend or the mesh backend will be scheduled first

#

I don't think I can just disable the mesh backend because I still use it for another type of entity

novel hamlet
#

How would this work? You would hit test every single entity, and if the point is in bounds it triggers an event for every entity it is inside?

#

I'm basing these events on the js pointer events as a relatively sane starting point.

#

pointerenter and pointerleave looks at the hierarchy from what I understand, so if the pointer is on an entity or any of its children, it will trigger

#

But those are not implemented yet.

novel hamlet
#

In the ui example in the repo, I handle this by making text entity Pickable::IGNORE, which allows picks to pass through the text.

fiery night
novel hamlet
#

You can also set the should_block_lower = false field on Pickable.

fiery night
#

What I did was this:

/// True if the given entity, or a descendant of it, is in the hover map for PointerId::Mouse.
///
/// This is used to determine whether to apply the :hover pseudo-class.
pub fn is_hovering(&self, e: &Entity) -> bool {
    match self.hover_map.get(&PointerId::Mouse) {
        Some(map) => map.iter().any(|(mut ha, _)| loop {
            if ha == e {
                return true;
            }
            match self.parent_query.get(*ha) {
                Ok(parent) => ha = parent,
                _ => return false,
            }
        }),
        None => false,
    }
}
novel hamlet
fiery night
#

Say I have a button with a caption. The button is a node, the caption is a child node. The caption is not lower than the button; the z-index is the same for both, and since the caption renders on top of the button, it's technically "higher"

novel hamlet
#

Right, so putting should_block_lower = false on the text will allow picks to pass through.

novel hamlet
fiery night
#

OK I see what you are saying.

fiery night
#

This gives the behavior that I want without having to tweak the styles of all the children.

novel hamlet
#

I think pointerenter is what you want for hover?

The pointerenter event fires when a pointing device is moved into the hit test boundaries of an element or one of its descendants, including as a result of a pointerdown event from a device that does not support hover (see pointerdown).

novel hamlet
fiery night
#

Are we talking about @barren tree 's case or mine? I don't have a problem that needs solving right now

novel hamlet
novel hamlet
#

So it will only run on the meshes that aren't using your custom backend.

novel hamlet
#

Because I agree it is surprising if you expect something like :hover. Unfortunately the terminology used in CSS/PointerEvents isn't always consistent, so it's difficult to even use the word "hover" and be on the same page. 😄

fiery night
# novel hamlet Yes, Jasmine's comment, but also trying to understand if this would avoid the "s...

What I mean is, for programmers building 2D widgets that have hovering behavior, whether they will expect hover to work like CSS (where an element is considered in a "hover" state because the pointer is over a child element), or whether they will expect hover to only apply when the pointer is over the element itself and not over a child.

I think that "least surprise" in this case is by having the UI framework emulate the CSS behavior, which is what I have done.

That being said, I would accept the idea that for non-2d-UI use cases, you might want it to work the other way: that is, to be able to hover over a child entity without having its parent to also be considered hovering.

barren tree
fiery night
#

I think it comes down to convention: in 2D UIs you don't generally see buttons inside of buttons, usually interactive widgets with hovering exist within containers that are not interactive (other than scrolling, etc.) - but for use cases outside of that, all bets are off. For example, in a 2D vector drawing app, it would be common to have a selectable item inside a selectable item.

novel hamlet
#

But I'm following the JS pointer event design, which is for 2d, unless I'm misunderstanding something.

#

If you want the child/parent thing in js using pointer events you would need js pointerenter/leave, which simply isn't implemented yet in mod picking.

fiery night
#

Not asking you to do anything 🙂

novel hamlet
#

All good!

novel hamlet
crude talon
#

Somewhat related question: it says in the backend.rs file that HitData.depth is the "distance from the pointer to the entity into the screen". Does this mean that the z-transform of an entity isn't meant to be passed in as-is?

#

Context being that I have an entity at -0.001 but it is registering as "above" a neighboring entity with positive z-values.

barren tree
#

But that ensures the hover dosen't get started until it's visible atl.

#

Or maybe this is better done a different way... idk

fiery night
#

Clicking on the backdrop also requests a close, same as hitting ESC

barren tree
#

Makes a lot of sense, thanks

crude talon
# crude talon Somewhat related question: it says in the backend.rs file that HitData.depth is ...

Still having this issue. Spent a few hours trying to figure it out but haven't made much progress. I basically made a custom backend for picking straight lines (based on a proximity threshold) and they keep registering despite the entities above them having blocking on.

Another thing I'm wondering about is whether there must be an entity that eventually captures a pointer. I just turned off input blocking on my background rectangle and now whenever my pointer isn't over any other entity, the first spawned entity gets hovered.

These issues together make me think that I'm misunderstanding something fundamental about how this plugin works.

buoyant escarp
#

It looks like pointer events only fire when they are hovering an entity. What if I want to do something when the a pointer interacts somewhere with no hover candidates?

#

Example: Scale the target when the user clicks and drags over a mesh, but zoom the camera when the user clicks and drags over nothing.

#

Is there a clean way to do this without having to replicate most of the pointer_events system?

#

Is the recommended solution to add a background entity/new pick back-end to receive pointer interactions?

hasty sail
buoyant escarp
#

Right, but if I want to handle any kind of complex touch gestures this kind of becomes a headache. I am having to basically replicate the pointer_events system to track the state of drags/clicks and so on. Was wondering if there was a better solution.

crude talon
crude talon
#

It seems to me like there's a difference with how depth gets calculated between the different backends. The sprite backend, and thus mine because I based it off that one, just sets the z transform of the entity as the depth in each HitData. The raycast calculates the depth as the distance from the near clipping plane to the entity. Is this intended or an inconsistency?

novel hamlet
#

Depth should be consistent within a camera. Note there is also an order field on hitdata, which should probably be the camera order.

crude talon
#

Yeah, that's taken into account

#

I managed to fix the issue in my own project by manually subtracting the near clipping plane from my entity z transform in the custom backend

novel hamlet
#

That sounds like a possible bug with the sprite backend?

crude talon
#

Yup

#

I can file an issue and/or a PR for you if you'd like? Tomorrow though, since it's 3am here

#

Do you need the issue or can I jump to the PR then? Disclosure, this would be my first time contributing to open source 🙂 so I don't know the conventions by heart yet

novel hamlet
#

(thanks for contributing!)

novel hamlet
# buoyant escarp Example: Scale the target when the user clicks and drags over a mesh, but zoom t...

Most of those events are built on the assumption that some entity is being interacted with. To keep the API somewhat unified, I would probably have a backend for this purpose (clicking on "nothing" can end up being complicated when UI and other interactions are involved), or I would hook into the InputMove and InputPress events. The latter would at least tie in nicely with the existing Pointer stuff.

#

I'm also open to suggestions.

buoyant escarp
#

I'm experimenting with a backend that always adds the window entity as a minimum-order hit. That part actually is super nice, because it lets multiple pointer interactions coexist (you can do a touch gesture against the background and interact with the UI at the same time). I'm now trying to implement a two-finger zoom-pan-rotate gesture for a 2d scene.

#

One hitch is that currently there's no way to send either a Drag event or a Zoom event but not both. I'm having a go at writing an event pipeline where multiple stages can filter and combine the pointer events before they are dispatched to the target entities.

novel hamlet
#

One hitch is that currently there's no way to send either a Drag event or a Zoom event but not both
Could you explain that a bit more please?

buoyant escarp
#

My multitouch zoom gesture sends a custom event when detected, but the individual finger touch pointers also cause standard mod_picking::event drag events to be sent. This is not ideal.

#

It would be nice to consume the drag events and turn them into a touch gesture event before things are dispatched to the picked entities. There’s no way to “un-send” events in bevy (that I know of).

#

Sorry if this doesn’t make sense, it’s a bit late here.

crude talon
#

@novel hamlet PR is up 🙂

novel hamlet
novel hamlet
# buoyant escarp My multitouch zoom gesture sends a custom event when detected, but the individua...

I've been thinking about this a bit more. I think what you want to do is replace the provided touch inputs with your own system. The current touch system is super simple - when a touch is added a pointer is spawned, it follows the touch, and is despawned when the finger lifts.

You would replace this with a more complex system that first detects if a group of inputs should be considered a single pointer or gesture. If it is detected as a group of fingers, it would spawn a single pointer that represents all of these, and attach a component so users know how many fingers were used. You could also use this system to decide if a multitouch should even get a pointer, or if it should instead be a gesture, which might not want to have pointer events.

#

This is something I would be interested in including in the library if you are interested in that.

novel hamlet
buoyant escarp
honest dirge
#

Is there an existing way to make Picking::IGNORE apply to all children too?
I'm doing object highlighting by spawning a child entity with Picking::IGNORE on an entity. That child ends up with has a number of its own nested children with Aabbs (spawned from GltfNode tree). Once that happens, those child Aabbs start firing triggering events because the top-most entity is a valid picking target.

Example video 6v0 is the top-level entity with picking, as the cursor gets close to the edge of the cell, I spawn the child cell (11v22) with Pickable::IGNORE (not shown, but I double checked it just now). A system then spawns a number of children under 11v22, including 13v21 which has Aabb and Mesh. This is the yellow edge we're seeing on screen. From the debug display you can see we're getting picking events from that innermost child (13v21).

Is the only way to walk the entities under that child and mark every Aabb entity with Pickable::IGNORE?

honest dirge
#

Hmm, from reading the docs, I feel like this should have worked to block picking events of child entities, but it doesn't appear to:

Pickable {
    should_block_lower: true,
    should_emit_events: false,
},
honest dirge
#

digging further, the RaycastSettings.early_exit_test is only getting called with the top-level entity (6v0), not the intermediate one that has Pickable.should_block_lower = true. Moving on to how early_exit_test is used.

honest dirge
#

Ok, in general Raycast::cast_ray() doesn't consider the entity heirarchy. So for an entity with Pickable.should_block_lower = true to have any affect, it must also have a Handle<Mesh> & GlobalTransform. And if I'm reading this right, it also needs to have an Aabb that overlaps all children Aabbs.

#

That's why this isn't working for me. I'm spawning the equivalent of a SceneBundle entity that doesn't have its own Mesh or Aabb.

commands
    .spawn((
        HighlightMeshTag,
        mesh_handle.clone_weak(),
        SpatialBundle {
            transform,
            ..default()
        },
        crate::mod_picking::Pickable {
            should_block_lower: true,
            should_emit_events: false,
        },
    ))
    .set_parent(entity);
buoyant escarp
#

Perhaps you could write a backend which wraps raycast and checks the parent hierarchy before registering the hit?

novel hamlet
#

This feels like a bit of an XY problem. What are you trying to accomplish here?

#

If you are trying to spawn a scene to highlight the top edge of the column being interacted with, and you don't want that scene to be pickable, you need to either:

#

The reason for this is very ECS-y. The components of an entity describe that entity. If we had to walk the tree for every property of an entity to emulate inheritance, things would get very slow very quickly. Even if it was made fast, it would add significant complexity, and is not super idiomatic IMO.

honest dirge
# novel hamlet The reason for this is very ECS-y. The components of an entity describe that ent...

Agreed on the complexity & performance points. I think the naming & documentation of Picking.should_block_lower gave me the impression that hierarchy was being taken into account. But lower here is spatially lower, not hierarchically lower.

Thanks for the require_markers note. It looks like either way I’ll be adding a system to traverse hierarchy adding a Pickable component. But it’s better performance to do this as a dedicated system than in the backends.

buoyant escarp
#

Should I open a PR fixing the touch pointer system to add the correct window if/when touch events are fixed?

novel hamlet
novel hamlet
novel hamlet
honest dirge
#

As far as the docs, I can throw together a PR to clarify spatial lower as opposed to heirarchical lower.

novel hamlet
#

Can these not be part of the serialized scene? E.g. save the gltf as a bevy scene and add the pickable components to that scene file?

#

tbh not sure how serializing the meshes in a scene works. 🙂

honest dirge
#

I'm not using Scenes. Setting up scenes for all the objects, widgets, items, characters I'll be making in blender is just a painful workflow. Instead I'm putting together a named node-based workflow. So instead of spawning SceneBundle with a Handle<Scene>, I actually spawn Handle<GltfNode> and have my own systems that spawn the meshes and materials found under the node. It's effectively a subset of the scene-based workflow, but allows me less headache in blender. It also allows me to use the actual node names from blender instead of widgets.glb#Scene12.

As for adding the Picking::IGNORE components to the entities before-hand, that won't work with my workflow. Scenes do store everything as entities in these independent World instances that seem to get merged in when scenes are added. It may be possible in that scenario. The GltfNode on the other hand is a single entity and I spawn entities from the struct elements within it.

Because I am also spawning some nodes that do need to be pickable, it's best for me to just fire an "it's loaded" event, and apply Pickable::IGNORE to the hierarchy. I'm sure it won't be the only time I need that loaded event.

novel hamlet
#

Makes sense!

solid breach
novel hamlet
#

Yes, this is because the events are independent types, and not a single type. Not certain what the correct solution is, because without unique types, it makes eventlisteners much worse.

solid breach
#

My first intuition was that I could reorder some system schedules on those independent event types, but I didn’t dive much deeper, I guess some solutions might impact eventlistener indeed.

buoyant escarp
#

Unify the events and then make an event-handler builder HandlerBuilder<Drop> with on_start, on_end, on_enter, on_leave, and on_over selectors and impl Into<event_listener::On>?

buoyant escarp
#

I was already thinking about this because a fixed enum PickEventState { Start, End, Over, Enter, Leave } and a single event for each interaction might also simplify gesture recognition a bit.

buoyant escarp
tepid sun
#

Hi!

I've got an issue - Is there any way to send two different events using the same type of pointer event? Like this;

On::<Pointer<Down>>::send_event::<MakeBattleMove>(),

                On::<Pointer<Down>>::send_event::<ResetClosestTile>()

This setup just leads to a crash. I could of course create a third event and then delegate from there, or just send event to ResetClosestTile from MakeBattleMove, but I'd prefer this clean setup

native vessel
#

You can call a system that sends 2 events maybe.

novel hamlet
#

And send as many events as you want.

tepid sun
# novel hamlet I'd use `On<E>::run()`

I assume you mean something like this;

 On::<Pointer<Down>>::run(
                    |event: Listener<Pointer<Down>>,
                     mut reset_event: EventWriter<ResetClosestTile>,
                     mut battle_move_event: EventWriter<MakeBattleMove>| {
                        reset_event.send(ResetClosestTile);
                        battle_move_event.send(MakeBattleMove(event.target));
                    },
                ),

It works, just wish I could do something like
On::<Pointer<Down>>::run(reset_closest_tile, move_in_battle)

buoyant escarp
#

wrap them in a tuple?

tepid sun
#

Sorry about the horrible formatting, no idea how to make it prettier haha

tepid sun
buoyant escarp
#

weird, I thought tuples of systems implemented IntoSystem

#

it does seem like that should work.

tepid sun
#

Yeah not sure, it says that it isn't implemented

buoyant escarp
#

TIL I was wrong about how that worked. Doesn't look like it would be easy to implement.

tepid sun
#

Shoot. Oh well, guess I'll just go with the above. Cheers!

scenic kiln
#

I'm trying to change the material on hover for a tile using bevy mod picking but I can't seem to get it to work.

I've got the following code:

commands.spawn((...,
Highlight {
  hovered: Some(HighlightKind::Fixed(hover_handle.clone())),
  pressed: None,
  selected: None
},
PickableBundle::default(),
MaterialMesh2dBundle {
  transform: ...,
  mesh: mesh_handle.clone(),
  material: materials.add(ColorMaterial {
    color: Color::NONE,
    texture: None,
}),
..default()
},
...

Even after adding the Highlight component the tiles still highlight using the default colors and materials for the plugin. Am I missing something?

novel hamlet
#

What kind of asset are you using?

#

ColorMaterial?

scenic kiln
novel hamlet
#

If you can put together a minimal reproduction, I can take a look. Otherwise it might take me a bit to get to this. What you have looks correct from what I can recall.

flint sundial
novel hamlet
#

By default, pickable things block lower pickable things.

flint sundial
#

what do u mean by lower in this context?

novel hamlet
#

iirc it was more for the interaction highlighting, without it, the text would receive the heighlight, so this was an easy way to express that I wanted the hit to skip the text and go to the button.

novel hamlet
flint sundial
#

ahh ok so the bubbling is relative to the entity's physical position rather than the parent/child hierarchy?

#

for some context this is relevant to me because if i were to dynamically add pickability to some entity e.g. to add a background color change on hover, doesn't this mean i would need to recurse down all descendants to add IGNORE to ensure hovering on any part of the entity will bubble up the hover?

novel hamlet
flint sundial
novel hamlet
#

Pickable is completely optional.

#

You don't need it to make an entity pickable.

flint sundial
novel hamlet
#

A backend makes entities pickable. This is entirely divorced from hierarchy. It's all about whether or not a pointer is hitting an entity. I.e. is the pointer within the bounds of that entity.

#

Bubbling is a different concept that allows an event that is targeting one entitiy, bubble up the hierarchy automatically, and trigger event listeners at any point as it bubbles up the hierarchy.

#

The only reason the text was marked as ignore was because text is a ui node that is drawn on top of the button, so it would be reported as the topmost hit under the pointer - there are potentially many ui nodes that the pointer is within - the text, the button the text is on, the node that the button is in, etc, etc.

#

So by making the text not pickable, the topmost thing under the pointer to the ui backend, is the button node.

#

Now, in addition to all of this, when the entity that is hit expereinces some event, like a click, or a hover, that event will also bubble up the hierarchy.

novel hamlet
#

Like an event listener in JS

#

The button, the text, the node all those are in, all of those are pickable automatically.

flint sundial
novel hamlet
novel hamlet
flint sundial
flint sundial
# novel hamlet It does bubble up

ok i was able to confirm this 🙏 getting warmer, i think i'm able to get my desired behavior by just setting all nodes to be IGNORE by default

flint sundial
novel hamlet
flint sundial
#

thanks so much for helping me figure this out, now adjacent question: can i set a particular event to not bubble up? e.g. without having to do Pickable { should_emit_events: false, ..default() } which applies across all events

novel hamlet
#

You can stop an event from bubbling any furth from inside an eventlistener callback, but you can't disable that behavior globally.

flint sundial
#

is there a way to specify that On::<Pointer<Over>> always runs after On::<Pointer<Out>>?

cold idol
#

Hi, I was trying to use bevy_mod_picking along with bevy_ecs_tilemap but it doesn't detect enitites created in with the tilemap plugin. Any idea why this happen or how to fix it?

cold idol
crude talon
#

Can't seem to get ui picking to work. This should be enough right? I also enabled the feature in the toml, though I saw that it's included in the defaults

    let top_bar = commands.spawn((
        ButtonBundle {
            style: Style {
                flex_direction: FlexDirection::Row,
                width: Val::Px(size.x),
                height: Val::Px(12.0),
                align_items: AlignItems::Start,
                align_self: AlignSelf::FlexEnd,
                justify_content: JustifyContent::Center,
                ..default()
            },
            background_color: BackgroundColor::from(col),
            ..default()
        },
        UiTopBar,
        PickableBundle::default(),
        On::<Pointer<Drag>>::target_component_mut::<Style>(|drag, style| {
            style.left = Val::Px(drag.event.delta.x);
            style.top = Val::Px(drag.event.delta.y);

        }),

    )).id();```
novel hamlet
crude talon
#

Hmm, shouldn't be

#

No entity is registering at that spot

novel hamlet
#

If you provide a minimal repro I can help.

flint sundial
#

just wanted to follow up on this #1038322714320052304 message i believe i'm having the same issue as #1038322714320052304 message

#

Bubbling the hover events won't work because the in and out events may get received out of order - so when you move the pointer from child A to child B, you may get the "in" for B before the "out" for A, which will confuse your state and make you think that it's "out".

novel hamlet
#

Yes, this is a known issue that will require adding coalesced picking events.

#

As well as work in the eventlistener crate

#

We also lack timestamps on events from bevy/winit that make this impossible to solve for long frames.

flint sundial
#

also i made the observation that the ordering of the systems stays the same during the runtime of the program, so it does seem to be "decided" at some point even now

novel hamlet
#

Yes, it just depends when the event sending systems run.

flint sundial
#

so it's possible to "luck" into the correct ordering, i wonder if there's a low level lever i can pull to just set it in stone

novel hamlet
#

The events themselves may be ordered, but the other issue is the callbacks themselves might be executed in a different order as well.

novel hamlet
#

All Over events are sent at the same time, and all Out events are sent at the same time. There is no way to interleave this correctly like Out Over Out Over, it will either be Out Out Over Over or Over Over Out Out

flint sundial
#

yes i'm fine with all outs -> all overs

novel hamlet
#

We can do that today with system ordering

#

If you wanted to, you could copy paste the definition of the DefaultPickingPlugins and order the systems however you need.

flint sundial
flint sundial
novel hamlet
#

I'm mostly surprised how many people are using eventlisteners, I added it as a nice little bonus, knowing this limitation existed.

#

Oops.

flint sundial
novel hamlet
#

Yup

flint sundial
buoyant escarp
#

Why does plugin ordering matter here?

#

They are pushing to a resource?

novel hamlet
#

The eventlistener callback execution order

#

You want guaranteed order that an On<Over> runs after On<Out>

buoyant escarp
#

Shoot. I'm working on plugin dependencies and it would totally mess this up. I should probably make an easy way to still run a bunch of plugins in sequence.

novel hamlet
buoyant escarp
#

But do these plugin's Plugin::build need to run in a specific order? Or does the order effect the functionality? Sorry, that's what I thought you meant.

novel hamlet
buoyant escarp
#

Ah cool, never-mind then. You had me worried.

flint sundial
#

plus identified another issue, which i can solve in the reactivity layer but may(?) be a logical issue in the pointer system; the issue is simply that Pointer<Out> does not fire on entity despawn, so if u are maintaining a hovered state and the hovered entity despawns, the unhover logic does not fire

novel hamlet
#

Unless there is a check happening on the entity somewhere, and the fact the entity no longer exists is preventing the event from sending.

flint sundial
novel hamlet
#

Yup

#

HoverMap

#

and LastHoverMap iirc

flint sundial
#

the despawn removes the Out listener so it can't fire?

novel hamlet
#

The event fires, but there is no listener to trigger

#

So if you look at the Pointer<Out> events, you will see that entity listed

#

But when the eventlistener plugin runs, it will see that event and go to the target (to trigger any callbacks), but the target no longer exists.

flint sundial
#

possible solution: store all Pointer<Out> callbacks in a resource, so the callback life is decoupled from the entity's, and when such entities despawn, the callback in the resource can be cleaned up the next frame by a system that listens to despawns (are there global despawn events already?)

scenic kiln
#

is there way to kill all events in flight? does <EventReader<MyEvent>>.clear() work for this?

novel hamlet
flint sundial
novel hamlet
#

I kinda thing this is expected behavior? If you despawn something before events can run, of course this will happen. This is an issue with despawning in general. If you have things you want to do prior to despawn, it's better to tombstone entities, do any cleanup, then actually despawn. That, or just defer despawning to the end of the schedule so callbacks have a chance to run.

#

This would also add complexity and open up other, more insidious bugs. What if your callback is (very likely) trying to mutate components on the entity the callback is on? This can't happen if you allow a callback to be decoupled from the entity it is on.

flint sundial
#

so i think it should be up to the user to e.g. safely mutate components

#

but i also think it makes sense to keep the callback handling symmetric + you've offered some alternatives that should work for me, i'll try them out 🙏

novel hamlet
#

prevent running systems that have nothing to do with the particular despawned entity
Why would they have nothing to do with the despawned entity? The entire point of event listeners is to associate behavior with a specific entity. If you don't want that behavior, you can use normal events, e.g. EventReader<Pointer<Out>>

#

Maybe I'm not understanding how you are using the feature?

flint sundial
flint sundial
scenic kiln
tepid sun
# 
/// Automatically records the "initial" state of highlightable entities.
pub fn get_initial_highlight_asset<T: Asset>(
    mut commands: Commands,
    entity_asset_query: Query<(Entity, &Handle<T>), Added<PickHighlight>>,
    mut highlighting_query: Query<Option<&mut InitialHighlight<T>>>,
) {
    for (entity, material) in entity_asset_query.iter() {
        match highlighting_query.get_mut(entity) {
            Ok(Some(mut highlighting)) => highlighting.initial = material.to_owned(),
            _ => {
                commands.entity(entity).insert(InitialHighlight {
                    initial: material.to_owned(),
                });
            }
        }
    }
}

If I have understood this correctly, the initial material is saved, and then the highlighting will be based on this initial state, even if we modify the material in another function.

Is there any way to update the 'initial' state, so highlighting with HighlightKind::new_dynamic actually reacts on the current state of the material?

novel hamlet
#

You can edit the InitialHighlight directly

tepid sun
#

I'll look into it now, thanks!

tepid sun
#

Not entirely sure how I should do it, bit lost with Handles..

materials.get_mut(color_material.id()).unwrap().color = Color::rgba(0.0, 0.0, 0.0, 0.5);
            initial_color_material.initial = Handle::Weak(color_material.id());

This compiles, but I run into crashes on the unwrap of the materials_get_mut, assumingly due to improper modification of the initial material.
Can I somehow directly modify the initial material's colour?

tepid sun
# 
materials.get_mut(color_material.id()).unwrap().color = Color::rgba(0.0, 0.0, 0.0, 0.5);
            initial_color_material.initial = color_material.clone();

This works! I completely forgot that the ColorMaterial already is queried as a Handle ..

Thank you! 🙂

steel imp
#

Re: bevy_eventlistener if I replace a listener it still seems to use the old callback:

// previously:
On::<Task<Perform>>::run(pickup_item)

// changed by:
            commands
                .entity(event.task)
                .insert(On::<Task<Perform>>::run(move_load_to_destination));

// still calls pickup_item... ?

Is there a correct way to replace an event callback?

#

Ah, removing the callback and then adding the new one the next frame works. It's just inserting over an existing one that does not.

novel hamlet
steel imp
flint sundial
#

should Pointer<Out> fire when the pointer moves from a ui border to ui background of some node?

willow yacht
# 
            // Callback systems require exclusive world access, which means the system cannot be run
            // in parallel with other systems! Callback systems are very flexible, but should be
            // used with care. If you want to do something complex in response to a listened event,
            // prefer to instead use `send_event`, and react to your custom event in a
            // normally-scheduled bevy system (see send_event usage below).
            On::<Pointer<Move>>::run(change_hue_with_vertical_move),
            // We can use helper methods to make callbacks even simpler. For drag-to-rotate, we use
            // this little closure, because we only need to modify the target entity's Transform:
            On::<Pointer<Drag>>::target_component_mut::<Transform>(|drag, transform| {
                transform.rotate_local_y(drag.delta.x / 50.0)
            }),

from https://github.com/aevyrie/bevy_mod_picking/blob/main/examples/event_listener.rs

a few questions (bear with me, the first two may just be general bevy questions (maybe all 3 are, i'm really, really new and really overwhelmed lol))

  1. both ::run and ::target_component_mut suffer from the non-parallelizable issue, right?
  2. in practice, should this be something I worry about? i've spent a while trying to find out other ways to implement this behavior, but haven't found anything that feels as clean to use
  3. just for laughs, how could I do something like that while keeping it parallel?

I see the example

            receive_greetings.run_if(on_event::<DoSomethingComplex>()),

but I'm wondering if there's a way to do something more like

fn move_system(mut sprites: Query<(&mut Transform, &Velocity)>) {

with one of the queries being something like On::<Pointer<Drag>>?

#

oh, actually, i may have thought of a pattern that works here, would be interested in feedback
on click, i insert some tag component into the entity that says like "Clicked"
then my system queries for <Transform, Velocity, Clicked> and if all three exist, it does the Transform and removes the Clicked tag

flint sundial
novel hamlet
novel hamlet
# willow yacht ``` // Callback systems require exclusive world access, which means ...
  1. yes
  2. Only if you are doing something that will take a while. However, as the docs say you can always take that long-running work, and do it in a normal parallel system, so there isn't really anything to worry about, it's trivial to switch to an event-based trigger where you actually need. it. I would just use callbacks until you actually have a problem.

i've spent a while trying to find out other ways
All you have to do is what the docs say. Take the expensive thing, and move it into a normal system that waits for an event to do that expensive thing. Your callback will instead just send an event to trigger the expensive thing in the normal system.

  1. I don't understand. The run_if, is like adding On::<Pointer<Drag>>, the problem is you need to specify which event you are reacting to, otherwise it would run for any drag on any entity. That's why it uses a special trigger event. You could always rename it something more useful like SquareDragged.
willow yacht
#

peepoNotes i see, the run_if thing actually clears it up a lot, its like the event example but you don't capture the extra info like which specific entity
alrighty so i won't worry about it unless its something expensive!

novel hamlet
halcyon forge
#

I already asked this in github discussion but since this place seems more active than there, I'll ask here too:

I am making a chess game, in that pieces are in depth 1 and board squares are in depth 0. When I want to move a piece, I first click on the piece and then click on board square to move.

But since the events only triggers for entity of higher depth, if I click on square with opposite piece, it triggers the click event for the piece. Say if I selected a white piece and I can't move the piece to square with black piece and capture it.

Is there a way based on condition I can temporarily disable click event for some entities in higher depth or make them kinda transparent, such that I can only select the lower depth entities?

novel hamlet
#

It depends what picking backend you are using whether or not this is supported. I'd look at the drag and drop example, which does this so that when dragging an object it doesn't trigger pointer events while the dragged object is being dragged.

#

You could also make the piece a child of the square, that way the event bubbles up to the square even if you click on the piece.

halcyon forge
# novel hamlet It depends what picking backend you are using whether or not this is supported. ...

I'm using sprite backend (ig that's what will be used if you use TextureAtlasSprite). I'll look at example.

For using it as children, I've considered just in the opposite direction, square as child of piece. I'll think of way you said and try around. Also it kinda makes sense that squares are independent of sprites.

Ofcoure, I always could do the workaround having a event handler system which calls the necessary function as per the game's data.

P.S: I thought I will copy all our conversation once it's resolved into discussion once it's done.

halcyon forge
#

I can probably use the Pickable::IGNORE, insert it when a piece is clicked and remove it when a piece is successfully moved.

novel hamlet
halcyon forge
#

I should've mentioned.

halcyon forge
hasty sail
#

Is PointerInteraction supposed to record any entity hits regardless if they have Pickable or not?

wraith rapids
#

RESOLVED: im setting up a scene with the DefaultPickingPlugins, then spawning an entity like this,

            SpriteSheetBundle {
                transform: Transform::from_xyz(x, tool_y, TOOLS_Z),
                sprite: TextureAtlasSprite::new(i as usize),
                texture_atlas: sprites.tools.clone(),
                ..default()
            },
            PickableBundle::default(),
            On::<Pointer<Click>>::run(test),
            ExpeditionPersist,
        ));
```  btw test is just a debug fn that logs a message
However, when i run the game I cant seem to get the pointer click to activate.
#

there are sprites underneath it but none of them besides this have the pickable bundle.

wraith rapids
novel hamlet
hasty sail
#

Makes sense, thx.

tepid sun
#

Heyo, I've run into an error being spammed out due to using LineList and I'm not sure whether it is possibly to silence it through the API or not. I have Pickable::IGNORE on the entity but it does not help, and I assume it would be through bevy_mod_raycast if anything

#

I don't want or need to have it register at all - I'm just using a LineList to draw an outline on other meshes

novel hamlet
tepid sun
#

Invalid intersection check: TriangleList is the only supported PrimitiveTopology

#

I'm trying to circumvent it by using TriangeList instead but I'm strugglign figuring out the maths 😅

novel hamlet
#

We should probably just remove this warning now that raycasting can be opt out instead of always opt in.

tepid sun
#

That would do! However I'm almost done figuring out the angles to do thicker lines with TriangleList, but I'm getting a crash using it hehe. It happens when I hover over any of the triangles.

thread 'Compute Task Pool (1)' panicked at C:\Users\godno\.cargo\registry\src\index.crates.io-6f17d22bba15001f\bevy_mod_raycast-0.16.0\src\raycast.rs:161:29:
index out of bounds: the len is 19 but the index is 19
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Encountered a panic in system `bevy_picking_raycast::update_hits`!
Encountered a panic in system `bevy_app::main_schedule::Main::run_main`!
#

My beautiful lines;

8e40a5f29736e11cf6f01e20c9ffc5ed.png
tepid sun
#

This doesn't make any sense to me.. adding that last line to the left made the crashing stop. Magic number?

4de851aaf2c9bedbccb5fd73de21b93c.png
novel hamlet
tepid sun
#

There are double lines on the left hand side of the (incomplete) hexagon

#

If you compare the first and second pic you can see that I just added two more triangles to form a new line

novel hamlet
#

My only guess is that your index list points to a triangle that doesn't exist. I have to look into the line where the error is occurring

#

Yeah, it looks like an indexing bug, I think in your code?

tepid sun
#

Hmmm... That could be it. I've never worked with shaders before so I probably messed something up. Just can't think of where at the moment

novel hamlet
#

Hm, actually this might be a bug in my lib

#

Because that branch is for non indexed lists which isn't often used

#

I think I see a bug

tepid sun
#

Haha well if that's the case - I'm happy with that, at least I perhaps could contribute something back 😅

novel hamlet
#

Hrm, actually this might be an issue on your side. Is your vert list a multiple of three?

tepid sun
#

Yup, 24 vertices now, 18 when it crashed (and before)

novel hamlet
#

Because I think my code is actually correct, but what's happening is you have a vert list that is not a multiple of three

tepid sun
#

I could copy paste my Vec in here but it's a horrendous mess to look at so that doesn't help. But yeah it's a multiple of three all through

#

Wait

#

Am I stupid

#

Let me check lol

novel hamlet
#

Try debug printing the vert list .len

#

I don't see how else this could fail. 😔

tepid sun
#

Oh my.. haha yeah it was all my fault. It didn't click first when you said that but I looked back on the error and realized that I always left 1 extra vert while I was building it out 😅

#

Works just fine. My bad. Thank you very much for your time! ⭐

novel hamlet
flint sundial
#

when i do a Query<(&PickingInteraction, &mut Pressable, &PointerPress), the pointer press is always completely false even if picking interaction is pressed, is this expected?

otherwise could we just attach PointerPress to PickingInteraction::Pressed ?

novel hamlet
flint sundial
novel hamlet
#

I don't really understand this query. The pointerpress is a pointer component, but a picking interaction goes on a pickable entity

#

It doesn't seem like any entity should have both of those components at the same time.

#

iirc there is no tracking component for pointer button presses that live on every entity, it would end up being a bunch of wasted memory. Only a handful of entities are being interacted with at a given time. Presses are intended to be used through events.

#

The PickingInteraction component was added as a compatibility shim for bevy_ui at one point, again iirc

flint sundial
#

the query shouldn't have PointerPress, that was my misunderstanding

novel hamlet
#

Your best bet is probably looking at PointerInteraction and PointerPress on each pointer entity, and getting the list of picked entities. This seems like something we should add in a central resource similar to HoverMap.

#

It looks like this kinda exists as the down_mapin events.rs, but it is a Local. We would need to change it a bit and make it a resource.

flint sundial
crude talon
#

I've come across some weird behavior with bubbling

#

I have a Node that has another Node as its child. Basically like a window and the top bar of that window. I have an On::<Pointer<Drag>> on the parent Node that mutates a component on it. When I drag on the parent, it works normally. When I drag on the child, it's supposed to bubble up to the parent, right? The child does not have any pointer event components on it. It correctly tries to trigger the drag, bubbling the event to the parent, but then it tries to mutate the component on the child and prints an error to the console because the child Node does not have the component it tries to mutate.

novel hamlet
#

The target is the thing that was interacted with, which could be a child

#

the listener is the entity that has the listener, which is the parent

#

Yeah, you are using target_component_mut

#

You probably want listener_component_mut

crude talon
#

Ooh

#

Got it, thank you

tidal nova
#

I'm trying to implement behaviour where an entity that's dragged off of a surface is deleted when dropped, but I'm running into some problems:

  • The "entity" to be dragged is actually a collection of sprites all under one parent, but the dragged entity of a DragEnter/DragLeave event is always the actually dragged entity, so I can't get the parent entity that is actually the one being manipulated by the drag (as the listener of the Drag event). I was able to work around this by making the biggest sprite into the parent entity, so that it's always the target, but is there no better way?
  • DragEnter and DragLeave events fire spuriously and seemingly randomly while the item is being dragged over the target surface, but don't consistently fire when it actually leaves/enters it.
  • Is there an easy way to filter events to only the Primary mouse button, or does it have to be done manually in every callback?
#

This is all 2D

novel hamlet
#

DragEnter and DragLeave events fire spuriously and seemingly randomly while the item is being dragged over the target surface, but don't consistently fire when it actually leaves/enters it.
This sounds kinda like you are hitting the entities you are dragging. Are you marking the dragged entities as non pickable?

#

Is there an easy way to filter events to only the Primary mouse button, or does it have to be done manually in every callback?
Currently you have to filter them in the callback.

tidal nova
novel hamlet
#

When being dragged.

tidal nova
#

Oh, no

novel hamlet
#

so I can't get the parent entity that is actually the one being manipulated by the drag (as the listener of the Drag event).
You can manually find the Parent of the dragged entity.

#

The reason it isn't bubbled is because the target of that event is the entity that is having something dragged onto or off of it

tidal nova
#

Yeah

tidal nova
novel hamlet
#

what?

tidal nova
#

I've got a background sprite, which I'm calling the surface. It's the thing I want to detect drags onto/off of.

novel hamlet
#

Yes, but you are dragging something, and it's following the pointer, right?

tidal nova
#

So I put the DragEnter/DragLeave on it

#

Yeah

novel hamlet
#

So the thing you are dragging will block hits from reaching the background

tidal nova
#

Ahh

#

I see... But I need the hits to go to the background regardless of anything in the way

novel hamlet
#

Right, so you make the thing you are dragging Pickable::NONE

#

This is how the drag and drop example works

tidal nova
#

But if I drag over something else that's pickable, sitting on the surface, will that trigger enter/leave for the surface?

#

Since that other thing will also block hits

novel hamlet
#

I don't follow

#

The only thing you are making not pickable is the thing you are dragging

#

It will have no effect on any other entities.

tidal nova
#

Right, but you said I wasn't getting hits correctly because the thing I was dragging was blocking them from hitting the surface right?

novel hamlet
#

Right.

tidal nova
#

But if there's another draggable entity on the surface, and I drag the entity over that one, then the other draggable entity—which hasn't been marked as ignorable—will get the hit

novel hamlet
#

Yes.

tidal nova
#

Meaning that I'll get a DragLeave from the surface?

novel hamlet
#

Yes.

tidal nova
#

Right, that's not what I want

#

I want to know when it leaves the bounding box

novel hamlet
#

I don't follow what you are doing. I'm going to need a visual or something.

novel hamlet
tidal nova
#

Either that or I just don't use drag events to handle going outside the bounding box

#

Which is probably the simpler approach

#

I'm on my phone right now so I can't get a visual

tidal nova
#

Update: Yeah, just manually checking the bounding box in the Drag event worked like a charm.

#

New question: Is there a way to smuggle a newly-spawned entity under the cursor during a DragStart?

The behaviour I'd like is that there's a "Spawner" entity, and if you click and drag the Spawner, it actually spawns a new entity for you to place.

steel imp
#

I've noticed that sometimes, even though a mouse up/down input event is fired, no Pointer<Click> happens and I miss clicks. Not sure why this would sometimes happen.

novel hamlet
novel hamlet
#

DragStart -> spawn entity and attach FollowCursor entity to it
system updates any entities with FollowCursor to follow the cursor
DragEnd -> remove the FollowCursor component from the spawned entity

tidal nova
#

Ah, thanks! I will try that

steel imp
# novel hamlet That is surprising. The plugin doesn't track events for this, it has a map of al...

Yeah I started to dig into the code to see where things were going wrong. Input events were definitely always being sent and received. I imagine it's an issue I've caused somewhere. While I run all my picking in PickSet, I do it myself because I have an interaction plugin that turns off entire types of picking depending on the current interaction state (and I have several new types of picking backend).

steel imp
#

fwiw, I figured this out. I was dispatching the pointer events in my custom pickers using a helper I have. that helper is deferred (requires apply_deferred somewhere after).

I wasn't doing this after PickSet::Backend, so I had oscillation in the pickers.

The events they produced weren't hitting every frame. and this confused the heck out of the over and hover maps.

Easy fix on my end.

novel hamlet
#

Thanks for the update!

tidal nova
#

this is really awkward when I'm also trying to have the spawner tray be in egui, because piping the removal of FollowCursor to the right place with dropping logic is proving complex and a bit redundant. Is there really no way to smuggle something into a dragged state?

#

ah I think I can stick it into the DragMap...

fiery night
#

@novel hamlet I just wanted to mention that recently I tried moving to the Bevy main branch so I could play around with one-shot input params, and I was unable to get mod picking to work with it (or to even compile). This is not an urgent need for the moment, but I imagine you'll have some work to do when the next version comes out.

novel hamlet
tidal nova
#

How do I turn off the debug thing over the mouse cursor?

novel hamlet
tidal nova
#

also my "smuggle entity into the DragMap" thing is apparently not working for the first click of the app's lifetime

#

it's getting into the dragmap fine so there must be something else broken...

#

any idea what that might be?

novel hamlet
#

Really can't say, not much to go off of.

novel hamlet
#

Then you wouldn't need to be hacking anything, if I'm understanding the problem.

#

The picking system would know that you are hovering something in egui.

tidal nova
#

But there are no entities until after I click on the UI control

#

and AFAIK there's no way to render an entity to an egui widget

novel hamlet
#

You don't need to render anything

#

You just need an entity

#

The hard part is that egui doesn't integrate well with the ECS, so things like panels don't have an associated entity.

tidal nova
#

yeah

novel hamlet
#

So you need proxy entities that you spawn for each of these interactible things in egui, so everything can interoperate.

#

You can also handle this by making egui derive the UI from these entities, so you don't need any crazy syncing.

tidal nova
#

I thought you basically couldn't assign ui to entities below the level of egui windows

#

I guess I could store Ui callbacks on entities

#

also wouldn't I still have problem then that it's the spawner entity that gets dragged, not the newly-spawned on?

novel hamlet
#

Spawn entities that represent the possible things you can spawn from the UI, including information like the name, icon, color, whatever.
In egui, run a query on these entities, and render corresponding egui widgets. If one of them is hovered in egui, you now know what entity that is.

#

These entities have nothing to do with the entities you are spawning as world objects

#

Entities are just IDs, they can be used for any data.

tidal nova
#

But when someone clicks on the spawner, it triggers a spawn

#

I want this newly-spawned entity to receive drag events after that

#

as if it had been waiting where the spawner was the whole time

#

The only way I can think of that would make that happen that doesn't require a smuggle is if the spawner entity spawned a copy of itself, then transformed itself into the entity it wants to spawn

novel hamlet
#

You can also just have an entity pre-spawned and ready to be used.

#

So when you hover the spawn button or whatever, that is the entity that gets the hover event, and the drag.

#

You can then insert whatever components you need onto it after the drag starts.

novel hamlet
tidal nova
#

yeah...

#

hm

#

would it be possible to add a method to ListenerInput<Pointer<DragStart>> that could change the targeted entity of the drag?

#

... that would still require me to add a backend to generate pointer hits though

#

OH

#

I think I've got it...

tidal nova
#

So I realized I could just generate PointerHits in the UI handler for egui, and I could also generate them targeting the newly spawned entity... but things aren't quite working yet

#

This is a pretty great setup if it works, but I'll need to debug

novel hamlet
tidal nova
#

I thought it was something more fancy...

tidal nova
#

@novel hamlet I got this working, thanks!

fiery night
#

@novel hamlet Is there a way to access the drag event and also inject a World? The issue is that systems that inject World can't inject anything else. More specifically, in my "callbacks as entities" and "mutables as entities" framework, I want to be able to modfiy a component in response to an event, where the component is attached to an entity that is neither the target nor the listener - but I also need some information from the event.

#

This works perfectly well for event handlers that don't need the event data.

fiery night
#

(Apologies if I've asked this before in a different context)

tidal nova
#

what do you mean "inject a World"?

#

Can you not just add a query for the event you want to your event handler?

fiery night
tidal nova
#

ah

fiery night
#

The problem with querying is that I only want to access a single entity, not all entities that have that component. (Also, the component in question is generic, so that makes things slightly more complex as well.)

tidal nova
#

Do you have the ID?

fiery night
#

Yes

tidal nova
#

You can use q.get(id) to just get the single entity

fiery night
#

Yes, but isn't that inefficient to query all entities and just use one?

tidal nova
#

It doesn't query all entities

#

There's a bit of pre-processing to load and lock the right archetypes, so if this is extremely hot code, that might have some cost, but taking in the entire World means you need a lock on the entire world which will quite likely cost you more because of bottlenecking

fiery night
#

So for example, I'm storing drag state DragState in a Mutable<DragState> where Mutable is a component. There are lots of entities with Mutable, although only a few with Mutable<DragState>.

tidal nova
#

Each instantiation is a different type

#

Mutable<DragState> and Mutable<WhateverElse> have no relation internally

fiery night
#

I understand that. It's not a big problem with Mutable<DragState>, but other kinds of components might have Mutable<bool> which would be more common.

#

Let me think about this some more.

fiery night
#

The problem is worse with callbacks. Callbacks need a World, they always run exclusively because you can't predict in advance what data they will mutate.

The dependency injection approach only works when data and code are closely coupled: the type of data accessed by the system is known at compile time. But a slider widget, for example, doesn't know what ultimate effect will be caused when the slider is dragged, it can only send out some kind of signal or event. In the previous iteration I used event bubbling, but events are a pain for a number of reasons. For one, the receiver of the event might not be a parent, might not be a UI node at all.

tidal nova
#

I'm using an approach of keeping a separate Ui component which pipes data in/out of the UI code

#

and then the UI code queries that component and runs its show method

#

Is it intentional that the total distance of Drag events does not correspond to the total distance that the mouse is dragged?

#

I'm noticing a few pixels missing, I assume it's because DragStart doesn't trigger until the mouse has moved enough to rule out jitter from a click, but is there any way to get those pixels back? It would simplify testing a lot, not to have to account for them

#

oh, hm, I guess some of this might be also caused by system ordering issues...

#

ah, nope, that's not it, the hit position is what I expect it to be, the position as of the click

#

oh, I think it's event ordering issues... fun

#

Hm, partly.

So I'm moving the cursor 5 units horizontally and vertical each tick. The first 5 units are ignored by the drag engine: I don't get a DragStart until the entity has moved 10 units, but the corresponding Drag event only has moved 5 units.

The other problem is that I'm dropping that Drag event on the same tick, so that's on me.

tidal nova
#

The DragEnd also shows a total distance moved of (45.0, 45.0) when it should be (50.0, 50.0)

tidal nova
#

... oh this is because of egui hit data being out of date by a frame, isn't it ugh

#

hmmmmmmmm

tidal nova
#

fixed it

tidal nova
#

I ended up needing to do this to fix the event ordering:

            // Ensure that deferred commands are run after [DragStart] events but before [Drag] events.
            // This is required to allow entities to transform themselves in response to [DragStart] and still pick up [Drag] the same frame.
            .add_systems(
                PreUpdate,
                apply_deferred
                    .after(bevy_eventlistener_core::event_dispatcher::EventDispatcher::<Pointer<DragStart>>::cleanup)
                    .before(bevy_eventlistener_core::event_dispatcher::EventDispatcher::<Pointer<Drag>>::build)
            );
novel hamlet
fiery night
storm peak
#

Hi all, i'm trying to use the bevy_picking mod as an easier implementation of raycasting and item manipulation.

fn manage_cursor(
    btn: Res<Input<MouseButton>>,
    key: Res<Input<KeyCode>>,
    mut window_query: Query<&mut Window>,
    mut controller_query: Query<&mut FpsController>,
) {
    let mut window = window_query.single_mut();
    if btn.just_pressed(MouseButton::Left) {
        window.cursor.grab_mode = CursorGrabMode::Locked;
        window.cursor.visible = false;
        for mut controller in &mut controller_query {
            controller.enable_input = true;
        }
    }
    if key.just_pressed(KeyCode::Escape) {
        window.cursor.grab_mode = CursorGrabMode::None;
        window.cursor.visible = true;
        for mut controller in &mut controller_query {
            controller.enable_input = false;
        }
    }
}

I've already locked the mouse, but I can see the pointer moving now that I have the mod eenabled

#

However, I only want it to make my pointer the center of the screen, and not moveable

#

I assume it has somethjing to do with explicitly setting the pointer's location, but I'm having some issues getting that to work.

 mut pointer: Query<&mut PointerLocation>,
 mut window_query: Query<&mut Window>,
) {

    for mut pointer in &mut pointer {
        let w = window_query.single().width();
        let h = window_query.single().height();
        pointer.location = Some(pointer::Location {
            target: RenderTarget::Window(WindowRef::Primary)
                .normalize(window_query.get_single_mut().ok().map(|w| w.position)),
            position: Vec2 {
                x: w * 0.5,
                y: h * 0.5,
            },
        })

The target is asking for a normalizedrender target, but when I try to add it I just cant seem to get it to work

storm peak
#

I figured it out.
For anyone that needs to do the same

fn manage_cursor(
    btn: Res<Input<MouseButton>>,
    key: Res<Input<KeyCode>>,
    mut window_query: Query<(Entity, &mut Window)>,
    mut pointer: Query<&mut PointerLocation>,
    mut controller_query: Query<&mut FpsController>,
) {
    let mut window = window_query.single_mut();
    if btn.just_pressed(MouseButton::Left) {
        window.1.cursor.grab_mode = CursorGrabMode::Locked;
        window.1.cursor.visible = false;
        for mut controller in &mut controller_query {
            controller.enable_input = true;
        }
    }
    if key.just_pressed(KeyCode::Escape) {
        window.1.cursor.grab_mode = CursorGrabMode::None;
        window.1.cursor.visible = true;
        for mut controller in &mut controller_query {
            controller.enable_input = false;
        }
    }
    for mut pointer in &mut pointer {
        let w = window_query.single().1.width();
        let h = window_query.single().1.height();
        pointer.location = Some(pointer::Location {
            target: RenderTarget::Window(WindowRef::Primary)
                .normalize(window_query.get_single().ok().map(|w| w.0))
                .unwrap(),
            position: Vec2 {
                x: w * 0.5,
                y: h * 0.5,
            },
        })
    }
}
tepid sun
#

Is there any way to 'force' a state? Let's say I have a hexagon map, and I want to highlight all of the tiles around the tile that I am pointing at with the mouse. I know I can just modify the color material for example, but it'd be neat if I can just force a Pickable entity to be in a certain state at will

tepid sun
tidal nova
#

Now I'm getting a crash as a result of Encountered a panic when applying buffers for system ``bevy_picking_core::focus::update_interactions``!:

error[B0003]: Could not insert a bundle (of type ``bevy_picking_core::focus::PickingInteraction``) for entity 17v3 because it doesn't exist in this World.
I guess this is because my DragEnd handler despawned the entity?

#

wait, but DragEnd runs after that system

tidal nova
#

believe I fixed it with

app.add_systems(
                PreUpdate,
                apply_deferred
                    .after(bevy_picking_core::PickSet::PostFocus)
                    .before(bevy_eventlistener_core::EventListenerSet),
            )
tidal nova
#

@novel hamlet you might want to consider including the above by default, as without it, event handlers won't pick up a PickingInteraction on the frame it's added.

#

Also changing the insert to be a try_insert

fiery night
#

@novel hamlet I know you are super busy, I'm wondering how much of a hassle it would be to provide a branch of mod_picking/eventlistener which works with the current bevy main branch?

novel hamlet
fiery night
tidal nova
#

Is it expected that modifying On<Pointer<DragStart>> doesn't work properly?

novel hamlet
#

wdym by modifying?

#

Replacing the component?

tidal nova
#

yeah

#

It continues to use the old event handler

#

I was able to work around it by having one call the other

#

but it's a bit of 🍝

novel hamlet
tidal nova
#

Want a 🎟️ ?

novel hamlet
#

I am very confused

#

The listener graph is built every frame

#

I don't see any caching across frames

#

I'm not sure how this is possible.

#

How are you overwriting the component? Do you have a MRE?

tidal nova
#

I'm removing it then inserting it again... I don't have an MRE but I will try to get one

#

What's the going odds on pebkac?

novel hamlet
#

I would think low, I'm not sure how it would be misused.

novel hamlet
fiery night
#

I want to make a top-level event handler which captures events that did not have stop_propagation called on them. Is this possible?

#

Here's the use case: when I get keyboard input, I look up the current focus element (as determined by bevy_a11y::Focus) and send a bubbled event to that target. This allows any widgets that are ancestors of that element to intercept and handle the key. However, if no widgets intercepted it, then I want a global handler to handle it.

#

The global handler will do things like tab-navigation.

novel hamlet
# tidal nova I'm removing it then inserting it again... I don't have an MRE but I will try to... 
use bevy::{log::LogPlugin, prelude::*};
use bevy_eventlistener::prelude::*;

fn main() {
    let mut app = App::new();
    app.add_plugins(MinimalPlugins)
        .add_plugins(LogPlugin::default())
        .add_plugins(EventListenerPlugin::<Foo>::default());
    app.update();

    let entity = app
        .world
        .spawn(On::<Foo>::run(|| println!("First Version")))
        .id();
    app.add_systems(Update, move |mut event: EventWriter<Foo>| {
        event.send(Foo { target: entity })
    });
    app.update();
    app.update();

    app.world
        .entity_mut(entity)
        .insert(On::<Foo>::run(|| println!("Second Version")));
    app.update();
}

#[derive(Clone, Event, EntityEvent)]
struct Foo {
    #[target]
    target: Entity,
}

Output:

    Finished dev [unoptimized + debuginfo] target(s) in 0.46s
     Running `target/debug/examples/removal`
First Version
Second Version
novel hamlet
#

So, I assume the UI root node.

fiery night
novel hamlet
#

If you want to express that there is some global root, then they should be children of that global root entity.

fiery night
novel hamlet
#

That parent doesn't need to have a node component or anything.

#

It's just an expression of Parent/Child

#

Any other solution is a hack that is ultimately re-expressing the parent/child relationship.

fiery night
#

ok

novel hamlet
#

I wouldn't be surprised if there are design issues in bevy_ui that disallow a parent relationship on your root UI nodes though.

fiery night
#

I thought maybe there was a way to scoop up the events via an EventReader.

novel hamlet
#

You can always fall back to an eventreader if you want.

#

On<E> is just reading events E.

fiery night
#

Right, so the question is, is there a way for the event reader to test if stop_propagation has been called?

novel hamlet
#

No, the concept of propagation exists solely during the event bubbling process.

fiery night
#

ok

novel hamlet
#

It is built on top of event readers/writers

#

which have no awareness of the concept

fiery night
#

in that case, it would probably be easier to just duplicate the tab navigation function in the dialog itself.

#

It's just a function call

novel hamlet
#

Just looked at the code. The listenerinput is a resource that only exists during the bubble_events system's runtime.

#

We could make an event that triggers when the event bubbles to the surface?

#

bikeshed UnhandledBubbledEvent<E>

#

As a sort of catch-all for events that reach the "surface"

fiery night
#

Let me think about it, I have some alternate solutions. Was just wondering whether this is possible.

novel hamlet
fiery night
#

I also need to invent a way to factor out On handlers into a reusable bundle. For example, both button and checkbox have similar logic for space/enter, so they have duplicate code. Simply putting this in a function is tricky because of all the closure captures (whether it's disabled and so on).

novel hamlet
#

My instinct would be to compose behavior from functions if possible, though I don't understand the capture issue.

novel hamlet
# tidal nova I'm removing it then inserting it again... I don't have an MRE but I will try to...

I made an actual test that passes as well. Lmk if I'm missing something.

#[test]
fn replace_listener() {
    use crate::prelude::*;
    use bevy::prelude::*;

    #[derive(Clone, Event, EntityEvent)]
    struct Foo {
        #[target]
        target: Entity,
    }

    let (tx, rx) = std::sync::mpsc::channel();
    let mut app = App::new();
    let entity = app.world.spawn_empty().id();
    app.add_plugins(MinimalPlugins)
        .add_plugins(EventListenerPlugin::<Foo>::default())
        .add_systems(Update, move |mut event: EventWriter<Foo>| {
            event.send(Foo { target: entity })
        })
        .update();

    let sender = tx.clone();
    let callback = On::<Foo>::run(move || sender.send("one").unwrap());
    app.world.entity_mut(entity).insert(callback);
    app.update();

    let sender = tx.clone();
    let callback = On::<Foo>::run(move || sender.send("two").unwrap());
    app.world.entity_mut(entity).insert(callback);
    app.update();

    assert_eq!(rx.recv(), Ok("one"));
    assert_eq!(rx.recv(), Ok("two"));
}
tidal nova
#

Yep, currently investigating

#

Aha!

# 

fn main() {
    let mut app = App::new();
    app.add_plugins(MinimalPlugins)
        .add_plugins(LogPlugin::default())
        .add_plugins(EventListenerPlugin::<Foo>::default());
    app.update();

    let entity = app
        .world
        .spawn(On::<Foo>::run(
            |ev: Listener<Foo>, mut commands: Commands| {
                println!("First Version... replacing handler...");
                commands
                    .entity(ev.listener())
                    .insert(On::<Foo>::run(|| println!("Second Version")));
            },
        ))
        .id();
    app.add_systems(Update, move |mut event: EventWriter<Foo>| {
        event.send(Foo { target: entity })
    });
    app.update();
    app.update();
    app.update();
    app.update();

    app.world
        .entity_mut(entity)
        .insert(On::<Foo>::run(|| println!("Third Version")));
    app.update();
}
# 
First Version... replacing handler...
First Version... replacing handler...
First Version... replacing handler...
Third Version
#

sec I think I see the fix

novel hamlet
#

Maybe command buffers aren't being flushed?

tidal nova
# 
     pub fn cleanup(mut listeners: Query<&mut On<E>>, mut callbacks: ResMut<EventDispatcher<E>>) {
         for (entity, (callback, _)) in callbacks.listener_graph.drain() {
             if let Ok(mut listener) = listeners.get_mut(entity) {
-                listener.callback = callback;
+                // Do not restore the callback if it has been replaced by the event handler.
+                if listener.callback.is_empty() {
+                    listener.callback = callback;
+                }
             }
         }
     }
#

... trying to make this into a test isn't working, I suspect because of command run timing

#

I'm actually not sure why in the non-test example, commands are being applied before the event handler is restored...

novel hamlet
#

commands are applied every time the callback is executed

#

to ensure they don't get missed

tidal nova
#

ahh, pebkac this time

tidal nova
tidal nova
novel hamlet
#

Yup, it fails without the fix 🎉

tidal nova
#

🎉

novel hamlet
tidal nova
#

ahhh

#

I didn't think of that, I'll try it with just an ordering constraint

#

... that didn't work. I'm not going to question things further right now.

#

@novel hamlet Do you want a PR from me?

novel hamlet
#

I just merged a fix for the mutating-in-place issue

#

I'll release it in a sec

tidal nova
#

Sweet ty!

novel hamlet
#

0.6.2 is live

tidal nova
#

thanks!

tidal nova
novel hamlet
#

@fiery night what crates do you need updated for bevy main? mod-picking depends on other crates for backends and such, which are obviously not ready for the next bevy version. I could update the internal crates, e.g. no backends

fiery night
fiery night
#

Been meaning to ask: is there any use in listening to PointerCancel if all I care about is dragging? In browsers, pointer-cancel is sent to tell the app that the current drag did not complete because the window lost focus, the mobile app went into the background and so on (this also cancels pointer capture). If a drag causes your app to go into some mode where you need to clean up after a drag is complete, then not listening for cancel events can potentially result in a drag that never ends. However, I do not know what the rules are for mod_picking: is it redundant to listen to PointerCancel if I am also listening to DragEnd?

#

Code that I write for handling drag tends to fall into one of two categories: drags that require a "finish" step and those that don't. An example of the former is rubber-band selection: the drag displays the rubber-band box, but doesn't actually select anything until pointer up. The latter is more like dragging a slider, value changes are applied immediately, drag end doesn't actually do much.

novel hamlet
#

Cancels can also happen in web and in mod_picking if a pointing device is lifted, I think that was the original case it was added for.

#

but drag end should handle that for you

tidal nova
#

Does DragEnd have a way to tell if it was a cancel or a proper completion?

fiery night
thick escarp
#

Is there already a somewhat complete/recommended example of dragging objects around a 3D world?

#

I have bevy_mod_picking set up to use rapier and have been using a On::<Pointer<Drag>>::target_component_mut::<ExternalImpulse> component to rotate the selected object with respect to the magnitude of the drag, but I am a bit lost on how to translate the XY components from ListenerInput<Pointer<Drag>> into the XZ components for the plane on which the dragged object is sitting

#

I guess I need to multiple the drag vector by the camera matrix or the plane matrix

thick escarp
#

I adapted some code that @steel island wrote as follows:

fn drag_system(mut gizmos: Gizmos, windows: Query<&Window>, cameras: Query<(&Camera, &GlobalTransform)>) {
    let (camera, transform) = cameras.iter().next().unwrap();
    let window = windows.iter().next().unwrap();

    let ray = window.cursor_position()
        .and_then(|cursor| camera.viewport_to_world(transform, cursor));

    if let Some(ray) = ray {
        let intersection = ray.intersect_plane(Vec3::ZERO, Vec3::Y);
        if let Some(distance) = intersection {
            gizmos.sphere(ray.get_point(distance), Quat::default(), 0.005, Color::RED);
        }        
    }
}

but now I want to integrate this into bevy_mod_picking, I can see that the DragStart event contains a HitData which in turn contains the entity id of the camera, but I am not sure about how to move that into the handler for the Drag event

thick escarp
novel hamlet
thick escarp
# novel hamlet I think all you need is ```rs drag_vector_world = camera.translation() + drag_x...

I had a quick look at this using this code:

.insert(On::<Pointer<Drag>>::run(|
    drag: Listener<Pointer<Drag>>,
    target: Query<(&DragTarget, &GlobalTransform, &mut ExternalImpulse)>,
    cameras: Query<(&Camera, &GlobalTransform)> | {
        if let Ok((drag_target, _, _)) = target.get_single() {
            let (_, camera_transform) = cameras.get(drag_target.camera).unwrap();
            
            let drag_vector_world = camera_transform.translation() + 
                drag.distance.x * camera_transform.right() + 
                drag.distance.y * camera_transform.up();
            
            tracing::info!("{}", drag_vector_world);
        }
    }
))

It doesn't seem correct though, since I am getting quite large values on the Y axis where I think they should be around zero...

novel hamlet
#

The values scale with the drag distance, which are screen space, you would need to scale them.

#

Also not sure if you want the total drag distance or the delta

thick escarp
#

I will investigate further, thanks for the hints

thick escarp
#

Looks like bevy_mod_picking might need a small update to be compatible with 0.13:

the trait `bevy::prelude::Event` is not implemented for `bevy_mod_picking::prelude::Pointer<bevy_mod_picking::prelude::Drag>
thick escarp
#

Updated https://github.com/allsey87/bevy_dragging_test/ which is a solution based on the Rapier backend that applies impulse to enable dragging an object around. I think it is correct in its current state although there are still some improvements that can be made:

  • Improve the mouse tracking (currently not working so well for low camera altitudes)
  • Make the impulse porportional to the object weight
  • Use PID control when applying the impulse (right now only P is used and there a bit too much overshoot)
thick escarp
hearty current
#

I'm seeing some weird behavior around selection. I can select some entities but not others and when I use the debug tool it does display an entity when hovering over it.

#

How can I go about debuging this?

#

I've been using PickSelection to find the picked entities using a query.

#

I should mention I guess these are sprites I'm picking

#

Do children of an entity contribute even if they don't have the picking bundle?

#

ah its not enough to remove the picking bundle you need to set the picking component to IGNORE.

#

problem solved. 🙂

gloomy path
#

I wish bevy_mod_picking re-exported bevy_eventlistener

fiery night
#

@novel hamlet How is it going on the 0.13 migration?

novel hamlet
#

I've started from upstream, eventlistener and mod_raycast are done. I've also included the change cart needed so we can upstream eventlistener.

novel hamlet
#

The only change that we should really get into eventlistener is the ability to add multiple callbacks to the same listener

#

I still am not sure how we ID the callbacks in a way that reuses bevy's existing idioms

#

Entities feel kinda weird here, but maybe that is fine.

novel hamlet
fiery night
#

The only reason for making an entity is that I can explicitly allocate and deallocate when I want, rather that depending on lifetimes.

#

SImilarly, I have signals/mutables which are also entities, kind of like your bevy_rx experiment.

novel hamlet
#

So I suppose registering a callback would spawn an entity, and if you want to get that id, you would need to use an entity command, something like

let callback_entity = listener_entity.on(Pointer::Click, my_callback);
#

^this also assumes we have a single pointer event type, to ensure correct ordering

#

...althought that could be a footgun, because you start with listener_entity and end up with the callback entity. Same type, completely different entities.

fiery night
#

In other words, use the Rust newtype pattern to return a struct that wraps the entity id.

novel hamlet
#

one option would be to use a builder pattern

novel hamlet
fiery night
#

Another advantage of wrapper types is that they can have generic params. For example, with Callback<P>, P is the type of the props argument of the function. This allows me to maintain type safety even thought the entity id itself has no type.

novel hamlet
#

would make it harder to misuse getting/modifying/removing an existing callback

novel hamlet
fiery night
#

I often use this pattern when I have a map of heterogenous types, the value type is an associated type of the key.

novel hamlet
#

something like
CallbackId<Pointer>
CallbackId<E: EntityEvent>(PhantomData<E>, Entity)

fiery night
#

I was thinking ListenerHandle but that works too.

novel hamlet
#

Yeah, that works for me

fiery night
#

You can also put methods on the wrapper if that is convenient.