#bevy_mod_picking

1 messages · Page 3 of 1

novel hamlet
#

yup, I was thinking the API would look something like

impl<E: EntityEvent> On<E> {
    fn remove<E>(&mut self, handle: ListenerHandle<E>):
    fn replace<E>(&mut self, handle: ListenerHandle<E>, callback: tbd) 
}
fiery night
#

SGTM

novel hamlet
#

And you could add symmetric methods on the handle type, as you said.

thick escarp
#

Both bevy_egui and bevy_rapier have been bumped to 0.25 which is compatible with bevy 0.13

gloomy path
#

Damn

#

Over fires before Out does. That's going to be annoying

tidal nova
#

Try something like this to customise the ordering.

(In theory this should work with just an ordering constraint and not an apply_deferred)

gloomy path
# 
.add_systems(
    PreUpdate,
    apply_deferred
        .after(EventDispatcher::<Pointer<Over>>::cleanup)
        .before(EventDispatcher::<Pointer<Out>>::build),
);
#

All this did was make Out firing after Over go from always, to randomly

#

Also, how do you make an ordering constraint between two third-party systems ferrisHmm

novel hamlet
#

To anyone wondering where the 0.13 update is, I've been struggling to make time to get this finished. I've completed the eventlistener and mod_raycast updates, which were required to be finished first, but I'm feeling very burnt out right now. There are quite a few PRs on mod_picking I would like to include for this release, but need time to review them. I'm also trying to get things set up to be upstreamed in the very near future.

There is a good chance I will merge the open 0.13 PR (https://github.com/aevyrie/bevy_mod_picking/pull/314), make a release, merge the other changes, and make another breaking release. That will help get a 0.13-compatible version out ASAP while I'm working on other improvements.

gloomy path
novel hamlet
#

They will need to be in the same schedule for this to work, which I assume is the issue.

gloomy path
#

They're all in PreUpdate, according to EventListenerPlugin<E>

novel hamlet
# novel hamlet To anyone wondering where the 0.13 update is, I've been struggling to make time ...

The other thing I wanted to mention is event ordering. This is my top priority before upstreaming, but it is non-trivial, and probably impossible without a bevy PR. The plan is to:

  1. Add the ability to add multiple callbacks to a single listener: #1038322714320052304 message
  2. Collapse all pointer events into a single Pointer event enum. (1) is needed to allow you to have, say, a Pointer::Click and Pointer::Hover on the same entity.
  3. Collapse the pointer event stream into a single stream, so we can track when a pointer down happens relative to a pointer move.
  4. Eventually, bevy needs to add timestamps or collapse its event stream, right now it is impossible to disambiguate event order within the same frame
#

If anyone wants to help get this crate upstreamed, these are the steps that are needed.

gloomy path
#

Eventually, bevy needs to add timestamps or collapse its event stream, right now it is impossible to disambiguate event order within the same frame

#

Yeah, this bit me as well

novel hamlet
gloomy path
#

I'm having to work around it for bevy_window::{CursorEntered, CursorLeft} by abusing the implementation detail that one apparently cannot fire twice in a row unless something is seriously wrong with the ECS

novel hamlet
fiery night
novel hamlet
novel hamlet
#

@marble pine is there any prior art in bevy w.r.t. how debugging behavior should be toggled? Is there an idiomatic solution? Fields on a setting resource? States?

marble pine
#

States are overkill unless there's serious setup/cleanup to do

novel hamlet
#

Someone PR'd a change with states and it feels way overcomplicated, and I want to revert to a simple field.

marble pine
gloomy path
#

TIL Select exists

#

This will make keeping track of UI element focus much easier when it comes time to implement widgets::text_input

gloomy path
#

Oh. Oh boy

#

A panic

tidal nova
gloomy path
# 
Resource requested by dso_nostalgia::windows::auth::interactions::close_button::{{closure}} does not exist: bevy_eventlistener_core::callbacks::ListenerInput<bevy_picking_core::events::Pointer<bevy_picking_core::events::Over>>
Encountered a panic in exclusive system `bevy_eventlistener_core::event_dispatcher::EventDispatcher<bevy_picking_core::events::Pointer<bevy_picking_core::events::Out>>::bubble_events`!
Encountered a panic in system `bevy_app::main_schedule::Main::run_main`!
#

It's missing

tidal nova
#

That should only exist during an On<Pointer<T>> event

#

Did you try to call the system outside that?

gloomy path
# 
.on_pointerout(On::run({
    move |
        mut commands: Commands,
        mut input: ResMut<ListenerInput<Pointer<events::Over>>>,
        mut images: Query<&mut UiImage, With<WindowFrame>>,
    | {
        input.stop_propagation();

        let source = input.listener();
        let image = images.get_mut(source).unwrap();

        image
            .map_unchanged(|i| &mut i.texture)
            .set_if_neq(button.clone());

        commands.add(PopCursorIcon { source });
    }
}));
tidal nova
#

Wrong event?

#

Looks like you're setting it for Out with an Over listener

gloomy path
#

on_pointerout can only take On<Pointer<Out>>

#

Ah, right

#

Damn

#

I did forget to change it

#

THIS IS YOUR FAULT RUSTC

tidal nova
gloomy path
#

IF YOU COULD JUST INFER CORRECTLY I WOULDN'T HAVE THIS ISSUE

tidal nova
#

coming next version: LLM-powered type inference

gloomy path
#

I'm serious about that

#

on_pointerout can only take On<E = Pointer<Out>>. On::<E>::target_commands_mut passes a ListenerInput<E> to the callback

#

For some reason, Rust can't infer E to be Pointer<Out> from the fact that on_pointerout forces it to be Pointer<Out>

#

But funnily enough, this is only when I deref it. It's able to infer it just fine right up until I do input.listener(), then suddenly it needs type annotations

#

YES

#

The close button works now :D

tidal nova
novel hamlet
#

Not at the moment, mostly just going through the motions of editing and merging.

gloomy path
#

Is there a way to stop propagation past a certain entity for all event types?

gloomy path
#

That is, apart from doing it manually by defining listeners for every event type I know about

thick escarp
#

Thanks for putting in the hours to get this done @novel hamlet @tidal nova @gloomy path - your hard work is really appreciated!

tidal nova
#

I've done nothing

#

I offered help was all

#

Please give my share of the credit to those who deserve it!

hearty current
#

I'm using the select feature which stores which entity is selected is there a way of temporarily pausing selection? One thought I had was to keep my own selection buffer that way I have more control but I wanted to make sure I wasn't missing something obvious.

#

Essentially I envision having different selection functionality based off of bevy state.

novel hamlet
#

The simplest solution without PRing/forking would be to assemble your own version of SelectionPlugin, but with a run criteria on that system.

gloomy path
#

How would I go about turning off multiselect entirely?

gloomy path
#

If you don't need event bubbling or callbacks, you can respond to pointer events like you would any other bevy event, using EventReader<Pointer<Click>>, EventReader<Pointer<Move>>, etc.

#

Ooh

novel hamlet
pure kestrel
#

@novel hamlet I see the 0.13 updates were merged, when do you plan a new release? Or is there more you want to do before making a new release and we'll need to wait longer? I am aching to upgrade my project to 0.13 : d

marble pine
novel hamlet
#

Yup, what Alice said.

marble pine
novel hamlet
#

I've opted to release anyway, but the rust gods are not having it.

#

Docs.rs is failing, I think the bevy_xpbd backend has pushed me over the 10 target docs limit.

pure kestrel
#

My upgrade to 0.13 was successful with that release so I am happy, thank you 🙂

novel hamlet
fiery night
#

@novel hamlet If I wanted to try out your multi-listener prototype, where would I go look?

novel hamlet
#

It doesn't exist yet

#

That's the first item on the list for the Purdue students working with Alice on the upstream

gleaming viper
#

I had a very basic question. I have a project that is using bevy_xpbd for the physics calculations and was going to set the bevy_mod_picking to use the XPBD backend. My question is, why would someone want to do this? What is the main benefit? The reasoning behind the question is, if the default picking solution is lightweight and performant, why would someone want to add the extra code complexity and readability by trying to make the mod picker us another solution?

#

Does using the XPBD backend increase the picking/selection performance and allows redundant code to be stripped out?

novel hamlet
#

extra code complexity and readability
What are you referring to here?

#

What is the main benefit?
It allows you to run picking across multiple sources, you don't have one backend, you have many. So, you can run picking on bevy_ui, xpbd, etc, etc, at the same time, and use the same picking API for everything.

gloomy path
#

^ The backends each tell bevy_mod_picking how to "speak" to a particular library and acquire data from that library

#

This allows for both splitting the implementation across backends, which allows you to only pull in the ones you need; and to also write your own picking support for any new library that doesn't have official support yet

#

It's an example of separation of concerns in action - bevy_mod_picking doesn't have to care about how the data it works with is acquired. It just has to ask the different backends to acquire the data for it, and then work with that data

#

Meanwhile, the backends concern themselves with how to read the data from the library they are built to interface with, and don't have to care about how bevy_mod_picking will use it

novel hamlet
#

Thank you for describing this much more effectively than I could. It's clearly still too early in the morning for me. 🙂

gloomy path
#

Speaking of

#

I need to figure out how to get bevy_picking_egui to not insta-panic the moment my app opens

#

bevy_egui assumes that the primary window will always exist, and so just uses the panicking Query::single to get its EguiContext - which panics if you ever close the primary window without immediately terminating the app

#

This presents a problem for me, because I need bevy_picking_egui to make dragging the inspector window not drag the app lmao

#

(On DragStart on the window backdrop, I call winit::Window::start_dragging.)

fiery night
#

@novel hamlet Here's a picking question I have been pondering. In my JavaScript game engine, which I'm in the process of porting to Bevy, there are many different kinds of picking that depend on the editor mode. Obviously, there's 2d picking for widgets (built into the browser). There's physics-based picking (which uses rapier/wasm) for picking objects in the game world. However, there are also a fair number of picking algorithms that pick "virtual" objects that don't actually exist in the game world. These virtual objects are often abstract grids. An example is the floor editing widget: there's a virtual plane which is set to the current floor height, and picking allows creating and dragging vertices for a 2d polygon. Similarly, the 'flatten' tool in the terrain sculpting mode defines a flat plane anchored at the point of first click on the terrain mesh, and as you drag, it picks the intersection of the picking ray with that plane.

So what I'm wondering is what would be the best approach here. One would be to artificially add invisible objects into the game world, and disable the other kinds of picking (such as physics). A different approach would be to write a separate picking system for these virtual objects, independent of mod_picking, and then have some logic that decides which picking result to use.

#

Another example: I want to have interactive gizmos with drag handles, similar to Blender. However, picking on the thin arrows of a gizmo will be too difficult for users, I'll need to have a larger hit region than what is displayed visually. This could be done by adding invisible physics colliders for the gizmo, but from an interaction standpoint it would actually be better to calculate the "closest" hit. This is similar to a 2d vector drawing app with drag handles: when you have many drag handles clustered tightly together, the picking algorithm chooses based on which one the mouse is closest to, not simply which one happeneds to be ordered on top of the others.

novel hamlet
#

These both sounds like things that can be done with custom picking backends

#

For the first case, you would do your virtual hit tests and report the order in a way that allows it to override other things. i.e. a picking "overlay". That would keep the feature pretty well scoped, and wouldn't need to interfere with other unrelated systems.

#

The second as well, though I would personally just add another set of colliders, rather than trying to compute a nearest screenspace, as it could jump to some other object in an unexpected way, and generally sounds expensive if you aren't doing this in a shader with JFA or something. Overlapping invisible hit volumes will behave pretty much the same as computing a closest hit in 2d.

#

Adding invisible colliders is pretty easy, it's something you can already do with the mod_raycast backend, or of course by using a physics collider + backend.

hard terrace
#

bevy_transform_gizmo doesn't have it's own thread so might as well ask here:

I've made a PR to get bevy_transform_gizmo working on Bevy 0.13. The examples now compile but when clicking on a object the engine panics with the following

thread 'main' panicked at ~/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wgpu-0.19.3/src/backend/wgpu_core.rs:3006:5:
wgpu error: Validation Error

Caused by:
    In Device::create_render_pipeline
      note: label = `opaque_mesh_pipeline`
    Error matching ShaderStages(FRAGMENT) shader requirements against the pipeline
    Shader global ResourceBinding { group: 1, binding: 0 } is not available in the pipeline layout
    Storage class Storage { access: StorageAccess(LOAD) } doesn't match the shader Uniform

I know nothing about wgpu and haven't touched the shaders code as far as I know. Any ideas?

novel hamlet
#

That's an issue with a shader. This doesn't have anything to do with mod_picking.

#

Iirc, it uses a vertex shader to draw the gizmo in front of everything.

#

So I assume something changed with the piplines, going off of the error message.

hard terrace
sand swift
#

I feel like I'm missing something obvious, but I can't seem to get event listeners to work? For example I have these two components on the same entity as a PbrBundle, but nothing seems to happen (also had issue when trying to send events):

PickableBundle::default(),
On::<Pointer<Over>>::run(|event: Listener<Pointer<Over>>| {info!("interacted");}),

Are there any known pitfalls that can cause this sort of thing to happen?

novel hamlet
#

That's not a lot to go off of. Do the examples work? If so, what's different between the examples and what you have?

sand swift
#

The examples do run (event_listener in particular)

#

let's see if I can create a more minimal example of the issue than the current mess that my code is...

sand swift
#

Too long to fit in a discord message, but here; this is with bevy_mod_picking = "0.18.2" and hexx = "0.15.0"

📎 main.rs
#

Ok, apparently the mesh for the hexagons is broken in some way, not sure what the issue there might be

#

So not really a picking issue

sand swift
#

Found the issue: the mesh was marked to only be used in the render world

fiery night
#

@novel hamlet Consider adding a note to the docs explaining this point.

fiery night
#

@novel hamlet I've got a puzzle: how to make mod_picking work with offscreen render targets. I have a new UI widget called "Compositor" which renders its children to an off-screen render target, and then renders the resulting image to the main window. The reason for doing this is to facilitate animated transitions: I can vary the opacity of the image, or apply other effects, to the composited image. This looks better than trying to vary the opacity of the child widgets.

#

The way this works internally is that the widget has a camera, an image, and an image bundle. The widget adjusts all of its children to have a TargetCamera of the compositor camera. The children are also unparented, so that they are laid out relative to the origin of the offscreen buffer. The resulting image bundle entity is then part of a normal ui hierarchy, using standard layout. In the case of a dialog, this means that the image is centered on the screen.

#

The problem, now, is that from mod_picking's perspective (specifically, the bevy_ui backend), these child widgets don't exist, since they are not reachable from the main camera.

#

I suspect getting this to work will require some changes to the bevy_ui backend. The question is, how to make that a general solution that doesn't just apply to my own use case.

#

One idea might be to add some sort of component as a hint to the backend - kind of like a portal. (In the React world, a "portal" is a way to arrange a DOM node so that it's location in the visual hierarchy is different from its location in the template hierarchy - so you can have a dialog or popup menu which is constructed as a direct child of the document root, even though its generating component is deeply nested within the app). So this ECS component would say, "I have virtual children, not using the normal Bevy Parent/Child relationships, which have these entity ids and this transform."

novel hamlet
#

I'm not fully up to speed, but my recollection is that we just need to flesh out the target camera logic. I'm not a huge fan of the API because it seems to add duplicate concepts for little gain. The good news is this kind of things is a direct goal of mod picking, so it should be as simple as making the bevy UI backend smarter with target camera.

fiery night
#

So there's a new component called RenderGroups which will be used for both UI and 3d scenes.

novel hamlet
#

Actually I think I see the issue, you just need to map the input to any images that are rendered to a texture. This is something I wanted to do for digetic UI anyway

#

I have to go for a bit, but please keep typing stuff out here. ☺️

fiery night
#

Within a RenderGroup, you have two choices: either a target (entity) or a layer (bit mask). There are advantages of each:

  • With a target entity, you can have an infinite number of layers for zero cost, but you can only pick one of them at a time.
  • With a bitmask, you can subscribe to multiple layers, but there's an allocation cost if the total number of layers gets above 64.
fiery night
#

There's also a coordinate transform involved, but it should be relatively simple. The composited image might be transformed in some way, but that transform will be reflected in the regular UI hierarchy (Transform/GlobalTransform) on the composite widget. The rendering of the widgets to the render target also has a transform, but again this is part of the UI hierarchy for the child widgets.

novel hamlet
#

I think in this case we want to inject a new input in the pipeline. Right now:

hw/sw inputs -> pointer positions -> picking backends -> events

Pointers are located on a render target, which backeds use to run hit tests.

To handle picking against a render target visible to another render target, we can make a virtual pointer that the first pointer casts against, to give us the position of the pointer on this floating image. Now that the offscreen render target has a pointer on it, the rest of the picking pipeline should work normally.

#

So, this would also work for diegetic (world space) UI, but it feels like overkill for this scenario. It seems like the better solution would be to have a camera rendering to the same window, and you could instead just apply compositing effects to the camera.

#

The other benefit to this is filtering. Rendering to an offscreen buffer, then showing that image on another camera is going to cause issues because the pixels of the buffer likely won't map dirrectly to the pixels on the camera, and you will likely end of with either shimmering, or a vaseline smoothed effect.

novel hamlet
#

So you can have a popup rendered to a different camera, and you could also apply effects to that camera, like blurs or transparency, and it should behave like you are compositing images, but it will avoid needing to render to an image and filter it seperately.

#

You can then do that for anything, not just UI - 2d, 3d, ui, whatever.

#

This would also avoid all the complexity of the input indirection, the picking plugin already knows that a single render target - that a pointer is on - is composited from layered cameras with their own viewports and order, and handles that ordering and sorting for you.

novel hamlet
marble pine
#

Ah, okay!

novel hamlet
#

To boil this down, cameras already provide a way to composite images (camera order) and filter the contents (render layers). The UI should really be reusing these concepts, we don't need to invent new ones.

marble pine
#

FYI @winged carbon as well

winged carbon
#

RenderGroups doesn’t invent anything, it just unifies behavior (propagation is somewhat new/complex, but mimics TargetCamera propagation).

novel hamlet
#

A direct example of this is the two_pass example in mod_picking. It takes two cameras, with two different views into the world, and composites them with camera order. The picking systems know how to handle composited cameras, so this just works the way you would expect. Regardless of the world position of the composited views, the composited order is what matters when doing the input sorting. This is how I would expect UI to work in bevy. Note this doesn't use any concepts from mod_picking, it's just respecting bevy's existing camera order functionality.

fiery night
novel hamlet
#

That's an implementation detail though, right?

fiery night
#

For the case of animating opacity, it's important that the alpha channel operate on the post-composited image. I don't know if you saw the recent PR which attempts to "propagate" opacity down the hierarchy, but that PR was the wrong approach, because it modulated the alpha channel of each child widget individually instead of modulating the alpha channel of the composited result.

novel hamlet
#

I guess my biggest question is: we already have a way to composite views that retains picking information - you can stack as many cameras as you want and you can easily calculate the pointer's position in any of these cameras - why would we invent something new here?

novel hamlet
fiery night
novel hamlet
#

There is no reason we couldn't make it possible to add compositing effects to composited cameras by rendering to a new buffer for each.

fiery night
#

As long as there's a buffer, what I want can work

novel hamlet
#

Yup, I understand you need that to do things correctly

#

But camera are already an abstraction that handles things like picking nicely, because it keeps everything in the same space. If we don't use cameras, we will effectively be reimplementing the same concepts to handle these cases.

fiery night
#

However, I can see a Bevy dev objecting to this with the argument, "but we already have that, it's called render targets"

novel hamlet
#

That loses a ton of information though, and means you will have filtering issues.

fiery night
#

I see

novel hamlet
#

As soon as you move to another render target, you've lost the connection between the window and the cameras.

#

They are now totally decoupled

#

And you need to add a bunch of plumbing to get that back.

novel hamlet
#

It was pointed out to me that if that texture is far from the camera or at an oblique angle, the rendered texture will need to be resampled in the orientation it is visible at. If it is at an oblique angle, you will now have awful aliasing issues.

#

This also means you need to start worrying about managing the size of that buffer manually. If that computer screen in the world is only taking up 10 pixels, it shouldn't be rendering to a fullscreen-resolution buffer.

mellow lava
fiery night
#

@novel hamlet I'm starting to play around with the raycast backend. Weird thing is, I can pick the standard 3d test objects (cube, sphere, etc.) but not the object that I am creating with a custom mesh. I have added Pickable component to it.

#

The fact that it's being drawn with a custom material shouldn't matter I would think.

novel hamlet
#

I am both sick and on vacation (allegedly) so not a ton of help right now.

#

The raycast backend is pretty simple, it just looks at the position and normal mesh attributes iirc

#

My only thoughts are that either the mesh is being unloaded from the CPU with that new feature in 0.13, or your custom meshes are missing an attribute, maybe normals?

fiery night
#

....aaaand with that, I now have hoverable gizmos 🙂 Well, gizmo-like things anyway.

young drum
#

Is there a way to set the highlight colors for pickable objects?

#

Ah there is

vernal token
#

Hi, just started using this crate for the first time. Im using the following with a Pbrbundle:

On::<Pointer<Move>>::target_component_mut::<Transform>(|mov3, transform| {
    println!("hover info: {:?}", mov3);
}),

Its just a simple cuboid. The info I need is where on the cube the mouse is "hitting". So a coordinate relative to the cube itself would be ideal, that way I can figure out which side of the cube the mouse is on. But im having trouble deciphering what the individual data fields represent.

young drum
#

What does this crate use raycast for? My terminal gets flooded when I use the debug grid plugin

image.png
# 
2024-03-28T03:56:41.242022Z ERROR bevy_mod_raycast::raycast: Invalid intersection check: `TriangleList` is the only supported `PrimitiveTopology`
vernal token
fiery night
#

New day, new problem: how to detect if the player clicked on nothing. I've got a scene with some objects - clicking on an object should select it. But clicking on the space between objects should deselect the current object. Problem is, the backdrop isn't an object.

buoyant escarp
#

Custom picking backend?

fiery night
#

That is one approach I am considering.

buoyant escarp
#

I have done it, and I recall it working

fiery night
#

Now, the situation is a bit more complex than that because there's also a 2d UI. The 3d view is a panel within the UI. In order to make picking on the 3d view work at all, I have to make sure that the portion of the 2d UI that covers the 3d view has picking disabled. But what would be better is if I could somehow make that one UI panel be behind, rather than in front of, the 3d view.

#

In any case, I'll see if I can come up with a "backdrop" picking backend.

fiery night
#

OK that wasn't too hard 🙂

#

I implemented a backend which always returns a hit on the entity with the BackdropPickable component. There can only be one such entity. The order is set to be less than the camera order.

vernal token
#

Hey I'd appreciate some help with something...

I tried to implement some basic dragging based on the example, but my little 3D "card" keeps flying (the deltas are too big). It seems that the delta values are scaled up or something, or maybe im not getting world deltas and instead its just screen deltas. Am I meant to convert this into a world delta? the drag example doesnt seem to need that. Although the camera setup is different, and the translation values of the cubes are way bigger than mine.

vernal token
#

Ok through trial and error i figured out a "drag correction" that gets divided into delta and more or less fixes the problem. I can settle for this if I find a way to calculate this and be more exact. Any thoughts?

vernal token
#

Actually I just discovered the correction doesn't work if I change the screen size. So now I have to incorporate that into the correction.

vernal token
#

aaaand unfortunately, that doesnt work cus it seems to be non linear. So im back to square 1 cus I need a solution that works everywhere.

vernal token
#

So it looks like i have a solution that involves using viewport_to_world. On its own i still needed a correction factor, but i can get rid of that if I switch to orthographic (which I believe is what I should have started with anyway).

vernal token
#

I wonder if there is a orthographic scaling that is 1:1 with the deltas and works with all resolutions. That way i can remove all the viewport_to_world calls. Its not really necessary, but it would be a cleaner solution.

novel hamlet
#

the example for dragging works because it is 2d

#

The drag values are the screens space cursor movement

#

mapping that into 3d will require projecting the cursor into the world.

#

I've done it for camera controllers a few times, but not sure if it is easily extractable into a small reusable thing.

vernal token
fiery night
# novel hamlet I've done it for camera controllers a few times, but not sure if it is easily ex...

I am wrestling with the same issue btw. The thing about dragging in 3d is that while x and y are known, the z-value (the picking depth) is arbitrary, someone has to decide where the rays stop, this isn't something mod_picking necessarily knows. In my case, when you start a drag there's a virtual plane which runs through the object being dragged, and which is parallel with the axis that you are dragging along. Distance in 3d is determined by intersecting two rays with that plane - one ray being the point where the drag started, the second being the ray representing where the pointer is now. Unfortunately, calculating those rays from the information in the drag event is hard.

novel hamlet
#

When I solve this problem, it usually looks like:

  • on start: pin the starting interaction in 3d space
  • on start: define the interaction constraint (plane, axis, etc) in 3d space
  • on move: hit test against the constraint, and compare the pinned starting position to this new position, to prevent drifting
#

I'd probably only use picking to handle the drag state of the handle I am interested in, and using that to kick off the hit testing logic.

#

You could use it to hit against the virtual constraint geometry, but that kinda feels like overengineering.

#

As it maps the pointers and camera to a 3d ray, so you can use your pointer IDs and other picking niceties, and it will handle all the annoying stuff like DPI and viewports for you.

fiery night
# vernal token thanks for the confirmation. I couldnt tell if I was doing something wrong. I'm ...

OK got it working (using rays and intersections to compute drag distances), code is here if you want to reference: https://github.com/viridia/bevy_reactor/blob/main/examples/complex/transform_overlay.rs#L99

GitHub
bevy_reactor/examples/complex/transform_overlay.rs at main · viridi...

Fine-grained reactivity framework for Bevy. Contribute to viridia/bevy_reactor development by creating an account on GitHub.

vernal token
# fiery night OK got it working (using rays and intersections to compute drag distances), code...

Thanks! My situation is not as complex as yours, so if I wanted to use the projective camera (and I do), Its still a top down view onto a "table", so with a correction value I can get a pretty accurate drag (very small amount of drift). But I might come back to this later and try this out for a "better" solution, cus the way im doing it now doesnt sit right with me, unless im in Orthographic mode which lets me ditch the correction all together and I get a drag with no noticeable drift. The only problem with the Orthographic camera is that it makes things look less "3D". Anyway, didn't mean to rant.

gloomy path
#

How can I trigger a handler from code, for example in tests?

vernal token
#

@novel hamlet I have a situation where OnOver makes something "pop up", and then OnOut pops it back down. If the mouse is on the very edge, it will trigger OnOver, but then it will pop up away from the cursor, triggering OnOut which pops it back down, back to touching the cursor again which triggers the OnOut another time.. rinse and repeat. I attached a video to demonstrate. Is there a way I can make OnOver trigger at a different time? Actually, what I really need is something like a deadzone around the object, so the cursor would need to go beyond the edge a little bit for it to trigger.

The only thing I can think of is not to use OnOver, and use OnMove and wait until its close enough to trigger OnOver, even if its a custom event and not OnOver itself. But I was thinking maybe u have a better way

#

Another thing that could help is a way to "reset" the OnOver event when it triggers too soon, to give it a chance to trigger again on the next frame and not wait for the mouse to actually leave and go back in.. So for a bunch of frames id be continuously rejecting it with the reset until its within a certain range.

#

Oh wait, would re-adding On<Pointer<Over>> to the entity reset it? That would be great.

Nope, didnt work.

young drum
#

Is there a way to disable highlighting?

#

It seems to not care what I set the GlobalHighlight resource to

buoyant escarp
vernal token
#

Or maybe thats beyond mod picking, im not sure what a bounding box is.

buoyant escarp
vernal token
buoyant escarp
#

It’s been a while since I used mod picking, let me see if I can be more specific. Give me a bit.

vernal token
#

Im so sorry for bothering u 😅 but i deeply appreciate it

#

It might be as simple as turning off visibility, but idk if mod picking ignores those or not.

buoyant escarp
#

I am sure mod picking has built in hitbox support I just don’t remember exactly how and don’t have a sec to look it up rn.

vernal token
#

No worries, im in no rush 👍

buoyant escarp
#

Oh no, it looks like you are using bevy_mod_raycast (probably) with Mesh components in 3d.

#

So what I would do is:

  1. Create a Rectangle and then call ::mesh to get a rectangular mesh.
  2. Add this mesh as a component to an entity along with Pickable. It won't be visible because you didn't attach a material. Add your visible card as a child of that entity, positioned so that they overlap.
  3. Use the invisible parent entity to position both together in global space.
  4. Add your handlers to the invisible parent entity, and have them access and effect the child entity.
novel hamlet
vernal token
novel hamlet
#

Either that or add a debounce to your logic, so you don't cancel the animation until it has finished the initial animation.

vernal token
young drum
#

is there a way to bubble a selection up to the first entity that has a specific component?

novel hamlet
#

Add an event listener component to any of those entities, and listen for selection events

young drum
novel hamlet
#

Yup

young drum
# novel hamlet Yup

This is correct right?

On::<Pointer<Select>>::listener_commands_mut(|listener, _| {
    listener.stop_propagation();
})

Does the entity I want the selection to end up on need any other components beyond the PickableBundle?

novel hamlet
#

It only needs the selection component, if I understand the question.

#

You could just add the component in the listener using a command if you wanted to.

young drum
hasty sail
#

Is it possible to use a BoxedSystem with On::<T>::run()?

#

I'm trying to do something like this:

pub fn to_callback<C: Component + std::fmt::Debug>(message: String) -> BoxedSystem {
    Box::new(IntoSystem::into_system(move |query: Query<&C>| {
        for c in &query {
            println!("{c:?}");
        }
        println!("{message}");
    }))
}

(just an example)

#

But On::<T>::run(to_callback::<SomeComponent>()) errors with:

the trait `bevy::prelude::IntoSystem<(), (), _>` is not implemented for `std::boxed::Box<dyn bevy::prelude::System<In = (), Out = ()>>`
#

Oh, it works by returning a plain impl FnMut instead.

vernal token
#

Oh i forgot, i need to add it to bevy and use the handle

buoyant escarp
vernal token
# buoyant escarp You work it out?

Yes! thanks for the help. I had to change some details (i.e Rectangle was not working out) but the main idea is still there and works like a charm.

buoyant escarp
vernal token
buoyant escarp
#

Interesting, thanks! This api could definitely be improved for this specific case.

vernal token
buoyant escarp
#

I think you're probably in the right and I've gotten my 2d and 3d primitives mixed up. I might make an example based on this when mod_picking gets up-streamed, since I'm sure it will be a common question.

young drum
#

Is there a way to get the screen pos of a hit?

young drum
#

Nvm. Just made it fire a one shot system which gets the pos from the window.

hard terrace
#

After updating bevy_mod_picking from 0.17.0 to 0.18.2 as part of updating to Bevy 0.13, multi-selecting doesn't seem to work with my code anymore, only a single selection

[dependencies]
bevy = { version = "0.13.0" }
...
bevy_mod_picking = { version = "0.18.0", features = ["backend_rapier", "selection"] }

fn main() {
    App::new()
        .add_plugins((
...
            DefaultPickingPlugins
                .build()
                .disable::<DebugPickingPlugin>()
                .disable::<DefaultHighlightingPlugin>(),
            TransformGizmoPlugin::default(),
            OutlinePlugin,
...

        commands.spawn((
            PbrBundle {
...
            },
            Collider::cuboid(0.5, 0.5, 0.5),
            PickableBundle::default(),
            GizmoTransformable,
            On::<Pointer<Select>>::target_insert(OutlineBundle {
                outline: OutlineVolume {
                    visible: true,
                    colour: Color::rgba(1.0, 1.0, 1.0, 1.0).into(),
                    width: 1.0,
                },
                ..default()
            }),
            On::<Pointer<Deselect>>::target_remove::<OutlineVolume>(),
        ));

fn setup_camera(mut commands: Commands, mut query: Query<Entity, With<Camera>>) {
    for entity in &mut query {
        commands.entity(entity).insert((
            GizmoPickSource::default(),
            RapierPickable,
        ));
    }
}

Any ideas?

tawny adder
#

Anyone know of an ergonomic way to use bevy_mod_picking to simulate a momentary switch, where the switch release constitutes of any Pointer<Up> that occurs anywhere following the initial Pointer<Down> on the target entity? I was thinking of the drag events, but they only work if there is movement after the initial "pointer down"

gloomy path
#

You can directly listen for Pointer<Up> events in an ECS system

tawny adder
placid sonnet
#

Is it not possible to modify the callback inside an On component?

gloomy path
tawny adder
gloomy path
#

That's a change in your requirements which I was unaware about

#

where the switch release constitutes of any Pointer<Up> that occurs anywhere following the initial Pointer<Down> on the target entity

tawny adder
#

Oh, true, only I have my big picture, and I wasn't specific enough 😅 What you suggest is good enough though for now, until I try to support multi-touch

tawny adder
#

The anywhere was meaning the release did not have to be on the entity, but I intended that it be still refering to the initial target

gloomy path
#

In that case

#

Do your thing on both Click and DragEnd

tawny adder
#

I think DragEnd does not happen if there was no movement...

gloomy path
#

That's what Click is for

tawny adder
#

Maybe listen for both then?

gloomy path
#

If the target of Pointer<Down> and the target of Pointer<Up> are both the same entity, you also get a Pointer<Click>. Else, movement must have happened, and you will get a Pointer<DragEnd>

tawny adder
#

Oh yeah, sorry I was missing what you were saying

#

Ahh, Pointer<Down> is always a guarantee

#

And I can use all of the others to handle the possible release scenarios

gloomy path
#

Your biggest problem here is what happens when you get both Pointer<Click> and Pointer<DragEnd>

#

It's possible to handle this by having them place a marker in the target's component list, and positioning a "clear" system after both event dispatchers

placid sonnet
#

hi i'm having an issue getting a basic callback to work, my code looks like

for (children, mut on_click) in &mut interaction_query {
    let mut text = text_query.get_mut(children[0]).unwrap();
    match state.get() {
        NetworkingState::Disconnected => {
            info!("updating on click");
            text.sections[0].value = "Connect".to_string();
            *on_click =
                On::<Pointer<Click>>::run(|mut connection: ClientConnectionParam| {
                    info!("clicked on connect");
                    connection.connect();
                });
        }
}

however when I click on the button, nothing happens.

(I added a On::<Pointer<Click>> component on the button)
Is modifying the On component supported?

gloomy path
# 
use bevy_eventlistener::event_dispatcher::EventDispatcher;
use bevy_mod_picking::event::{Click, DragEnd, Pointer};

.add_systems(
    PreUpdate,
    clear_releases
        .after(EventDispatcher::<Pointer<Click>>::cleanup)
        .after(EventDispatcher::<Pointer<DragEnd>>::cleanup)
)
gloomy path
placid sonnet
#

so modifying the component directly doesn't work?

gloomy path
#

I don't know

#

I'm just suggesting a way that I know does work - though you'll need to apply commands after the system finishes

tawny adder
gloomy path
placid sonnet
#

It doesn't work either; I can see via the inspector that my Button entity has the On<Pointer<Click>> component, but when I click on it nothing happens

#

oh i didn't add the default picking plugins ...

gloomy path
#

Sanity check: did you add the DefaultPickingPlugins when you constructed the- No you didn't

tawny adder
fiery night
#

@novel hamlet How difficult would it be to make resource change detection work on HoverMap? Currently HoverMap gets mutated every frame, whether the pointer has moved or not, so it always reports changed.

novel hamlet
#

Possibly complex, would need to diff the previoushovermap

#

But doable because that previous map exists

fiery night
#

Not too important, I can work around it. Mainly I've been debugging reactions, seeing which dependencies are changing frequently, hover map was one.

vernal token
#

@novel hamlet @buoyant escarp So following up on a question i had before, where u told me to decouple the thing being animated from the thing being "picked", by wrapping the the thing being animated with an invisible parent (which doesnt get animated).

I replaced the thing being animated (a pbr) with a SceneBundle (with a gltf). The problem is that now its not giving me the entity of the invisible parent being picked, but instead one of the children generated by Bevy for the gltf. This seems to be the intended effect, according to the gltf.rs example. So how can i make it so that the scene bundle and any of its children cannot be picked?

#

Well, im assuming thats how I want to even solve it. I suppose i can work my way up to the parent from the entity it gives me, but that would SUCK lol.

vernal token
#

Oh hold on, my mistake. It sometimes gives me the parent first, and other times it doesnt. But assuming theres no crash, i expect id be getting both every time. All I gotta do is ignore the one i dont want.

vernal token
#

Yeah im sorry, this was my fault. Shoulda just used if-lets and live in that hell lol.

Actually even when doing things more correctly, the fact that the children are still pickable forces the infinite loop problem i was having before, except now its worse cus it doesnt matter where the cursor is.

novel hamlet
vernal token
novel hamlet
#

@marble pine once the current eventlistener stuff is done, I can try to make the time to get the other work done for upstreaming mod_picking. If we can, it would be nice if someone else can shephard bevy_eventlistener through the bevy PR process.

#

If we wait on me for everything, it will probably take forever.

#

I'm thinking the best way to get this done is to upstream it without doing any integration work. Like, just keep it as a blob of inert code but in the bevy codebase. Switching bevy_ui over to mod_picking will be its own work, and I'm worried it could get stuck in review for a long period, as there is a ton of room for opinions there.

#

We could also keep Interaction and mark it for deprecation, then figure out what pattern people want to use instead, to reduce controversy and breakage.

marble pine
marble pine
novel hamlet
novel hamlet
#

How it works:

  • disable the default input system
  • send InputMove when the texture is hovered in egui - the event includes the render target of the pointer
#

That's pretty much it. The picking pipeline handles everything as designed; the backends see that a pointer is over a render target, compute the hit tests using any cameras using that target.

novel hamlet
vernal token
#

😆

gloomy path
hasty sail
novel hamlet
hasty sail
# novel hamlet This is using egui. You could do the same with bevy_ui if you wanted, sure.

Right, I asked because last time I tried render targets + bevy_ui it doesn't seem to work with bevy_mod_picking. Relevant issue: https://github.com/aevyrie/bevy_mod_picking/issues/325

GitHub
bevy_ui backend does not work with `TargetCamera` · Issue #325 · ae...

Here's a minimal reproduction code. It's a combination of 2d/pixel_grid_snap.rs and ui/button.rs from the official examples: Expand use bevy::{ prelude::*, render::{ camera::RenderTarget, r...

novel hamlet
#

All it's doing is rendering to texture, so as long as you can render that in the UI, and figure out where the pointer is over that texture, that's all you need to do this.

hasty sail
#

Yeah, I needed to write my own backend for it to work. Did you need to make a custom solution for your demo above?

spare sandal
novel hamlet
#

Yeah, you don't need a backend in this case, you need a new input system that feeds the backends. The input plugin tells the backends "there is a pointer at this location on a render target". The default input plugin can only tell the backends about where the mouse and touch inputs are over a window, so you need to add your own input system that also tells the backends that there is a pointer on top of the render-to-texture viewport render target.

#

Once you feed this input data into the picking pipeline, the backends can pick that up and should just work ™️. They will see the render target, find the cameras that write to it, and do any raycasting or whatnot using that camera.

#

If you are using bevy_ui, you will probably need the default input plugin as well as an input plugin for viewports. The default plugin will help you detect that a pointer is over a viewport in bevy_ui, you then just need to take that information, and turn that into InputMoves and InputPresses on the viewport's render target.

#

This will add a frame of latency, unless you are careful about how you order things, or run some plugins in the picking pipeline twice.

hasty sail
#

That's an interesting approach, never thought of doing something like that. I think I'll give it a try. Btw, will you put the demo above into the repo's examples? I think this is def some useful info.

warm plank
#

If I wanted to use a Text2DBundle outside bevy_ui, what approach should I use? Do I need to make a new Bundle with the pickable and text2Dbundle and use that?

novel hamlet
novel hamlet
warm plank
novel hamlet
#

I guess it just depends on what you are doing? Defining a new bundle with the components you want seems pretty simple.

warm plank
manic bridge
#

Hey I have 2 questions:
a) Is there a way to disable picking for a specific camera?
b) And a way to enable picking for a camera that's not rendering anything to the screen? (I know CameraOutputMode::Skip can do it but it will still perform work and render to intermediate buffers)
EDIT: Ah just found RaycastBackendSetting which I think has what I need
EDIT2: I'm not sure actually...
EDIT3: So, RaycastBackendSetting solves a) for me as I can just prevent interaction with the entities rendering on that camera.
But for b), it doesn't look possible since the code directly checks for camera.is_active
I think I'll just use CameraOutputMode::Skip for now, and later make my own backend since it looks really easy to do 👌
EDIT4: Warning: With CameraOutputMode::Skip, the intermediary rendering textures get displayed on any camera placed afterwards :^( (Setting Camera::order to 100 does the job though. This is so hacky 😅 )

novel hamlet
manic bridge
novel hamlet
#

Yeah, something like that

fiery night
novel hamlet
#

The reason it does this is so that it matches how cameras are being composed.

#

But that's ultimately pretty easy to change if you fork the backed to do what you need for your use case.

novel hamlet
#

My M.O. to this point has been keeping the interface for backends very simple and unopinionated, so it's infinitely flexible - you can change the behavior however you want because you can write custom backends.

#

But the default backends that the plugin provides respect camera settings as a sort of "best practice" that isn't enforced anywhere.

manic bridge
novel hamlet
novel hamlet
manic bridge
patent arrow
#

Hey, I was looking at the drag_and_drop example, specifically the dragging, and I was wondering if there was a way to extract out the transform stuff into a system

#

I've been trying to come up with a way of maybe continuously feeding the drag.delta value into a component but haven't had much luck doing that

novel hamlet
#

On<E> callbacks are just systems

patent arrow
#

sorry I think I worded it poorly

#

just for my own organization I'd rather not have the logic be within the on<E> callbacks, so I wanted something like this

fn move_target(mut query: Query<(&Dragged, &mut Transform)>) {
    for (drag, mut transform) in query.iter_mut() {
        transform.translation.x += drag.0.x; 
        transform.translation.y -= drag.0.y;
    }
}```
Where `Dragged` is a component that looks like `pub struct Dragged(Vec2);` and the `Vec2` is the `drag.delta`
#

i mostly have it working because I just modify the dragged component with target_component_mut, the only issue is when I stop the drag it keeps moving

novel hamlet
#

But you can stuff move_target, the system, into a listener, to get the exact behavior you want

#

I guess the other question is why you want to add this extra layer in the middle?

#

You'll need to reset the dragged component every frame, otherwise it will keep dragging after events stop firing

patent arrow
#

yeah I do that

patent arrow
novel hamlet
#

dragging is inherently an event, representing it with a component, which is retained, is going to make your life more difficult unless you have a need for it.

patent arrow
#

the main issue i have now is that drag end is set up with commands_mut because I remove a component, would it be possible to add another drag end listener that also does target_component_mut?

#

i'll go back to moving it all within the listener im just curious

novel hamlet
#

those are just convenience methods that all use run under the hood

#

if you want more complex behavior, just use run to run a full-fat system

#

block_attack is a full system, you can do whatever you want in that. To access the event data, you need to add the Listener or ListenerMut system param, but that is the only "special" thing that is different from a normal system.

patent arrow
#

oooh I see!

#

thank you this is useful

novel hamlet
#

e.g. ListenerMut<Event> is just shorthand for ResMut<ListenerInput<Event>>

patent arrow
#

got it working :)

#

that example helped

hard terrace
young drum
#

Is there a way to handle when nothing is clicked?

buoyant escarp
#

You have to add a picking backend that always returns a hit behind everything else.

young drum
#

is there one built in?

buoyant escarp
#

For some reason I thought I had pushed one, but I guess not.

#

I can post my code for this end of day, if I remember.

young drum
#

Thanks. Might hack something with a skybox in the meantime.

buoyant escarp
#

It was like 30 loc tops I would just need to find it

granite oyster
#

I'm skimming through the docs, is there a way to let a pointer event pass through to a UI node beneath the top one? I'd like to be able to detect hover events for both buttons when they overlap

#

Ah, you can just disable should_block_lower in Pickable

vernal token
#

So im assuming theres a way to customize/disable keys from triggering picking events? I noticed that the drag works on both mouse buttons and the scroll wheel. Some may only want a subset of those.

novel hamlet
#

That information is available in the event and you can filter

vernal token
#

oh cool

bold timber
#

I'm a bevy beginner trying to use the picking mod's example on my already working model spawning;
example:

fn setup(mut commands: Commands) {
    commands.spawn((
        // Spawn your entity here, e.g. a Mesh.
        // When dragged, mutate the `Transform` component on the dragged target entity:
        On::<Pointer<Drag>>::target_component_mut::<Transform>(|drag, transform| {
            transform.rotate_local_y(drag.delta.x / 50.0)
        }),
        On::<Pointer<Click>>::add_command::<DeleteTarget>(),
        On::<Pointer<Over>>::send_event::<Greeting>(),
    ));
}

mine:

commands.spawn(SceneBundle {
        scene: asset_server.load("models/fridge.gltf#Scene0"),
        On::<Pointer<Drag>>::target_component_mut::<Transform>(|drag, transform| {
            transform.rotate_local_y(drag.delta.x / 50.0)
        }),
        ..default()
    });

But I get

error: expected one of `,`, `:`, or `}`, found `::`
  --> src\main.rs:46:11
   |
44 |     commands.spawn(SceneBundle {
   |                    ----------- while parsing this struct
45 |         scene: asset_server.load("models/fridge.gltf#Scene0"),
46 |         On::<Pointer<Drag>>::target_component_mut::<Transform>(|drag, transform| {
   |         --^^ expected one of `,`, `:`, or `}`
   |         |
   |         while parsing this struct field

Can anyone help or point me towards what I did wrong ?

novel hamlet
#

The scenebundle is a type from bevy that does not contain On

#

Additionally, this is invalid syntax

#

The difference is that in the example, it is adding a bunch of components in a tuple, like

(
  Foo,
  Bar,
  Baz,
)```
#

But a scenebundle is a struct, and it has a list of known fields

SceneBundle {
    scene: my_scene
    transform: my_transform
    ..default // use defaults for everything else
}
#

You basically want a combination of these things

spawn((
    SceneBundle {
        scene: my_scene
        ..default()
    },
    On::<Pointer<Drag>>::run(/* stuff here */),
    FooComponent,
    BarComponent,
));
#

You were trying to put the On component inside the SceneBundle, but (1) the scene bundle doesn't contain it, and (2) that syntax isn't correct.

bold timber
novel hamlet
bold timber
young drum
buoyant escarp
# 
// Inserts the window entity as the a hit behind everything else
fn update_window_hits(
    pointers: Query<(&PointerId, &PointerLocation)>,
    mut output_events: EventWriter<PointerHits>,
) {
    for (pointer_id, pointer_location) in pointers.iter() {
        if let Some(Location {
            target: NormalizedRenderTarget::Window(window_ref),
            position,
        }) = pointer_location.location
        {
            let entity = window_ref.entity();
            let entry = (entity, HitData::new(entity, 0.0, None, None));
            let order = f32::MIN; // The window goes behind everything else
            output_events.send(PointerHits::new(*pointer_id, Vec::from([entry]), order))
        }
    }
}
#

This is what I had a few months ago. No idea if it works any more (or if it worked at the time).

# 
app.add_systems(PreUpdate, update_window_hits.in_set(PickSet::Backend));
#

Should do it. This gives you the actual window entity itself, which I think is kind of cute.

#

@young drum

young drum
#

thanks I'll see if that does it

gloomy path
#

ListenerInput::<E>::listener() is the entity this callback belongs to, right, right?

novel hamlet
gloomy path
#

Good

hushed plank
#

I have a question about the gltf example: Clicking on any mesh that's a part of the scene sends an event with the entity of the clicked mesh. Is there a way to make it so that the event sent is associated with the root of the SceneBundle; that is, is it possible to make the event send the same entity when any part of the helmet is clicked on?

granite oyster
#

That's the difference between target and listener in things like target_mut or listener_mut. target is the thing that got picked, listener is the closest listening ancestor to target

novel hamlet
hushed plank
#

Thank you! That's exactly what I needed.

tame wharf
tame wharf
#

Here is how I've done so far, using basic Bevy'sEventReader<MouseButtonInput> and the PointerInteraction component:

fn deselect_target(
    mut commands: Commands,
    mut buttons_events: EventReader<MouseButtonInput>,
    target: Option<Res<Target>>,
    pointer_interaction_query: Query<&PointerInteraction>
) {
    if let (Some(mouse_event), Some(_)) = (buttons_events.read().last(), target) {
        if mouse_event.button == MouseButton::Left && pointer_interaction_query.get_single().unwrap().get_nearest_hit() == None {
            commands.remove_resource::<Target>();
        }
    }
}
fringe quarry
#

Hello,
I am trying to apply the On<Pointer<Click>> event from the gltf.rs example to my own scene (contains meshes loaded from different file format) and it is not reacting to clicks, not hovering.
When I load a gltf file it works, does it work only on the gltf asset somehow? I have examined the two entities and mine looks little bit different, is that a problem?
Here is a screenshot of my model as a scene in world inspector

image.png
novel hamlet
#

All it requires is a hierarchy (parent child), and a listener (On<E>) somewhere in the tree above a child that you want to listen for events on.

#

And of course the picking plugins

#

There is nothing referencing gltfs at all

fringe quarry
#

Do the children of the scene count? It does not seem to work for the scene I am loading - when loading the meshes I create a world, spawn the pbr bundles into it and then create the Scene from the world

novel hamlet
#

What do you mean about creating the scene?

#

If this is serialization related, the On<E> component has no way of persisting behavior yet.

novel hamlet
#

But if it is a scene handle, that will probably make a "break" in the hierarchy

#

Because many scenes can have many parents via cloning the handle

fringe quarry
#

Ah, that might be it, I will experiment in changing my loader to include the Parent relations

flint sundial
#

is it intended behavior for parent entity listeners to "retrigger" per child? e.g. if i have a parent with a On::<Pointer<Over>> with 2 children with their own On::<Pointer<Over>>'s, when i hover from one child to the next, should the parent's listener trigger twice?

this is in the ui context btw, and in this case the children are entirely spacially contained within the body of the parent

#

like it makes sense to me that the event bubbling would result in this behavior, but i think the parent has only been "hovered" once, and i think that's how the browser's DOM handles it (although i could be wrong, will check in a bit)

#

otherwise it seems like either the parent must track the hover state of all children, to avoid retriggering from a child bubble, or all siblings must track the hover state of each other, to turn off bubbling when a sibling is hovered

novel hamlet
#

I think you are actually asking abou enter and leave though, which are different

#

enter and leave are aware of their children states, and don't retrigger. In this case what you care about is the pointerenter and pointerleave on the parent, I believe. That will not retrigger when moving between children, because the pointer never entirely left it.

flint sundial
novel hamlet
#

I expect you will need ordered events first 🙂

flint sundial
#

ahh gotcha, is that part of mod picking upstreaming?

novel hamlet
#

yeah, @buoyant escarp is working on that part of mod picking right now

flint sundial
#

o awesome 🙏

novel hamlet
#

It shouldn't be too hard to add enter/leave to mod_picking, it's more a question of performance.

#

it would suck to be doing all that work if no one is even listening for those events, although that is getting into the territory of observers

gloomy path
novel hamlet
#

The "no one" here was meant in the sense of computing this for every interaction as opposed to lazily when a listener/observer is present.

gloomy path
#

Oh

novel hamlet
fringe quarry
still rover
#

Hi, imagine I have an infinite grid with no explicit entities that can be clicked and I want to send a ClickGrid(Tile) event if a click is not caught by UI or models.
How do I detect if a click event is caught by mod_picking or not?

novel hamlet
#

It sounds like you do want it to detect clicks, and you want a backend for your tile system.

#

This is the situation backends were designed for.

#

It could be as simple as reporting a hit every frame, but at a very far depth. Entities and/or UI in front would absorb the hit first if they report hits at a nearer depth or closer camera order.

still rover
vernal token
#

So i been using DragEnter lately. I noticed that it gets triggered when the pointer enters, not when the actual entity enters. Looks like thats what was intended. But if I wanted it to trigger as soon as the entity enters, would I need to figure that out myself or does the crate have a way of doing this more easily?

vernal token
#

I suppose one trick I could do is have something invisible around the thing im dragging into, and have it interact with that instead.

novel hamlet
#

it gets triggered when the pointer enters, not when the actual entity enters
What do you mean?

vernal token
novel hamlet
#

Right, that is the job of a physics/collision library.

bold timber
#

Hey, I tried googling it but I found conflicting responses. Is there a way to make bevy mod picking work with sprites ? If not what is the recommended way ?

novel hamlet
#

It's built in, there is even a sprite example

bold timber
novel hamlet
#

I'm not sure what you mean. The example shows that the plugin supports picking sprites.

bold timber
novel hamlet
#

Please read the docs. There is nothing needed to make the sprites special.

#

If you want to add behaviors, you can add listeners, or manually react to events, but the docs and other examples cover this.

#

Sprites are supported in the same way 3d meshes or UI are.

bold timber
#

okay I will try, thank you

bold timber
#

Hey, Say I want a system that can only select one entity at a time. I guess I would have to store which entity is selected in a Resource, and modify that resource everytime an entity is clicked on. But I'm not sure how to access the resource inside bevy mod picking event callbacks.

novel hamlet
#

the callbacks are just systems, so you access the resource like you would any other system.

granite oyster
#

Any way to add both a target_insert and target_remove for the same listener? Or is that just a target_commands_mut scenario?

novel hamlet
#

Anything more complex than a single one of those helpers is probaby best expressed as a full bevy system using run, though target_commands_mut should work too.

granite oyster
#

I remember at some point you had a chart that showed performance of the various callback styles, but I can't find it anymore

novel hamlet
flint sundial
#

can i make a PR to derive Copy for HitData? looks like there's a lot of cloning that can be avoided

flint sundial
novel hamlet
#

Copy doesn't change the perf of those structs, but it will make it a breaking change to add a non-copy type to them in the future.

#

Also want to be careful implementing it on types that are mutated, to prevent copying when moving is desired.

#

That can lead to the situation where you think you have mutated something, but you've actually just mutated a copy, and the original is unchanged.

gloomy path
#

Copy can be quite the footgun in that regard, if just applied willy-nilly to everything it could possibly apply to

flint sundial
novel hamlet
granite oyster
#

Wait... that's not 100% right...

#

Weird... so Copy: Clone but... it doesn't use it at all thonk

#

Anyway, off topic for this channel, sorry

gloomy path
#

If you derive both of them, the Clone impl will just become ```rust
impl Clone for MyType {
fn clone(&self) -> Self {
*self
}
}

# 
// module header...

struct Blah(i32);
#[automatically_derived]
impl ::core::marker::Copy for Blah { }
#[automatically_derived]
impl ::core::clone::Clone for Blah {
    #[inline]
    fn clone(&self) -> Blah {
        let _: ::core::clone::AssertParamIsClone<i32>;
        *self
    }
}
#[automatically_derived]
impl ::core::fmt::Debug for Blah {
    #[inline]
    fn fmt(&self, f: &mut ::core::fmt::Formatter) -> ::core::fmt::Result {
        ::core::fmt::Formatter::debug_tuple_field1_finish(f, "Blah", &&self.0)
    }
}
still rover
#

Unfortunately we current don't have an ImplicitCopy trait to specialize on.

flint sundial
#

if i've already manually sorted Over and Out events (out always after over) by modifying the interaction plugin what's the best way to emulate Enter and Leave?

novel hamlet
fiery night
#

For enter and leave events, I've found it easier to use the hovermap resource than to rely on events. I've used this pattern for a couple of different things, but I'll use cursors as an example: I have a system which run when the hovermap resource is changed. This gets the current hover entity, walks up the ancestor chain until it finds an entity with a Cursor component. It then writes the content of that component to a second resource which is the "current window cursor". A different system then updates the winit cursor image based on that resource.

The advantage of this approach is that it's relatively cheap:

  • we only walk the ancestor chain when the hovermap resource is changed.
  • we only mutate the window cursor resource if the new cursor is different than the previous one.
    Also, with this approach there's no chance of missing an event (which could happen if some intermediate entity wants to listen to in/out events and calls stop_propagation), or if we forget to handle pointercancel and the window loses focus or the mobile app gets pushed into the background. Polling hovermap means we always know the correct state.
flint sundial
gloomy path
#

You can also use the PreviousHoverMap as a robust way of determining changes in hover state from the previous update.

#

I take it that to determine whether a changed happened at all, a *previous_hover_map == *hover_map would suffice?

fiery night
#

However, that won't tell you what changed.

gloomy path
#

As does the HoverMap's own code

#

(It's mutably dereferenced on every frame, regardless of whether it actually needs to be updated or not.)

fiery night
#

So, to answer your original question, I don't do an equality comparison on the entire map, since that would require iterating through the whole map.

#

Instead, I just extract the value for the entry I care about and compare that, which is cheaper.

#

Normally, the hover map contains information about multiple pointing devices (if there are any) but for hovering and cursors I only care about the mouse/trackpad.

gloomy path
#

I get the feeling that bevy_mod_picking isn't happy about me overwriting Handle::default() in the Fonts collection

image.png
hasty sail
novel hamlet
hasty sail
hasty sail
# novel hamlet Yeah, you don't need a backend in this case, you need a new input system that *f...

Also I can confirm that this method works well for me, at least for UI. I'm currently concerned with how it works with other backends (e.g. sprite, xpbd). I think the UI backend works well because it considers the render target of the cursor and ui. But with the other backends, it looks like you can end up accidentally clicking on two different things at the same time (with the real cursor and the custom one).

novel hamlet
#

You don't need a custom pointer iirc, you can send this event to drive the mouse pointer.

#

So if the mouse is over some viewport, the input system only sends mouse moves to that pointer location until it leaves the viewport

hasty sail
#

Hmm I see, so you send a new InputMove with the same id, but with a different Location::target? But still if a backend doesn't consider the render target at all, wouldn't this still cause extra clicks?

novel hamlet
#

Backends must consider the target, otherwise they have no context for where the pointer is (camera, texture, etc).

#

If they didn't, then things like multi window support would be broken

hasty sail
#

AFAIK, with the sprite backend you can click on a sprite that is render-targetted to a texture with a cursor that has a window target. But I will double check again.

novel hamlet
#

That doesn't really have anything to do with this though? The fact that a sprite is an image has nothing to do with where the pointer is located.

#

This only matters when you are talking about a pointer moving from one target to another.

hasty sail
novel hamlet
#

The overlay is spawned for every pointer, so if you have multiple pointers, you would get multiple overlays

hasty sail
#

How does it prioritize when there's different targets for each pointer though? I'm sending new mouse events (with different position and target) after the original ones, but the overlay (correctly?) only shows the overlay for the one I'm sending.

novel hamlet
#

You should be replacing the built in input system plugin with your own. It's just an event stream, so it will look at the latest event, which is why it works, but that is probably brittle and there is no guarantee that won't break in strange ways in the future.

#

So, it's not that there are different targets for each pointer, it's just that your event happens to be the most recent location that was received.

#

From the plugin's point of view, every frame the pointer is flipping between the window to the location you are specifying, it just happens to skip all earlier events.

hasty sail
#

Ah yeah, that's kinda sucky I guess. I guess I should vendor and modify the bevy_picking_input since it's kinda small, though I usually don't like doing that.

novel hamlet
#

it's in its own plugin so you can just disable it and add your own copy

#

it's like this by design - the input plugin can't be everything for everyone, so it is just a small, default impl that you can copy-paste and modify depending on your use case.

hasty sail
#

Alright, I'll give it a go 👍. Thx for the advice.

copper wolf
#

Is it strictly necessary that bevy_mod_picking use bevy_egui 0.25? Other crates I use with egui are up to date, and having this crate means I need to use out of date versions of them so there won't be duplication and everything works well together. I was considering cloning the repo and updating the manifest myself

marble pine
copper wolf
#

👍 🙂

copper wolf
#

Although now that I think about it, I wonder why it hasn't been merged if it's such a small fix

marble pine
#

Aevyrie's been quite busy, and struggling with the burden of maintainership

#

Which is part of why #1236111180624297984 exists

novel hamlet
#

Yup, I'm just v busy

#

It's a small fix, but I usually take care to do a lot of manual testing with any release.

copper wolf
#

Oh, sorry if I came off as pushy. Love your work aevyrie 😃

novel hamlet
#

No worries. 🙂

halcyon dust
#

Is there a way to change the color of the debug text? I have a bright/white background and it's barely visible...

#

I didn't find any API for this and I cannot query the constructed text because it has no special marker component... (looking at debug_draw for bevy_ui)

spare sandal
halcyon dust
#

okay, so I missed nothing :/

spare sandal
#

it would be pretty simple to add a config for it though

halcyon dust
#

yup, maybe I'll add a PR, let's see... don't want to interfere with upstreaming, but I guess as it's easy, no effort would be wasted

spare sandal
#

it should probably have a dark transparent panel by default to make that better

halcyon dust
#

Another question... I have a bevy_ui Node that wraps other nodes (think like an egui window) and I want to move that... so on the topmost node I do: ```rust
// Top-level grid (app frame)
cmd.spawn((
NodeBundle {
style: Style {
position_type: PositionType::Absolute,
display: Display::Grid,
/*
left: Val::Percent(25.0),
top: Val::Percent(25.0),
*/
width: Val::Percent(50.0),
height: Val::Percent(50.0),
grid_template_columns: vec![GridTrack::min_content(), GridTrack::flex(1.0)],
grid_template_rows: vec![
GridTrack::auto(),
GridTrack::flex(1.0),
GridTrack::px(20.),
],
..default()
},
background_color: BackgroundColor(Color::WHITE),
..default()
},
PickableBundle::default(),
On::<Pointer<DragStart>>::target_insert(Pickable::IGNORE),
On::<Pointer<DragEnd>>::target_insert(Pickable::default()),
On::<Pointer<Drag>>::target_component_mut::<GlobalTransform>(|drag, transform| {
let current = transform.translation();
let new_x = current.x + drag.delta.x;
let new_y = current.y + drag.delta.y;
let new = Vec3::new(new_x, new_y, 1.0);
let new = GlobalTransform::from_translation(new);
*transform = new;
}),
//On::<Pointer<Over>>::listener_insert(BackgroundColor::from(colors::BUTTON_HOVERED)),
On::<Pointer<Over>>::target_insert(BackgroundColor::from(colors::BUTTON_HOVERED)),
On::<Pointer<Out>>::target_insert(BackgroundColor::from(colors::BUTTON_NORMAL)),
UiInventory,
//Interaction::None,
Name::from("Inventory"),
))
.with_children(|builder| {

#

This works, but I can only move the individual pieces. How can I make it so that when I drag the parent, everything inside moves accordingly?

#

(Also it's ugly, but I was trying to get it to work with GlobalTransform because messing with Transform didn't work for the UI stuff :/)

novel hamlet
bold timber
# 
On::<Pointer<Click>>::target_component_mut::<occupable::Occupable>(|_, occupable| {
    occupable.selected = true
}),

Hey, rn this is working fine, but I would like to rework this so that instead of modifying the component, it store the clicked entity inside of a resource.
I don't know how/if it's possible to do that here so i'm asking.

gloomy path
#

Then just call On::<Pointer<Click>>::add_command::<YourCommand>() instead

bold timber
#

I ended up making an event listener loop, but this could be cool, i'll probably try it soon, thanks a lot

novel hamlet
#

Callbacks are just bevy systems, so you can do this the same way you would in a normal system.

eternal pumice
#

I have a picking problem I'm not sure how to tackle.

I'm using bevy mod picking successfully and I'm using the debug text where as I hover pickable things I see white text with entity info and positions etc.

I have a GLTF scene that I spawn but I don't see anything when hovering over it.
In egui inspector down the scene tree I see meshes with Pickable.

Any idea where to start looking?

eternal pumice
#

Ah I tried adding ShowAabbGizmo to everything, and for some reason the model I'm using looks fine but has tiny bounding boxes

novel hamlet
#

That would do it. Sounds like you are using the raycasting backend, which uses AABBs to accelerate hit tests.

eternal pumice
#

I've been noticing that if I have an egui window in Bevy and I click somewhere in the egui window and something pickable is behind the egui window it gets selected

#

Any advice on how that could be avoided?

novel hamlet
#

Use the egui backend

#

This is demonstrated in the example

halcyon dust
novel hamlet
#

I think you would need to change the apsolute position of the node, not the transform? Unsure

#

More of a bevy_ui question

marble pine
halcyon dust
#

Style only has left, right etc though... Hm, so it should work as one unit if I don't mess with GlobalTransform and Transform? That helps 🙂

marble pine
halcyon dust
#

I'll try to cook something up with Position::Absolute

marble pine
#

There's some non-obvious way to do it, because Flexbox is great

halcyon dust
#

Oh, lol

#

Thanks!

paper quartz
#

is posible to drag a 3d asset?

halcyon dust
obsidian carbon
#

Hi @novel hamlet , sorry to bother. Could I ask for a bit of help please? I have been debugging why picking stopped working when I updated from 0.19.0 -> 0.20.0 and I have made several discoveries. First of all, it is working. At least your part. I used sprite_backend as a template and modified it to work with Lunex. The weird thing is, that Sprites work as expected, but when I try to pick something without a sprite, it gets blocked. Weird, because it should not have an effect if entity has sprite or not. I don't query for these components. Do you have any idea from where this might come from?

image.png
#

This is incredibly weird, because they are ignored or blocked by something, but there should be nothing

#

This is the line of code that is being set off resulting in blocking

obsidian carbon
#

Update: It was not that

#

For some reason after updating Bevy does not feed entities without sprite to mod_picking

#

If I give it image it works

#

But I don't want to do that, I want something like a hover zone

#

So, could there be a new optimisation that could potentially be the cause? Like skipping entities that cannot be rendered?

#

I solved it just now

#

Entities without the new SpriteSource component introduced here are excluded from mod_picking backend and thus hit detection is not even happening.

granite oyster
#

Is there an idiomatic way to enable/disable groups of listeners? Example use case, when a unit is selected, cells should respond to right clicks to tell that unit to move there. Should I:

  • insert On<Click> onto every tile when a unit is selected?
  • enable a pointer used specifically for tile selection?
  • add tiles to a "picking layer" and enable that layer?
  • something else?
novel hamlet
#

I would just use normal events for this. The unit movement system would check if a unit is selected, then read pointer events, and see if any right clicks were of map cells.

# 
if selected_units.is_empty() {
    return
}
if let Some(clicked_cell) = pointer_clicks
    .read()
    .filter(|press| press.button.is_secondary())
    .filter_map(|press| cell_query.get(press.target()))
    .next() 
{
    selected_units.move_to(clicked_cell)
}
obsidian carbon
#

Hi, I have been trying to port your "render_to_texture" example to work with Sprite and Raycast backend, but I cannot wrap my head around it. From what I have gathered, it works by listening to events on texture_entity and creating new events from the camera? Though I cannot find the code that does that. Can I ask for some clarification on how this thing works please?

novel hamlet
#

All it's doing is driving the pointer inputs based on mouse hits on the render-to-texture.

#

It's drawing the image in egui, and egui returns the pointer location over that image. Once we have that, we send an InputMove to the picking system, and that's pretty much it.

#

This replaces the default input system. The default input system always assumes pointers are located in the window. This example is showing how you can override that behavior, and instead tell the picking system that the pointer is over a texture instead.

There is nothing you need to do to make the sprite or raycast backends work, they already work in this example.

#

The picking pipeline looks like

inputs (where are the pointers) -> backends (what are the pointers hitting) -> focus (sort all the hits) -> events
#

Because you are changing the input of the pipeline, and telling it the pointer is on an image, everything else will "just work", because that is how backends are designed.

gloomy path
#

Honestly, my main issue with bevy_mod_picking is that drag events don't prevent click events from occurring, which has been consistently painful for dragging-based interactions in UI :c

meager sky
#

pretty new to Rust and Bevy but greatly experienced with UI and game engines.

Anyway a couple of questions

I can spawn an entity with On::Pointer<xxx> {} callbacks to get pointer intersect and button events. But it doesn't seem there is an easy ( or a nice sample ) way to get access to the Queries in these call backs. Is there a way ? Is there a way to get access to the "world" in these call backs ie: making one exclusive ?

Also I have read there is a "bubble up" methodology that uses a parent child relatioinship - specified by a tranform hierarchy ?

I would want for a child if hit to be able to optionally stop propagation of the event up the tree.

Possible ? is there a nice sample or doc on how to use the bubble up methodology ?

Thanks !

novel hamlet
# meager sky pretty new to Rust and Bevy but greatly experienced with UI and game engines. A...

But it doesn't seem there is an easy ( or a nice sample ) way to get access to the Queries in these call backs. Is there a way ? Is there a way to get access to the "world" in these call backs ie: making one exclusive ?
Yes, callbacks are systems, so you can use any kind of normal bevy system you want, e.g. On::Pointer<Click>:run(my_cool_system)

Also I have read there is a "bubble up" methodology that uses a parent child relatioinship - specified by a tranform hierarchy ?
Look at the bevy_eventlistener crate and docs

I would want for a child if hit to be able to optionally stop propagation of the event up the tree.
Yes, you can call listener.stop_propagation().

meager sky
meager sky
# novel hamlet I'd also take a look the the event listener example in mod_picking: https://gith...

Ooook - well getting close. I have used the run() function to have a closure run as a "system" when getting the event ( for now just an Over for testing ) My desire is on the Over
if there is an "Armable" data field present add the pointer to it. If not create one.

But I am a bit stuck on getting a mutable reference to the Armable as I keep getting not mutable errors on the assignment of the query result to the fields in the tuple
--> if let Ok((_transform, mut hovered)) = query.get_mut(event.target()) {
"hovered" needs to be mutable. I get

= query.get_mut(event.target()) {
^^^^^ cannot borrow as mut

I have tried variations of mut and &mut

If there is a better category to post this in let me know 🙂

commands.entity(id).insert((
DroppedImage, // for now for testing
Armable { ..Armable::default() }, // empty unless there is apointer over it.

On::<Pointer<Over>>::run(|event: Listener<Pointer<Over>>, query: Query<(&Transform, &mut Armable)>, time: Res<Time>| {

        // since added to this should I count on it always being present ?

// if let Ok(hovered) = query.get_component_mut::<Armable>(event.target()) {
if let Ok((_transform, mut hovered)) = query.get_mut(event.target()) {

            if let Some(pcd) = &mut hovered.data {
                pcd.over_count += 1;
            } else {
                // let me = over.listener();

                info!("#### new PCD {} ", event.target.index() );

                hovered.data = Some(Box::new(PointerControlData {
                    entity: event.target,
                    ..PointerControlData::default() }));
            }

        } else {
            // the entity does not have the components from the query
        }
    }),
meager sky
obsidian carbon
#

The pipeline I use here is 3D camera rendered to a texture and then composed in 2D camera + Bloom => Final

obsidian carbon
#

Seems like it is working on WASM, so it is probably system ordering issue... Before which system does it need to run?

gloomy path
#

Does bevy_picking_core::pointer::Location use logical coordinates or physical coordinates?

fiery night
#

@novel hamlet What's the current best practice for detecting a double-click?

novel hamlet
#

I don't think there is one yet. Winit doesn't expose OS double click times yet afaik

meager sky
#

working on my pointer driven tools.

finally got enough Rust in my iron poor blood to get some basics going.
we pretty much want to respond to the gamut of pointer/multi-touch events in our program. This is in some way re-doing things we have worked on for many years.

so in creating an entity and putting the listeners on it, I now have this test:

commands.entity(id).insert((
DroppedImage,
Armable { ..Armable::default() },
...
// note time is there just to test using a resource argument
On::<Pointer<Over>>::run(|event: Listener<Pointer<Over>>, mut query: Query<(&Transform, &mut Armable)>, time: Res<Time>| {...}
On::<Pointer<Down>>::run(|event: Listener<Pointer<Down>>, mut query: Query<(&Transform, &mut Armable)>| {...}
On::<Pointer<Up>>::run(|event: Listener<Pointer<Up>>, mut query: Query<(&Transform, &mut Armable)>| {...}
On::<Pointer<Out>>::run(|event: Listener<Pointer<Out>>, mut query: Query<(&mut Armable)>| {...}
On::<Pointer<DragStart>>::run(|event: Listener<Pointer<DragStart>>, mut query: Query<(&Transform, &mut Armable)>| {...}
On::<Pointer<DragEnd>>::run(|event: Listener<Pointer<DragEnd>>, mut query: Query<(&Transform, &mut Armable)>| {...}
}

I omitted the contents of the closures, and I assume they can also be "static" methods ?

It would be nice to factor the list out into a bundle or something to set that all up. since we will have a bunch of different entity bundles for visible primitives/meshes.

How might I bundle the On::xxx listener declarations nicely ?

meager sky
# novel hamlet I don't think there is one yet. Winit doesn't expose OS double click times yet a...

hmm am in the early part of re-doing an deep interactive program and was wondering if there is going to at some point reasonably high res timestamps sub msec or at a minimum precision msec on the pointer events. Also when moves with these stamps in an "event stream" will be read from hardware ( if supported ) at a higher sampling rate that the FPS. What we had in the older system was "pointer update" items that provided a read only handle to an array of moves, or when we have a evet reader would also have an "on_frame_complete" event after the last event for the frame was read. Think implementing a rapid "zen" brush painting tool etc.

meager sky
novel hamlet
novel hamlet
meager sky
novel hamlet
meager sky
# novel hamlet We use winit for all of that stuff, and it is all there afaik, someone just need...

well I am slowly finding out what is in there. Our old software was based on only two listener methods on_pointer_over( pointer, bool entering) and on_hit() then had a single tool ( in the rust way would be a struct and traits ) that would install it's listeners on the "touch" ( basicly read that pointers inputs into itself ) to get subsequent moves/exits until exit/up/end and when done would cleanup it's mess and go away ( attached to the touch with an Arc ). this way an entity bundle doesn't need to have a fat bundle of pieces and the tool is transient and only uses resources when active. And thinkgs like move events only get distrubted to entities that have requiested them. pardon long windedness :). anyway we woudl often have a zillion little 4 pixel square things and could zoom or move in the space so keeping chunks of stuff out of the entities is good then allocate the tool only when wanted.

anyway for now getting a MVD ( minimum viable demo ) going with bevy mod picking 🙂

novel hamlet
meager sky
novel hamlet
#

If you want, you can also reach into the internals of the plugin and ignore the frontend. The HoverMap and PreviousHoverMap (plus DragMap iirc) are the source of truth for all the events.

#

That level of abstraction just gives you some big resources that store maps of "which pointer is hovering what entity" in one place.

#

The drag one is similar, but it also keeps track of drags, obviously, which can last many frames.

meager sky
meager sky
novel hamlet
meager sky
novel hamlet
#

Just looked at the release history... just shy of 4 years. Yikes.

novel hamlet
#

Interesting to hear about projects with that kind of legacy.

meager sky
# novel hamlet Interesting to hear about projects with that kind of legacy.

One encounters a lot of different use cases when one has been in UI R&D and bespoke interface design for major graphics tools. I have always wanted to get game quality interaction into real tools or as I have put it "high performance computing for the game of life" I think somewhat the iPhone broke the ice for many as proving that fluid high framerate interaction and rewsponsive interfaces can make many tools better - though alas the quality of much of it at the consumer level seems to be getting worse 😦

meager sky
# novel hamlet Very cool. 🙂

BTW late here so bed soon.
So yes I have spent way too much time doing infrastructure like bevy_mod_picking since 1986, and for me needs to be driven by wanting to do something with it.

Do you have any desires of what you woudl want to do with it ?

novel hamlet
#

I make CAD software at work, which is what I really want to be working on.

meager sky
# novel hamlet I make CAD software at work, which is what I really want to be working on.

I worked on the AutodeskAnimatorPro product And some on 3d Max for Autodesk 1987-88-89. ( Earthquake in SF 😮 ) then Caligari for Windows (3d modelling scene design package).

Is your picker/event module used by your employer's CAD software ?
is your CAD interest in things like designing things like Aircraft and Hospitals, Industrial process and product design. There is such a wide gamut of specialized products and tools out there.

I sent a discord "friend" request if you want to discuss any of this by a DM thread, do you know Ida Borisova ? We have been using her Rust/Bevy tutoring help service to get up to speed. I would be interested in what your design thinking and design goals for bevy_mod_picking have been.

meager sky
novel hamlet
#

Moving this conversation to DMs. 🙂

echo yew
#

I have a bevy UI layout in which I added a pickable action on a node that is positioned absolutely within another node. This works as expected. But as soon as I add pickable to the parent node the child node's interaction no longer works.

e.g what I want is:

+------------+
| A          |
|  +----+    |
|  | B  |    |
|  +----+    |
|            |
+------------+

I want Drag actions both on A and B, triggering separate events. B is a child of A, and is positioned absolutely.

I followed the example in which I set IGNORE on drag start and default on drag end, and then do stuff on Drag.

When adding interactivity only to B it works as expected, when I also add it to A then A works as expected, but B does not.

I can see that the A event is not triggered when I initiate a drag on B, but for B the drag only works for a split second and then stops.

Adding some debug logs, the following seems to happen:

With only events added to B:

  • DragStart fires, Drag fires repeatedly, DragEnd fires

With events added to B and A, dragging B:

  • DragStart fires, Drag fires once, DragEnd never fires

I'm assuming there's some unwanted cross-over between A and B, but I don't know how to avoid that.

#

A bit more debugging:

Pickable::default() on A doesn't break things for B, neither does On::<Pointer<DragStart>>::target_insert(Pickable::IGNORE), and On::<Pointer<DragEnd>>::target_insert(Pickable::default()).

But as soon as I add On::<Pointer<Drag>>::send_event::<FocusShift>(), then dragging for B stops working. There's nothing special about the system handling FocusShift, it's completely decoupled from the system handling the drag logic for B.

meager sky
#

so wanting to stop propagation of a event up the tree from a child On:: callback.

on the child entitiy I have this:

    On::<Pointer<Down>>::run(Armable::on_down_event),

It all works fine but now I want to stop propagation to the parent, and need to have a mutable ( Listener OR event or whatever is needed )

pub fn on_down_event(
                     mut event:  Listener<Pointer<Down>>,
                     mut query: Query<(&Transform, &mut           Armable)>,
                     camera_query: Query<(&Camera, &GlobalTransform), With<PoolCamera>>,
                     ) {
    // same problem with or without the mut after let 
     let mut e = &mut event;

    // for now ignore other buttons
    if (e.button != PointerButton::Primary) {
        return;
    }

    e.stop_propagation();
   // more stuff to do here
}

I get this error and not sure how to do it 🙂

e.stop_propagation();
|         ^ cannot borrow as mutable
flint sundial
flint sundial
gloomy path
#

When working with the Bevy UI background, does bevy_picking_core::pointer::Location use logical coordinates or physical coordinates?

novel hamlet
#

logical

#

Though I'm not sure what you mean by

When working with the Bevy UI background,

#

The Location is the location of the pointer on a render target (window, image, etc)

meager sky
gloomy path
gloomy path
novel hamlet
#

The event data generated by bevy_picking_ui are just hits, nothing else. Everything else like deltas are derived from the pointer location.

meager sky
#

This might not be the right forum for this but 🙂

I want to target an event ( maybe a custom event ) to a specific entity, and then have that entity have an On::MyEventType::run(a_system/closure) to receive and consume it.
Any pointers on how to learn how do this?
That is create->load->dispatch. ... receive

Thanks.

buoyant escarp
#

look at bevy_eventlistener for now

#

note that this crate may be replaced with in-engine features in a future release

meager sky
#

And 🙂
I have some nice UI dragging/scaling things working but for multi-touch and "forklift" mode I have needed an "on pointer input frame complete" event which,

In the multi-touch case AFTER all touches (pointers) have moved for this frame, resolve the action ( multi-finger translate rotate scale etc ) for the frame, and update visuals, transforms etc.

I assume I need to set the system that does this to run after bevy-mod-picking finishes it's event distribution.

So I need to set my system to run after that system ? what system is that ? thanks !

meager sky
novel hamlet
slim dawn
#

I'm using modpicking together with the mesh from the bevy_hexx package, and it refuses to trigger events on those meshes.
Normal quads still work, if I add a SimplifiedMesh it also works.
Has this ever come up before? Is there anything a mesh should conform to to be pickable?

novel hamlet
#

Entirely depends on the backends you are using

#

The built in mesh raycasting backend should work for any bevy mesh, as long as the primitive topology is supported

meager sky
# novel hamlet It's not clear how you are using multi touch in the context of picking.

thanks for the Pick set link. I'm not quite used to the bevy docs yet 🙂
As far as multi-touch - I am now racing for a clients demo in Rust and Bevy which I am new at 🙂
The old system I had which I am using bits of a "transform controller" would receive the events from multiple fingers/mouse/etc. and keep them in a map/list as there could be many in a single "main render thread" frame. Then after it was known no more down,up,or moves would come in, do all the math etc to finish the frame to update the visuals, come up with a new transform, add triangles to a caligraphy stroke, deal with momentum or whatever. I'm not doing this at the moment as the demo is for now only using one pointer.

slim dawn
#

Weird, started hapening after updating both bevy from 0.12, hexx 5 versions and mod_picking 2 minor versions 😅 . I'll try to debug manually by ray casting the meshes to figure out what's happening. If you haven't got any reports it should be somewhere in my code.

novel hamlet
#

Make sure you don't have multiple versions of bevy in tree

slim dawn
#

ah it was me all along:
RenderAssetUsages::RENDER_WORLD on the mesh makes it not pick anything.
RenderAssetUsages::all()
fixes it

#

It makes sense I guess, not sure how often people run into this. Would it deserve a comment somewhere? should i make a github issue and close it immediately?

novel hamlet
slim dawn
meager sky
#

If a better place to post this 🙂

Using the examples seen about as a guide I am trying to create a custom "TestEvent" and all the needed bits so I can add a listener for it to a test entity. ( for now, real use later )

    On::<TestEvent>::run(| event: Listener<TestEvent> | {
         info!("we got the custom event !")
    }),

The TestEvent is as follows:

#[derive(Clone, Event, EntityEvent)]
// #[can_bubble]. for now not interested in bubbling for this event.
pub struct TestEvent {
    #[target] // Marks the field of the event that specifies the target entity
    target: Entity,
}

The "listener plugin" added to the "app"

        .add_plugins(EventListenerPlugin::<TestEvent>::default())

I get the error below for the line above:

----------- ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait bevy_app::plugin::sealed::Plugins<_> is not implemented for bevy_eventlistener::EventListenerPlugin<TestEvent>, which is required by `bevy_eventlistener::EventListenerPlugin<TestEvent>: Plugins<_>

Possibly a versionitis issue ?

In the cargo file:

bevy = { version = "0.12", features=["jpeg"] }
bevy_math = "0.12.1"
bevy_mod_picking = "0.17.0"
bevy_eventlistener = "0.7.0"

Thanks !

novel hamlet
#

you need to be using bevy_eventlistener 0.6

meager sky
runic glade
#

if i only want entities to be selectable, do i have to add the entire PickableBundle or will only PickSelection be necessary

fiery night
#

I'm trying to debug a situation with DragEnter/DragLeave. When the mouse enters the element, I get a flurry of about 3-4 enter/leave messages. If I move the mouse 1 pixel, I get another pair of drag/leave messages. I don't think this is a mod_picking bug, more likely a problem with the way I have set up my entities. These are ui nodes BTW.

novel hamlet
#

Presumably that means the thing you are hovering is being repeatedly blocked/unblocked by something else

novel hamlet
#

However most picking components are completely optional, and you might only need the pickselection component

coral finch
#

Is bevy_mod_picking compatible with big_space, without any specific configuration on either side? I feel like I'm probably just doing something dumb but wanted to make sure it's supposed to work before going down a rabbit hole

runic glade
fiery night
novel hamlet
novel hamlet
fiery night
#

I'm having more problems with event ordering. I'm trying to drag a connection from A to B, and here's the sequence of events:

Terminal::DragStart: 179v1
Terminal::DragEnter: 164v1
Terminal::DragLeave: 164v1
Terminal::Drop: 164v1
Terminal::DragEnd: 179v1

179 is the A and 164 is B. As you can see, when I release the mouse button I get a DragLeave event before I get a Drop event. DragLeave and DragEnter are used to validate the connection, so by the time I get a Drop it thinks the connection is invalid and ignores the drop event.

#

This works sometimes and fails sometimes, depending on the order of events.

#

Sometimes I get this order, which also doesn't work:

Terminal::DragStart: 179v1
Terminal::DragEnter: 164v1
Terminal::DragLeave: 164v1
Terminal::DragEnd: 179v1
Terminal::Drop: 164v1
#

My code uses the enter and leave events to display visual feedback as to whether the drag operation is on a valid target or is on empty space. However, with the drop event coming last, it thinks that every drag operation ends with the mouse on empty space.

solid breach
fiery night
# solid breach <@301060831314182146> I'm not sure if it's still 100% relevant but this might be...

Right, I'm aware of the event ordering issue, but I'm wracking by brain trying to come up with a workaround. For simple dragging of a single entity, I can set a state variable on start/end, and ignore events when the variable is not set. That works because I can mostly ignore "end" events. But in the more complex case of dragging between two entities, that approach doesn't work - I need more information about what happened during the drag, but getting the events out of order means that the information I need is gone, or at least indistinguishable from a canceled drag action.

novel hamlet
#

As far as I am aware, it's impossible, because that information does not exist until the plugin switching to the unified (or timestamped) event stream

#

I think the unified input stream was merged for 0.14, so that could be a good follow up PR in the upstreaming

marble pine
#

It was!

meager sky
#

so I have a group of handlers.
How might I make a bundle of them ?

    On::<Pointer<Over>>::run(Armable::on_enter_event),
    On::<Pointer<Down>>::run(Armable::on_down_event),
    On::<Pointer<Up>>::run(Armable::on_up_event),
    On::<Pointer<Out>>::run(Armable::on_exit_event),
    On::<Pointer<DragStart>>::run(Armable::on_drag_begin_event),
    On::<Pointer<Drag>>::run(Armable::on_drag_move_event),
    On::<Pointer<DragEnd>>::run(Armable::on_drag_end_event),

thanks!

novel hamlet
#

This is more of a bevy question - you would do this the same way you make any bevy bundle.

#

Something like

#[derive(Bundle)]
struct ArmableBundle {
    over: On<Pointer<Over>>,
    down: On<Pointer<Down>>,
    // ...
}
meager sky
novel hamlet
meager sky
meager sky
novel hamlet
meager sky
novel hamlet
# 
fn add_armable_listeners(mut entity: EntityCommands) {
    entity.insert((
        On::<Pointer<Over>>::run(Armable::on_enter_event),
        On::<Pointer<Down>>::run(Armable::on_down_event),
        On::<Pointer<Up>>::run(Armable::on_up_event),
    ));
    // could add if statements, only add some of the components, etc
}

fn my_system(commands: Commands) {
    let mut entity_commands = commands.entity(some_entity);
    add_armable_listeners(entity_commands);
}
meager sky
granite oyster
#

I'm getting a weird interaction with egui and mod_picking (bevy 0.13) where egui complains fonts aren't available until the first call to Context::run. The call stack goes through mod_picking::debug::debug_draw_egui. It doesn't always happen, sometimes it just loads up fine, but in those cases, I'm not getting pointer hits on egui panels (which is why I turned the debug flag on mod_picking)

#

Is this a known issue? Or has anyone had issues with egui/mod_picking before? I'm still trying to boil it down to a min reproducible example, but it's hard bc it's not consistent 😭

#

Hm, doesn't look like debug_draw_egui is ordered relative to egui's own draw systems? Let me turn on ambiguity detection and see if it's upset

#

Oh heck

#

That's a lot of ambiguities

#

Back to the drawing board.

novel hamlet
#

It doesn't manifest in the examples that use the debug draw?

granite oyster
#

You mean the mod_picking examples? Haven't gotten around to running them but I have gotten my own project down to what I consider a minimum repro:

image.png
#

lemme go look at the examples

granite oyster
#

iirc

novel hamlet
novel hamlet
granite oyster
#

At any rate, the "hit entity" doesn't change when I move the mouse on and off of the egui panel

novel hamlet
#

The issue is that you can have multiple cameras overlapping, which takes higher precedence than depth

#

Or, in the case of egui, there is no egui camera, but it draws on top of everything else

granite oyster
#

Ah, and that's why the order is just f-off high

novel hamlet
#

Exactly.

granite oyster
#

(on v0.19.1 tag btw)

novel hamlet
#

wat

granite oyster
#

Which means the selection doesn't work as advertised either. The egui window claims you can select a 3d object, then click the egui window without deselecting it

#

🤷

novel hamlet
#

lovely

#

It is working on 0.19.0, and 0.20, time to go spelunking

novel hamlet
#

The backend only supports bevy_egui 0.25, but the main crate deps opened this to support 0.25 to 0.27

granite oyster
#

Ohhhhhhhhhhhhhhhhhhhhhhhhh

novel hamlet
#

I need to go make a release for 0.19.2

granite oyster
#

Sure enough, my project is using 0.27

novel hamlet
#

A fix was merged but it happenned in the middle of the bevy RC and got missed

#

I'll yank 0.19.1

granite oyster
#

Yup, that's definitely what it was. I locked bevy_egui to 0.25 and everything's perfect

#

Ah, that'd be what was causing the crash as well 🤔 because 0.27 got its fonts init'ed but 0.25 didn't I guess?

novel hamlet
granite oyster
#

Alright, gonna go eat dinner. Thanks for the help! I'll keep an eye out for 0.19.2

novel hamlet
#

Oh, looks like I did publish 0.19.2 for bevy_picking_egui, already, a cargo update should fix it for you

#

I'll yank the 0.19.1 version, that was the only step I missed

#

Done, an update should bump. Note that only bevy_picking_egui was bumped to 0.19.2, everything else will be at 0.19.1

meager sky
#

so I have an entity that has listeners added to it for enter/exit and all the rest, for arming/dragging/etc functionality. This works fine.

it has children that are simply visuals which overlap and are visually on top of (overlap but go outside as well) the parent, which I hoped would have pointer events from Enter/Exit/etc bubble up to the parent with the listeners installed.

then in that parent I need to know which of the children was hit, as well as having the parent itself being known as the target, one way or another.
At the moment I just put a quad as a child but the parent listeners are not getting the bubble-ups ( I assume bubbling is hierarchy based, and independent of camera relative z position ) of the child.
Visually all is good, but the parent in not getting any events from the child, the child apparently is eating the pointer events.

Am I missing something really stupid here ?

let child_id = commands.spawn((PbrBundle {
        mesh: quad_handle.clone(),
        material: child_material,
        transform: Transform::from_xyz(-30.0,0.0,0.05), // a little bit in front of parent
        ..default()
    },
)).id();

commands.entity(parent_with_listeners).push_children(&[child_id]);

The other thing is once I get that working can I make an invisible mesh entity as a hit bounds and not have it render ?
Or does one need to render it with a very high transparency with alpha 0.0 ?

I notice I have the parent of the child is also a child of a "scene" entity which will get all events that are not explicitly "stop_propagation"ed, and it WILL get the down/drag/uo etc events when the pointer is over the child of the parent_with_listeners !

thanks !

granite oyster
#

By setting visibility to RaycastVisibility::Ignore

#

This is assuming the raycast backend. With others, I think visibility isn't taken into account the same way? Like xpbd and rapier don't check visibility I think

meager sky
granite oyster
granite oyster
#

cargo run --example egui --features=backend_egui,backend_sprite crashes as well 😭

novel hamlet
#

Yanking does not apply to data in the git repo

#

You would need to be on 9f94f40be35939c8d524a3e3a853f1469996485e

granite oyster
#

bevy_picking_egui-v0.19.2

novel hamlet
#

cargo tree -i bevy_egui ?

novel hamlet
#

My guess is your lockfile is stale

granite oyster
#

🤔 I thought I did a cargo update

#

Let me check again

granite oyster
novel hamlet
meager sky
#

is there a way to Explicitly forward a Pointer<Xxx> event to another entity/target ( after setting the stop_propagation() ( or not ) this might be a work around for the bubbling apparently not working.ie: my previous question,

granite oyster
#

Couldn't you just change the target and EventWriter::send it again?

#

Might be a one-frame delay though I guess

modern marten
#

hey
I have a On::<Pointer<Click>>::target_commands_mut(DO_STUFF) listener on an element
this element also has some children
the problem is, when I click on a child, then DO_STUFF happens on the child, instead of the parent element like I intended
is this intentional? how can I have the listener only run for the element for which is was declared? do I need to explicitly set up some bubbling?

gloomy path
#

You want listener_commands_mut

modern marten
#

oooh

#

thanks, that was it

#

and is there a way I can change a particular child, even if only the parent was clicked?
I want to change button text if the button was clicked, even if the cursor wasn't directly on the letters

modern marten
#

can I combine different helpers somehow?
I want to use both On::<Pointer<Click>::listener_commands_mut and On::<Pointer<Click>::listener_component_mut (since it has access to state), but can't
I thought maybe I can chain these two methods together somehow, but no dice

novel hamlet
#

It's just a tiny bevy system

proper scarab
#

If I'm not mistaken this crate will be integrated within Bevy soon, but do you know if will it change much in the process? I would like to use it now and I wonder if that will mean rewriting a lot once it becomes part of Bevy, in which case it might be better to just wait

buoyant escarp
#

There will be a pretty direct and simple upgrade path, it’s a priority.

still rover
#

does bevy_mod_picking's raycast backend support view layer and camera ordering?

novel hamlet
#

(unless they go out of their way, or don't use a camera, all backends should respect camera order)

hearty current
#

For bevy_mod_picking has there been any thought about targeted keyboard events? Click to focus an entity and then keyboard events target the focused entity?

#

I'd be willing to add some if there is interest. 👍

#

I imagine the focus could just be "selected".

#

It could get more opinionated by shifting focus up and down the bevy hierarchy with focus shortcuts too.

marble pine
#

There's a whole story there with "focused elements" that I think we can do something really nice with

hearty current
#

I'll add something in my own code base for now then. 🙂

marble pine
fiery night
granite oyster
fiery night
#

@marble pine @granite oyster A few months ago I offered to upstream my focus system, and the general sentiment / feedback I got was opposed to it, and I kind of understand the reasoning: my focus navigation system is very "desktop-centric" in that all tabbable fields are in a linear sequence, much like a web page. For games, however, focus navigation is often spatial - hit the up button to focus the next upward button on the page for example. There are multiple different models of focus traversal: spatial, graph-based, etc. No single algorithm works for all designs.

marble pine
#

(and I'll need to get a screen reader or two set up to test)

novel hamlet
hearty current
#

and to be fair bevy_mod_picking kind of has focus via the selection stuff.

novel hamlet
#

Yeah, that seems pretty UI specific, I don't really have any opinions on how that works from a picking perspective

#

mod_picking has a few things that sit on top that are kinda just best-effort-get-things-working, but if upstreamed should be part of larger systems.

#

selection and highlighting come to mind. The former with a focus system, the latter with some styling system. idk

hearty current
#

Maybe not the right place to ask but are you cool with me making PR's to make On clonable? I assume making the picking bundle clonable should be fine.

novel hamlet
#

Iirc the reason it isn't Clone is because it could cause confusing behavior with systemstate.

#

But that might depend on implementation

#

If it's trivially clonable and the cloned listeners don't share state, it's probably fine

hearty current
#

share state as in move data into them?

novel hamlet
#

Can't recall the details atm. 😅

novel hamlet
hearty current
#

Its critically for me as I need to be able to clone bundles which include On. I used an Arc<RwLock<T>> and it works fine but I'm not sure I am hitting an edge case.

novel hamlet
#

That allows things like event read tracking and locals

hearty current
novel hamlet
#

I'd say try it and see what happens. 🙂

hearty current
#

I mean I did and it works, but there might be some edge cases. For example if a user calls the system > clones > calls the clone system I could see some weirdness because the system state is "modified", but if they cloned > call system > call cloned system it would be fine.

#

essentially you probably want to clone before a system runs in a given frame, and I'm not sure how you would enforce that.

#

you could just say "clone" at your own risk shrug

hearty current
#

Ah so I remembered how I solved this exact issue in kayak.

#

you keep track of the ticks between

#

but you always reset the tick between systems

#

this works fine for UI like Kayak but I don't know how that would work for bevy_eventlistener

sleek sage
#

Hi, how do I trigger an event for a virtual pointer? And is there anyway to make the virtual pointer area bigger? I'm trying to create a virtual pointer at the middle of the screen and I need to know which object it's hovering

novel hamlet
#

A pointer is a single point

#

There is an example for controlling a virtual pointer.

meager sky
#

On:XXX events

I got some of our UI working with the bevy_mod_picking using the On::Pointer<XXX> events posted to entitites.

Now we have upgraded to rust 0.79. and Bevy 0.14 and things have broken.
what is the new way ?
seeking docs and samples so I can get it working and move forward again 🙂

novel hamlet
#

Nothing has changed in mod_picking

meager sky
novel hamlet
#

Nothing I am aware of, no.

#

All the examples and tests work on the newest version, so I'm not aware of any breakages.

meager sky
meager sky
# novel hamlet Nothing I am aware of, no.

Hmm - it seems Brad commented out things that didn't compile and one is where I created a custom event type and it now gives errors on the line:

 .add_plugins(EventListenerPlugin::<TouchEvent>::default())
|      ----------- ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `bevy::bevy_app::plugin::sealed::Plugins<_>` is not implemented for `EventListenerPlugin<page_root::TouchEvent>`, which is required by `EventListenerPlugin<page_root::TouchEvent>: Plugins<_>`

So this seems to be an issue with changes to the event system ...

novel hamlet
meager sky
novel hamlet
#

If a plugin does not implement Plugin, that is because you are probably using an old version, so EventListenerPlugin implements Plugin for bevy 0.13, but not 0.14

#

Yes, the library that is giving you EventListenerPlugin

#

run cargo tree -i bevy

#

there are likely multiple versions of bevy in your app

#

@marble pine think we can get some sort of version checking in the Plugin trait? This seems to happen all the time.

meager sky
novel hamlet
#

Oh, this might be an issue with your TouchEvent?

meager sky
novel hamlet
#

TouchEvent must implement EntityEvent

#

Plugin is only implemented for EventlistenerPlugin<E> when E: EntityEvent

meager sky
novel hamlet
#

That hasn't changed since last year.

meager sky
novel hamlet
#

You are probably missing the target annotation.

#

Or you can implement this manually, yes

meager sky
marble pine
meager sky
novel hamlet
marble pine
novel hamlet
#

@meager sky published patches to help with the plugin error you saw, as well as providing an error on the macro.

#

You can update with cargo update -p bevy_eventlistener

meager sky
eternal pumice
#

I'm using the egui backend and I notice that if I change the resource EguiSettings's scale factor, the debug mod picking stuff gets displaced proportionally- is this a known issue?

hot mountain
#

Is it possible to disable picking for certain cameras, or is picking always enabled for every camera? I think the UI overlay (2d camera) is interfering with picking results in the main game (3d camera), and so I want to see if disabling picking for the 2d camera makes any difference. (using avian backend)

#

Ah, probably RenderLayers.

novel hamlet
novel hamlet
hot mountain
novel hamlet
#

Yup

copper wolf
#

Don't know if it's a solved problem, but rendering an egui::Slide at the top level, clicking on it will select an entity through the ui. Hovering doesn't seem to have this problem

#

egui::Windows work fine too

#

Also the ui generated by egui_dock seems to be completely ignored by the plugin

novel hamlet
#

The backend uses

ctx.get_mut().wants_pointer_input()

unsure if there is a better way to detect when egui wants the pointer.

shy night
# novel hamlet The backend uses ```rs ctx.get_mut().wants_pointer_input() ``` unsure if there i...

That's the solution I use and it's pretty reliable in all my testing. Ended up adding some polish by wrapping it in a plugin: https://gitlab.com/somniumteam/conspiracy-board/-/blob/master/src/ui/egui_capture.rs?ref_type=heads

The plugin just uses that function and the one for keyboard input to manage the existance/non-existance of some resources, which makes it easy to use with run conditions E.G. .run_if(not(resource_exists::<IsEguiCapturingMouse>))

copper wolf
#

How can I go about picking an entity through code? I have a hex grid and want to be able to pick a circular region of the map when clicking at it's centre

novel hamlet
#

Look at the documentation on backend, or the documentation at the top level of the crate.

#

It also really depends on what you mean by "pick a circular region of the map"

#

It sounds like you want to mark neighbor hexes as selected, but I'm just guessing based on context clues.

bold timber
#

I'm having trouble finding out why sometimes my clicks on sprites are registered and sometimes they arn't. So i'm wondering if there are some debug features that would show me the entity i'm currently hovering, maybe with a ui rect or something idk ?
I tried activating the debug and highlight features in my Cargo.toml but it didn't do anything (i'm a beginner)
I am using the DefaultsPickingPlugins for now

copper wolf
#

Yeah, I wanna mark the tiles around the one I clicked

novel hamlet
novel hamlet
novel hamlet
bold timber
novel hamlet
#

You might need bevys default font feature.

#

But I thought that was enabled with the debug

bold timber
#

There is now text on my cursor

#

seems to be working

#

off to debugging I guess

copper wolf
#

What I want is, I want to replicate what happens with the tile I clicked with the neighbours, so they get highlighted by the plugin as though I had multiselected them manually. I wanna know what components to add/change to mark those tiles as selected so the plugin takes care of the rest

#

Instead of clicking all those tiles manually, I want to mark them all automatically

bold timber
#

btw is there a way to get more info on the hit entity, like components or a name or smth ?
Hard to know who Entity 14 is.

copper wolf
#

Here, this blue region is the selected, but I had to click each cell automatically

image.png
#

Don't know if that's clearer or if you need more info

shy night
#

I ended up making a system that updates a resource every tick, which holds the hex coordinate currently under the mouse pointer. It converts the coordinates using your math of choice and then rounds it to integer coordinates.

Since everything in my use case was snapping to the center of the grid, I was able to iterate over all gridded entities and just check which one had the matching position by strict equality.

For getting a radius of hexes, I made a coroutine that yielded spaces in a spiral pattern, and used iterator functions to take however many spaces I needed. You'll need a bit of extra math to convert it in to accepting a radius.

Here's my implementation, but all the math is intentionally biased toward a particular axis alignment which generates flat-topped hexes: https://gitlab.com/somniumteam/mechronomicon/-/blob/master/src/hexagons/mod.rs?ref_type=heads#L145

copper wolf
#

That's not the problem, I have something similar set up in my project. What I'm wondering is what components change when marking an entity as selected, so I can do the same with the tiles around it so the plugin can take care of the rest

#

Thanks either way, but I found what I was looking for

bold timber
#

Does anyone know how bevy_mod_picking works for 2D ? I'm seeing some entities picked up that are not visible so I wonder on what basis it works

novel hamlet
bold timber
#

is there a way to get a visual feedback as well like a rect or smth ?

novel hamlet
#

The egui debug overlay shows drags, but that's about it

novel hamlet
copper wolf
#

Yeah exactly that, thanks

bold timber
novel hamlet
bold timber
#

What does you mean by highlighting

novel hamlet
#

Changing the material on hover, click, select.

copper wolf
#

This. This is what I wanted to accomplish. Thanks for the help 😄

image.png
novel hamlet
#

Nice!

granite oyster
#

Why doesn't the sprite picking backend return a world intersection position?

novel hamlet
granite oyster
#

Is there a good chance of getting it cherry picked for 0.19? I'd rather not upgrade to 0.14 just yet

novel hamlet
#

That is a breaking change, so I may have to revert if it breaks things for someone.

#

But I can cherry pick, yes.

granite oyster
#

Oh shoot, hadn't thought about that

bold timber
#

Hey, Can anyone explain or point me towards an explanation of how the crate works for 2D ? Does it simply try to hit sprite rects ?
I'm having issues with invisible entities interrupting the click

granite oyster
#

pretty sure it just uses the sprite box yeah. If by invisible you mean alpha = 0, you'll want to actually make the entity not visible to avoid being picked up by the sprite backend

#

Visibility::Hidden will do it

bold timber
#

I'm not sure what those entities, guess i'll have to add a name to everything

granite oyster
#

Not sure what you're saying, are you just saying you don't know which entities are blocking the picking?

bold timber
#

shouldn't be hard to figure out, and I heard adding names is good practice (I now see why)

granite oyster
#

Gotcha 🙂

bold timber
#

Thanks for the insight

#

Hey, found the culprit, it's my custom spritesheet animating component that has a huge size compared to the rect it actually renders.
I'm not sure how to fix that in particular but for now, is there a component to disable bevy_mod_picking on an entitity ?

granite oyster
#

That's weird about the enormous spritesheet tho 🤔

bold timber
#

Pickable::IGNORE works great, thanks for the fix

gloomy path
#

RIP, PointerButton doesn't support other pointer buttons, which means I can't make M4/M5 useful to those who have them

bold timber
granite oyster
novel hamlet
marble pine
#

What's the best way to disable On callbacks when the game is paused (using a state)?

#

Since I don't think I can slap run conditions on these right now

novel hamlet
#

Easiest would be to add the check in the callback system.

marble pine
#

Works like a charm

granite oyster
#

@novel hamlet are you interested in supporting arbitrary rotation on sprites? atm it only works correctly with Z rotation. I'm adding world hit position to the sprite backend and I can go ahead and add this as well if you think it's worthwhile 🙂

#

XY rotation on sprites isn't super common to be fair, but it also seems like a valid use case

novel hamlet
#

That's why the sprite example looks so wild, it's really more of a test for all the crazy permutations that sprites can be in. There are so many places where scaling, offsets, etc can happen; it is really hard to catch all of them.

granite oyster
#

I threw a question into the pr description about pre- or post-sorting sprites/hits

hallow river
#

Is there any way to make bevy mod picking do picking with respect to something projected onto a sphere?
I have ui that i have projected onto the inside of a sphere for use as a VR desktop environment

marble pine
hallow river
#

I see, okay

#

thanks!

novel hamlet
hallow river
#

yes

novel hamlet
#

It should be a lot easier then. All you have to do is do the math to hit test where the pointer is in the sphere rect, and all your picking math can be normal 2d rect math.

hallow river
#

okiee

eternal pumice
#

Per-camera highlight settings is not a thing right? I have a use case where I have a debug camera which would use highlights, and "real" cameras which should not.

Taking a gander at impl I guess it won't work since it seems to swap out assets instead of overlay it. Please correct me if wrong

novel hamlet
#

That's correct. The built in highlighting system is pretty rudimentary. Theoretically, if you store information about render layers or camera entities inside the asset, it's totally possible.

eternal pumice
#

I suppose other routes could be either a gizmo based highlight, or a render graph node that runs late with settings extracted on a per-cam basis

novel hamlet
#

We could switch from asset-handle-based to component-based, to make it completely flexible. All it does is handle the boilerplate of swapping between component values on an entity when it is hovered/selected/clicked

granite oyster
#

Is there any way to order event listeners? Barring that, is there a way to stop an event from getting to a lower target? To be clear, this isn't propagation since the listeners are in separate hierarchies. Basically I have A and B that both listen for Pointer<Down> (and overlap). I either want a way to make sure A fires before B, or if B fires first, to mark the event as consumed so it doesn't reach A

#

or tbh there's probably just a better way to architect this... idk

#

Hmm, callback "priority" (or order) is probably too niche a feature to add

granite oyster
#

Alright, I think the call is just to disable one of the handlers when I don't want it active

novel hamlet
#

The order is currently indeterminant, because they are independent events.

acoustic kettle
#

Is there some way I can tell if a Click as part of a Drag?

For context, I'm reacting to both clicks and drags on a 3d mesh, and doing some stuff based on where that 3d mesh was clicked, or doing something different if it was dragged.

What I am seeing is that after I finish a drag, I receive both a Click and DragEnd, which would be fine, but they arrive on separate frames -- DragEnd arrives 1 frame earlier. So I can't really see a way to easily disambiguate them.

It seems that I also get DragEnd even on a movement-less drag. So I suppose I could respond only to DragEnd and just apply a distance threshold for clicks? edit: Hm, that doesn't seem possible -- DragEnd doesn't contain the pointer position.

novel hamlet
#

Unfortunately these kinds of useages aren't really possible without the ongoing upstreaming work, which will include collapsing the events down. The Click and DragEnd arriving on different frames sounds like a bug.

#

Yeah, dragging does not enforce any thresholds on amount of movement to consider a drag as started.

#

DragEnd includes distance dragged

acoustic kettle
#

The Click and DragEnd arriving on different frames sounds like a bug.

Can't repro in a minimal way, so must be something weird I'm doing.

#

Ah yeah, makes sense. Need to consume both events streams.

hasty sail
#

Does PointerInteraction::sorted_entities contain the parent entities? Or just entities that directly overlaps with the cursor?

novel hamlet
#

It does not include ancestors

runic glade
#

is there any point in using ButtonBundle over NodeBundle if i’m using this crate to handle interaction stuff?

fiery night
#

Has anyone attempted to trigger an observer from within a bevy_eventlistener event handler? For some reason when I try to inject Trigger<MyEvent> into the handler, it no longer considers the closure an IntoSystem and won't compile.

novel hamlet
novel hamlet
buoyant escarp
novel hamlet
#

I assume they are trying to use Trigger<MyEvent> as a SystemParam in the event listener's callback system?

buoyant escarp
#

What's why I want to see the code, I don't understand what they are trying to do. Trigger<T> is created by the observer dispatcher as an input to the system, not a system param. It's not something I would expect to see anywhere but in an observer.

#

If they are just trying to trigger an observer from within an eventlistener callback, they should just use commands.trigger(MyEvent)

#

or perhaps trigger_targets, if they want to trigger the observer on the same entity as the eventlistener callback.

novel hamlet
#

Thanks, that is helpful. I'm ignorant to all of that, and I thought you might have a much better inkling.

fiery night
buoyant escarp
fiery night
wispy sierra
#

Is there a way to check if something was pressed/hit/interacted with?
I want to interact with interactable objects but move around the camera when it's something I can't interact with

novel hamlet
#

Look at the docs and examples.

wispy sierra
wispy sierra
novel hamlet
#

If you detect a mouse click and there are no picking events received.

wispy sierra
#

Ok thanks

wispy sierra
#

Is there a way to shorty disable/hide an object for interaction? I don't know if its an avian collider or mesh and want to ask before I write the logic twice and add a distinction everywhere

novel hamlet
wispy sierra
novel hamlet
#

You can't click things if they aren't hoverd.

wispy sierra
#

:0

#

Thanks 🙏

novel hamlet
runic glade
#

is there any way to have drag events register even if they aren't on a specific entity? i want to be able to click and drag in order to translate the camera

flint sundial
novel hamlet