Picking Objects In 3D

[woven jetty]

I am just starting out with programming in Bevy using this great tutorial https://www.youtube.com/watch?v=B6ZFuYYZCSY&list=PL2wAo2qwCxGDp9fzBOTy_kpUTSwM1iWWd. I have gotten the mini game running etc, and I am now adapting it to my needs. Specifically I would like to select the spaceship or asteroids. I have brought in bevy_mod_picking to aid me, but it appears that selecting SceneBundle objects is non-trivial. Do you have any tips on how to associate the high-level tag Spaceship with the picked meshes? Because right now I am able to get the picking to work by recursively going over the meshes, but the connection to the top level tag is lost, so whenever I get the click selection it is useless because I can't associate it with the transform of the space ship.

[viral barn]

This example will show you how to do this: https://github.com/aevyrie/bevy_mod_picking/blob/main/examples/gltf.rs

Also: https://github.com/aevyrie/bevy_mod_picking/blob/main/examples/event_listener.rs

The key concept is that you can stick an event listener, On<Pointer<Click>> on your Spaceship, and any clicks on children of the spaceship will bubble up the entity hierarchy and trigger the event listener on the spaceship.

At that point, you will have access to anything on the Spaceship, like its Transform, as well as information about the child mesh that was clicked from the event.

All of this is covered in the top level docs. https://docs.rs/bevy_mod_picking/latest/bevy_mod_picking/

[woven jetty]

@viral barn Thank you for the response - I have found those examples, but unfortunately I am not able to get the transform of the spaceship. I am getting the target entity, which happens to be some entity down the hierarchy from the spaceship. I also can't quite grasp how do I tag the selected spaceship if I can't get to it...

[woven jetty]

This is how I ended up getting to the transform:

fn receive_greetings(mut greetings: EventReader<DoSomethingComplex>,
                     parent_query: Query<&Parent>,
                     transform_query: Query<&Transform>,
                     spaceship_query: Query<Entity, With<Spaceship>>)
{
  for event in greetings.read()
  {
    info!(
        "Hello {:?}, you are {:?} depth units away from the pointer",
        event.0, event.1
    );

    let mut entity = event.0;

    while let Ok(parent) = parent_query.get(entity)
    {
      entity = parent.get();
      info!("Got the parent: {:?}!", parent.get());

      if let Ok(transform) = transform_query.get(entity)
      {
        info!("Location: {}", transform.translation);
      }

      if let Ok(spaceship) = spaceship_query.get(entity)
      {
        info!("Got to the spaceship!");
      }
    }
  }
}

I am not quite happy to traverse the entity hierarchy, but maybe that is the intended way of working with ECS... Still learning.

[viral barn]

You want the listener, not the target.

[woven jetty]

Thank you - this did the trick!

[woven jetty]

@viral barn I do find it problematic to query for the "picked" state though. With events I can do something when clicked on, but how do I run something continuously on the selected component? The event, although bubbles up still does not mark my top entity as PickSelection::is_selected as true, so now I need to have a way to store this state on click and somehow detect when my component is deslected and also deselect it.

[viral barn]

Then you can add a component like this and toggle it with an event.

[woven jetty]

would the recommended approach be:

1. Add selected tag on click
2. Remove selected tag on ..., what is the event that tells me the object was deselected? The deselection example seems to be targeting a different case.

[viral barn]

If you are already using the selection components that mod_picking_selection provides, you can just add that, and toggle the selected state.

[viral barn]

For 2), that's really up to the behavior you want, and why this isn't built in.

I guess my question is, what are you trying to accomplish? If you want a custom selection behavior, you could just copy paste the bevy_picking_selection plugin, and modify it to fit your needs. The crate is very, very small.

All it really does is checks if something is clicked on, and selects it, then deselects other entities based on whether or not ctrl/shift are being held, or if the entity that was clicked is able to trigger deselection of other entities.

If you want to add the ability to select a parent based on the state of a child, you need to figure out how that relationship works. Presumably, if you are trying to select a parent from a child there is some reason for this. Are their multiple children? What happens if one child is selected or deselected, but the other siblings aren't?

[woven jetty]

I have multiple entities that should be individually selectable. Every time I select one I want to draw things around the selected object. For that I need to have the condition of being selected to be easily query-able. I also want to be able to deselect any object by clicking on empty space.

[viral barn]

That is something the built in selection plugin already does.

You haven't mentioned parenting in your comment though, your original question was about selection in a hierarchy.

[woven jetty]

Sorry - I think we are thinking in terms of slightly different things. The objects I am talking about are individual spaceships, so they don't have parent-child relationships. I am thinking I'll attach a click even to window and see what happens.

[woven jetty]

I've been looking through the bevy_mod_selection code and can't quite find the bubbling up code yet. I guess I expected to see hierarchy traversal one way or another, but it does not seem to be happening. What I really want I guess is the "piercing through the hierarchy" of "is_selected". In other words - if my meshes are all selectable and my root entity also has pickable bundle on it then once I click on the mesh the event would bubble up, but also set "is_selected" on all Pickable entities to true, so I could easily query for each one.

Is this something that could be instrumented with the current state of the picking mode and I don't see how to do it?

[viral barn]

I'm confused, you are talking about bubbling up a hierarchy, but here you say there are no parent child relationships.

[woven jetty]

The spaceships don't have parent child relationship with each other, but each spaceship has a top tag Spaceship with the SceneBundle spawned with it. This SceneBundle is what contains the mesh that I end up clicking on. When I click on the mesh I do get the bubbled up event with the entity pointing to my Spaceship tag, but then at the level of Spaceship the PickableBundle is not marked as is_selected. Here is the actual code:

  commands.spawn((
    MovingObjectBundle {
      velocity: Velocity::new(Vec3::ZERO),
      acceleration: Acceleration::new(Vec3::ZERO),
      collider: Collider::new(SPACESHIP_RADIUS),
      model: SceneBundle
      {
        scene: scene_assets.spaceship.clone(),
        transform: Transform::from_translation(location)
                             .with_scale(SPACESHIP_SCALE),
        ..default()
      },
    },
    PickableBundle::default(),
    VisionObjectBundle::default(),
    Spaceship,
    Health::new(SPACESHIP_HEALTH),
    CollisionDamage::new(SPACESHIP_COLLISION_DAMAGE),
    On::<Pointer<Click>>::send_event::<SpaceshipSelected>(),
  ));

I want the PickableBundle at this level to also be marked as selected once the event bubbles up.

Now I also see how the bubbling up works - through EventListenerPlugin

[viral barn]

I don't think this changes my earlier recommendation then.

The provided selection plugin only updates the entity that was selected. If you want to add bubbling behavior, you will need your own selection plugin that manages the state of the selection component.

[hard tulip]

@viral barn Sorry to revive an old conversation here, but just wanted to ask a quick question about the gltf.rs example you linked...

Is there a reason why the make_pickable() system needs to be added to the Update schedule? Why can it not be run during startup after setup, by using .chain() ?

[viral barn]

Because the loading of a gltf scene is asynchronous, it could finish on frame 3, 50, or 100, etc. Running it in Update is just a simple way to ensure that things are make pickable as they are spawned.

[hard tulip]

@viral barn Thank’s for being so helpful in your replies. This makes sense too. I tried chaining it to the end of the other startup systems and was curious why it didn’t work. Thanks for the insight!