Picking through a transparent overlay camera

[faint lava]

I have several cameras, each rendering to a different area of the window, and on top of all of them I have a big 2D camera viewport (spanning the entire window) that I render the GUI buttons on. The last one is mostly transparent except for where the buttons are. If they're rendered in that order then picking in the undermost cameras doesn't work. If I change the order of the cameras so the world entities are drawn last then picking works, but now my GUI buttons are drawn behind the world entities. That looks terrible. How do I fix it so that the picking continues working even with the large (and mostly transparent) GUI camera viewport on top of everything?

[fast hound]

try looking into https://docs.rs/bevy/latest/bevy/picking/struct.PickingBehavior.html

[faint lava]

Thanks for the tip! I tried adding PickingBehaviour to the overlaying camera, but it has no effect 😦

PickingBehavior {
should_block_lower: false,
is_hoverable: true,
},

Perhaps it is not the camera entity that is in the way? I wonder how to figure out what entity need to be given this Component?

[fast hound]

not on the camera, but on the nodes

[fading jungle]

The last one is mostly transparent except for where the buttons are
That giant ui node is blocking the cameras below it

@final pasture @radiant adder another case of this issue

[fast hound]

oh, this is a known issue?

[faint lava]

Aaaaaah! Yes OK now I understand - each and every relevant node that exists in the GUI layout hierarchy need to be given the above PickingBehaviour! It works now if I do that!

That camera viewport was apparently "hidden" below 4 nodes of UI layout panels, so 4 different UI panels had to be given those PickingBehaviours before the underlying camera views were "dug up" and became pickable. I should probably have been able to figure out I had to do that, if I had just realized the fully transparent UI panels were actual things that took up screenspace, but since they're entirely transparent and only there for layout I had even forgotten they existed. 🫣

Thanks for the help!

[fast hound]

now, should a Node that does not have one of Text, Button or UiImage be pickable?

[faint lava]

IMHO: Having such nodes consume picks by default is a footgun. But removing pickability entirely decreases freedom of the creator. So IMHO it makes most sense to make it possible to be both pickable and not pickable, but have the default be no rather than yes.

On the other hand it also makes sense to have all entities behave the same, so if pickable is on by default on everything, it's a little weird and confusing to have some specific types of entities be the opposite by default. So I can see arguments both ways here.

Perhaps it is sufficient and suitable to leave it as-is, and instead just clearly inform the creator in API code comments and in examples something like "Yo! Beware that these transparent layout nodes consume pickings, and if you don't want that then you need to specify so."

[radiant adder]

Yeah; we've been debating UX for a while on this

[fading jungle]

We could make the UI backend require the PickingBehavior component, and add it as a required component on text, button, uiimage

[radiant adder]

Definitely on button, maybe on the others

[fading jungle]

You probably want those things to block

Well, maybe not text, idk

[radiant adder]

Right yeah

[fading jungle]

Well idk, users are going to expect nodes with a background to block

It kinda feels like bevy UI is over generalized at times like this. So many components to do simple things.

Without more structure, I'm not sure what to do other than make UI picking opt in, but that has its own set of problems.

[fading jungle]

That train of thought is pretty much exactly what we went through. 😅

I'm still curious if tying pickable by default to nodes with a background/fill makes the most sense.