:rainbow: Rainbow Team Relations :rainbow:

[cloud plover]

I have tried to be diplomatic, and have discussed this pattern with you in the past, but this pattern of interpersonal behavior needs to change.

(To be clear, I do think that our initial relations implementation is incomplete, and look forward to building upon it)

[wary fjord]

For my personal interests what I need is still missing (many-to-many), but at this point any step no matter the size is a step nonetheless

[cloud plover]

Yeah, I just ran into needing many-to-many for directional navigation today 😅

[echo ore]

Could just provide join tables smh 🙄

/j

[cloud plover]

I'm also in the middle of adding EntityHashSet as a possible RelationshipSourceCollection, to support more efficient relationships with unordered "child analogs"

[viscid python]

I'll say my final peace on this. I've always been more excited for those "complex internal refactors" more than every supported rushed impl, because they're not just bricks for a good relationships impl they're also features. Hooks & observers was one of these. From everything I've seen have those refactors have never gotten as much support as this.

[cloud plover]

Okay, so coming back to this, I'm running into a bit of a block: https://github.com/alice-i-cecile/bevy/blob/fe9570461000c2d1231daebd902fb5f13f0e4be1/crates/bevy_ecs/src/relationship/relationship_source_collection.rs#L67

We need an iter method that returns an owned, double-ended iterator of Entity

But EntityHashSet::iter() returns &Entity as the item

I can't just create a temporary Vec and return an iterator over that, because then it complains about temporary values

My options are:

1. Change the trait definition
2. Change EntityHashSet::iter to use a custom iterator or something
3. Some secret third thing that I've missed

[echo ore]

And I assume .copied() violates the need for a static iterator?

[cloud plover]

the trait bound `entity::hash_set::Iter<'_>: DoubleEndedIterator` is not satisfied
the trait `DoubleEndedIterator` is implemented for `core::iter::Copied<I>`
required for `core::iter::Copied<entity::hash_set::Iter<'_>>` to implement `DoubleEndedIterator`

[echo ore]

Ahh ok so hash_set::Iter just doesn't implement DoubleEndedIterator at all

[crude oracle]

DoubleEndedIterator is currently used by some of the Query extension methods. If we decide to drop those we could drop that requirement

[cloud plover]

Makes sense. Can we add an alternate iterator method on EntityHashSet or something?

[echo ore]

Hmm, since HashSet makes no guarantees around ordering, why not fake the double-ended-ness and just return next() when calling from the front or the back?

[cloud plover]

I do like those extension methods 😅

[cloud plover]

That seems nice

It'll mean a custom iterator type for EntityHashSet, but that's fine

[crude oracle]

On the topic of relations, I personally dont see the current impl as particularly rushed or compromised. I'm still pretty bullish on relationships being normal components.

[cloud plover]

I think that an iterator over EntityHashSet returning a plain Entity seems pretty sweet

[echo ore]

I personally think the worst you could say is there's more to add to the current API to allow many-to-many, query integration, etc. I'm just glad we have even the first piece of a blessed relations API to work on now

[ancient spear]

I've seen you mention the idea of "value components" before. Is the idea that (for fragmenting relations) things would stay normal components, but have the component id decided at insertion time based on the value?

[crude oracle]

Internalizing storage does give us the ability to do arbitrary magic on the inside, but by forcing ourselves to use normal ECS APIs as the foundation we make the API much more natural in terms of spawning / reflecting / etc.

Then for enabling things like "fragmenting relations", by considering it from the perspective of "what features does the existing ECS API need to support this", it encourages us build those in a way that benefit non-relations scenarios.

"Value components" are one of those features. "Fragmenting relations" can just be components whose values affect their component identity (thus fragmenting the archetype). We can use "value components" on non-relation components for cases that might benefit from fragmentation.

[crude oracle]

Yup!

[ancient spear]

very cool! I wonder if this could be used for enum discriminants as well

[crude oracle]

Thats one of the ones I'm interested in / thats one of the first scenarios I'd try to make work outside of relationships

[ancient spear]

it would be really nice to have proper closed sets for common "filter" components 🙂

[crude oracle]

I also like this approach because we can build things out incrementally in a reasonably uncontroversial way

Rather than needing some big new special cased "architectural pillar"

[echo ore]

I'm really curious to see how the Query API will change in response to value-components.

[crude oracle]

And if we do ultimately hit a wall and decide that a special cased pillar is necessary, we won't really have wasted our time because we built generic improvements

[cloud plover]

I'm bullish on first-class indexes in the ECS, and think that we could get a lot of the "queries are aware of relations" benefits using them

[crude oracle]

(and we will have unlocked a bunch of relationship scenarios along the way)

[tall mountain]

unlocking relationship progress like hearts in a dating sim

[ancient spear]

I'm guessing it might end up something like this, to avoid doing the more flecs-style dynamic query contruction in systems:#ecs-dev message

[somber girder]

Is there a reason why this needs to be owned? EntityBorrow could be used for this!

[ancient spear]

So you could maybe have: Query<Entity, (), ChildOf> to group by the "discriminant" of ChildOf, then you could iter_with_target(target: Entity)

[cloud plover]

Hmm, maybe! I'm trying not to make too many changes at once in this PR

[somber girder]

For a natively DoubleEndedIterator hashset iterator, that is usually what IndexSet is used for

[cloud plover]

Alright, draft PR is up: https://github.com/bevyengine/bevy/pull/17447

[somber girder]

Why is there a need for the doubleendedness here?

[cloud plover]

Extension traits for traversal

We could probably actually move that bound...

That will need one more associated type but eh

[wary fjord]

That DoubleEndedIterator impl would be a bug according to the C-Bug label

[somber girder]

yeah, that is quite a strange impl

[cloud plover]

Yeah lol, I think we should move the bound instead tbh

[noble panther]

What's needed for performing queries on entities targeting a given entity, as efficiently as them just having a marker component?

I basically want this:

fn my_system(query: Query<Entity, With<BelongsToA>>) { /* ... */ }

but in a way where I don't need to encode the filter at the type-level, and can instead query for all entities with a BelongsTo(A) relationship (where A is the entity). This should ideally be as fast as the statically typed version, and let me use different targets at runtime

Is this basically fragmenting relations (with value components) and relationship queries?

[ancient spear]

#ecs-dev message

[noble panther]

(Context: I'm wondering if I could make physics worlds entities, and assign entities to them via relationships, and then perform queries for each world's entities as efficiently as filtering by a marker component for each world, but without being constrained by types 😛)

[viscid python]

If I understand you correctly

struct MyEdge {
    data: Foo,
    entity: Entity,
}

changing entity here would effect the archetype? Component field modifications are non-structural changes. Archetype moves are structural changes that have to be applied at a sync point. What's your plan here because forcing the component to be immutable is really poor ergonomics if you just want to modify data.

[ancient spear]

honestly I don't find that too terrible, especially since you could have another component alongside if you really need quick edits for relationship edges

how common is it for relations to have non-target data anyways?

[viscid python]

you can't have "another component along side" if you allow a relation arity greater than 1

(one to many, many to many)

[cosmic elk]

There's actually a bunch of use cases for "value relations"

@cloud plover 's directional navigation demo is a case in point.

[crude oracle]

I haven't fully explored this space yet, but one of my first thoughts is "relationships dont directly store data and instead associate to it via required components"

[echo ore]

Once again, you can use a join table to implement this while a more streamlined API is built. A -MyEdge-> MyEdgeData -MyEdge-> B for example

[crude oracle]

(although this definitely doesn't work if you have more than one of the same relationship type on an entity)

Solving that problem "generically" could be something along the lines of "table data keyed to the value of another column identity":

#[derive(Component)]
#[requires(FriendInfo)]
#[component(value)]
#[relationship(relationship_target = Friends)]
struct FriendOf(Entity);

#[derive(Component)]
#[component(identity = FriendOf)]
struct FriendInfo {
   strength_of_friendship: usize,
}

entity.get_keyed::<FriendInfo>(FriendOf(me));

[echo ore]

Another option could be to have a new trait, Fragment, which describes how a component should be assessed for fragmentation.

This has a nice benefit where you could fragment as Transform-like component with a coarse grid, so entities "near" each-other get put into the same archetype automatically

[ancient spear]

could be an extra Discriminant type on Component defaulting to (), where Relationship: Component<Mutability = Immutable, Discriminant = Entity>

[echo ore]

That's definitely a possibility! Outside relations there's definitely value in value-components and a way to specify how the component should be discriminated

[ancient spear]

I wonder if there's a way to refer to an enum discriminant without constructing an actual enum variant first

like discriminant!(Foo::A) which would fill the variant with all zeros and transmute, then call mem::discriminant, but that's UB

[viscid python]

Ok but what if you don't know the key & you just want the edge data + the entity?

[crude oracle]

Perhaps something along the lines of Query<Keyed<FriendOf, FriendInfo>>, which returns all pairs of both components?

[viscid python]

I was trying to figure out if this design would allow for entity.get_keyed::<Foo>(bevy::Wildcard)

[crude oracle]

I think that query qualifies right? And we could definitely built similar logic into the entity-access apis

[crude oracle]

And given how this is implemented, you could have more than one piece of data keyed on a given relationship:

Query<Keyed<FriendOf, (&FriendInfo, &FriendshipQuality)>>

[viscid python]

Bit more verbose but I think so. If the entity API allowed for wildcards it'd just be missing entity.get_target::<Foo>(n) (n denotes which edge you want when you have an arity greater than 1)

[crude oracle]

Given that correlating data to relationships isn't something that all relationships need, and given that someone might want to correlate multiple pieces of data to a relationship, I think this is a reasonable tradeoff

And we can also build shorthands, or encode "implied associated data" in the type system if we want

I see no reason why wildcards couldn't be supported (although idk if id phrase it as a "wildcard" parameter)

We should consider just making it a separate iterator api

Reasoning about "relationship arity" is something I'll have to do another day (this is my weekend). I firmly believe that is a solvable problem one way or another

[ancient spear]

I like Keyed for this. If we were to generalize to value components in general, we'd need to separate components with Discriminant = () that have exactly one occurence on an entity, and general target types that might occur multiple times.

i.e. QueryData becomes QueryData<Target = ()>

I see the vision 👁️

[cloud plover]

"spring strength" is my go-to example, or "path length"

[noble panther]

technically any physics constraints between two entities (though I probably wouldn't implement them as relations)

[wary fjord]

generally anytime you need to describe the relation itself rather than either entity

[echo ore]

How friendly are these two friends? What position does this employee have at the company, etc.

[crude oracle]

@viscid python I will echo what @cloud plover said here. The way you've approach critique over the years has been exhausting. Dissenting opinions are welcomed and encouraged, and in this particular case your experience with relationships is super valuable. But lets look at how this conversation played out:

1. You came in really hot, in a way that strongly implied that we (and largely I in this case) are taking shortcuts / being short-sighted / are not providing real value / are doing this for the hype without understanding what we're doing. This was clearly intended to trivialize the work done and imply that we (and largely I in this case) did a bad job. The vibe here was "punitive".
2. People shared why the current implementation is valuable for our immediate goals, and we showed that it serves a foundational piece of a pretty well thought out path towards elegantly improving both Bevy ECS and relationships in a low risk incremental way.

I'm pretty used to dealing with this type of thing in a way that doesn't hurt me (after many years of experience dealing with challenging situations), but a lot of people aren't. If this was anyone else, your behavior could have driven them away from the project for good.

I'd like your takeaways from this to be the following:

1. Your feedback is welcome (even when it is a dissenting opinion), but only if it is given constructively.
2. Just because something isn't implemented the way you want it to be, doesn't necessarily mean it is bad/wrong. When in doubt, give people the benefit of the doubt and approach interactions with some level of humility.
3. The way you treat people matters, and many of us don't enjoy the way you've interacted with us. I value your insight, but I don't want to deal with bad vibes to get it. And I definitely don't want those bad vibes scaring people away.

[viscid python]

I've decided I will no longer be participating in bevy development. I thought I'd be less frustrated & more patient in these conversations because I'm no longer using bevy but I guess not. I've always tried to keep it constructive & technical but I think I've just been suppressing negative emotions which... even if I didn't slip up at wouldn't be enough. My mistake was thinking it would have been enough but it wasn't & was just lazy. I guess I have to put more effort into being light hearted & less serious. Sorry to anyone I've made disheartened. I just wanted to see the best relations impl & the best version of bevy & I hope you all prove me wrong & achieve that.

[crude oracle]

Makes sense! Thanks for letting us know where your head is at. You’re welcome back whenever, but if interacting here involves suppressing a lot of negative emotions, it does probably make sense to take some space.

[cloud plover]

@somber girder I got the "move the trait bounds" approach to work 😄 https://github.com/bevyengine/bevy/pull/17447/commits/0955a157cd880c060665e753b2e4bf851b413da9

I think that's a lot more correct TBH

Just uh, mildly insane

impl<'w, 's, D: QueryData, F: QueryFilter> Query<'w, 's, D, F> {
    /// Iterates all "leaf entities" as defined by the [`RelationshipTarget`] hierarchy.
    ///
    /// # Warning
    /// For relationship graphs that contain loops, this could loop infinitely. Only call this for "hierarchy-style"
    /// relationships.
    pub fn iter_leaves<S: RelationshipTarget>(
        &'w self,
        entity: Entity,
    ) -> impl Iterator<Item = Entity> + 'w
    where
        <D as QueryData>::ReadOnly: WorldQuery<Item<'w> = &'w S>,
        <<S as RelationshipTarget>::Collection as RelationshipSourceCollection>::SourceIter<'w>:
            DoubleEndedIterator,
    {...}

[somber girder]

Nice! Those do indeed look ominous 😆

maybe it'll look a bit cleaner with some type aliasing?

[cloud plover]

Yeah; I'll see what I can do there. Does Rustdoc use those yet?

[somber girder]

IIRC rustdoc does, but rustc errors dont

[wary fjord]

is rustc planning to use them for errors?

[somber girder]

We should expect every bound/trait/type we write to appear in errors eventually

[cloud plover]

https://github.com/bevyengine/bevy/pull/17447/commits/9221e27d788b78db680cdd4bf41d78a72cf0bb10 It really does, thanks 🙂

[wary fjord]

I wonder if the relation query methods should use Item<'w>: Borrow<S> instead of = &'w S

[somber girder]

On that note, why the rename from Iter to EntityHashSetIter?
convention usually is to do entity::hash_set::Iter instead so users can choose to disambiguate if they want

[cloud plover]

@somber girder do you think that IndexMap is worth implementing as a storage backing, or should I stick to EntityHashSet, Vec and SmallVec?

[cloud plover]

It needed to be publicly exported, and the module itself isn't public

I could have changed that, but that would be breaking, so instead I chose to disambiguate in the type name

[somber girder]

I think so, indexmap is basically HashMap + order which allows for some nice things, like slicing

also, I am currently on the PR for that very EntityIndexMap right now

[cloud plover]

Oh sweet. Let's wait on that to add the impl and only do it for the specialized Entity variant then

[somber girder]

hmm, would pub use hash_set::* in the proper mod.rs not work? Then the various iterators could be mentioned with consistency (same for hash_map)

[cloud plover]

You can, but that then it's use entity::Iter, which is bad.

[somber girder]

true

[ancient spear]

can you rename in a reexport?

[cloud plover]

Yeah

I think we should just expose the modules, but that was a bit more than I wanted to do in this particular PR

[somber girder]

Currently, that type is a hashbrown HashSet copy with an extra generic, so diverging on the names would mean we'd be more commited to wrapping the original

[cloud plover]

Easy to do though

[somber girder]

which by itself isn't really bad, just somewhat more verbose than using the original hashbrown type

I did investigate whether that extra Hasher generic could be added to the iterators in hashbrown proper, but surprisingly, that is actually a breaking change - which means more tangible usage would be nice before proposing it

[cosmic elk]

Being able to express critical views without hurting feelings is a subtle skill. Many people learn this skill instinctively, but some people (like me) need a guide, a teacher. Learning how to navigate disputes and tense situations has been a life-long journey for me. One of the most valuable, eye-opening resources for me is learning Non-Violent Communication (NVC) cnvc.org.

Being "light-hearted" is not really a solution in the long-term, people can see through the humor and recognize it for what it is - a mask. A better strategy is to be able to learn to instinctively recognize the difference between factual judgements and moralistic judgements, and learn to use language that concentrates on the former, while leaving out the latter. It's the difference between saying "I don't think this will work" vs. "This is a stupid idea".

[somber girder]

In a similar vein, I've found this blog post nice:
https://smallcultfollowing.com/babysteps/blog/2023/09/27/empathy-in-open-source/

[somber girder]

https://github.com/bevyengine/bevy/pull/17449

[cloud plover]

There we go: https://github.com/bevyengine/bevy/pull/17447 should be ready for review finally

Took me ages to figure out why bevy_ecs_compile_fail_tests wasn't working: I was missing a feature flag in bevy_ecs's own dependency on smallvec that was enabled elsewhere in the broader bevy tree

[marble relic]

Dang, I know cargo (rustc?) got a feature a bit ago where it would tell you which feature was missing as a compile error but I don't know if it works 100% of the time

[native yacht]

very cool that relations has shipped good work yall :3 its very cool to look back on the stuff that was "theorized" a couple years back and see it actually exist now and work

[somber girder]

Its interesting how that happens!

[noble panther]

Spitballing here, but would it make sense to use graph edge terminology? Where RelationshipTarget would be renamed to RelationshipIncoming:

#[derive(Relationship)]
#[relationship(incoming = Children)]
pub struct ChildOf(pub Entity);

#[derive(RelationshipIncoming)]
#[relationship_incoming(relationship = ChildOf)]
pub struct Children(Vec<Entity>);

With RelationshipSources, I believe the confusion was that while the component stores the sources, it lives on the target entity, and conversely with RelationshipTarget, it's confusing that the type doesn't actually store the target, it stores its sources.

With RelationshipIncoming, I feel like it's a bit less ambiguous, as it indicates that the component simply represents incoming edge(s), which also implies that it must live on the target entity.

Arguably, this is similar to RelationshipSources, but imo it reduces the confusion around sources being in the target component or the source component being on the target entity by not even using those terms, while still clearly describing the directionality. "Incoming" also works for both one-to-one and one-to-many relationships as it can be interpreted as both singular and plural.

[gusty granite]

For what it is worth, I've been keeping track on bevy ecs development for years. And I used to be very frustrated by it. This has made me pull back completely for bevy development to be clear. I now scroll the discord just to see if there is suddenly a tangible progress.

I think commenting on how one frustrated user has chosen to communicate about these issue is valid, but it is not the whole picture.

There are some core contributors who have responsibility of part time contributors burnout on ecs things. And frankly, it could be an issue with how bevy ecs is governed alongside the rest of the engine.

I do find it disappointing that after years of free contributors bowing out of bevy ecs development, and after many many eruptions on the discord... Only community conduct is raised as the problematic aspect of this.

[devout sedge]

Just to chime in again: We now are making tangible progress again, because it turns out that there are pieces missing in the ECS that are causing issues for some of the nice things we want to land. We've had a lot of discussions, a lot of bikeshedding, and that takes a toll on everyone involved. It wasn't helped that some needed refactors merged into main too quickly, which then caused regressions that had to be resolved after the fact, and naturally, it made contributors wary on doing such work.

But naturally, it couldn't remain stalled forever, and now stuff is getting landed in a much more incremental way that is getting us where we want to. Folks are naturally going to get excited for this. What is actually demotivating is then being told that is not enough and not good somehow. We should not let Perfect be the enemy of Good, and also we should try to not put others down just because what had landed is not quite the exact vision of how to proceed with things.

Burnout happens regardless of whether a project runs well or not. We get frustrated with things because we care, but that is also the danger. We can care too much and lose sight of the bigger picture and also of our own mental state. We have to manage our own expectations too, and also be ready disengage when we get too frustrated or burnt out. This is not a bad thing, and it shouldn't stop folks from returning either once things have progressed or they feel ready to approach the problem space once more.

Place a bit more trust on others. And also remember to look after yourself as well. This is a long-haul project and it isn't going anywhere. People will come and go, and that's okay. Anyways, end of my wall o' text.

[safe shoal]

I'd not really call what I've seen in this discord or even this specific topic over the last year "eruptions", if anything the ways conversation works out when conflict and critique arise is what drew me into being active at all here. lots of open source projects decide to just "man up" and "get it done" and become fiefdoms where nobody wants to explain how something works to you and contribution is ad-hoc, messy.

it's not a mistake or sign of weakness that complex systems design takes time, and it's dishonest to view things as not having taken big leaps even recently, even with a re-formalisation of "already existing features" that enables designing next steps.

[safe shoal]

I do not aim to be sycophantic. but nor do I think most people here aim to be. It's just that I have been working with bevy on my own load-bearing projects in the last few months and I have felt the changes in what is possible in practice.

[unborn quail]

Hey @alpine patrol were you able to look at the relations rfc?

[echo ore]

Can definitely co-sign this. I'm currently doing a massive ground-up refactor to allow Bevy to run on no_std targets. While I have some pushback, I would say it's all been entirely constructive and in the best interest of the project. Maybe one day I stop contributing, but that's just how hobbies work...

[alpine patrol]

Uh, what is this in reference to?

[unborn quail]

this guy: https://github.com/bevyengine/rfcs/pull/79

[alpine patrol]

I don’t think so. Why do you ask?

[unborn quail]

I vaguely remember that you offered to review it a couple of months ago when I was complaining about that progress was slow.

But it might've been someone else, I can't quite recall

[alpine patrol]

I probably did and forgot, let me add it to my list, sorry about that.

[unborn quail]

it's fine

I was just reminded of it

found it!

my memory is doing better than I thought

[crude oracle]

I do think RelationshipIncoming is nicely unambiguous. I also agree that it is very similar conceptually to RelationshipSources. Going out of our way to avoid using the source / target terminology we've defined in some ways avoids confusion, but it also introduces a "new" concept ("incoming"). I personally think that whatever we use should employ the source/target terminology, to encourage people to talk about and think about it in those terms

RelationshipTarget has the benefit of having the same "is a" naming style as Relationship, so there is parity

I think RelationshipSources is slightly less ambiguous, and makes it feel more like a collection.

But ultimately we may want explicit one-to-one relationships. In that way RelationshipTarget feels more generic

[noble panther]

Yeah, it's just that "Children is a relationship target" feels inherently misleading to me personally, because:

1. The component stores the source entities instead of storing the target.
2. The component is not the relationship target, the entity is. The component is just tracking incoming edges / source entities that have a relationship with that entity. The entity would still be the target of the relationship without this component existing.

Relationship clearly "is a" relationship, and a component implementing the trait is the source of truth for that relationship, but for RelationshipTarget I think that is less true

[cosmic elk]

I have a couple of other random ideas on naming:

• One term I thought of is RelationshipPartners, although perhaps it has unwanted overtones, and doesn't strongly indicate directionality.
• Another term would be Related. We've already introduced this term in things like despawn_related(). However, in the latter case, related means "all entities which are transitively reachable" not just the nearest.
• Other terms might be "reciprocal" or "inverse" as in "Relationship(inverse = ChildOf)".

[keen mural]

Just a lurker reading back through history as I do, and wishing I could make a thread-in-a-thread for this. 😅 I feel like this doc comment is missing the case of non-tree DAGs, am I missing something?

[vivid cave]

One idea I don’t think I’ve seen suggested - what if we didn’t need a Children component or a RelationshipTarget trait at all?

I propose we have only one trait, Relationship, that is implemented on the source of the relationship i.e. ChildOf.

To query the inverse of a relationship, instead of having a dedicated component for each relationship type (Children), bevy would automatically add a generic Related<ChildOf> component to the target of the relationship.

I feel this is conceptually much simpler. There’s a single trait and type defining each relationship. All children entities will have a ChildOf component, and all parent entities will automatically have a Related<ChildOf> component.

This also lends itself well to one-to-one. Related<T> can generically contain one or many elements depending on the specific implementation of the Relationship trait.

@crude oracle hate to ping you, but I’m not sure you’ll see the above if I don’t, and I feel the idea is worth considering

[crude oracle]

The component stores the source entities instead of storing the target.
This is kind of inherent to the space if one component stores the target reference and one component is the target.

The component is not the relationship target, the entity is. The component is just tracking incoming edges / source entities that have a relationship with that entity. The entity would still be the target of the relationship without this component existing.
I do see your point, but as a counter-argument, how do we mark an entity as "being" something? Via components 🙂 (ex: "this entity is a enemy, so I'm giving it the Enemy component")

[crude oracle]

One term I thought of is RelationshipPartners, although perhaps it has unwanted overtones, and doesn't strongly indicate directionality.
In addition to the potential unwanted overtones, I think it suffers from the same general problem of "introducing a new concept that is not discussed or used anywhere else when talking about relationships". If the goal is to erase the "target" terminology, I think going in the direction of "sounds the least like a new concept name" is the move. I don't want people to look at relationship partner and be like "what the heck is a partner and how does it relate to a relationship source or target".

Other terms might be "reciprocal" or "inverse" as in "Relationship(inverse = ChildOf)".
If this internally uses a different trait, thats even more confusing. And if it "specializes" the Relationship trait I think that breaks down functionally, as Relationship behaviors are "different". Possibly doable but it would introduce architectural complexity to solve a naming problem, which feels wrong.

[crude oracle]

This has been proposed before and I see the appeal. My biggest concern here is that the upcoming spawn APIs (bevy native and BSN), and "hierarchical relationship spawning" in general largely hinge on having a nice name for the related collection:

world.spawn((
  Foo,
  Children::spawn((
     Spawn(Bar),
     Spawn(Baz),
  )),
  Reactions::spawn((
     Spawn(Hi),
  ))
));

world.spawn((
  Foo,
  children![
    Bar,
    Baz,
  ],
  reactions![
    Hi,
  ]
));

bsn!{(
  Foo,
  [ // Children is implied
    Bar,
    Baz,  
  ],
  Reactions [
    
  ]
)}

Without those names this gets nastier:

world.spawn((
  Foo,
  Related::<ChildOf>::spawn((
     Spawn(Bar),
     Spawn(Baz),
  )),
  Related::<ReactionTo>::spawn((
     Spawn(Hi),
  ))
));

world.spawn((
  Foo,
  related_child_of![
    Bar,
    Baz,
  ],
  related_reaction_to![
    Hi,
  ]
));

bsn!{(
  Foo,
  [ // Related<ChildOf> is implied
    Bar,
    Baz,  
  ],
  Related<ReactionTo> [
    Hi,
  ]
)}

"evergreen relations" got around this with type aliases to a "generic collection". I'm pretty strongly against this as it:

1. Requires almost as much typing to make a type alias as it does to derive a trait on a new type.
2. Type aliases don't play nicely with reflection, scenes, or rust analyzer

[cloud plover]

I think the other problem with this design is that it makes configuring the storage type a lot harder: you need to add a second generic

[vivid cave]

Another idea is to have Related<T> be a trait instead of RelationshipTarget .

impl Related<ChildOf> for Children {
}

[rose pendant]

Fwiw in fragmenting relationships there's only one relationship ("ChildOf"). In my non-fragmenting design I used fragmenting relationships instead of generics, so entities have a (Parent, ChildOf) and (Children, ChildOf). The user API only ever uses ChildOf though

[elder eagle]

wrt the IsA vs HasA component naming convention, i don't think HasA can go away (eg "has a Transform" vs "is a Transformable" or w/e where the latter is very awkward)
so i don't think it's worth trying to standardize on one or the other. a super-component with a bunch of required components could use the IsA convention (although imo HasA is still fine there), while a simple component could use HasA

i haven't read much context here so i may be off-base

[crude oracle]

Agreed (generally). I think it might be reasonable to be principled about it in a smaller context (ex: relationships + relationship targets). But I also don't think we need to

[elder eagle]

there's also the IsDefaultUiCamera component, which uses IsA explicitly via the Is prefix. although i assume that name was chosen because DefaultUiCamera is already a system param

[versed spade]

https://github.com/bevyengine/bevy/issues/17492

I opened up an issue about relationship ergonomics. I guess it's doable from my understanding, but I'm not sure.

[versed spade]

@cold mason Would you be interested into adding a macro to mevy like the one I described in #17492? We concluded that the idea has some significant drawbacks, but it may be appealing for some people. It basically saves roughly half the work for defining relationships.

[alpine patrol]

so where does this group go now btw? how much of the minimal-relations-api doc is implemented now?

do we need an updated proposal for next steps?

[left summit]

I wouldn't be against starting a new working group, but that's just my opinion. I feel it could be good to continue without all the history

[ancient spear]

double-rainbow 👀

[tight salmon]

Value Relations. Priceless

[scenic fox]

🌈^2 Team Team Relations

[unborn quail]

This working group was created to build out flecslike fragmenting relations (or at least a minimum version of it). This very roughly follows the https://github.com/bevyengine/rfcs/pull/79 rfc and in terms of progress on that, we've removed the render world blocker with regards to a retained render world.

Next steps are to optimize data structures such that the can deal with a large number of components / archetypes. This is general optimization work which tbh is more general to the ecs as a whole

Things like archetype deletion and queries as entities fall under this

[alpine patrol]

So you see the goal as being largely unchanged, yes? That’s great.

[unborn quail]

Largely

It's just very slowgoing because there are very few people working on this / know about the next steps necessary

I sadly don't have the time

Moreover, because it's optimization work a lot of benchmarking is needed and I feel like that's a bit of a barrier for people

It is at least for me

[alpine patrol]

Going slowly is understandable. Let me read over the proposal, I have time finally.

[unborn quail]

@cloud plover If you could read through it again that would also be appreciated. Maybe by getting this merged, we get some more / new people working on it

[cloud plover]

Remind me on Sunday or Monday please? Currently weekend for me

[unborn quail]

Oh that's right

I will

[unborn quail]

poke @cloud plover

happy sunday

[cloud plover]

Happy Sunday 🙂

[cloud plover]

Alright, taking a look at this now

[cloud plover]

@unborn quail I left a series of comments on RFC 79 🙂 My primary complaints:

1. This needs to be updated: a) in terms of "what still needs to be done" b) "how does this co-exist with non-fragmenting relations" and c) "why do we need both". To be clear, I do think there's a good chance we need both.
2. I really dislike the "pair" terminology, and will insist on finding an alternate nomenclature.

I'm generally nervous about the very rigid assumptions that our current component storage makes (dense, monotonic, small), and think that those are worth addressing regardless of the final relations API that Bevy lands on

[runic shadow]

I think it would be difficult for anyone to re-write the RFC such that it would align with bevy's future plans without more direction from cart on what his "fragment on values" implementation is intended to look like. Questions like "Does each value get it's own component id? Or is this special cased in the archetype storage?" have a big impact on what the technical requirements to implement it would be.
I'm on the same page as far as bevy's ECS internals currently being quite brittle due to the distribution component ids have to have to keep things performant. In an ideal world I would be able to attach to bevy with an inspector, define a bunch of dynamic components and spawn entities with them and then go and then delete everything without that world instance being poisoned in any way.

[cosmic elk]

I'd like to see more concrete use case examples

[runic shadow]

For what part?

[cloud plover]

I definitely agree with this, but I also can't sign off on this as an ECS-SME without at least a rough plan of how this all fits together

[runic shadow]

Of course, a lot has changed since the RFC was first written.

[cosmic elk]

The "motivation" section talks about the need for better hierarchies, more features/options, but it is all very abstract

[cloud plover]

(especially in the context of non-fragmenting relations)

[runic shadow]

Well at the time this was written the only Parent-Child hierarchies existed, only having that option was pretty restrictive. In the non-fragmenting implementation support has now been expanded to arbitrary relationship types so that motivation is gone.
On the other hand there is still no way to express in a query "give me my parent's transform", fragmenting relationships make this easier to implement (and more performant) since the hierarchy is stored entirely in the archetype, the parent for a given archetype isn't going to change out from under you. It makes it easier to query down a hierarchy like for transform propagation as you can just order the archetypes/tables you visit since you know each level of the hierarchy is in it's own archetype.
There are a bunch of places that the non-fragmenting implementation is using hooks internally, which incur additional overhead compared to if the ECS storage was natively aware of relationship ids, with the pair id design setting a relationship is the exact same operation as setting a ZST component (which also simplifies the dynamic API surface, only need to expose one operation).
If you wanted to fragment on values you could store them as a (Type, Value) pair of ids so you can query for all entities with any value of that type, or all entities with a specific value, and the clean-up is easy since if the type is dynamic and you delete it you can easily find all (Type, *) pairs to clean-up.

Most of the features can be implemented with a non-fragmenting design, there are just performance and UX considerations.

[rose pendant]

I think queryability in general is mostly lacking in the non-fragmenting implementation

[cloud plover]

Yeah, I agree with that analysis

You can work around it by using multiple queries and calling query::get

But that's not great ergonomics, and it obscures a lot of what's going on

[rose pendant]

Use cases for more interesting relationship queries have been discussed plenty and have been written about

Since flecs 4.0 those query features have also been used quite a bit- so there's prior art showing it's useful

Also obligatory link to https://docs.larian.game/Osiris_Overview which shows the same kind of tech being used by an AAA studio

[runic shadow]

One example I have run into personally when writing a flecs-vello integration was being able to make a query for all shapes and their parent scene, which made it really concise to write systems that rendered out all the shapes to the scene. Using two queries works, it just feels so clunky by comparison, and it's probably non-trivially slower.

You can do all the advanced query-ing stuff on the hooks based impl, but it seems much more painful to implement. A lot of bespoke code before you even think about how you are going to support dynamic APIs as well.

[timid mountain]

For that specific part, would this PR be the only thing needed?

[runic shadow]

There's also the sparse set in table storage PR: https://github.com/bevyengine/bevy/pull/14928

Those are the main two things off the top of my head, there are also a few other minor usages of component id as a vec index that would need to change.

[cloud plover]

Ah good, I was going to point out that location too 🙂

But yeah, I'm happy to move forward with both of those PRs if y'all can get two approvals on them

I would also be happy with a ComponentId -> Identifier (or whatever) rename today, even before changing any of the data that it stores

It's generally quite confusing right now, since they're also used for resources

[unborn quail]

I've read back the discussion and concluded three things:

• I still think that having both fragmenting and non-fragmenting relations is a good goal to strive for, but how that API will look like is a challenge, and this is something I'd really like @crude oracles input on.
• I need to really look into the fragmenting relations implementation and its API to see what it solves and what it doesn't.
• There are quite a few no-brainer things from the RFC we should still get to, but sadly I don't have much free time atm and likely for the next five months

[cloud plover]

On the topic of RFC #79:

Closing this out: I don't think that editing this is a particularly good way to get to an implementable design doc. Too much has changed, there's too much history here, and there's too many implicit assumptions about the path we should take. I'm not at all opposed to archetypal relations (and actively want to support things like large numbers of archetypes and clearing empty tables and archetypes), but I want to see a user-focused path / design for how and why Bevy should get there.

I'm leaning towards closing this working group and moving discussions back into #ecs-dev as well; we don't really have a clear plan or scope currently, and I don't want to fragment discussion on more incremental improvements

But I'll wait a day or so for y'all to disagree with me on that

[rose pendant]

we don't really have a clear plan or scope currently,
I thought this group was making progress on the things outlined by @runic shadow in his RFC for removing blockers

which seemed like a reasonable/correct list to get to fragmenting relationships

[cloud plover]

Very much agree with this point

The problems are that a) folks have not laid out a clear argument for why that's the path Bevy should follow, beyond "flecs has shown that it works" and b) no one seems actively interested in writing that design doc

It will be months until Bird is free to do so, and these groups should not languish indefinitely

That's not intended as a knock against flecs, or the technical ideas behind fragmenting relations

[rose pendant]

Hm but people have been making arguments in favor of fragmenting relationships for the past three years 🤔

Neither here nor there for me, if bevy no longer wants to implement fragmenting relationships then closing the WG is fine

which would make sense if it no longer aligns with the future direction you/cart want to take it in

[cloud plover]

That's not what I'm saying here: Bevy is officially neutral on "should we implement fragmenting relations", and has never endorsed implementing fragmenting relations. The working group at this stage is "let's see if you can come up with a compelling proposal", and the status has been "no proposal has been approved, the existing proposal has serious problems, and no one is going to write a replacement"

If someone is interested in making that argument, we can spin a new working group up

Or do it without one

But we shouldn't let this working group just languish

My personal unofficial stance is that Bevy should have fragmenting relations, unified under a single API, with a perf-optimization switch. Just like table vs sparse storage

[left summit]

my opinion is that working groups should have a limited scope, and finish in a reasonable duration. we kinda fail with this one from that point of view, and continuing with a new working group with a well defined scope that can be done in max 2 releases would be better

maybe taking several working groups to reach the end of what we can do with relations

[alpine patrol]

There are quite a few no-brainer things from the RFC we should still get to, but sadly I don't have much free time atm and likely for the next five months

I'd hate to have this work be lost though. This working group has not been a failure; yall removed several of the significant blockers to relations, and any additional work items should still be tracked somewhere.

[rose pendant]

Makse sense, this is the 2nd time a "reset" happens though, and the last time things started out a bit aimless as the new(er) folks that wanted to work on it didn't have much of the context of the previous discussions. If this WG / James' RFC is closed that'll likely happen again

[left summit]

I definitely didn't mean that this working group is a failure. there has been progress, and getting a list of followup task is an output!

[alpine patrol]

i'm still on team fragmenting personally, i want to see this fully investigated and prototypes at minimum. i think we could probably keep this group open, as long as people are willing to work on updating the design proposal.

imo it's really a question of if there's community members willing to allocate time to that design/proposal/prototyping work. if not, then a new working group won't do much either.

[cloud plover]

I'd prefer a new RFC and working group at this point 🙂

There's a ton of history here that really isn't particularly relevant

[rose pendant]

This has been mentioned a few times- what kind of history are you referring to?

[cloud plover]

1000s of messages of backscroll, and arguments about what our goals should be, which doesn't incorporate "Bevy has non-fragmenting relations" into the discussion

[rose pendant]

Hm, but this WG is not unique in that

Next generation scene/UI also has 3.4k messages. There are a handful others as well

Anyway I'll drop out of the discussion as I don't really have a stake in this

I'll just wait and see what happens 🙂

[safe shoal]

to appeal to the ritual, can threads be renamed and if so can the thread be renamed to reflect fragmenting relations? would this work well enough as a sign of a sea change towards fragmentation?

[runic shadow]

The only way fragmenting relations is going to happen in my mind is to have some place to track it (whether it's a WG, RFC or issue) so that the relevant changes (which have their own self contained benefits, such as the two PRs mentioned earlier) can be merged iteratively and then a final design can be worked on after most technical blockers are complete.

The amount of time it takes to review and merge the requisite ECS changes means that no design is going to still be relevant by the time it's implementable (as the ECS APIs are constantly being iterated on) unless it's a top priority for maintainers.

If someone were to do a new design now, by the time it could be implemented it would also need to take into account bsn or some other new development.

[cloud plover]

The only way fragmenting relations is going to happen in my mind is to have some place to track it (whether it's a WG, RFC or issue) so that the relevant changes (which have their own self contained benefits, such as the two PRs mentioned earlier) can be merged iteratively and then a final design can be worked on after most technical blockers are complete.
Definitely agree: I just think we should move to an issue

[runic shadow]

I'm all for a tracking issue with the relevant information to make it approachable

[cloud plover]

Alright 🙂 I'm actually writing one up; let me move that to a HackMD and we can quickly collaborate to get the details right?

https://hackmd.io/@alice-i-cecile/SygJprBu1l

I'm splitting out the more contentious ComponentId / Identifier changes; those should be done directly as part of fragmenting relations, as they don't have value before then

[runic shadow]

Yeah, I think if we got to the point where you can clean-up archetypes and component IDs without leaving an impact on the World we would be in a good place to tackle the actual relationship part

[cloud plover]

As discussed in https://github.com/bevyengine/rfcs/pull/79 and laid out by @james-j-obrien, Bevy's ECS architecture is not optimized for use in worlds with a very large number of archetypes. Fundamentally:

bevy_ecs currently operates under the assumption that archetype and component ids are dense and strictly increasing.

While this is a blocker for efficient implementation of fragmenting / archetypal relations, it's also worth exploring and fixing in its own right.

There are several steps to this:

• [] Remove the sparse set in table storage https://github.com/bevyengine/bevy/pull/14928
• [] Replace the FixedBitSet in Access / FilteredAccess https://github.com/bevyengine/bevy/pull/16784
• [] Query and system caches need to be easier to access.
• [] Query and system caches must be updated to reflect changes to archetypes
• [] Create and expose tools for deleting component ids
• [] Create and expose tools for deleting archetypes

This is a tracking issue: as more related work comes along, please link it here and bother Alice to update this issue description.

There's the current contents; @runic shadow do you think that's sufficient for an initial tracking issue?

[runic shadow]

Yeah seems like a good start 👍

Means that when we merge the two linked PRs we have an actual place to track next steps, which is the main goal

[cloud plover]

Aye aye 🙂

Thanks a ton

[runic shadow]

We will probably need more detailed motivations for each point to make clear why they are important/worth pursuing to a new contributor (not that I would expect this to be a first stop, but when discussions start about these features we always get questions)

IMO the problem with these ECS changes is they all come across as very academic and intimidating

[cosmic elk]

The problem with long-standing WGs is that they are impenetrable for new users. No one has the time to scroll back and read 10,000 messages to get all the context of the discussion. And without a continuous flow of new users, the effort dies.

Ideally, a complex WG would maintain a "work product" / RFC / summary which would be pinned to the discussion in a way that is easy for new users to find, without having to scroll to the top. This would include links to blog posts or other necessary context.

[alpine patrol]

especially without an up-to-date proposal to function as a landing page.

[cloud plover]

Yeah 🙂 And sometimes hard to motivate. I really liked the one line from your RFC about "bevy_ecs currently operates under the assumption that archetype and component ids are dense and strictly increasing.". This is a super succinct summary of the technical problem that needs to be solved.

[runic shadow]

Its succinct but most users don't care about that detail, they want stuff they can see using

[rose pendant]

I understand- so one course of action could've been to make sure that this WG meets those criteria. I personally don't think closing the WG will bring bevy closer to fragmenting relationships

This is mostly based on seeing similar efforts die/get rebooted in the past

[cosmic elk]

In an earlier comment, you posted a link to your blog post about the use cases for relations. But that link isn't in the RFC, which means it gets lost in the torrent of discussion and floats away.