:rainbow: Rainbow Team Relations :rainbow:

[rose pendant]

Yeah, it'd be panic, incorrect or UB

[tight salmon]

Even with per instance counting, it would still result in a runtime panic

[rose pendant]

What James is suggesting is a different mechanism where a query optionally does not return results that alias

(or warns, or panics)

That effectively means that a query result can be "wrong"- it's not returning all results that match the query

[polar vortex]

🤔 what if it's ordinarily a panic, and we only extend the query plan if the user asks? (both rust and whatever scripting environment would have access to that API)

the unsafe iterator wouldn't change the query if you had opted into the extra checks, it just yields pointers instead of references

[tight salmon]

Yeah I wouldnt agree with an skipping version, but panic is reasonable

[rose pendant]

A database that can panic on a query is not really feasible though

[runic shadow]

That's why I like being able to warn and continue safely

[rose pendant]

Say you wanted to use bevy as a game server/backend querying engine

[polar vortex]

I mean, something else being talked about in #ecs-dev was error handling

[tight salmon]

Is the simple Sanders example above something that would cause the panic in question?

[polar vortex]

could catch_unwind every system lol

[rose pendant]

Yes, but what's a bigger issue is that while you're entering queries in an explorer-like environment, it's very plausible that you accidentally enter a query that aliases

Having your server panic on that would be a non starter

[tight salmon]

Do you return the aliasing results in Flecs or?

[rose pendant]

Yup, it's no problem in C/C++

[polar vortex]

it's an unsafe world out there

[rose pendant]

Only according to Rust's narrow definition of unsafe 😉

[polar vortex]

narrow but useful

[runic shadow]

In a dynamic scenario like that you would probably have all the code operating on trait objects and pointers anyway so you might not care about opting into an unsafe/checkless iteration

[polar vortex]

(I'm mostly joking anyway)

[runic shadow]

I really don't like my apps to crash though, I'm a big warning guy where possible

[rose pendant]

Yeah, I just mean that the UB that can occur with mut aliasing in Rust doesn't happen in C/C++

[tight salmon]

It does, the language just doesnt prevent you from doing it so you dont notice until it happens

[rose pendant]

In my previous position where flecs was used as a backend service serving up hundreds of queries/sec panics would've been a major problem

[runic shadow]

That's not true, the compiler doesn't assume they don't alias so there's no miscompilations

[rose pendant]

Exactly

[runic shadow]

You can do plenty of UB in c/c++ but not by holding 2 mutable references to the same thing

[tight salmon]

doesnt.....don't, double negatives brain hurt 🤕

[runic shadow]

Sorry, the compiler flag is noalias so my brain always has to work in "don't alias" speak

[tight salmon]

If you access on two seperate threads?

[runic shadow]

It's very conditional as to whether it's UB

In rust just holding two mutable references to the same thing is UB

[tight salmon]

Right, but its for this very reason no?

[runic shadow]

Yes, that's one of the reasons

But it also tells the compiler that mutable references won't alias, so it can make optimisations based on that assumption

So whenever you do it it could potentially be UB, in c/c++ it's only UB under more specific conditions

[rose pendant]

If the code that's evaluating your queries doesn't run on multiple threads though, or the code doesn't modify the mut refs, you can't run into issues

[echo ore]

The aliasing issue only really matters for mutable data, so read-only queries can always provide the correct results. In addition, queries which join within a single (known-acyclic) relationship type will also not alias. So we only really need a good workaround for queries which access data mutably through non-acyclic joins

Not "never happens", but still a pretty particular use-case

[runic shadow]

~~Determining ~~ Enforcing acyclic-ness is pretty hard though, this would be an easy way to find it only when it matters and not just crash

[rose pendant]

As soon as you start allowing for people monkeying queries together in a web UI you'll see stuff that's a lot wilder than mut aliasing :p

[echo ore]

A few pages up I propose how we can enforce acyclic relationships (where the relationship wants that of course)

[rose pendant]

A lot of effort went into the flecs query engine/DSL to prevent cases where the query engine would just plain lock up because the number of evaluated possibilities is huge

[echo ore]

Yeah I've done my fair share of SQL horrors haha

[rose pendant]

It's also not just for acyclic relationships. You can have aliasing in a query without cycles

[somber girder]

it'd be pretty weird for query results to differ based on whether you iterate mutably or not, it'd be best if it was either consistent or be signified as different somehow

[rose pendant]

Position, Position(foo)

E.g. find all entities with Position, return Position for entity foo

One of the entities returned for the first term will be foo

[runic shadow]

My only problem with that is it seems super expensive to have running in release mode, but it is possible

[tight salmon]

Is each term (, separated) fetched/searched sequentially in order?

[echo ore]

That's an existing issue that Bevy resolves by determining if two queries overlap, which is currently blocked at the system parameter level. I guess I'm moreso talking about what you can squeeze into a single query (in the Bevy sense of the term)

[rose pendant]

Not necessarily. The query engine can reorder terms to improve performance

[rose pendant]

This is a single query

[echo ore]

Could be made very cheap by keeping an ancestor cache, which might be useful anyway

[polar vortex]

every term has variable or fixed source, if we make those explicit, the query looks like this

Position($this), Position(foo)

one of the results will have $this == foo

[runic shadow]

Would that help for creating new hierarchies though? Populating the cache the first time isn't fun.
I haven't really thought deeply about it though so maybe it would be fine, don't want to derail too much.

[echo ore]

You'll have to forgive me I don't have any personal experience with Flecs! When I say Query I mean what you can currently cram into a Bevy Query<D, F>'s D and F parameters.

[runic shadow]

Well you can't currently do any of this, but this is equivalent to a single bevy query

[rose pendant]

With bevy's current queries you can't run into these aliasing issues

[runic shadow]

(although since it's so simple you can obviously split it up)

[rose pendant]

I'm assuming a future iteration of the bevy query engine that's more flecs like

(in that it supports multi source queries, variables etc)

[tight salmon]

Super derail, but did we figure out how to keep system parallelization along with needing Query Builders for Systems?

[echo ore]

It definitely could be expensive, especially if you add a parent to the top of a hierarchy. My thinking is we add in the hooks/etc. required to enforce the invariants, but feature gate them so users can opt in/out

[polar vortex]

did that need figuring out?

[runic shadow]

If it's optional we couldn't rely on it for safety though

For complex traversal queries I don't know how to iterate them in parallel without aliasing

It's a very tricky problem

So probably you just cant

[echo ore]

It's how integer overflow works in Rust so I don't see why not honestly

[tight salmon]

I mean, if you want a query to contain an explicit entity, for example maybe relations or the query in the example, you would probably need to be in system with access to other information to base that entity off of

[runic shadow]

There's a big gap between "without this feature you can alias in safe code" vs integers act different

[runic shadow]

You need to build the queries before you are in the system, otherwise it's pretty difficult

[tight salmon]

So like a setup state -> runtime state

[runic shadow]

You use the system builder and pass it the query builder, there's an API in bevy currently that lets you do that

[polar vortex]

Well, eventually, a system builder API should appear that you can use to build a system that just has a complex query from the get-go. I think it's feasible (but probably pretty expensive) to produce a fine-grained Access for it.

[echo ore]

Personally, I'd like to just put the invariant enforcement in and actually measure the performance impact. I don't believe it would be substantially worse than the other costs associated with adding components to a hierarchy (archetype moves, etc.)

Especially since the complexity is proportional to the depth of the hierarchy, and I'd like to see some indication of how deep a reasonable worse-case-scenario is.

100 ancestors? 1000? I imagine the editor's UI system would surface this value

[somber girder]

hmm, what exact approach are you referring to here again? I can't find your original message

[runic shadow]

Personally I don't like the idea of performance degrading with depth at all, but benchmarks are always the answer to performance questions anyway

[echo ore]

Traverse ancestors til root or you return to start when adding a relationship

[somber girder]

and this is to prevent duplicates?

[echo ore]

This prevents all cycles (an entity being its own parent/grandparent/etc.)

Being able to assert that means you can traverse the entirety of an acyclic relationship graph in parallel

Since it is known that every entity can only be reached via a single path

[somber girder]

is it only that, or are there more benefits?
I am struggling to see why this couldn't be done with a hashset, which wouldn't degrade in performance with scale, is there something I am missing?

[echo ore]

I believe that's the main benefit performance-wise. The other would be a correctness one, but that's specific to the semantics of whatever relationship (my Foo relationship must be acyclic because reasons)

[echo ore]

The hashset would need to be stored on all entities (a hashset of its ancestors) and whenever a parent relationship is added, you'd need to traverse all children to update their hashsets

This is potentially much more costly since you have to iterate all Children, whereas the up-traversal check only has to iterate Parents

Basically checking a chain vs checking a tree

[tight salmon]

If expecting the normal build pattern though of build parents, then build children, checking children would be cheaper then checking parents quantity wise no?

[rose pendant]

cycle detection is one of those problems that gets engineers stuck in analysis paralysis

the irony

[somber girder]

I haven't thought this to the end, so I am not sure, but why store a hashset on every entity? Could it work to maintain one for every known connected subgraph?
Assuming the "we want to maintain an acyclic invariant for this graph" during insertion, we would have access to more than just what is on the entity itself, no?

[tight salmon]

Query<Solution<AllMyProblems>>.await

[echo ore]

That is true, but if we're considering the worst case scenario, parent traversal scales with the depth of the hierarchy, whereas child updating scales with the entity count of the hierarchy, which can be way worse (ever parent has 5 children, depth of 5, you're looking at 5 to the 5 checks instead of...5)

[tight salmon]

Would you have to do the check anyways to know the subgraph?

[echo ore]

That might work! I'm certain there are caching techniques I'm unaware of which could make the checks even faster, but my naive would just be check parents, and that's already fairly cheap in my opinion

[somber girder]

To know the subgraph, you would basically be doing a few HashSet::contains calls on the existing subgraphs, which is not expensive, and certainly doesn't need you to iterate through anything

[echo ore]

This wouldn't work if you were moving an existing child from one part of the graph to another

At least I don't think so!

I'm not an expert, and I do believe there is some caching strategy

Just not sure what

[somber girder]

The way I am imagining it, you would do a contains for both the child being added and its parent for each subgraph.
"Moving a child" would consist of "removing" then "adding", which on their own aren't anything new.
The more complex situation would be a removal causing two newly disconnected subgraphs, I gotta think about that one a bit
(joining two subgraphs is just a union)

[echo ore]

That does make sense. All I know is, whatever technique we propose to speed it up has to be better than what I'm proposing, which scales with hierarchy depth. Not because I think it's the best, but just that I know it works, and it's very simple

[rough gyro]

Rounding up the current API – there's also the event propagation mechanism. That currently expects that:

1. There is a component on the entity which represents the relation
2. That component implements Traverse
   So any replacement thing needs to either rework that mechanism or have a component that can be used for that.

// We enable propagation by implementing `Event` manually (rather than using a derive) and specifying
// two important pieces of information:
impl Event for Attack {
    // 1. Which component we want to propagate along. In this case, we want to "bubble" (meaning propagate
    //    from child to parent) so we use the `Parent` component for propagation. The component supplied
    //    must implement the `Traversal` trait.
    type Traversal = &'static Parent;
    // 2. We can also choose whether or not this event will propagate by default when triggered. If this is
    //    false, it will only propagate following a call to `Trigger::propagate(true)`.
    const AUTO_PROPAGATE: bool = true;
}

[wary fjord]

It's actually pub trait Traversal: ReadOnlyQueryData, not just a plain component

So however we want to model grabbing relational data as query data should work

[rough gyro]

Right, okay, that's a good point

[rough gyro]

The problem of having connections between nodes and them possibly separating into subgraphs (or two separate components as in graph terms components) is called Dynamic Connectivity. https://en.wikipedia.org/wiki/Dynamic_connectivity . There is at least outils for Rust that implements algorithms like this: https://docs.rs/outils/latest/outils/ . I think this is currently the state of the art algorith, but not sure what outils implements: https://arxiv.org/abs/1708.03962

[alpine patrol]

If we pre-hash entity ids (which I think we do) then we can deploy a bloom filter to speed up ancestors membership check into basically a bit-and

In a correctly structured hierarchy we should never actually find an inclusion in the set of ancestors so we can optimize the shit out of that path, and flag it as hot for the compiler.

[echo ore]

I didn't know about bloom filters, really cool! Yeah that sounds entirely workable to me, with the added benefit that every entity stores a precomputed list of its ancestors, which may speed up some code

[alpine patrol]

precomputed ancestors list is one way to speed up transform propagation.

we get all entities with changed transforms, then use the ancestor lists to find find greatest common ancestors and start propagation from there, rather than always starting from roots, for both an in increase in parallelism and a reduction in the number of total nodes visited.

so that would be basically ideal for me

i've been thinking about doing it anyway tbh

[alpine patrol]

Btw, if someone works on this, it’s important that the set of ancestors (and potentially the bloom filter) for each entity contain the entity itself. Lets us do clever stuff with set intersections. So it’s not so much the set of ancestors as it is the set of entities which cannot be children (the anti-children).

Then if the intersection of the anti-children sets of two different entities is non empty then one is the ancestor of the other.

Ordering them as a list would also be helpful

[rough gyro]

Okay, time to get this show on the road. I wrote the first question on the minimal interface design document here: https://hackmd.io/getbapR5TTCNVuBlNju1Yw?both#Q1-Should-relation-changes-be-atomic

For this, I filled in the description myself, and also a suggestion, as I have a guess that this is pretty uncontentious and we'll just all agree. However, please don't let that discourage you if anyone feels different about this. Please comment and discuss if warranted. If you just want to agree, just toss some emoji on this comment and I'll change the suggestion to a decision after a few days if it looks clear to call it.

[rose pendant]

and there isn't a generally accepted consensus as to what they exactly are
** cough **

[rough gyro]

I'm hoping at least some stuff is clear, and starting from the easy end. We'll get to the difficult ones later 😉 But of course, I might be proved wrong here too...

[rose pendant]

Few other relevant documentation links:

• https://www.flecs.dev/flecs/md_docs_2Relationships.html
• https://www.flecs.dev/flecs/md_docs_2FlecsQueryLanguage.html
• https://ajmmertens.medium.com/building-games-in-ecs-with-entity-relationships-657275ba2c6c
• https://ajmmertens.medium.com/a-roadmap-to-entity-relationships-5b1d11ebb4eb

The linked doc is just the component traits manual which doesn't actually describe relationships itself

But ways in which you can tweak relationships

[rough gyro]

Yeah, I didn't gather any comprehensive list, I just stashed it there from this discussion so that I don't need to remember it elsewhere

And I have read all the mentioned pages at some point in time as well, but flecs has gone forward since the last time I looked at it

[viscid python]

It's not that there isn't a consensus as to what they are it's that there isn't a version of the consensus that looks like what it'd look like in bevy. The API desired is very like flecs except there's some things that have to be renamed because we use different concepts & some APIs totally altered because Rust & C++ have different metaprogramming capabilities. But the functionality from a theoretical standpoint I would say there is consensus on.

Additionally something that's missing is a clear roadmap. There have been work items identified & drafts made but they're kind of sparse & don't go beyond an MVP. Sander's "roadmap to entity relations" is more of a roadmap to make your own roadmap than a roadmap you can use for your engine. Bevy's got a lot of stuff to refactor in addition to the things it has to add.

[rose pendant]

Agree with all of that, still think that there is some consensus on what they are 😉

[viscid python]

Yeah I would say there's a strong consensus on that actually. Atleast among people familiar with the subject matter.

[rough gyro]

Oh man... Hooks only get deferred world, so they can't make immediate changes to the world. Sent this message about it: #ecs-dev message

That means that if in a relationship implementation, hooks were to drive consistency, then components can't be the storage for relations. A hook that runs when Parent is added cannot instantly add Children to the parent entity. It can only do so through commands which means there is a observable inconsistency. So if the Q1 decision suggestion stands, it has implications.

When I looked at the pull by @echo ore, I saw all the modifications happened in commands, but mentally just filed it under the common thing of adding wrappers for changes in Commands. But it kept nagging me in the back of my head a bit, so went and checked in the end.

[echo ore]

So at first I was concerned about this, but I'm less so now. In the current hierarchy implementation, you can only add a Parent (or Children) if you have full world access (since it potentially moves archetypes, etc.). So you're either in an exclusive system (commands will be applied after it runs, including the hooks), or you're mutating the hierarchy via commands anyway (and thus the new hook commands will also be applied during the same deferred application)

The one instance where it can have an observable delay is when you're editing an entity that already has a Parent/children, since you could update it without the archetype move, and thus without exclusive access to the world

[rough gyro]

Yeah, I don't think it's a terribly big problem either – but if we decide it's okay to have this small mismatch at times, then we can't have the requirement that everything is always atomic. We must relax that requirement.

[rough gyro]

Okay, with all the discussion on the other side, I'm now convinced that: 1) hooks and observers need to use commands to add/remove components, 2) commands added from hooks and observers should be automatically flushed in correct depth-first order. There are currently bugs why it doesn't act like this, but I think we can assume those to be fixed fast. So that means the relationship implementation can use hooks and/or observers to enforce consistency in a way that is "atomic". So Q1 decision doesn't force our hand in the implementation method.

[polar vortex]

By "consistency", do you mean an invariant like "acyclic" or the relationship's cleanup policy? If you meant cleanup policies ("deleting parent deletes all of its (grand)children"), IMO we shouldn't occupy the hook slot ourselves.

The ECS should have a built-in routine that is automatically invoked when entities are deleted.

To cleanup efficiently when an entity is to be deleted, we'd want to figure out what archetypes will be affected and then move their entities in bulk (and emit OnRemove).

Also, for relationships like ChildOf whose cleanup policy is to delete all entities in possession when the target gets deleted, we probably need to emit OnRemove and delete archetypes/entities from the "bottom" up (from the leaves up to the entity being deleted).

[alpine patrol]

Can you expand on this? With relations, will recursive despawn become the default behavior?

[viscid python]

For hierarchy probably

Also yeah flecs doesn't use hooks or observers for relation edge cleanup

[alpine patrol]

Why not retain the current despawn_recursive command and have it just remove the components by default?

[viscid python]

Because it's a default that never made sense

[alpine patrol]

I do hope we can keep the scope of the relations changes minimal

[viscid python]

The only reason it wasn't made the default behaviour was because it would be difficult to opt out of. That changes with relations because you can define your own relations.

[alpine patrol]

I see. I still suspect transform propagation being tied to a specific relation will complicate that

I’m not sure if it will really be all that easy to opt out of in some specific cases.

we should probably evaluate that change separately, I’m not opposed to it.

[viscid python]

Transform propogation would use up traversals which means

• You don't need Transform components on every entiry
• You can easily "propogate" arbitrary shit with up traversals

[alpine patrol]

Say more

[viscid python]

What's missing?

[alpine patrol]

How it works? Why don’t I need transform components on every entity?

[viscid python]

Because up traversals allow finding the first ancestor with the component instead

[alpine patrol]

Ah, gotcha. You’ll still need to use parenting for the default transform propagation to work through, no?

[viscid python]

You'd have to use something

[alpine patrol]

Which locks you in to that. It feels like deletion propagation should be a totally unrelated relation.

[viscid python]

It doesn't. Up traversals are a very simple query to write.

[alpine patrol]

Transform propagation may not be

[cloud plover]

I think that the existing despawn_recursive design is bad, and opting out of it should be explicit. The reason it exists is because of module boundaries: bevy_ecs isn't hierarchy-aware

[viscid python]

What about it would be difficult? Relation queries would remove any unsafe.

[alpine patrol]

Mm, makes sense. I’m not opposed to this change, I’m concerned about making semantics changes during the switch to relations, and making migration even more complex.

[cloud plover]

Yeah, that's fair 🙂 Might make sense as a two stage thing, in either direction

[alpine patrol]

Maybe it won’t be then, that would be great!

[viscid python]

It's not as big a change as you think because I guarantee in 90% of cases where hierarchy is involved despawns are already done with despawn_recursive

[alpine patrol]

I’m thinking about my work codebase and I think it’s might be more like 60 to 70, which is why I bring it up. I’ll check and see, maybe it’s less common than I suspect.

[cloud plover]

Fascinating; I've never intentionally done despawn (unrecursive) on a parent and wanted the "please detach my children but keep them alive"

[alpine patrol]

Maybe I am misremembering, it’s possible. I’m dead sure I’ve used it.

But generally you can just detach first and it’s not a huge hassle, I guess.

[rose pendant]

Would also be interested in use cases for this

[polar vortex]

for child of it should, but it's a cleanup policy that any relationship could have, child of isn't a special-case

[polar vortex]

when it comes to deleting the target of a relationship, you can either

• remove the relationship from all entities that have it
• delete all the entities that have it
• throw some kind of error (warn, panic, etc.)

if components (and relation kinds) are entities, then there will also be a policy for deleting the component/relation itself

child of is one that would delete the children upon deleting the parent, but many other relations would just remove the relationship (remove would be the default policy)

[rough gyro]

I mean in general using hooks to implement relations. If the hooks were not "atomic" (in the sense that the world isn't observable in an inconsistent state), then any implementation would have to be fully baked in to the engine - and at the moment it isn't, at all, bevy_hierarchy is separate from bevy_ecs. How components are actually used and which components have hooks etc. isn't set in stone in this API design.

As for cleanup policies, that's a complex topic indeed that goes more into the advanced design pretty fast. Not sure how far we want to design that part for the minimal relations that could replace the current implementation of Parent/Children.

[rough gyro]

• Reparent, as in when a parent is despawned, move all children to grandparent.

FWIW, I've always hated Parent/Children as a hierarchy, because it doesn't actually mean anything. Parent/Children for Transform/GlobalTransform is obvious – but that's just TransformParent. Then UI has nesting of elements which is also Parent/Children but it's not exactly the same, it's more a tree of elements like DOM. And then every game can use the same thing for something custom to their own. And the cleanup policies for these possibly differ – so debating what should be the default cleanup policy for Parent/Children doesn't really make sense in my opinion. I'd be happier if we can get relationships into a state where making all these relations distinct is easy, so then the same relation won't be overloaded with independent things. Then it will be a much simpler discussion to discuss what's the right cleanup policy for UI hierarchy specifically, for example.

[viscid python]

As was already mentioned flecs does not use hooks for cleanup. The simple ones will be baked into the engine & anything more complex is up to the user to implement. That is adequate. Something baked that is good is better than something you can customize every aspect of that is crap.

[rough gyro]

Sure, but is it a requirement for the next iteration of bevy_hierarchy that it must not use hooks? We are aiming for minimum requirements here, not implementation design for the complete relations stack

[viscid python]

Reparent, as in when a parent is despawned, move all children to grandparent.
I don't know where this was originally from but in my entire year making aery back when it was still my uni project this was the one cleanup type that

• I never found a concrete use for
• Was the thing that created the most headaches

When your cleanup creates new edges thats when things get messy and complex. It makes cleanup indeterministic because it then matters what edges you cleanup first. It made the cleanup algorithm extremely slow & complex. That's not the case if all your cleanup (the ones we provide) are only destructive. That's why when I was prepping it for release the only cleanup I provided for aery was:

• Nothing: Just remove edges (no dangling)
• Counted: Edge counted cleanup (despawn a parent if all its children are despawned)
• Recursive: Recursively cleans up (despawn all children of a parent with the parent)
• Total: Both counted & recursive

[viscid python]

If hierarchy uses it then this is also the mechanism for other relations.

MVP probably won't have cleanup.

[rough gyro]

I am expecting that the next step for hierarchy is built on components and hooks. And the step after that is fragmenting relations. Fragmenting relations will be built in to the ECS, but the next step will just be a more consistent parent/children.

[viscid python]

Those can be treated as orthogonal

[rough gyro]

Every thing in my world has Environment which means the general conditions that surround that part. For example a habitat. If the habitat gets despawned, all it contains would be directly exposed to the habitats environment, hence reparented. They don't suddenly vanish.

[viscid python]

Is this a mechanic from your game?

[rough gyro]

Yup, but it's heavily unfinished, so more a planned mechanic.

So can't tell if it really works to build it like that

But environment mechanics certainly exist - environment dampens radiation coming from outside, for example

[viscid python]

In either case I don't think it would be something you should be using hierarchy for & this falls outside of "simple cleanup we can provide".

So if you did want that it would fall on you to implement.

The provided cleanup shouldn't try service everything just the most common things.

[rough gyro]

Yeah, but you wanted an example where reparent is a useful cleanup mechanism, so this is one.

[viscid python]

Yeah but the complexity it introduces makes all cleanup worse. For that reason alone I don't think it's worth it. The scope of the provided cleanup types should be simple. Also this is something you can probably use hooks for. You would have a despawn hook for Enviornment that just checks if it has descendants & ancestors & makes the ancestor adopt the orphans.

OnDelete($e), Enviornment($e), ChildOf($e, $p), ChildOf($c, $e) :- ChildOf($c, $p)

[rough gyro]

I'm not arguing it should or shouldn't be included in standard cleanups, I just wanted to say that it exists and has use cases. What should be included as standard is a separate discussion.

[viscid python]

There's probably use cases but as demonstrated it's easier to make work when it's not the responsibility of edge types like the example I just gave which instead makes it the responsibility of Enviornment

[polar vortex]

I mean in general using hooks to implement relations.

As for cleanup policies, that's a complex topic indeed that goes into the advanced design pretty fast.

🤔

Using hooks/observers to "implement relationships" seems like a detour to me.

Since relationships will just be components, I don't think adding or removing them involves anything new (yet).

But when the target entity is deleted, the world is left in an invalid state. Since we're cutting off the version numbers to squeeze the indices of two entities into one component ID, the ECS itself has to maintain an invariant that relationships only exist between still-living entities.

Otherwise, because entity indices are recycled, a user could spawn a new entity with the same index and cause leftover relationships to appear "valid" again, when they aren't.

This WG can tackle it later if its priority is to just get something out, but cleanup is ultimately necessary because the world is left in an invalid state without it.

The hook/observer infrastructure was needed for queries. Because relationships will lead to way more archetypes as well as archetypes being deleted in potentially large quantities, iterating a list of all archetypes created/deleted since the query last ran was not going to scale well. Better for queries to receive alerts when likely-matching archetypes are created and deleted.

I may be a little behind, but IIUC, the remaining work to be done was:

• Use the component index for the initial construction of queries.
• Use observers to keep queries caches up-to-date.
• Change how archetypes and edges between archetypes are stored so that archetypes can be deleted efficiently.
• Implement the cleanup policy routine.

• Reparent, as in when a parent is despawned, move all children to grandparent.

🤔 I think that would be too context specific. The policies I listed would be valid regardless of properties like exclusivity, symmetry, etc. Nothing added or replaced, just removed.

Debating what should be the default cleanup policy for Parent/Children doesn't really make sense in my opinion.

Is there an engine / editor program in popular use where deleting a node doesn't also delete the subtree underneath it? That's the kind of nesting child of is meant to express AFAIK. I don't think there is much to debate.

[rough gyro]

Using hooks/observers to fix current Parent/Children implementation is something that I thought was already agreed that it should be done. It's bug https://github.com/bevyengine/bevy/issues/12235, and it was scheduled for 0.15 and now postponed to 0.16. Currently the situation is that there are bugs in Parent/Children like 15575, manipulating either of those components without using the blessed API will break consistency and despawning requires always explicitly using despawn recursive or otherwise again things can break.

I really don't think all fixes to the current state of things shouldn't be gated behind a full relations implementation. @cloud plover can you chime in here please?

[cloud plover]

I didn't have full consensus for that proposal, to be fair

But I would like to avoid blocking incremental improvements indefinitely 😦

It is tech debt, but so is the existing bespoke hierarchy impl

[polar vortex]

Oh, if it's something to fix a bug, then alright. Like I said, I'm pretty behind on all the conversations.

[cloud plover]

Yeah, the existing API for Parent/Children is really easy to misuse

I think that we should be able to remove despawn_recursive now-ish using hooks

Which should make the migration easier

[polar vortex]

I see now. Sure, for existing Parent/Children impl, it makes sense.

[rough gyro]

So, the plan, in my opinion would be:

1. Agree on API and guarantees for Parent/Children that is future-proof
2. Reimplement Parent/Children with this API so that we can fix all the bugs
3. Implement minimal fragmenting relations

[cloud plover]

Yep, that seems ideal to me

[rough gyro]

Personally, I'd really love to also see: 2.5. Implement generic minimal relations without them being fragmenting (ignoring performance for now) and expose them as experimental feature to users

[cloud plover]

I'm also okay with fixes that don't conform to the ideal API, so long as they mostly don't break the current API

[polar vortex]

Didn't we retire the "fragmenting" and "non-fragmenting" terminology?

[rough gyro]

What's the new terminology?

[polar vortex]

Hmm, nothing really, just relationships.

Putting both the relation and target in the component ID means different pairs will be different components and result in different archetypes.

It's just a given at this point.

I think "fragmenting" carries a somewhat negative connotation while suggesting that there's a viable alternative, which IMO isn't the case.

[rough gyro]

Do you mean that the only type of relationships Bevy would support are relationships where each relationship target gets their own archetype? If so, that limits the usefulness of relationships quite a lot.

[viscid python]

It doesn't it makes them vastly more useful because queries don't have to access component data to get edge information

[polar vortex]

Not quite. Bevy could make whether or not zero-sized components—which most relationships would presumably be—fragment tables an optional property.

I don't at all see how it limits their usefulness considering the only implementation of relationships has worked that way for years.

What's disadvantageous about being able to directly find all entities with a specific reactionship pair?

[viscid python]

The thing everyone is ever concerned about whenever they mention fragmentation isn't archetype fragmentation it's table fragmentation which can be eliminated by reaching further parity with flecs which iirc does this with sparse components.

It's not "fragmenting" vs "non-fragmenting" it's archetypal vs component

[rough gyro]

Okay, let's assume there's a relationship which is for a Train, and there's NextCar and the inverse of that, if named PreviousCar. In such a relationship, the NextCar entity is different for every entity. This means that there's exactly one entity per archetype. This means that even though there's only a very small amount of systems that care about train car ordering, every system will be slowed down by the fact that entities are no longer cache local but instead allocated separately each one.

[viscid python]

and the inverse of that, if named PreviousCar
You don't need this. Relational queries can go in both directions.

[rough gyro]

Okay, right, so if table fragmenting is separate from archetype fragmenting, then I don't really mind. But Bevy must deal with the fact that there's as many archetypes as there are entities.

[cloud plover]

Yep, that's the broad plan

[rough gyro]

Yesyes, that was just an example that if it were named, like it is in Parent and Children

[viscid python]

Children only exists as a hack to make hierarchy viable

[rough gyro]

It's part of the API

Sorry, gotta go for now, bbl

[viscid python]

Like without it you cannot find all entities that have (Parent, e). Which is not a problem for archetypal relations because:

queries don't have to access component data to get edge information

[polar vortex]

Table fragmenting can be addressed its own. That's kind of another perk of relationships just being ordinary components. If we do an optimization for zero-sized components, then everything that counts as one would benefit.

And in the first place, have to put the content in the ID for them to be zero-sized.

[viscid python]

These discussions are also always sooooo loaded with assumptions. If it makes sense for you to model a train like this in the first place you're likely trying to leverage the self introspective capabilities that come about from the ECS getting these prolog like features. Meaning each of your cars likely has other properties with more relations your game inspects. The likeliness of them being in the same archetype was low to begin with. There would have been no benefit from the edges being in components either because if you need resolver capabilities resolvers with the component approach are going to be slow af.

Linked list situations aside the fragmentation you get with hierarchies (which will probably be far more common) is perfect because all of an entities children are in the same table.

[polar vortex]

I think it's totally fair to just be immediately skeptical after hearing "more tables with fewer entities".

But that probably wouldn't penalize query iteration that much. Not every system is matching hordes of entities to begin with.

AFAIK the primary cost of table fragmentation is that table moves are pretty expensive, so ideally we do them less often.

Reducing table fragmentation could in certain situations make iteration "worse" if a query iterates its results archetype-by-archetype instead of table-by-table. It'd be unlikely that an archetype's entities are contiguous within the table.

We can probably avoid that most of the time though.

[viscid python]

I mean I was skeptical as well but after you think about it more you realize it's not as bad as you think & in many cases it's actually even better. Like the random access you get from table jumps when iterating queries is already what we have with current hierarchy! People just overlook it because "they're in the same archetype".

[polar vortex]

Yeah, I've heard that the heaviest part of transform propagation is performing the traversal? Having the relationship expressed in its ID would enable optimizations like sorting a query's cached results by hierarchical depth (or similar).

[alpine patrol]

Traversal is expensive, but the big issue currently is that we have a hard bound on parallelization (per tree) and visit things we don’t need to (visiting all nodes from root even if there are no changes).

Memory coherence is not the primary issue

Cached toposort would be very nice as well though

[polar vortex]

In any case, the "have the relation and target in the ID" approach is the most capable AFAIK.

It enables queries to perform traversal efficiently and without access conflicts.

Building queries and performing cleanup are straightforward because we can directly lookup all archetypes that have a specific pair (or a specific part of a pair).

Similarly, reducing fragmentation in this scheme is straightforward (configurable even) compared to adding it in some other scheme. (Fragmentation can sometimes be beneficial, like with transform propagation, assuming there's no cached topological sort coming from something else.)

[rose pendant]

Perf overhead does show up in some other less obvious places, like when you frequently instantiate (prefab) hierarchies

That ends up creating/deleting tables, which needs to notify observers, which has to update query caches

That + improvements in query iteration speed is why I'm working on a new hierarchy storage that's dedicated to that use case

[rose pendant]

Linked lists are not a great use case for relationships. You'd be better off having a NextCar component with an entity handle

In the latest flecs version those fields are also queryable (though not indexed, so slower)

[obsidian bough]

Coming back in after being out for a short while, whats the status on component hooks based relationships?

Nevermind, I checked the github. Seems that got cancelled. Shame

[echo ore]

There was a need to further design the target API for relationships first which my PR had skipped over. @rough gyro has started working on a design document (I believe pinned) which should hopefully allow for someone to make either a follow-up in a similar vein, or just help design true relationships

[rough gyro]

Sorry, I've been busy with work so have left this linger for a while, but getting back to it again

@polar vortex There are at least three ways to implement relations:

1. Entity as value inside components
2. Graph as a Res
3. Relations in archetypes
   I bet I can find use cases for each of these where they outperform the others. This "archetypal" relationship method is not only possible one by far and when we are discussing alternatives or implementing things, we need names for these things. If you don't want to call it fragmenting relationships, I'm totally fine with that, but it needs a name to distinguish it from the alternatives. Also, I don't think Bevy users need any other name except "relations" or "relationships", as the storage of those is an implementation detail – but us here certainly need a name for it.

[rough gyro]

See, that's the thing. These "linked list" relations and other kind of relations need the exact same features as the rest. There needs to be a way to traverse them in the opposite direction (automatically indexing or inverting the relation) and that needs to be kept consistent so there's never a mismatch. And often they should somehow also react to despawn logic. And that is at least as important of a use case as parent/child relations, if not more import as there's going to be a lot more of those and usually just one meaningful hierarchical relation.

They can be called whatever, but then we need a new word for relations that are of this type.

[viscid python]

You have not stumbled into new information. These have been known for years. 1 & 2 are essentially the sameing thing. Resources are just a special kind of component.

I bet I can find use cases for each of these where they outperform the others.
Sure but we only care about the one that serves the most users & is good enough for the biggest majority of use cases & that's the archetypal version. It has a big leg up because queries do not need to access data to perform operations. The resource version is particularly bad for a lot of use cases because all of the information is in a single component which creates a lot of API friction. That is essentially what any plugin that provides a "context" is doing. Eg. the egui integration.

[polar vortex]

We actually had a similar conversation a little over a year ago.
#1123375029070082179 message
I think bevy needs the direct archetype look-ups that each (relation, target) pair being a different component provides, so I don't really have any personal interest in other methods.

Beyond than that, this has been argued over for at least two years. To my knowledge, none of the alternatives reach the same ease of cleanup and search as each relationship having a unique component ID. The work this WG has done so far is helping us achieve that implementation.

AFAICT "each relationship has a unique component ID" puts us in the best possible position.

I'm not saying that's perfect by itself. We may want to incorporate a different representation in special circumstances.

For example, if I had to guess, @rose pendant's upcoming optimization for traversing child of during transform propagation sounds like it'll internally substitute ChildOf(parent) with Placeholder(depth) and use a graph data structure to remember which entities are related. (edit: 🤔 Either that or entities will have both pairs to fragment on both parent and depth.) The end result being that archetypes group entities by their depth in the hierarchy, making it easy for a query to sort its cache on depth.

Even that still relies on the relationship having a unique component ID.

Like, based on my own analysis, everything points to "each relationship pair should be a different component". flecs being capable of what it is and @viscid python having made aery and then being more in favor of what flecs did afterwards just strengthens my confidence in this.

[viscid python]

To add to this when you try to do everything you'll end up with something that can do nothing or everything badly. We should be doing as much as possible & archetypal relations let us do the most.

[viscid python]

Also I want to reiterate this point. A lot of the analysis around approaches that store the edges in data instead of archetypes to prevent fragmentation are shallow & fall apart when you think beyond the first thought.

[polar vortex]

I look at it more like "being consistent will pay off."

Just like entities are the centerpiece of functionality (i.e. why making things entities is almost always beneficial), archetypes are the centerpiece of optimization.

The main cost bevy's ECS is setup to minimize is the cost of finding entities that have a certain set of components.

Queries are setup to find and remember a set of archetypes. If we treat that as a design philosophy, it feels quite natural to say that different pairs should be different components.

That way relationships capitalize on what's already been built.

But it'll still be possible to "skirt the law" in certain circumstances to boost performance.

e.g. Transform has a characteristic access pattern—random access sprinkled here and there followed by a linear, all-encompassing propagation pass—so opting to fragment archetypes on depth instead of (🤔 or maybe in addition to) the exact parent would make sense.

[rose pendant]

They can still be called relations, but the current way that relationships are stored doesn't lend itself for linked list style relationships. So if you wanted to go hard on those you can, but you'd likely have to implement a dedicated storage for it

Though there are special cases where even linked list style relationships can work well with the current impl, if you're careful about how you use it it can be performant

[rough gyro]

Agreed. My only point of contention was that we shouldn't say "oh just put an entity into a component", as that doesn't give the inverse relationship. I don't think this needs to be extremely performant, but the automatic consistency needs to be there in the user API.

[rough gyro]

I know. But to be clear, I'm not trying to advocate for a specific implementation - but just that we need words for these things so we can discuss them when needed, for example in the migration plan. So until someone tells me a better name than fragmenting relations, that's what I'm calling the implementation method where each relation target gets their own archetype.

[rough gyro]

1 and 2 are quite different. Components for entities are stored in a linear table, so smallvec can store some amount of entity ids inline. A shared resource like petgraph would store them as adjacency list which is a linear list of edges where the edges form a doubly linked list, for example. So resources allow for vastly different storages where as entities in components is bound to what components can do.

And yeah, I know none of this is novel. And to reiterate - I'm not trying to mess up your plans on relations. But we need to have a common language and to be able to discuss these without reductionist arguments on only fragmenting relations existing.

[viscid python]

"Archetypal relations"

And the only thing you can make a migration plan for is hierarchy

Everything else will be too use case specific & will be up to the user to port

[rough gyro]

I'm fine with Archetypal relations. The only downside is that it might be mistaken to mean the archetype graph instead, but that's easy to clarify. And I think that's a fine name also because it specifically says relations are in archetypes, but doesn't say that Tables are fragmented, so that can be a separate thing.

[rough gyro]

And is "Table fragmenting" a good way to refer to Tables being split per entity target?

[polar vortex]

Yeah, I think it's OK to call it fragmentation. That's what it is.

It's just... tables being fragmented by relationships will be a consequence of tables being fragmented by zero-sized components (which no longer needs to happen now that OnAdd observers exist), not relationships themselves. Like, it'll be exacerbating an existing design issue, not creating a new one.

I think I dislike the term "fragmenting relations" because I can't see the word "fragmentation" being viewed in a neutral light.

Like, the point of archetypes is fragmentation, but I feel like the average person who hears "fragmentation" will first process it as a bad thing (e.g. hard drive fragmentation) before they get the rest of the context.

IMO a specific relationship being a distinct component is the most natural expression of the bunch, so I don't want users to think it was an "Option B" kind of idea.

Even without archetypes, the data-oriented aspect of ECS encourages users to divide a game's state into components whose granularity matches how the data is queried and used by systems.

[rough gyro]

I have no such issues viewing fragmentation as negative. However, what about Split Tables? Almost as correct, and no negative connotations as such.

Or! Partitioned tables? Databases call the same thing partitioning often. Partitioned per parent, concretely in this case?

[polar vortex]

I think it's best to say "table fragmentation" if you want to talk about it. Partitioning has already been used to describe something else that fits it much better. I just have a dislike for the term "fragmenting relations".

If you must distinguish it from alternatives (even though IMO there is no serious contender), "archetypal relations" or "'relations as identifiers' vs 'relations as data'" feel better to me.

[rose pendant]

iirc it was the bevy community that came up with the term when people were still against it :p

[polar vortex]

right, kinda why I'd like to avoid it now

But anyway, if someone was explaining relations to an unfamiliar user, I'm more concerned with how table fragmentation would be linked to relationships. I don't want the unfamiliar user to come away thinking "relationships are bad because table fragmentation".

because it's not true

If the largest majority of relationships will be zero-sized, then the table fragmentation will be a consequence of bevy starting out lacking observers and choosing to track changes for all components.

Now that we have observers, Bevy could (and should) decide to exclude zero-sized components (which can't change) from having change ticks. Bevy could additionally decide to drop "added" ticks entirely (which would just cement the reason to exclude zero-sized components).

That would eliminate table fragmention for zero-sized components, thus from zero-sized relationships as well.

Likewise, if we consider non-exclusive relationships that hold data, those must necessarily occupy different table columns.

So that leaves exclusive relationships with data. Technically those could occupy the same table columns independent of target, but that's probably easier said than done.

[viscid python]

The problem with "fragmentation" is it over emphasizes tables being split. They're not being split we're creating new groups for the entities. Bevy users also by default imagine edges top down instead of bottom up. So they'd focus on Children & immediately think of the worst case scenario. Instead of Parent which can group all siblings together.

[rose pendant]

Doable as long as you can guarantee that the type is the same for all pairs regardless of the target

[viscid python]

It's also not as beneficial as you think

[polar vortex]

yeah, this is more how I'd want users to see it, "you can group entities together by parent", "the ECS can find all entities with the same parent is as quickly as it can find all entities that have a T", etc

not "it's a shame those perfectly good archetypes/tables had to be split up"

[polar vortex]

I wasn't trying to say it is/isn't valuable. I wanted to emphasize that relationships mostly won't be a new source of table fragmentation.

Zero-sized relationships only fragment tables if zero-sized components fragment tables.

Non-exclusive, non-zero-sized relationships must fragment tables because 1 entity gets 1 row in a table. (edit: Unless we'd want to try consolidating entities with the same number of the relationship into the same tables. The "assuming same data type" condition would then apply.)

Exclusive, non-zero-sized relationships are the only ones where we could point out that different pairs could share a table column... assuming that the different pairs still hold the same data type.

[rose pendant]

You could also call them indexed relationships, because the component index indexes which archetypes have which pairs

which speeds up queries

[polar vortex]

hmm, I like that

[rose pendant]

Indexing is the main reason why the relationship storage has been designed this way

[polar vortex]

well, I like that it puts focus on the main utility

yeah

[opal torrent]

Users can already avoid fragmenting tables by defining components with StorageType::SparseSet, right? (Or am I getting the terminology confused?) What is the advantage of making zero-sized components non-fragmenting that users can't already get from sparse storage?

[wary fjord]

ZST components shouldn't need to have space allocated and should just set some bits in a bitset, but we currently do

[viscid python]

I just read this. Fragmentation is still useful even when edges don't have data. And also for non-relational uses of compoments. Zst optimizations & fragmentation are orthogonal.

[viscid python]

Columns aren't allocated if that's what you're referring to. That's just thanks to rust but there's still optimizations bevy could do.

[polar vortex]

That's right. The advantage is people shouldn't have to think about it? Zero-sized components have nothing to store... except change ticks, but ZSTs can't change, and users have OnAdd to detect their addition.

[polar vortex]

Archetypes don't have to share tables, but they might as well share if they can. It's a trade. The shared table will have rows from different archetypes jumbled together (so iteration is less than ideal) but entity's components won't need to be copied if you just change a ZST.

[viscid python]

but they might as well share if they can
Nah this would make for a bad default imo

[polar vortex]

if apps trend toward giving entities more components on average, moving them will cost more. I think not having to move the rest of an entity's components each time you add/remove a marker will generally pay off more than perfect dense iteration.

but idk for sure

like, I have the vague impression that as apps scale up, structural changes will become more common, while if queries increase in complexity / number of terms, they'll match fewer entities

so I think it makes sense for archetypes that only differ on sparse or marker components to share tables by default

it could be configurable like storage type, but I do think sharing is the better default

[echo ore]

Seems reasonable to me. Perhaps an archetype move could be delayed until there's enough entities to justify the move? Since at that point, you would start getting the benefits of dense storage

[polar vortex]

idk what you mean by delay an archetype move. An entity can only belong to one archetype at any time and each archetype is linked to one table.

A move is only "delayed" when, like, commands are sitting in the queue. Once the entity's set of components changes, it has to change archetype immediately.

But if the entity can remain in the same table, then moving archetypes is cheap (swap remove an Entity value out of the old one and append it to the new one).

[viscid python]

This is only the case for components that get changed frequently. Majority of data in an app will be read only most of the time.

[echo ore]

Sorry might be a half-baked thought. I believe table is the term I was looking for. If there's only, say, 4 entities with a particular component, then it makes sense to keep them in their normal tables and store those components sparsely. Once there's enough entities (some arbitrary threshold), then they can be moved to the more specific table that densely stores that component

[polar vortex]

I get the impression doing that could over-complicate the implementation. Right now, storage type is solely determined by the component. If it could depend on the entity, that'd have to be recorded somewhere that could survive the entity moving between archetypes (and then cleared when finally added to new table).

[viscid python]

Also majority of mutations aren't structural mutations.

[echo ore]

I mean it could be stored in the sparse table itself yeah? Since that's effectively a list of the entities that are pending a move to a dense table.

(In my proposal, not currently)

[polar vortex]

what is a sparse table?

[echo ore]

Sorry wrong term, the sparse storage

[polar vortex]

what I meant was, how would you know if an entity's component value is stored in the table where it ought it be or if it is still temporarily stored in sparse storage?

[echo ore]

Naive answer? A new storage type that's an enum with the variants like SparseTemporary and DensePermanent

[polar vortex]

Wouldn't that entail adding a set of vectors to every archetype (one per SparseTemporary component present in the archetype) to track this for each entity in the archetype?

If an entity moves from one archetype to another, we'd have to copy those bits. Like, it's kinda reintroducing the copying issue that motivated sharing tables just with less data to copy.

Since this is all very hypothetical to us atm, I don't think it's worth a deep dive without measuring how not fragmenting on marker components would affect performance.

We're just guessing how much we think the density of same-archetype entities within tables is worth rn.

[echo ore]

Yeah it's all just guesswork I agree, and I'm very unfamiliar with the way the ECS stores data internally. Guess the point is if the fragmentation is an issue there are things we could look into

[polar vortex]

hmm, should I not assume users would change the relationships between entities more often than they'd add/remove other components?

if "big" apps have more components attached to their entities, then adding/removing a relationship would mean all the ones that have data need to be copied into a new table

the cost of transferring an entity's data to its new table scales with the number of components the entity has and the number of intermediate moves (which is why command batching is important)

[viscid python]

It's usecase specific. If they're using relations to do things like making a ui hierarchy or model features on maps in a game they'd change pretty infrequently. If they're instead trying to do something like baldur's gate 3 & make their game self introspective then they will change a lot.

[viscid python]

Thing is apps (even big apps) don't spend most of their time being big.

I think it'd be fine as an opt in just as sparse storage is opt in atm.

[polar vortex]

I'm all for letting the user decide, but I'm not convinced "markers fragment tables" should remain the default behavior.

The per-frame impact of "time spent moving an entity" seems like it'd be a bottleneck more often than "time spent on additional cache misses" when iterating queries (just the ones that iterate archetypes, not the ones that would iterate tables directly).

but I'm just guessing

[viscid python]

It depends on the "per frame" thing actually being something that happens a lot per frame. That Baldurs Gate 3 example is the most frequent one I could think of but even then it's turn based & the amount of changes that happen every frame probably wouldn't cause a bottleneck.

[polar vortex]

yeah basically, it doesn't have to be present every frame, but like the upper bound on the time one frame can take

one frame occasionally taking too long can turn into "game is a stuttering mess" reviews

[viscid python]

that's an easier problem to diagnose anyway cause you can see "hmm frame's dipping when I do this" vs trying to diagnose poor dense iteration

[cloud plover]

I'm generally in favor of keeping performance costs localized for this reason 🙂

[polar vortex]

@rose pendant, what's been flecs' default behavior when it comes to tags and (dense) table fragmentation?

By default, would archetypes that only have different tags share a table or not? Is there a "trait" the user can add to the component to opt in or out?

(if it is configurable, when are users encouraged to change it?)

[viscid python]

Also is this behavior the same for zst relations like ChildOf?

[rose pendant]

I don't do this yet, but plan is to make tags non-fragmenting by default, and have a trait that allows you to opt-in into fragmenting

ChildOf would be an example of a tag/relationship that'd be fragmenting

I’m building the prefab storage refactor first though. One disadvantage of the table/archetype split is that you can still get a lot of archetypes even though components are dense

In bigger apps there can be a significant amount of overhead when creating archetypes, because it needs to notify observers, update query caches etc. which takes longer the bigger the app gets

So I’m prioritizing something that can drastically reduce table creation first

I’m also wondering whether the table/archetype spit by itself is enough. If you have a game where you both want to make heavy usage of relationships for game logic as well as dense iteration for perf critical systems, you wouldn’t want one to impact the other

Relationships for game logic benefit from fragmentation, perf critical systems (like transform etc) don’t

[unborn quail]

Looking for reviews for https://github.com/bevyengine/bevy/pull/16591

trying to adopt https://github.com/bevyengine/bevy/pull/14385 afterwards

[runic shadow]

Only taken a quick look so far, one minor note is I think many_components_and_systems needs to be public.
I ran it a few times and it was a little noisy (swinging +-10% each way) but my setup is far from ideal.
I do like the idea of having a benchmark of a highly populated world, and it does that well, once we start breaking things down in a flamegraph/tracy it should help give a clearer picture of how things look under load. The benchmark does have a lot going on though so it doesn't seem like it would cleanly show a before and after impact of most changes and there's possibly room for a more targeted benchmark for archetype access specifically. For particularly invasive changes it seems like a good sanity check to ensure there's no massive performance black hole for a more "realistic" app that isn't seen in more artificial benchmarks.

[unborn quail]

How do you think such a targeted benchmark would look like? I suppose you could just generate a whole bunch of accesses and compare them for conflicts

[cloud plover]

Yeah, if needed we can cheat and use more internal APIs

[unborn quail]

Hey @wary fjord, are you willing to help me out with queries as entities?

and maybe also quickly review https://github.com/bevyengine/bevy/pull/16591, if you're so kind? It only takes 2 minutes

[wary fjord]

sure, anything specific?

[unborn quail]

mainly just advice

like many dumb questions

replacing QueryState with Entity in Query is a pain

because of all the lifetime shenanigenry

[wary fjord]

Yea sure, I'll write up how I would approach it

[unborn quail]

Like, I've already divided it up for me in the following way:

• Make QueryState a component and have an Entity reference in Query (one PR)
• use observers to add / remove archetypes (different PR)

the first step is a big one, the second one I should be able to do

[wary fjord]

I think you don't need to change the layout of Query at all to do queries as entities, actually.
In my mind, you just need to change what the lifetime 's refers to in things that return queries, instead making them tied to the world's lifetime ('w).

So, to put that into code the SystemParam impl would look like:

unsafe impl<D: QueryData + 'static, F: QueryFilter + 'static> SystemParam for Query<'_, '_, D, F> {
    type Item<'w, 's> = Query<'w, 'w, D, F>; // <--- instead of Query<'w, 's, D, F>
    // ...
}

Since we're moving the QueryState into the world, we would of course change the SystemParam's State to Entity

unsafe impl<D: QueryData + 'static, F: QueryFilter + 'static> SystemParam for Query<'_, '_, D, F> {
    type State = Entity;
    // ...
}

When you're implementing init_state, you would still create the state and initialize it, but you would insert it into the world and store the Entity as the state.
And then when you're implementing get_param, you simply use the stored Entity to grab a reference to the QueryState, which gives you a &'w QueryState resulting in a Query<'w, 'w, D, F>.

The problem obviously arises then that you can't implement new_archetype for Query, so you'll actually immediately need the observer event at this point.
How its implemented beyond here is beyond what I trialed, but I believe this is a much simpler way of going about it.

That will save you a lot of work mucking around with changing Query's functions

[unborn quail]

that sounds good

I'll have to look at the original code in order to see how it should be changed

[wary fjord]

One thing you might notice is that we don't have row-access contraints, only column-access contraints, so we can't technically prevent anyone from accessing the QueryState in the world currently, but we can make it really annoying to by using a non-pub Component to store the QueryState for an MVP

[runic shadow]

Yeah, the QueryState needs to be hidden/immutable

[unborn quail]

yeah, I'm not worried about that as much

like, if you're accessing QueryState, you get what's coming to you

[tight salmon]

I think Joy suggested Arcing it, so that multiple systems could share the same QueryState

[runic shadow]

If you are only taking an immutable reference you don't even need to arc it to share state

It just means you also can't do in place archetype updates

You could still arc it if you didn't want to do the component lookup each time I suppose

[tight salmon]

Hmm wonder what the overhead for adding row access would be, besides for extra memory it would only be checked after matching column access. or vice-versa whichever is faster i suppose

[polar vortex]

the thing to keep in mind about QueryState is that it's only mutated when updating its cache

when we run a system, that system never needs exclusive access to the query state, it's just iterating or fetching the results

if cached queries have to be entities then QueryState doesn't need to expose any &mut self methods in its public API

boxing/arcing the state of a query or system avoids overhead from having to look up the entity and do the "hokey-pokey"

you don't want schedules wasting time looking up systems every time they run, you want them to cache pointers. same goes for systems. you don't want systems wasting time looking up their queries every time they run

likewise, because running systems doesn't mutate the query state, many systems could share queries, which would reduce the amount of observers needed

[cloud plover]

I'm chewing on the ordered children problem a bit more

From my understanding, flecs style relations are not ordered

Which is sensible, and a good default for most cases

But without a stable order that can be manipulated, our UI layout will fall apart

And we may struggle with very noisy diffs during serialization

The latter is a bit easier: we just need a stable sort strategy

[alpine patrol]

simply add ordering relations between relations /s

the relations themselves end up in archetype tables, right? why can't we use table order?

[cloud plover]

That seems reasonable to me too, but I'm suspicious since @rose pendant gave me the impression that this is a very hard problem

[runic shadow]

Generally you want the table/archetype to have ids in an order that makes them easy to compare and search i.e. monotonically increasing, hence by default not supporting ordering. Plus it would probably be complicated to re-order children with an implementation like that. It's a very interesting problem that's hard to produce an efficient implementation for, would be interested in Sander's thoughts (I believe I've heard him speak on it before and perhaps how it would likely be easier using the implementation he's putting together for the prefab storage refactor), I've also spent a lot of time stewing on it.

[cloud plover]

Are there any other use cases where we really care about the ordering of children? @tall mountain @twin tide does animation rely on this?

[tall mountain]

Don't think so 🙂

[left summit]

nop for animation

[tall mountain]

Animation doesn't even "see" parent-child relationships directly for the most part; the hierarchy is just established "in advance" and then becomes irrelevant until transform propagation iirc

[alpine patrol]

There was a thought that child order could be used for the (as yet still hypothetical) 2d transforms.

But that could be worked around

[tall mountain]

for automatically managing z-index or something?

[rose pendant]

Depends on what you mean by ordering. My take on it:

• BFS order: that's easy with relationships since you can easily compute a depth for each archetype
• DFS order: bit harder but still relatively easy
• Ordered by component values: bit harder, still doable, but very expensive
• Declaration order (as in a UI template): easy for trivial use cases, hard for complex UIs

[alpine patrol]

ye

[rose pendant]

he's putting together for the prefab storage refactor
Yeah working on that atm

[cloud plover]

Declaration order is what we need here

[alpine patrol]

not strictly insertion though, right? we still want to be able to insert ui nodes between other nodes during runtime.

[rose pendant]

The key things I'm trying to solve for is to differentiate between different sections of:

• static entity lists (I know exactly which entities will be generated)
• conditional lists (which entities are generated depends on some condition that I need to track)
• dynamic lists (the number of entities I need to generate depends on some value that I need to track)

A template could define itself as something like

<static entities>
<conditional entity>
<dynamic list of entities>
<static entities>

[cloud plover]

Yep

[rose pendant]

That's an example of a dynamic list

[cloud plover]

I'd be tempted to solve this with placeholder ghost nodes TBH

cc @cosmic elk

[rose pendant]

I'm taking a different approach that's more integrated with the data structure that stores the template AST

[alpine patrol]

it seems to me that none of these model what we want. we want the left relations from any given entity to have an ordering which is not defined based on component values but held within the ecs itself, with an api to re-order and manipulate stuff.

[rose pendant]

The last one is what you want (declaration order)

And for drawing you probably want a reverse BFS

[cosmic elk]

I'm going to propose a simple solution that no one will like: don't use relations for children.

[cloud plover]

I think there's a high chance that MVP relation doesn't get used for children intially

But we will definitely be overwhelmed with complaints if we don't swap eventually

[alpine patrol]

⚔️-Controversial! if relations can't model the current semantics of children (the current main relation-like thing) then i will be left with serious doubts about their general applicability.

[cosmic elk]

I have plenty of use cases for relations other than children

[tall mountain]

That's fair, but the main reason that other things aren't modeled with relations is because they don't exist

[rose pendant]

Whether you use the feature or not is kind of irrelevant for how the problem should be solved. You still need a per-parent list of children that lets you do efficient incrementalization

[tall mountain]

(and plenty would require less invariant management than the parent/child hierarchy)

[runic shadow]

People store Entity/Vec<Entity> in their components for all sorts of reasons, and if it was more ergonomic to do queries across those entities people would avoid it less

[cosmic elk]

I quite liked @echo ore 's re-implementation of children that avoided a lot of the book-keeping, using observers and hooks to manage the lists

[alpine patrol]

we are working on shipping (some) of that

perhaps the right way to do this is with an entity-id linked list.

each entity stores it's "first child" separately, and each child stores the next sibling.

[unborn quail]

finally, we found a use for a linked list! /s

[rose pendant]

nope

I mean you can do it like that. It's just not very efficient

[rose pendant]

I think this'll be easier to scale ^

Basically store the three different kinds of lists in different vectors that are separate from each other

So you can grow one without affecting the others

[viscid python]

This is like 1 use case... if anything this usecase is the niche one. Fragmenting relations also aren't mutually exclusive to entity lists. Everything else that uses hierarchy except parts of bevy_ui that need ordering could still be moved to relations.

[rose pendant]

You can then annotate your template AST with things like "this scope is going to update dynamic section 3"

[alpine patrol]

just to float another silly ⚔️ idea, could we use two different storages for ordered and unordered relations (like sparse sets vs archetypes)? The entity list is kind of like a sparse set.

so we could keep exactly the current semantics, but access it through a consistent api along with unordered relations?

[rose pendant]

Doesn't work because you need ordering per parent, and sparse sets aren't fragmented on parent

[runic shadow]

You could have a different underlying implementation for a given class of relations though

[alpine patrol]

that's what i'm saying, we could keep them as entity lists instead of putting them in tables.

[rose pendant]

fwiw this is roughly how frameworks like leptos work

[viscid python]

I still think a cached sort at the query level that doesn't reorder tables could work here? Even if it loses a lot of the gains of fragmenting relations it would still be about as efficient as it's currently done & with a nicer API.

[runic shadow]

Yeah, the easiest way to avoid this problem is to fall back to a simple and inefficient ordered implementation

[cloud plover]

Yeah, that seems fine 🙂

[unborn quail]

we're gonna get relations specialisation before actual specialisation

[cloud plover]

Yeah, I think that's my preference, especially if we can paper over it in the API

[alpine patrol]

or actual relations

[unborn quail]

I have mentioned this before, but the ratio of discussing relations vs implementing them is a bit lobsided imo

[runic shadow]

So is the barrier of time and expertise between the two

Very true though

[unborn quail]

I'm not going to be the discussion police

[viscid python]

cause it's easier to get nerd sniped than it is to sit down & do either the complex or mundane work to implement them

[unborn quail]

but sometimes I'm like, hey, maybe leave a review

[alpine patrol]

well according to the tag this group is still in the needs-design-doc stage.

[unborn quail]

not that I have anything up for review currently

but y'know it's the idea

[alpine patrol]

maybe if we had a doc, people would be able to find the work to do more easily

i for one have no idea how i would contribute to this initiative

[rose pendant]

My 2 cts would be to loop in whatever BSN is doing for templates/UI since that's basically the front end for this. That should tease out the different use cases/perf issues

[unborn quail]

I mean you can greenlight https://github.com/bevyengine/rfcs/pull/79 today

[alpine patrol]

sweet i'll git it a shot

[unborn quail]

this is what I've been working from for the past months

[runic shadow]

There was an attempt to flesh out API requirements here: https://hackmd.io/getbapR5TTCNVuBlNju1Yw, but I think it ran out of steam

[wary fjord]

I just have more personal impetus for things more immediately helpful than sitting down and researching relations stuff
Who knows, maybe I'll get through my list and then want to start tackling it lol

[runic shadow]

Yeah one of the problems is there's a lot of very mundane refactor work that needs to be done, Bird! and perwirink have done a good job at tackling a few good chunks of it

Retained render world was a huge piece of work that was also partially relations motivated

[viscid python]

It got scope creeped really quickly. There's stuff on there that I've said before is well beyond an "mvp".

[unborn quail]

(primarily)

[unborn quail]

all performance improvements are purely bonus

[runic shadow]

Yeah that's why the original RFC touches on such little API, even to implement that requires eliminating all the technical barriers

I sat down and spent some time trying to get to components as entities last weekend, but it's more work than I expected

Mostly due to the remaining sparse sets (on component ids)

[unborn quail]

my current hope is that I can take away enough 'mundane' refactors so that some better developers come back for the big impact stuff

(that means you @runic shadow, I don't feel like being subtle)

[runic shadow]

Personal gripe but I hate the way that bevy constantly passes around parts of the world as seperate references. I had to spend like 30+ minutes just passing &mut Entities through all the component registration code.

My main conclusion after that was we probably need to pursue: https://github.com/bevyengine/bevy/pull/14928 first

Just to stop memory usage exploding on high component ids

[unborn quail]

that one is on my list to adopt and merge

but I'm doing the access bitset removal first

[runic shadow]

That's also a really important one yeah

[unborn quail]

memory really does explode, with the new stress test I noticed that having only 20K entities and 2K components takes up a good 8 GB of ram easily

and every entity has at most 10 components

I ran out of memory multiple times in testing

[runic shadow]

That does seem pretty terrible actually.

[rose pendant]

What does the stresstest do?

Randomize components?

[runic shadow]

Bird can correct me as they're the author but it adds a random set of components to a bunch of entities and creates a bunch of systems that perform busy work on a random selection of the components.

[unborn quail]

yes, it's an entropy generator

[rose pendant]

I wonder how much ram that'd take up with me 🤔 probably also not insignificant

You'll end up generating a lot of archetypes if you add components one by one

[unborn quail]

https://github.com/bevyengine/bevy/pull/16591

here's the link

code should be fairly readable

[runic shadow]

There's definitely a very interesting archetype graph being generated in there

That's probably one of the least realistic parts of the test given that most real applications always add the same components together and in the same sort of order

[viscid python]

Flecs has archetype gc tho

[unborn quail]

I considered making it 'more realistic' but the problem is that there are many many many many ways in which realistic apps are very different from what I can randomly generate

[runic shadow]

Yeah, it's not something that I think is easily rectified, still feels like 8GB is a lot though

[unborn quail]

anything I could add that isn't just running a snapshot of an existing game, would be unrealistic, so I elected to keep it simple

[rose pendant]

Yeah, you could keep RAM in check by calling table cleanup while creating

[viscid python]

Which like "realistically" will happen in sync points in apps. I think it would be quite unlikely for an app to spawn everything it needs in 1 deferred point.

[rose pendant]

Not sure how representative it is, but just did a quick test with 20.000 entities/10 randomized components. Takes up 9MB

Creates 1024 archetypes

[unborn quail]

yeah, but the 10 components come from a pool of 2000 components

[wary fjord]

do we have any tools to measure where we're taking up a lot of memory?

[rose pendant]

Ah ok, let me try that

[unborn quail]

(it'll be a lot lot worse)

[rose pendant]

437 MB, 181484 tables

[alpine patrol]

valgrind would be my go-to

[unborn quail]

laughable

as in, it's laughably bad how bevy performs

[wary fjord]

If we're seriously taking 8GB of RAM, we should investigate where

[unborn quail]

I'm trying to reproduce it

wait a minute

[wary fjord]

also how big are the components

[runic shadow]

u8

[viscid python]

inb4 loop in wrong place

[rose pendant]

Repeating that I'm not sure how representative this is. I'm just creating the entities, doing nothing with systems/queries

[runic shadow]

Like it's not thaaaaat bad in comparison, we already know a bunch of data we are storing that flecs isn't

And I've read a lot of flecs code, there are no bytes being spared in there

Given that I haven't seen any major recent attempts to get memory usage in the ECS down, not an unachievable place to start (assuming it's even close to representative)

[rose pendant]

(the reverse is also true though)

[runic shadow]

Very fair

[rose pendant]

My component index records are pretty chunky

[runic shadow]

But I'm thinking of access sets specifically just being a bunch of cruft

[viscid python]

In my own project anyway the more I use the ECS (ie. the more components & entities there are) the less data each component has.

[rose pendant]

(ignoring for a sec that last week I stripped 8MB from world creation from bad usage of a sparse set)

[wary fjord]

we should do memory usage benchmarks at some point, we have plenty of speed ones (not to say we shouldn't add more speed ones as well)

[runic shadow]

Haha yeah I did see that PR, but I've been recently going over the archetype graph and component index code specifically and kept running into a lot of clever use of spare bits

[echo ore]

I'm hoping the no_std-ification will allow for some embedded users to advocate for memory optimisation options. Obviously everyone benefits from it, but embedded is disproportionately effected by it

[rose pendant]

Graph edges are also larger than I'd like them to be. If I had to guess that's where most of the memory is going

Haven't come up with good ideas to make them smaller though, beyond maybe compressing pointers

[unborn quail]

I've ran the test. This is for 50 000 entities and 1000 components

better part of 10-12 gigs

[rose pendant]

50k entities is 1GB for me (with 2000 randomized components)

[unborn quail]

I'll do 20 k entities and 2000 components next

it will crash

[rose pendant]

50k works but 20k crashes? 🤔

[unborn quail]

oh nvm, it doesn't

[echo ore]

20k with double the randomised components I believe

[rose pendant]

oh 50k/1000

[unborn quail]

this is actually slightly better

I'm gonna drop the number of systems down to 1, see if that does anything

[echo ore]

Sounds like it's a per-entity overhead. Have you tried with only 1 or 2 randomised components?

[unborn quail]

okay, yeah it has nothing to do with systems

barely a dent

[rose pendant]

50k/2000 components with empty table cleanup is 160 MB

[unborn quail]

I'll try with 10 components next

yeah, it seems like we're leaving some memory on the table

[rose pendant]

Only 2 orders of magnitude

[unborn quail]

if we're hitting several gigs over mb

[viscid python]

it's likely N choose K complexity (I forget the expression for that N!/(K!(N-K)!))

[rose pendant]

== without table cleanup
20k/2000 components: 436 MB
50k/2000 components: 1 GB

== with table cleanup
20k/2000 components: 65 MB
50k/2000 components: 160 MB

(table cleanup is called after each created entity)

[unborn quail]

yeah, I'm running with 10 components and 1 mil entities and the memory is barely noticable (don't see it on the gigs graph) so probalby closer to the megabytes

[rose pendant]

Yup, that's to be expected

The components/entities themselves barely take up any storage

10m * a byte is 10 MB

[unborn quail]

meaning that we scale poorly with the number of components in the world

which is expected, but good to verify

[runic shadow]

Yeah, proves the value of the test

[unborn quail]

is is very funny that the fps drops through the floor as all the systems can no longer run parallel to each other as they are all conflicting

[runic shadow]

I'm not surprised we have a huge amount of memory caught up in metadata, but it would be good to start cataloging where most of it is spent

[unborn quail]

I don't know how I would go about finding that out

but that'd be a good idea

[rose pendant]

That sounds like a problem for relationships

[unborn quail]

hence the test

[rose pendant]

Yeah, but I mean that sounds like a much bigger problem than RAM usage

RAM usage is probably relatively easy to fix

[unborn quail]

we are talking about 800 systems all accessing the same 10 components

so it's not exactly suprising

[rose pendant]

Intuitively I'd expect that if you have more archetypes you get less contention 🤔

[echo ore]

Perhaps using tracing to log the lifetimes of some of the likely culprits?

[viscid python]

yeah but only if you have the second part of a pair (R, *) which you won't for all queries

[twin tide]

it doesn't care

[rose pendant]

Why is that?

[unborn quail]

yes and with the current test we see that

very few archetypes -> more contention

[rose pendant]

I mean for this specific case, which doesn't have any pairs

[viscid python]

It's the way bevy systems disambiguate access conflicts. It goes off type/query term information & it needs more runtime borrow checking for this not to be the case.

[rose pendant]

Ahh gotcha. You have 10 total components, not 10 added components

Ok that makes more sense

[unborn quail]

that's right

[rose pendant]

Though still that means you have 1024 archetypes, which is probably more than average for bevy

[unborn quail]

the whole thing is that we scale terribly with 1000 total components

the number of components added to every entity is between 1 and 10

[rose pendant]

Right, you flip a coin 10 times, 2^10 = 1024

[unborn quail]

yes

and we flip an additional coin for each component (on the system side) to see if it's mutable or not

[rose pendant]

I guess the number of systems is what does it here

[unborn quail]

yup

800 is many

especially if none of them are paralizable

[rose pendant]

Let me try that

How much FPS are you getting out of that?

[rose pendant]

Are those frame times?

[unborn quail]

50K entities, 10 total components, 800 systems

nope

fps

[rose pendant]

oh lol

[unborn quail]

frames per second

I will say that the entropy machine I implemented is not the nicest thing to run

[rose pendant]

what if you use a single threaded executor?

[unborn quail]

wait a second

have to actually go into the code

not much better, but I'm not running the best benchmark here

actually a factor worse

but I have a bunch of stuff open, so it's hard to say

[rose pendant]

What are the systems doing?

[viscid python]

it rebuilds the schedule graph each frame right? I'd say that's the problem more than the executor

[unborn quail]

it sems to be mostly using a single thread

[viscid python]

the flamegraph would be interesting to look at if it's not a total pain to setup on your system

[unborn quail]

Given an entity (c1, c2, c3, c4, ..., cn), we calculate: (1^1 + 2^1 + 3^1 + ... + c1^1) + (1^2 + 2^2 + 3^2 + ... + c2^2) + ... + (1^n + 2^n + ... + cn^n)

in modulo 255 arithmetic

it's a generalization of triangle numbers

and we assign the resulting number to all mutuable components in the query

(I wasn't lying when I called this an entropy machine)

[rose pendant]

With 1M entities, 10 randomized components, 800 simple systems that just increase each component every frame I'm running at ~2400 FPS single threaded

[unborn quail]

impressive

I'm quickly running a flamegraph

but than I'll go to bed

[rose pendant]

Wait, those systems were filtering on 3 components so not iterating all entities

When I iterate all entities I'm running at ~210 FPS

[unborn quail]

hmm

[rose pendant]

I doubt whether enabling multithreading is going to make a difference here 🤔 probably will make it slower

Yeah no difference

[unborn quail]

Took a bit (had to install perf)

[runic shadow]

Seems like roughly a quarter is in the executor with the majority of that being access check related

[unborn quail]

yup

which means that cart was right when he said that this specific part of Access<T> is performance sensitive

so that's good to know

(I'm multithreading again btw)

[rose pendant]

Would contention show up in the flamegraph?

How can updating a few bits take up that much time 🤔

[unborn quail]

you do it very often

[runic shadow]

And with 21000 components there would be a lot of bits

[unborn quail]

and the minute detail that access is currently a bitvector

with 1000 components, that's a lot of bits

[rose pendant]

Yea but I'm updating 1 million components and still running at 210 FPS vs. 0.1 FPS

Maybe I don't understand how the algorithm works, but that seems excessive

[unborn quail]

I'm using the base parameters again: e = 50k, components (total) = 1000 and systems = 800

and I'm getting 90-100 fps

[rose pendant]

~1000 bits x ~1000 archetypes right?

[rose pendant]

(that makes more sense)

[unborn quail]

access vectors are stored on a per query basis, so assuming 1 query per system it's 1000 bits x 800 = 0.8 mil bits

but since every access stores about 5 bitvectors you end up around 4 million

[runic shadow]

Well it would be per system for this comparison right? Same difference with one query per system

[unborn quail]

yes

I believe it's one query per system, but because I don't know the internals completely there might be some secret queries hanging around

[rose pendant]

What's the difference with the 0.1 FPS benchmark? Didn't that also have 50k entities, 1000 components, 800 systems?

[unborn quail]

no the 0.1 benchmark was a 50k entities, 10 total components and 800 systems

[rose pendant]

Ah right right

Non intuitive that more components means better performance but makes sense

[timid mountain]

How much RAM usage do you get with the bitset version? (Or was it already with the current code in main?)

[unborn quail]

Current code in main

[slate shoal]

It's actually more bits because the scheduler has a bit per component per archetype

[runic shadow]

Is the scheduler still doing archetype component access checks? Would be interested in seeing how much capacity for parallelisation we would lose with just component checks.

[cloud plover]

Yes

[cloud plover]

If we swapped to this we could precompute an order / allowed parallelism

[opal torrent]

There's an intermediate option of using the filtered access sets. That gets you parallelism for things like With<T> and Without<T> without the size scaling with the archetype count. The checks would be more expensive, though.

[cloud plover]

We should also consider running the access checks ahead of time, rather than last second before the systems are dispatched

And special-casing an early branch for "nothing is running so don't bother running a check"

[polar vortex]

In a theoretical sense or in currently running bevy apps?

archetypes >>> components, so coarsening the granularity of the "locks" from "these T buffers" to "all T buffers" is a objectively throwing away a lot of potential parallelism.

But who knows if a schedule contains parallelism that can be realized or if it saves significant execution time? The ideal cases are like, you get to run 2+ lengthy systems at the same time.

But width in a flamegraph is not execution time, right? AFAIK bitset intersection operations should be cheap (it's entirely sequential scans of integer arrays + SIMD instructions)

🤔 just a lot of them I suppose

[runic shadow]

Yeah I mean in practical scenarios how much is gained from that parallelism, how many systems in real apps are doing significant work in the same part of the schedule on the same components but in different archetypes

I imagine there are some generic systems that we have a bunch of versions of all running on similar components but slightly different archetypes based on some generic type argument.

[polar vortex]

Yeah, I can't think of any sequences that can happen in parallel off the top of my head. I do think the odds would increase if users add a bunch of related plugins and those all assign their systems to the same sets.

we could collect samples and extrapolate, but we'd never be able to make a decision with total confidence

fwiw, I'd personally be OK with not having (or not frontlining) parallel system execution, especially if the CPU-side of rendering reduces to "upload data to GPU"
and if a user is writing their own systems, the easiest gains are probably gonna be parallel query iteration or manually-submitted async jobs

parallel system execution has some other downsides, like not (really) supporting stepping

[timid mountain]

Could it be an option left to the user?

[polar vortex]

yeah, but I think it's more of a UX consideration

specifying dependencies is unnecessarily verbose if the systems are just going to run as a sequence, insertion order would be much more concise

[rose pendant]

last time I argued that it was a very controversial thing to say, have things changed since then?

explicit over implicit etc

[cloud plover]

No I still disagree strongly with Joy here :p

[rose pendant]

kk :p

[cloud plover]

.chain is great though; very glad we have it

[polar vortex]

hmm, I may have gone off-topic with that

users already get a schedule-specific choice between serial/parallel execution, so ig a more appropriate question is which should be default

how a schedule arranges systems when no dependencies are specified is a separate thing

but my opinion on that is "if parallel system execution goes away or gets heavily sidelined, specifying before/after dependencies seems unnecessarily verbose"

[alpine patrol]

(why would parallel system execution go away ?!)

[rose pendant]

To that point though, a focus for me atm is to remove as many of the idiosyncrasies of ECS as possible. IMO switching to an ECS is ideally just a change in how data is organized. It should come with as few as possible additional limitations. I don't see it going mainstream otherwise

Those things include allowing for structural changes while iterating, seeing changes immediately, and also things like scheduling annotations

I'm currently working in a large org with hundreds of people from different backgrounds, and I estimate that about 5% are knowledgable about ECS

Any barrier to entry is guaranteed to cause annoyance

</offtopic>

[polar vortex]

I'm speaking hypothetically. There's no need to get rid of it, but say it's more common for scheduling to actually degrade performance. Then nobody will use it, then why keep it around?

Similarly, say its ideal beneficiary (the renderer) moves work out of systems and into shaders to be mostly/entirely GPU-driven.

I just think the UX cost of before/after everywhere is only justified by parallel execution, since there's basically no performance difference between serially executing in insertion order and in topological order.

In those circumstances, I think it'd make sense to normalize not specifying dependencies at all.

IMO

but yeah, in hindsight I don't think this particular point goes with the earlier discussion about the granularity of access checks so my bad

[viscid python]

Bevy defaults to trying to let users get some parallelism even if it's unideal (which it often is because it just goes off access which is not an indicator of if something is a good candidate for parallelism). I think that's a bad default because a good default should make things users want to do frequently easier. Sequential operations are far more frequent than parallel ones & adding dependency labels to everything complicates this.

[rose pendant]

(and apparently in some cases means you'll be running at 0.1 FPS instead of 200 FPS)

(for reasons that are less than intuitive)

[viscid python]

Also because of this default it's used parallelism as an unproven performance crutch which has left a lot of things unoptimized

[noble panther]

I don't really get "the UX cost of before/after everywhere". Every instance where I use them is for ordering relative to a system set somewhere, not forcing specific systems to run sequentially, so I don't see how insertion order would be relevant there. If I add a bunch of systems at once and want them to run sequentially, I just chain them

I am somewhat skeptical of Bevy's defaults re: parallelizing systems though, most systems tend to be small (and keeping them small is encouraged) so the benefit is somewhat dubious. Last I tested, the parallel executor just made perf worse

but I'd want representative benchmarks to really judge

[wary fjord]

There's probably knobs we could add to oportunistically batch groups of systems to run in parallel rather than single systems

[noble panther]

Could we just have an API to explicitly specify groups of systems to run in parallel (if possible)? I almost never want individual systems to run in parallel

[alpine patrol]

It may be hard to benchmark this, but it’s not hard to see that basically everyone uses the parallel scheduler currently. I suppose that might change in the future if it got worse, but I would be against any change that introduces degradation.

[wary fjord]

tbh what I want most for concurrency is just rayon or something as powerful. parallel query iteration is nice but its only 1 of the things I'd like to do in parallel, and quite limited in what it can do (i.e. just a simple for_each)

[rose pendant]

and want them to run sequentially, I just chain them
this is a bit harder to do across plugin boundaries

[alpine patrol]

Work stealing would be nice. But removing core default functionality seems bad, even if it’s backed up by benchmarks, which I am skeptical of.

[noble panther]

Yeah in those cases you need to use before/after (not against systems though, I try to use system sets if I'm ordering against something in a different plugin or module). What's the alternative to specifying the dependency though? You need them to run in a specific order, but you can't really rely on insertion order since the systems are added in different plugins

[polar vortex]

I didn't mean anything fancy by it. Just that it's more to type. The alternative in my example is add_systems basically having an implicit chain.

"do things in declaration order" appears elsewhere (UI), so it's not crazy crazy

[noble panther]

Yeah personally I think I'd be fine with making add_systems chain by default and having parallelization be the opt-in version, e.g. (a, b, c).multi_threaded(). But again, needs representative benchmarks

[rose pendant]

the parallel executor just made perf worse
Yep, basically my point from a while back. I (still) haven't seen conclusive evidence that the executor improves performance consistently. I have seen conclusive evidence that there are a few degenerate cases where its performance absolutely craters (@unborn quail's benchmarks from yesterday)

seems bad, even if it’s backed up by benchmarks
This is what I'm really having a difficult time understanding 😅 literally the only reason you'd want a parallel executor is to improve performance. If benchmarks show that it doesn't do that and it gets brushed off, there's something wrong with how decisions are made IMO

[rose pendant]

In my case it's phases (basically system buckets) + a mechanism that lets you specify dependencies between plugins

[alpine patrol]

I am arguing not to remove a core piece of our execution model even if it inefficient on some benchmarks. We may have people who rely on parallel execution or whose behavior is significantly different from the benchmarks. They will not appreciate getting the rug pulled out from under them.

Change the default, sure. But remove it?

[rose pendant]

Sure, but understand that you can use that argument (there may be use cases ... but we don't know) to block any progress

[alpine patrol]

I don’t think this is an implementation detail, we’re talking about one of bevy’s core properties, for which there would be essentially no migration.

[rose pendant]

There is this a priori assumption that systems are a good primitive for task schedulers. This has never actually been established

[polar vortex]

the nice thing is that how a schedule orders systems when no dependency information is provided is an independent variable (like, it's not coupled to another decision)

in the "systems are entities" world, those can be two different relationships

like, we'd attach a relationship like (RunsUnder, <schedule>) to indicate that the system will run as part of the schedule

and attach 0+ (<schedule>, <dependency>) "relationships" that adjust its position in that schedule, if there are none, we could choose to fallback on insertion order

where <schedule> and <dependency> are currently living entities

[viscid python]

If anything I would argue these decisions should be made now while it's still before 1.0

[alpine patrol]

I agree; I think the decision we should make is not to do it.

[noble panther]

I don't think people are necessarily arguing about removing parallel scheduling entirely, just changing the default, given sufficient benchmarks (or at least I am)

[polar vortex]

I didn't mean to equate allowing systems to be implicitly ordered with removing parallel execution. I just wanted to say that the emphasis on specifying dependencies derives from parallel execution.

[viscid python]

Bad defaults aside I also think the parallelism model is deeply flawed

[polar vortex]

so if parallel execution ever takes a backseat (i.e. stops being default), I think users will be like "why do I have to type all that?"

[rose pendant]

Why do you think that though 🤔 There is little data that supports being so sure about it

[polar vortex]

I agree systems might not be the right unit of parallelism since game logic is so sequential, but the model itself is a common way for databases to execute transactions in parallel (but the base speeds are like an order of magnitude slower because they're writing changes back to disk, so it actually helps)

[rose pendant]

the base speeds are like an order of magnitude slower
this is the thing that makes all the difference

[noble panther]

I am arguing not to remove a core piece of our execution model even if it inefficient on some benchmarks.
Playing the devils advocate, I don't see why we would have to keep parallel execution as the default even if it is more efficient on some benchmarks 😛

(I have not even seen any such benchmarks in representative scenarios, but have seen scenarios where the parallel executor does noticeably worse)

[alpine patrol]

Frankly because I don’t think synthetic benchmarking would be enough here. I would want us to be hearing this from commercial teams using and deploying games and apps using bevy. To my knowledge we are not hearing that.

[unborn quail]

The parallel scheduler is one of those capabillities which is nigh impossible for a third party to recreate if we drop it, which is why I think it's unpopular to drop it.
It's one of those things which might be usefull and given Bevy's promise to be most things to most people I don't think it's gonna happen. I do foresee the default changing though.

Ideally we'd want this scheduling to perhaps be done by a third party crate, but that's not possible with the current architecture

[rose pendant]

Yea but usually you default to the simplest solution unless you have data that supports a more complex solution. Not the other way around

[alpine patrol]

I think commercial teams care a lot about performance and are cognizant of the opportunities afforded to them. If not, we should ask them and see what they come back with. Withholding a positive response from them, there’s no way I could (personally) consider removing a good choice.

[noble panther]

Does Tiny Glade use Bevy's scheduler? Would be interesting to hear if they have data on this

[unborn quail]

I'd love to strip out the complexity that the scheduler brings, but there needs to be a viable alternative for teams that do want that capabillity

[rose pendant]

If you were to apply that decision process to every breaking change in Bevy the project would deadlock instantly

[alpine patrol]

I would not try to apply this to anything less weighty. Am I not allowed to say that some changes are more significant than others?

[unborn quail]

So I'm not saying it can't / shouldn't be done, I think it should, there's just the question of how

[rose pendant]

^
@alpine patrol how many datapoints would you need to see for you to change your mind?

you're never going to hear back from all commercial developers, so in absence of that is there anything that would convince you otherwise?

[polar vortex]

I just wanna make it clear that I was never seriously arguing for ripping out the ability to execute systems in parallel. I was just using that as an example to say that specifying dependencies would seem weird in a world without it.

I'm more of the mind to just reduce potential parallelism—by coarsening access or just making serial execution default.

Separately, if we make serial execution default and tell people to exercise their own judgment about using the parallel executor, then I think Bevy should allow schedules to fall back on insertion order in the absence of dependencies.

(I actually think it'd be nice no matter what's default, but definitely if serial is.)

Also, from the preceding discussion, are the access bitsets really too big?

10,000 archetype components would be around a kilobyte

[wary fjord]

SystemMeta is like 320 bytes I think, excluding heap-allocated

[polar vortex]

ig there are multiple bitsets per Access and multiple Access instances per system

but I don't think an app would reach a gigabyte that way

🤔 did we stop using bitsets?

I kinda remember that being discussed months ago

[runic shadow]

Definitely still using bitsets, there's an open PR for transitioning way from it

They are only going to be one part of the memory usage, I imagine the archetype graph is also quite big, hard to say blindly.

[polar vortex]

system params can have individual command queues too, might account for something

not sure if that influence was present the benchmark tho

[unborn quail]

You can test it out yourself @polar vortex, since the benchmark is in main under examples/stress_tests/many_components

[viscid python]

Would there be anything stopping nuking the Commands system param & having a commands() function that gives access to a thread local command queue?

[tight salmon]

Thread local commands would be nice, you could easily impl OnMutate for muteable components that way.

[slate shoal]

We probably want commands to have a consistent ordering for determism. Right now we run commands in topological order for systems

[tight salmon]

There would be order per-thread right? just not between threads?

[viscid python]

Not seeing where that would be different because systems run serially would enqueue their commands serially & systems run in parallel are already indeterministic

[slate shoal]

Commands are applied single threaded. The order commands are run at that time is deterministic per run.

[viscid python]

The thing users would care about is when they have serial systems. They'd be enqueued serially when using thread locals & would be applied in the same order.

I would say trying to rely on any kind of command ordering beyond that is way outside of the API contract & hard to even do anything useful with?

[tight salmon]

It is non-deterministic at compile time, although I guess deterministic at runtime, but I doubt anyone would use that information for anything currently.

[slate shoal]

Some users have asked for command ordering to be consistent in the past. It was a property we were careful to preserve with the stageless rewrite

[tight salmon]

That sounds like they are relying on a foot gun waiting to go off, commands in parallel ran systems shouldn't be deterministic, if users want that they should order their systems.

Also, just realized this isnt #ecs-dev whoops 👀

[viscid python]

Oh same. Lol. This took a long detour.

[polar vortex]

No, pretty sure there isn't. Having fewer queues would help do batching. And being thread local would avoid needing the mut in mut commands: Commands.

[polar vortex]

Bevy could merge every thread's queue into one queue (at the sync point) to put them in strict chronological order, but you don't actually want to uphold strict chronological order.

When Bevy is flushing the command queue and gets to a command acting on an entity (e.g. insert T to e1), ideally we incorporate all other commands acting on e1 as well, as a "batch operation" on e1.

The order between commands affecting the same entity is preserved. And the batches take effect in the order that the first command affecting each entity appears.

[viscid python]

There is one thing I'm unsure about and that's how this would work with multiple worlds

[polar vortex]

wdym? the impl shouldn't use thread_local!, just something like ThreadLocal

[viscid python]

Ah ok

[honest eagle]

Hey new bevy user here. I'm working on a project right now that needs relations, I was wondering about what the progress with the official relations support is. Would it be worth it for me to code my own implementation? Is relations support coming soon (0.16/0.17)? Has a specific syntax been decided on?
Thanks in advance

[sturdy zealot]

If you need relations right now consider using: https://github.com/iiYese/aery

[honest eagle]

Idk how I forgot third party crates exist

Thank you

Do you know if it uses the same/similar syntax to what bevy will use when first party support is added?

[sturdy zealot]

That's up in the air at this point. It should be close enough and hopefully easy to port once a final api is decided on

[honest eagle]

Okay, Thank you!

[keen shale]

Rainbow sounds interesting - I have been working on re-writing things we have done in C++ and Java to do interactive interfaces to large graphs of nodes edges - tables- grids in different "windows" and scenes often in different transform hierarchies etc and how to do it with ecs and bevy our current path - so am interested :).

One thing we have is multiple sets of "relationships" between nodes and have features like:

highlight/or some other visual change all items connected in a relationship graph ( one of several ) Highlight all the links/edges in that connected component keeping track of the distance from the origin ( often moused over ) node. likewise undo it when mouse out. ( pointer-ray in the case of a VR hand controller )

Highlight a transform element parent based on whether a child (recursive) is "armed"

Etc 🙂
How to traverse/modify nodes in such graphs in rust a new thing for me esp with borrows etc and adding onto it entities in bevy ECS

[viscid python]

If you have serious graph workloads I wouldn't advise using bevy.

[keen shale]

I figure in some ways - BUT dealing with the scene/transform graph does have to be dealt with in the render thread.

[viscid python]

highlight/or some other visual change all items connected in a relationship graph ( one of several ) Highlight all the links/edges in that connected component keeping track of the distance from the origin ( often moused over ) node. likewise undo it when mouse out. ( pointer-ray in the case of a VR hand controller )

Highlight a transform element parent based on whether a child (recursive) is "armed"
Is similar to my UI requirements

Doing it with bevy is painfully unsustainable in its current state

No one really has an idea of when an MVP will land & the current rate of progress is very slow

[keen shale]

I do see that ..

[viscid python]

And that's MVP. Doing the stuff you've described will require more advanced features which will leave you waiting a while.

[keen shale]

I did develop all the features in the old user interaction engine but rust and ECS are new to me compared to 30+ years with java/C++/etc.

[viscid python]

If you have deadlines & haven't invested yourself into bevy too much already I would recommend flecs for this kind of project if losing the other elements of bevy isn't a big deal for you.

[unborn quail]

yeah, idem

[keen shale]

FYI - here is a page on pior work done with didi and Brad Paley.
https://didi.co/docs/BradPaley_June2021.pdf

[viscid python]

lol that is very similar to my project

[keen shale]

hehe -

[viscid python]

#showcase message

There's an editor video in the thread

the graphs aren't obvious without a detailed explanation but yeah it's trying to address the UX issues of dealing with "graph"y data

Very DAW-like stuff

Sorry getting off topic here

[keen shale]

A question is how easy it is to use the parts of bevy we need ( as with any library ) That is rendering/shader/gpu/math. that is webasm and native distributable and then build our scenes UI on top of that :). Of course better to have it done an cooperate with a community. It's hard to take so many steps back after 35 years of work 🙂

[viscid python]

The UI is going to be the hard part

Especially if you don't have finalized designs that are completely feature frozen

You essentially end up writing a lot of hacks to get around bevy missing relations which is incredibly slow to iterate with

[keen shale]

I have thought maybe like a shared ( networked shared UI/workspace ) system I worked on the renderer needs to be it's own thing and all interactions with it are posted as pretty much write only messages. But a key is being able to deal with the scene graph, dynamic reparenting, and having links (edges) between nodes in different transform spaces / windows to track where they are attached without frame delay.

My transform tree cached the Global transform and they were only updated when read or if a parent node was altered. ( of course they were read to stuff the GPU when rendering )

Especially if you don't have finalized designs that are completely feature frozen

Well a lot of this is to have a testbed for things as proposals for people to pay us for haha research followed by development, every thing is always an experiment for how to do the next one better

[ancient spear]

One thing I'm really excited to experiment with post-relations is components-as-entities + something like flecs traits

instead of the generic component + plugin per generic variant thing we do now

[opal torrent]

I put together a draft of this, using FilteredAccessSet to be more precise than just components, and precomputing everything during schedule init.
It doesn't seem to hurt any on many-foxes!
And I'm pretty sure anyone who needs the parallelism it loses can recover it with a few Without filters.
Is it worth pushing up as a PR? What sort of testing would make us comfortable making a change like that?

[runic shadow]

Definitely seems worth pushing it somewhere, would be interested in what affect it would have on the new benchmark when contention is super high.

[opal torrent]

Here it is: https://github.com/bevyengine/bevy/pull/16885

[cloud plover]

It all leads back to relations

[wary fjord]

Honestly, I've been thinking for a while that we would need some maintainer to lead relations design work for it to make progress

[echo ore]

Is there an SME-Relations?

[wary fjord]

yea, Sander 😂

[echo ore]

Might be worth deputizing someone (probably not Sander as I think they might be busy haha)

[alpine patrol]

we do and it's cart

[wary fjord]

wrong reply?

oh I guess that could apply to either mine or bushrat's message

[rose pendant]

Yea I'm too busy implementing relationships

[alpine patrol]

for your consideration

[naive ravine]

dear lord

[tight salmon]

Relations related: #ecs-dev message

[wary fjord]

Discussion thread on discord here (I didn't notice it at first): https://discord.com/channels/691052431525675048/1319864421992108093

[wary fjord]

My version of bushrat's hook-based relationships PR: #1319864421992108093 message

[timid mountain]

So what is the status on this?
Is the non-fragmenting relations effort related to the fragmenting relations project that is the core of the WG?

[tight salmon]

So work on non-fragmenting is happening first, because it is easier, and after that fragmenting relations will attempt to be rescoped around non-fragmenting with value components (A component whose value is included in its ID). AFAICT

[timid mountain]

Would we still have "component as entities"? i.e. would the relations only be of the form <Component, Entity> or could we also have <Component, Component> relations?
Is there a doc describing non-fragmenting relations?

[tight salmon]

Component as entities is seperate and will come after either form of relations, at least that was the plan.

[unborn quail]

I sadly hit a block once more (so much to do, so little time, and sometimes I can't get my brain to coorporate) so I'm not working much on it in the near future

[alpine patrol]

Non-fragmenting relations are basically just a big lookup table stored in a resource, which is maintained using hooks and (in this case) immutable components.

[timid mountain]

So the fragmenting relations could still continue in parallel?

[left summit]

yup, work can be... fragmented between those two efforts 🥁

[rose pendant]

Sounds like carts plan for fragmenting is slightly different from what's been discussed thus far though

[timid mountain]

yea i didn't read everything about cart's plan so i'm a bit confused about I can do to help, what the current plan is for both fragmenting and non-fragmenting

[runic shadow]

Yeah I don't know what the intended implementation for fragmenting on values would look like

[rose pendant]

imo the most straightforward way would be to do a mapping from value -> entity, so relationship pairs can still be (entity, entity), or at least (component, entity)

That'd keep storage/query code simple (and fast)

And would mean that the existing work on fragmenting relationships is still usable

[runic shadow]

With an implementation like that it's effectively just an extension to fragmenting relationship pairs, but I felt like the phrasing was presenting it as an alternative, maybe I misinterpreted it. it's been a while

[rose pendant]

Yup, I don’t think that’s how cart wants to implement it, that’s how I would do it

[runic shadow]

I guess you could just collect hashes and hope to avoid falling back to any custom equality behaviour, but you'd still need to store the actual values somewhere (if you have empty archetypes) and be able to run the comparisons

[unborn quail]

Oh I love this

I'm fully in favour of this, I'm a Big Entity shill through and through

although I will probably put queries as entities higher in the priority list than systems as entities

[cloud plover]

Fair 🙂

The ordering was pretty arbitrary

And assets-as-entities is also missing

[unborn quail]

for the simple reason that queries as entities has some excellent work already done by flecs, while systems as entities has not

flecs does not have systems as a first class object, while bevy does

so there's more design work there to make it really work

[cloud plover]

I hope the guard rails / rules are a good compromise for type safety and teaching, without restricting the generalization that y'all are excited about

[unborn quail]

yeah, what we're basically doing is the rust monad trick

like, Result, Option and a bunch of other things are monads, rust just never names them as such

I wonder if resources as entities makes query access checking any cleaner

there's a lot of code duplication going on there between resources and components

I've thought it over and the answer is yes

that'll save a hundred lines or so

also btw @cloud plover, I currently study in Utrecht so while I may not be able to be there for the entire RustNL week, I can probably be an impromptu guide if you need

[cloud plover]

Oh lovely; definitely come out for dinner with us at least

[unborn quail]

yes, I'm thinking of taking the week off to go to the conference, but my thesis is also not going to write itself

I'll see, I'll definitely get tickets

[timid mountain]

I would really love Resources as Entities to stop having to special-case everything for resources

[cloud plover]

Yeah, it really sucks for networking

[rose pendant]

What do you mean by that? Systems in flecs are also entities

[unborn quail]

my mistake, I got a couple things mixed up

yeah, I just messed up

[rose pendant]

It's the other way around. Queries in flecs can be associated with entities, systems are always entities. The system entity is always the same entity that's associated with the system query (the latter part might be different in bevy because of how systems/queries are related to each other)

[cosmic elk]

The title says "everything" but it does not seem to include the discussion of "assets as entities".

Poor assets, always getting overlooked 😦

[cloud plover]

True; or plugins. I was scoping this to ECS internals for now 🙂

But the same principles apply

[echo ore]

Damn that's exciting

[crude oracle]

Relationships PR is merged!

https://github.com/bevyengine/bevy/pull/17398

[twilit grotto]

Woah that was fast

Congrats! Exciting times

[devout sedge]

I'm already seeing how I can make use of it in bevy_rand.

[devout sedge]

I'm already feeling my brain wrinkling as since I made my global entropy sources to entities instead of resources, I can make them also have relations

[gusty granite]

Is this the end of relationship?

Like.. Is the feature fully implemented now?

[devout sedge]

No, this is like part one of many more parts

AFAIK, fragmenting relationships is still on the table, but that involves further work, while non-fragmenting relations unblocks a whole bunch of stuff that benefits from it

[unborn quail]

nope far from

[viscid python]

Gonna be honest I don't see what it's unblocking? Like the mentality before this was "Oh relations aren't necessary for UI or the editor we can just use hierarchy" & this finally acknowledges that they are necessary if you want to maintain your sanity... but this is not meaningfully better than hierarchy?

[devout sedge]

I'm already able to make use of it though? Being able to define custom relations that are separate to the Parent/Child hierarchy semantics and behaviour is still an improvement

[viscid python]

That doesn't get you that much further. You can't do any interesting stuff without relationships being a construct queries can reason about.

[devout sedge]

Maybe, but an improvement is still an improvement, despite not getting us all the way there.

[unborn quail]

if cart things it's usefull for stuff than I'm going to take his opinion on this

but I agree it's far from a full relationships impl

[noble panther]

My understanding is that the primary first-party goal here is to make hierarchies, observers, and reactions use relations, allowing them to be spawned without archetype moves while generally making things more optimized and natural. So mostly motivated by the scene work, while still making basic custom relations expressible, even if the cool stuff is not there yet.

[viscid python]

Queries being able to reason about relationships isn't even close to "all of the way there". I'm not saying this isn't "all of the way there" I'm saying this is barely ahead of where we are & people are already letting their expectations run wild.

[unborn quail]

yeah, this is a case where the release notes should temper expectations when we get there

although much more than that we can' really do

if people wanna hype, they'll hype

[devout sedge]

Because folks now have a new tool, even if massively incomplete. Anyway, I'm getting the feeling we are looking at this from two massively different perspectives, so I'm just going to step back because I don't think we are going to see eye to eye on this.

[alpine patrol]

I think this is a pretty big technical improvement for a large part of our existing code-base and an incremental step towards fragmenting relations. I don't really get why every feature has to be "unblocking" something now, progress doesn't always happen in big jumps.

I know that I am excited to see this applied the observer-observed relationship, to the problem ui reactivity, to speeding up and simplifying transform propagation, and possibly to other niche things like the camera-window dynamic or renderlayers.

All of these are now things we can investigate. Perhaps some of them will have additional blockers, perhaps some of them could be more complete/more efficient with fragmenting relations.

But to say that it doesn't provide any improvements seems to me to be jumping the gun a bit.

[rose pendant]

The original issue for tracking fragmenting (I think) relationships got closed though, is this still on the table? 👀

[cloud plover]

Yeah, this is still on the table

It's just that the original issue was quite cluttered at this point

And more focused issues are required to be useful

I definitely encourage y'all to try out the current impl, and make issues / PRs for functionality that you run into (and cite prior art if that's helpful)

And also to start migrating various internal systems to use these. One of the big advantages of having something blessed is that it should make migrations to future evolutions (new little APIs, fragmenting relations) way less painful than a dozen ad hoc implementations

[tall mountain]

From my perspective, making the existing hierarchical relations more watertight is also quite a boon

[cloud plover]

We can also reuse all of the existing traversal code for arbitrary relation types now

[cosmic elk]

@crude oracle @cloud plover Do we want to document the naming conventions for relationships?

[cloud plover]

Once we have some, yes 😅

The bike shed is still hot

[noble panther]

I know there was some bike shed around this but I don't love the trait RelationshipTarget being for components that store the source entities 🤔

relationship_target = Children reads like "children are relationship targets" when in reality they are the sources. Children is stored on the target entity, but the actual children are the source entities

though I don't really love the old relationship_sources name either, it's tricky

Also I feel like we could at least cut the relationship_ prefix so it's just #[relationship(target = TargetedBy)]

it should be clear enough from context that it takes a type implementing RelationshipTarget

[viscid python]

Which is why I never said that ? My own crate would be redundant if that were the case. Like this is being read as "we did relationships guys" while all of the things that make relationships powerful are sidelined under a hand wavy use of the term "fragmenting". People seem to be more excited to check a box instead of understanding what that box means.

[viscid python]

What happens when users try to do things these "relationships" can't do that relationships should be able to do? Like for instance grouping? They're going to go back to those bespoke impls.

[echo ore]

Not trying to start an argument, but if you say "this is not meaningfully better than hierarchy" and push back against people saying "an improvement is still an improvement", it does read like you're saying this isn't an improvement...

[viscid python]

It's marginally better. That's about it. I could not use this to replace my own bespoke impl in my game cause it adds nothing I'd need.

[cloud plover]

Sure. And we'll steadily improve them. What's the point you're trying to make? This group is focused on trying to improve things, simply complaining about how inadequate the existing work has been done is unproductive.

[ancient spear]

not everyone needs true relations though

the hierarchy stuff is a common enough pattern, and having a robust implementation of that is quite nice I think

[alpine patrol]

Like this is being read as "we did relationships guys" while all of the things that make relationships powerful are sidelined under a hand wavy use of the term "fragmenting".
i don't think that's what's happening at all.

[echo ore]

And hopefully we'll be able to add what you're missing during the 0.16 development cycle and/or beyond! Please feel free to open issues and/or submit PRs for the functionality that's missing. aery is proof you've got the skills to do so, and now we have the beginnings of a blessed API to extend

[viscid python]

I disagree. The contentment with the way things are as "the bevy way" & people not being interested in other solutions has been the biggest source of debt accumulation that makes anyone who tries to work on these things burn out.

[cloud plover]

Speaking personally, the number one reason why anyone trying to work on relations burns out has been toxicity and negativity from certain members of the community.

I will defer to the other <@&1064695155975803020> to make the call on any moderation action here, but I am incredibly frustrated by the years of petty sniping and derailing that you've done, and I think it would be wise of you to reflect on the impacts of your actions, both personally and on the project.

I have been chewing on, and working towards this problem for literal years. I'll pick it up, work with interested contributors, and try and drive something forward, before it gets bogged down in controversy for "not being good enough", with a rant about how we just need to do these seven complex internal refactors before we can actually ship anything.

And then people give up, and I move on to other work that doesn't have this problem