#very slow ecb (entity command buffer) in debug mode

1 messages · Page 1 of 1 (latest)

bleak onyx
#

Hello!
We have updated from Entities 0.16 and Unity 2020 to Entities 1.0 and Unity 2022.
We solved a lot of problems and had a good release.
However, we still have one very annoying problem.
If the editor is in debug mode (for example to use breakpoints), then the ECBs begin to waste an incredible amount of time.
For example, adding one component to one entity can take about 0.5 seconds. If you enable deep profiling for analysis, you can see that when working with buffers, EntityQueryManager.TestMatchingArchetypeAll is called tens of thousands of times.

Our project is very large, we have about 1000 components and 3000 systems.

For the test, we tried to transfer the piece of code causing this problem to a new project on the same version of packages and unit. The problem is not reproducible. There are no calls to EntityQueryManager.TestMatchingArchetypeAll at all.
Yes, the new project does not have the same scale as the original project, but there should probably be at least one call when working with the buffer.
Apparently the problem is in some extensions we wrote over version 0.16, but I can’t figure out what exactly could be causing such a problem.

Perhaps someone has encountered something similar or has ideas on how this could be fixed?

` private EntityCommandBufferSystem TestCommandBuffer =>
testCommandBuffer ??= World.GetExistingSystemManaged<BeginSimulationEntityCommandBufferSystem>();

    private EntityCommandBufferSystem testCommandBuffer;

    protected override void OnUpdate()
    {
        var ecb= TestCommandBuffer.CreateCommandBuffer();

        Entities
            .WithAll<TestComponent1>()
            .WithAll<TestTagComponent1>()
            .WithNone<TestTagComponent3>()
            .ForEach((Entity entity) =>
            {
                ecb.AddComponent<TestTagComponent3>(entity);
            })
            .WithoutBurst()
            .Run();
    }

`

south marsh
#

Just sounds like it's causing a sync point

bleak onyx
# south marsh Just sounds like it's causing a sync point

Yes, I know synchronization points very well. This is exactly what happens. The problem is resources spent on this, and only in debugging mode and only with a combination of DOTS 1.0 and Unity 22.3. And only for the project that has completed an update, but not for a new created project.

river vigil
#

Your best bet - move all your sync points to a very beginning of frame

#

so there are no jobs scheduled yet

#

And that would just mean that sync points will be free

bleak onyx
# river vigil Resources spent on this equal resources spent on jobs themselves. Main thread is...

You are right. But my problem only exists under certain conditions and is not reproducible in production. Only in the editor in debug mode. This disables all additional debugging tools, such as the Journal Entity.
Outside of this mode, my buffers are practically free 🙂 Or in this mode, but on versions of Entities below 1.0 🙂

Unfortunately, in this case there is no waiting for Job to complete. Such delays are usually clearly visible during debugging. In this particular case there is nothing like that.

river vigil
bleak onyx
#

attaching a screenshot of an example of buffer operation trace

river vigil
#

thus probably the reason why it's slow

#

hierarchy is useless

#

use timeline in multithreaded contexts

bleak onyx
# river vigil hierarchy is useless

The hierarchy, as far as I can see in testing, clearly shows that the very fact of waiting takes place.
But to understand what we expect, I agree, we can’t do without a timeline 🙂
If you think the timeline will help us, I will collect the latest data and attach it.

river vigil
#

and from this you can figure what to do

#

best is always - just get rid of sync point

#

no logic benefit is worth potentially 30% fps

bleak onyx
#

We discussed in private messages, but just in case I'm attaching this for public.

#

I'll continue to do research 🙂

bleak onyx
south marsh
#

oh editor does run a lot faster in release mode

#

but there have been a lot of ecb improvements since then but it still helps a lot, highly recommend for all artists/designers to be in release mode

#

you seem to be making a lot of structural changes?

#

how many operations and entities are you affecting in your ecb

#

sorry re-read your original post

#

you have a decent amount of systems/components

#

but how many archetypes do you have?

#

have you edited the package to allow yourself to have more than the regular cap?

#

(i.e. is it 67k?)

bleak onyx
# south marsh but how many archetypes do you have?

Right now I'm just trying to clarify the issue with archetypes.
We independently create about 300 archetypes.
But if you believe the output in debug mode, Entities finds hundreds of thousands of archetypes 🙂
At the moment, I have added the sources of the Entities package to the project and added log output to the archetype registration code for testing. In release mode, the log is output 1000 times, in debug mode 1,000,000.
And I think this is exactly the problem. But I still don’t understand where Entities gets them from in debugging mode. And why, with similar data, I don’t see this in the newly created test project.

south marsh
#

yeah that many archetypes is a huge problem

#

but usually you would crash at around 8-12k archetypes

#

which is the limit by default

#

unless you've edited the package to increase the size of the archetype storage

bleak onyx
#

We tried not to make changes to the package source code. But now, apparently, we’ll try 🙂 Because we see a little more crash in analytics than in previous releases, but we didn’t suspect that we were exceeding number of archetypes.

bleak onyx
south marsh
#

i only use ecb for instantiation and destroying entities

#

i no longer have any add/remove components

#

so i have no performance problems with ecb and keep a really low archetype count

#

(but this took a new project, no way the previous project could ever be updated to this architecture)

river vigil
#

yeah, unity did a really great job at removing need in structural changes