archived-dots

[gusty comet]

yeah i feel like i'm a beginner-mediocre coder at best, so im trying to take parts of dots and implement it where I can...I still need to learn the jobs system since I would like most stuff to be multicored...

[rustic rain]

I never actually learnt OOP the good way, kek

straight into ECS

1 year total experience in programming here

[robust scaffold]

The journey is very long. Honestly, I recommend sticking with mono for now. And get really good at ShaderLab / HLSL (not the shader graph basically). DOTS is CPU side compute shaders.

Burst is nice dont get me wrong. Hell, before 0.50, I was rolling my own ECS/DOTS with a pile of native arrays. Pure burst and engine code.

You dont need DOTS to do burst. But you wont get the most out of it without it.

[gusty comet]

lol, unfortunately shader code is beyond me, I got damn good at Shader Graph funnily enough, and I can only imagine they will tweak it to be more performant in future, when they do hopefully it will result in instant performance gains, but I did hear SG is expensive

at least im not stuck in Bolt or something, im glad i learned to script basic stuff properly XD

[robust scaffold]

It's not the fault of shadergraph, it's the limitation of visual scripting in general. The majority of shader performance improvements is understanding the "hidden" rasterizer between vertex and fragment shaders. Shadergraph basically lets the user code entirely in fragment and thus runs code every pixel whereas the text version of shaders lets you have full control. Or at least as much as possible by the compiler.

[gusty comet]

yeah i was pissed when I couldnt use the scene depth node to dictate the strength of my vertex displacement for water...PISSED

[robust scaffold]

Sure, you can hypothetically run vertex code that is passed to fragment inside a shadergraph but the only visual scripting shader language I've seen that has it possible is Unreal's. And it's really clunky.

[robust scaffold]

That's an issue with DX9 legacy code.

[gusty comet]

are they getting off DX9?

or is that 2022?

[robust scaffold]

We're already 2 versions ahead by 2019 LTS, DX11.

Shadergraph uses tex2d() samplers, as is default. But that's DX9 language.

[gusty comet]

so i should be able to mix fragment and vertex stuff?

somehow?

[robust scaffold]

no.

That's mesh shaders, DX12+ preview.

[gusty comet]

guessing thats 2022

[rustic rain]

Vulkan is the only way

[robust scaffold]

Unity does not support mesh shaders yet. And probably not until 2024 or 2025.

Vulkan barely supports it.

[gusty comet]

it's only logical

[gusty comet]

it was suggested i could export a heightmap of the terrain and use that....but i thought using that would be using a texture and therefore vertex + fragment issue again?

[robust scaffold]

Dx11 introduced Texture2D<float4> and explicit samplers. But shadergraph does not use it as it has little backwards compatibility.

With Texture2D, you can sample textures in vertex code. Not with tex2D.

[gusty comet]

oh so i have to use a different node? ahh TIL

[robust scaffold]

No. Shadergraph does not support Texture2D. They compile to tex2D.

[gusty comet]

OH I see...bah....

[robust scaffold]

Shadergraph is basically a bunch of code samples stringed together.

[gusty comet]

hmmm......i need to find some way of dictating the strength of the displacement in specific areas...

not that this is DOTS related, but XD

[robust scaffold]

For example, this is my vertex to fragment data package. I do 80% of the computing in vertex, so 2 times per 500 pixels or so, then do 20% of the computing in fragment.

[robust scaffold]

DOTS and shaders are extremely similar.

For example, a shader's v2f is the "chunk component" in DOTS.

The pixels / fragments are the "entities". The vertices are the "chunks".

[gusty comet]

0-0 , seems like black sorcery to me...

Although an array looked like sorcery a year ago XD

dots is powerful, im betting 1.0 is going to have quite a bit, since they said they were going to make hybrid approaches a focus too

[robust scaffold]

It's 200 lines of linear algebra, trigonometry, vertex projection, and matrix multiplication

[robust scaffold]

Nah. It honestly wont change much from 0.50

For beginners, it'll probably actually be slightly easier to understand, seeing the IAspect interface and code generation, but for actual performance? You'll be hand writing IJobEntityBatches

[gusty comet]

was planning to try jobs soon, and add burst ontop of it.....assuming thats possible...like I said i'm just starting out

[robust scaffold]

Yep. Really easy. Take a NativeArray<int>(). Pass it through an IJob. Add [BurstCompile]. Schedule. Complete. Tada, multithreading.

The only package you need for that entire sequence is Burst. No DOTS. No Hybrid. Works right out the box and shipped with the editor.

[gusty comet]

hopefully itll be easy to get the hang of multithreading, its so important nowadays

[robust scaffold]

Multithreading is the easiest part of DOTS. Understanding ECS design is the years long process.

[gusty comet]

especially since its rare to get less than a six-core now, it's kinda the new baseline i've noticed when people invest in rigs

[robust scaffold]

Which is why I'm rolling my own physics engine. Completely multithreaded physics. Combined with completely multithreaded game data and multithreaded rendering (kinda, not fully done). Means a completely multithreaded game.

[devout prairie]

is it a 2d only engine?

[robust scaffold]

Kinda. Still uses float4x4s for LtW matrixes.

A true 2D engine would cut that down to 3x3. With rendering inputs to support it.

[devout prairie]

i think i read you say you're rewriting the transform system also right

[robust scaffold]

Already done. Just disabled the built in transform and substituted my own.

[devout prairie]

nice, the transform system always felt a bit of a rats nest to me

parenting also seems to be as slow as hell

[robust scaffold]

Parenting = inheritance = not proper DOTS design

[rustic rain]

parenting is random access

so generally you'd avoid that

[devout prairie]

i always try and avoid yep

[robust scaffold]

For my "inheritance", I add tags to entities to specify who takes what inputs.

[rustic rain]

there's no inheritance in ecs though

only composition

[devout prairie]

reading up on latios framework docs is interesting, i haven't tried it but he seems to know his onions.. i think he's also rewritten transforms, i think maybe does do parenting but differently, has deep dived into the skinning shaders etc, and implemented a bunch of optimizations to unity physics..

guys like kornflaks, tertle and the latios guy should take up positions in unity 🙂

[robust scaffold]

Nah

I barely know what I'm doing most of the time

[devout prairie]

single handedly taking on the whole framework

[robust scaffold]

I've just been here for too long

[devout prairie]

haha doesn't sound like it

so what's stopping your physics engine being 3d instead of 2d?

[robust scaffold]

3D takes a lot of compromises and optimization that causes jittering instability

2D has the advantage of basic geometry and float2 vectors for significantly better stability.

And my next plan is to branch into custom netcoding as well and 2D has a lot smaller data byte sizes compared to 3D.

[devout prairie]

i guess more simd'able also with those float2's

[robust scaffold]

Not really

At least in my experience with burst. Burst doesnt recognize that float2s can be packed together into a float4 for SIMD. Only floats can and well, no one is developing a 1D physics engine.

[devout prairie]

hah

[rustic rain]

can always implement it yourself

with Reinterpret

[robust scaffold]

So there's no benefit for 2D vs 3D in simd sadly. Chunk sizes can hold more entities though.

[rustic rain]

will be a bit of pain

[devout prairie]

are there challenges with determinism when building a physics engine and multithreading etc?

[robust scaffold]

Then you gotta worry about padding and all that.

[rustic rain]

but that would be a huge benefit in some crucial systems like parenting

I'd say, it'll be a must have

[robust scaffold]

If you do any changes at all to floats in Burst, it's deterministic.

if you touch a float outside burst, it breaks determinism.

Well, you can set it to ints. Float -> int and vice versa is deterministic.

[devout prairie]

how about solving collisions etc in parallel, i'd think that could be a challenge

[robust scaffold]

Well, unity did it so it's possible.

[devout prairie]

ie collisions being a kind of inheritance in a way

true

[robust scaffold]

Unity's built in 2d physics engine is burst job multithreaded.

Collisions only occurs between 2 colliders and only those 2.

[rustic rain]

why parallel collisions are a problem?

sounds extremely simple to thread it

since it's all read only

[devout prairie]

so that's why unity physics breaks everything down into pairs i guess

solve all the pairs in parallel, and then assimilate the results i guess

[rustic rain]

btw

does 2D float3x3

has depth value?

for drawing

[robust scaffold]

No depth. Only layer sorting.

[rustic rain]

yeah

layer sorting

[robust scaffold]

a true 2D rendering system local to world does not include sorting. That is done at draw call or a compute shader generating the call order.

But you could, with the Z value.

[rustic rain]

well

I feel like it's important one

[stiff valve]

I am trying to do click to move system with DOTS. Can someone please give me a hint on how to do raycasting in DOTS, please?

[rustic rain]

since you need to be able to tell the difference between certain objects and their heights

[robust scaffold]

I mean, it's not that difficult. Binary sort by a layer value.

2.5D might be different.

[robust scaffold]

https://docs.unity3d.com/Packages/com.unity.physics@0.51/manual/index.html

[stiff valve]

Thanks

[robust scaffold]

Like a per-pixel Z value sorting will not be possible in a 2D rendering system.

[stiff valve]

Unity physics samples are not avalaible(

[rustic rain]

nah, per mesh sorting

[stiff valve]

I think I found them

[rustic rain]

so if for example your character is built from several sprites - you can sort them all

[robust scaffold]

https://dots-tutorial.moetsi.com/unity-dots-physics/use-dots-physics-for-collisions
This is for object collisions but raycasting uses the same concept.

[stiff valve]

thanks

[robust scaffold]

Each one will then need their own l2w and their own rendering draw call. It cant be merged into a single mesh if you want a single drawprocedural to render it with sorting.

[rustic rain]

I don't put too much on procedural

[robust scaffold]

And if you want each vertex to have their own sorting, that wont be possible in a 2D engine like godot.

[rustic rain]

simple instancing works great with 2d

[robust scaffold]

DrawProcedural is basically the simplest draw call possible in a graphics driver.

[rustic rain]

as long as you have atlasses

[robust scaffold]

It maps straight to CmdDraw in Vulkan and such.

[rustic rain]

it'll be nice and fast

yeah, but you lose all features of render pipeline

[robust scaffold]

Unity instancing is a "mid level" command. They handle a lot of the buffer passing

[robust scaffold]

Thats the thing, you can roll your own, superior version.

[rustic rain]

bruh

might as well just go and use C++ and write all from scratch

[robust scaffold]

https://media.discordapp.net/attachments/300454903250419712/1002784559638782063/unknown.png

With DrawProcedural, you basically are.

DrawProc and DrawProcIndirect are the two "actual" draw commands available in C++ drivers. Everything beyond that, you can do yourself.

What unity does for you that I think they do perfectly is setting up the windows, the viewports, the scissors, and fundementals.

And the render target pool as well. Thats why Im not moving to implementing my own engine.

[rustic rain]

yeah, but it's not in my best interests to reinvent the engine

[robust scaffold]

There's the engine, and then there's fluff. Cut away the fluff, keep the engine.

Especially 2D. Unity has bad fluff. Their sprite packer works though. I dont see myself reimplementing that. And they have an algo for merging colliders into 1 composite that I also use. Which is why my tilemaps will have built in physics2D colliders despite everything else running on my own physics and custom colliders.

[robust scaffold]

What I really want though is buffer chunk components

[viral sonnet]

closest to that, out of the box, is a persistent nativestream

[robust scaffold]

Or an unsafe list in a chunk component

Ugh, i think thats what Im gonna do

[viral sonnet]

ah on chunk level, yeah that works too

just somewhat annoying to handle

honestly, i wouldn't do that. kind of a roundabout way to save it in a chunk comp

[robust scaffold]

or just swallow the extra data and do per entity component

Ugh, random access pain

Or, attach a buffer to the source of the data and iterate per source, rather than per destination.

That will need unsafe access to the component data though... which is also random access.

I guess Entity -> Entity relationships are just bad in general.

Unless I restructure my entire lighting data system....

[mortal crow]

Is it possible to change dynamically the maximum length of a limited distance physics joint?

[robust scaffold]

I dont see why not: https://docs.unity3d.com/Packages/com.unity.physics@0.51/api/Unity.Physics.PhysicsJoint.html
There's a set accessor to the properties. You'll need to calculate the tangent then multiply by your new length

[robust scaffold]

Code for burst, not for readability.

[robust scaffold]

Hidden upside of writing my own physics loop, I can now specify exactly where camera position update is conducted so no more shaky sprites when camera is following a player.

[mortal crow]

@robust scaffold I can't modify it into the inspector and my code does not seem to do anything in game or inspector:

  public partial class CraneInputHandlingSystem : SystemBase
  {
    private NativeArray<Entity> _jointEntities;

    protected override void OnStartRunning()
    {
      EntityQuery allPhysicsJointQuery = GetEntityQuery(ComponentType.ReadWrite<PhysicsJoint>());
      _jointEntities = allPhysicsJointQuery.ToEntityArray(Allocator.Persistent);
    }

    protected override void OnUpdate()
    {
      var                 input  = GetSingleton<CraneInputData>();
      NativeArray<Entity> joints = _jointEntities;

      Entities
        .WithName("CraneInputHandlingSystem")
        .WithAll<ActiveCraneTag>()
        .ForEach((Entity entity, in CraneMechanicsData craneMechanics) =>
                   {
                     if (input.Grabber > 0f && joints.Length > 0)
                     {
                       foreach (Entity joint in joints)
                       {
                         if (HasComponent<PhysicsConstrainedBodyPair>(joint))
                         {
                           var bodyPair = GetComponent<PhysicsConstrainedBodyPair>(joint);

                           if (bodyPair.EntityA == craneMechanics.Grabber)
                           {
                             var physicsJoint = GetComponent<PhysicsJoint>(joint);

                             physicsJoint.SetLimitedDistanceRange(new Math.FloatRange(0, 50));
                           }
                         }
                       }
                     }
                   })
        .Run();
    }
  }

[robust scaffold]

Im afraid I have no experience with DOTS physics.

[robust scaffold]

Thats also a lot of nested if statements. Ya gonna kill burst with that logic.

[molten flame]

@mortal crow You are not setting the component back onto the joint entity.

The authoring tools for joints are "a bit shit"

Also as KornFlaks says, try to make use of return statements to clean your nested blocks and give burst a hand.

[robust scaffold]

Side note, Debug.DrawLine is the only visual debugging tool available to a job. Not bursted but it works:

[robust scaffold]

Wait, Debug.DrawLine does work with burst

[rotund token]

yes

most things in debug do

(it's terribly slow though, i highly recommend a proper draw library)

[robust scaffold]

Yea. This is just to verify that the AABB and points correspond properly

[timber ivy]

what do you mean by that?

[drowsy pagoda]

Doing this in a IJobEntityBatchWithIndex job. It's fields:

[ReadOnly, DeallocateOnJobCompletion] public NativeArray<UnitGuid> unitGuids;
[ReadOnly, DeallocateOnJobCompletion] public NativeArray<LocalToWorld> transforms;
[ReadOnly] public BufferTypeHandle<InventoryElement> inventoryBufferTypeHandle;
[ReadOnly] public BufferTypeHandle<EquipmentElement> equipmentBufferTypeHandle;

Does that mean that the query will only return units WITH those buffers? I have units that don't have those buffers and I just need to check if they exist.
Does that mean I have to use BufferFromEntity<InventoryElement> instead? Or is this way valid (meaning it will return entities without those buffers too)?

[rotund token]

the query determines what entities are you returned

you can use the chunk and do chunk.Has(BufferTypeHandle) or var accessor = chunk.GetBufferAccessor(BufferTypeHandle); then check accessor.Length > 0

to determine if the entities have the component

no need to check independent entities

[drowsy pagoda]

Gotchya. That's perfect.

[rotund token]

the chunk has the data

[drowsy pagoda]

Thx

[rotund token]

how have you marked your sources to be in the navmesh collection

all it does is this

                {
                    Matrix4x4 localToWorld = Matrix4x4.TRS(transform.position, transform.rotation, Vector3.one);
                    var worldBounds = GetWorldBounds(localToWorld, GetInflatedBounds());
                    NavMeshBuilder.CollectSources(worldBounds, m_LayerMask, m_UseGeometry, m_DefaultArea, markups, sources);
                }```

[drowsy pagoda]

Using a NativeParallelMultiHashMap, does anyone know if I map.Dispose(jobHandle) if it will dispose of the nested NativeArrays as well? Or does this mean I have to implement a for loop to dispose of them manually? If latter, then how can I do that in background thread?

[rotund token]

what nested array?

[drowsy pagoda]

is it array or list? From what I understand, a NativeParallelMultiHashMap is a key, NativeArray<value> type deal? or no?

Oh! Wait a minute, nevermind. I think I understood what it is now. The keys are allowed to be "duplicated", so the "array" is stored that way. I see...please ignore my dumb question lol

[timber ivy]

A yep got it working now thank you very much!

  List<NavMeshModifier> modifiers = new List<NavMeshModifier>();
    List<NavMeshBuildMarkup> markups = new List<NavMeshBuildMarkup>();
    List<NavMeshBuildSource> sources = new List<NavMeshBuildSource>();
    private IEnumerator RebuilNavMeshAsync()
    {
        Bounds navMeshBounds = new Bounds(new Vector3(observer.transform.position.x, 0, observer.transform.position.z), new Vector3(100, 20, 100));
        modifiers = NavMeshModifier.activeModifiers;
        

        for (int i = 0; i < modifiers.Count; i++)
        {
                markups.Add(new NavMeshBuildMarkup()
                {
                    root = modifiers[i].transform,
                    overrideArea = modifiers[i].overrideArea,
                    area = modifiers[i].area,
                    ignoreFromBuild = modifiers[i].ignoreFromBuild
                });
        }




        NavMeshBuilder.CollectSources(navMeshBounds, surface.Value.layerMask,
                        surface.Value.useGeometry, surface.Value.defaultArea, markups, sources);


        var navMeshUpdateOperation = NavMeshBuilder.UpdateNavMeshDataAsync(
            navmeshData, surface.Value.GetBuildSettings(), sources, navMeshBounds);
        yield return navMeshUpdateOperation;     
            

    }

[drowsy pagoda]

@rotund token Trying to wrap my head around your bufferAccessor.Length > 0 check. Say there are 1000 entities, but only 200 of them have that buffer. I'm running this job in parallel so using .AsParallelWriter() on that map. In my loop, when I check for buffer accessor length, how do I know which index of the accessor belongs to the index of the current entity in the loop?

[rotund token]

isn't the chunk passed into the IJobEntityBatch

?

all entities in that chunk will have the exact same archetype

so the answer is all of them

[drowsy pagoda]

Trying to figure out this error...

Oh!

I understand now. Forgot about each batch is same archetype deal. Gotta lay off the weed.

let me try

Ok, it now displays the same error, but only a single one. I currently am testing this with 6 entities, only 1 has that buffer. So before the accessor length check, the error would come up for the 5 that didn't have the buffer. After that fix, it now displays that error for the one with the buffer?

What else might be triggering that error?

[timber ivy]

I am looking at the docs for a job system and saw you have to call job.Complete(); or something after scheduling it. Is calling complete just stalling the main thread till the job is finished?

[drowsy pagoda]

Yes.

[timber ivy]

Then whats the point of jobs?

if it just pauses the main thread

[drowsy pagoda]

You don't "have" to call job.Complete after schedule.

Ideally you'd never need to.

[timber ivy]

then how do you know when to free the resources?

like this is the example in the docs

NativeArray<float> a = new NativeArray<float>(2, Allocator.TempJob);

NativeArray<float> b = new NativeArray<float>(2, Allocator.TempJob);

NativeArray<float> result = new NativeArray<float>(2, Allocator.TempJob);

a[0] = 1.1;
b[0] = 2.2;
a[1] = 3.3;
b[1] = 4.4;

MyParallelJob jobData = new MyParallelJob();
jobData.a = a;  
jobData.b = b;
jobData.result = result;

// Schedule the job with one Execute per index in the results array and only 1 item per processing batch
JobHandle handle = jobData.Schedule(result.Length, 1);

// Wait for the job to complete
handle.Complete();

// Free the memory allocated by the arrays
a.Dispose();
b.Dispose();
result.Dispose();

[drowsy pagoda]

Well, some jobs that take too long, you can set a timeout and call job.Complete when timeout reaches limit. Just example.

[timber ivy]

is there a handle.oncomplete event?

[drowsy pagoda]

They just show you all you can do with it. The code between where you schedule and where you complete is up to you.

No. Unfortunantely there is not.

[timber ivy]

Ok so in the sample above id just set a timeout to call complete()?

[drowsy pagoda]

Alternative would be

while (!jobHandle.IsComplete)
  await Task.Yield();

[timber ivy]

what would that do?

[drowsy pagoda]

that will literally wait for the job to complete before executing the rest of the code block WITHOUT stalling the main thread 😉

[timber ivy]

oh cool

thanks!

[drowsy pagoda]

You just have to make sure your method that has that has async keyword. Like: private static async void MyCustomMethod()

@rotund token were you able to see this question? If not, let me know if you can help me with this.
#archived-dots message

[rotund token]

you're using the wrong index ^_^'

what is your code

[drowsy pagoda]

Hang on, adding a check for index, I'll ding you when I post it.

[rotund token]

going for a run in like 2 min

so thats about as long as you have ^_^'

[drowsy pagoda]

Ooff, guess I missed it. Just get back to me when you can.

public void Execute(ArchetypeChunk batchInChunk, int batchIndex, int indexOfFirstEntityInQuery)
        {
            var inventoryBuffer = batchInChunk.GetBufferAccessor(inventoryBufferTypeHandle);
            var equipmentBuffer = batchInChunk.GetBufferAccessor(equipmentBufferTypeHandle);
            
            for (int i = indexOfFirstEntityInQuery, j = 0; j < batchInChunk.Count; i++, j++)
            {
                var unitGuid = unitGuids[i].Instance;
                
                unitSnapshots[i] = new UnitSnapshot
                {
                    guid = unitGuid,
                    infoGuid = unitGuids[i].Info,
                    location = transforms[i].Position,
                    rotation = transforms[i].Rotation
                };

                var inventoryCount = inventoryBuffer.Length;
                if (inventoryCount > 0 && j < inventoryCount)
                {
                    var inventory = inventoryBuffer[i];
                    foreach (var inventoryItem in inventory)
                        unitInventory.Add(unitGuid, inventoryItem.unitInfoGuid);
                }

                var equipmentCount = equipmentBuffer.Length;
                if (equipmentBuffer.Length > 0 && j < equipmentCount)
                {
                    var equipment = equipmentBuffer[i];
                    foreach (var equipmentItem in equipment)
                        unitEquipment.Add(unitGuid, equipmentItem.unitInfoGuid);
                }
            }
        }

[rotund token]

lucky for you i got a call from my boss

var inventory = inventoryBuffer[i];

you should be using j not i

you're indexing the chunk

var equipment = equipmentBuffer[i];

same here

not sure what you're i is used for these

infoGuid = unitGuids[i].Info,
location = transforms[i].Position,
rotation = transforms[i].Rotation

and why're your passing it in separate from the chunk iteration

[drowsy pagoda]

Oh snap!

Oh, initially I did ComponentTypeHandle, but because I'm executing this job outside the System's OnUpdate chain, it gave me problems. So did source gen. So I had to go old fashioned 😛

It worked. Thanks.

[drowsy pagoda]

Because they were not retrieved using ComponentTypeHandle. They were pulled from query.ToComponentDataArrayAsync

[timber ivy]

InvalidOperationException: WaitJob.op is not a value type. Job structs may not contain any reference types.

is there a way to get around this?

trying to do something like this

public struct WaitJob : IJob
{
    public AsyncOperation op;

    public void Execute()
    {
        while (op.isDone == false)
        {
            
        }
    }
}

[rotund token]

not without pinning your class

and passing in the ptr

[timber ivy]

Is that a performance hit?

[rotund token]

not really

i mean if you pinned too much memory (huge amounts) at once you'd have some issues

but 1 litle thing would be fine

[timber ivy]

What about lots of small things?

[rotund token]

you'd be releasing this as soon as job is done anyway right

[timber ivy]

Yes but I have my doubts this is going to perform well being called 100s of times a second

public struct WaitJob : IJob
{
    public IntPtr op;

    public void Execute()
    {
        var handle = GCHandle.FromIntPtr(op);
        
        while( ((AsyncOperation)(handle.Target)).isDone == false)
        {
            
        }
    }
}


 public async Task<GameObject> GetVehicle(string name)
    {
        var waitFileLoad = new WaitJob();
        
        var req = AssetBundle.LoadFromFileAsync(Path.Combine(Application.streamingAssetsPath, "vehicles"));
        GCHandle requestHandle = GCHandle.Alloc(req, GCHandleType.Pinned);


        waitFileLoad.op = requestHandle.AddrOfPinnedObject();
        
        var jobHandle = waitFileLoad.Schedule();

        

        var j = new WaitJob();
     

        if (req.assetBundle == null)
            return null;

        var loadRequest = req.assetBundle.LoadAllAssets();

        GCHandle loadRequestHandle = GCHandle.Alloc(loadRequest, GCHandleType.Pinned); 

        j.op = loadRequestHandle.AddrOfPinnedObject();

        var h = j.Schedule(jobHandle);


        while (!h.IsCompleted)
            await Task.Yield();

        requestHandle.Free();
        loadRequestHandle.Free();
        return ((AssetBundleRequest)loadRequestHandle.Target).asset as GameObject;

    }

Gonna profile it now and see how it runs tho

[rotund token]

there's no real cost to it

it just means the GC can't move the memory and has to work around it

my question is

why are you doing this in a job?

you're in a task anyway

why not just wait in the task

[gusty comet]

What is the most appropriate datastructure (i.e. container) for an ordered queue?

I assume list is a linked list, which is nice - but it has no insertAt

[timber ivy]

Yeah I actually just changed my code to this. I think this is better

public async Task<GameObject> LoadVehicleAsync(string name)
    {
        var req = AssetBundle.LoadFromFileAsync(Path.Combine(Application.streamingAssetsPath, "vehicles"));
        
        
        while (!req.isDone)
            await Task.Yield();

        var bundle = req.assetBundle;

        if (bundle == null)
            return null;

        var assetLoadOp = bundle.LoadAssetAsync(name);

        while (!assetLoadOp.isDone)
            await Task.Yield();

        bundle.Unload(false);
        return assetLoadOp.asset as GameObject;
    }

Well that crashes unity lmao

Do you have to call async methods a certain way or something?

[rustic rain]

how do you call it?

[timber ivy]

Just like a normal method

LoadVehicleAsync from a non async method

I guess whats happening is its not happening on the background since its call is not being awaited

But I can't follow the chain of the calls and make all of them async so idk what to do in this situation

[rotund token]

you shouldnt need to await it

but if you're returning the result you can't use it without waiting

[timber ivy]

        while (!assetLoadOp.isDone)
            await Task.Yield();

        bundle.Unload(false);
        return assetLoadOp.asset as GameObject;

isn't that waiting there?

[timber ivy]

Ok you're right

the task isn't finishing before the code gets back to the current block

and the return value isn't ready yet which is crashing editor

[rustic rain]

why not use Addressables?

or is that some kind of integration with dots?

[solemn hollow]

oh man creating a blobasset for a complex nested datastructure feels so bad.

arrays of arrays are a pain to get out of those blobs too. i cant just copy out the array of arrays because those blobarrays inside can only be inside a blob :S

as far as i can see i have to recreate the whole structure at runtime again to actually work with it. seems like singleton entity would have been much better here. but then again nested arrays are a problem...

[devout prairie]

wondering is there any advantage of a blob over just some native arrays/hashmaps/whatever as members of a SystemBase?

[rustic rain]

can be stored unmanaged

[solemn hollow]

i use blobs so i can create that datastructure at conversion when the game is built.

[karmic basin]

Can't you flatten your data ? Like you would for a relational database vs NoSQL database ?

I mean from relational to NoSQL

Kinda like working with hashmaps everywhere and using IDs to access data

[solemn hollow]

i have no clue about databases so there might be an easy way i dont see. i got something like a List inside a List (flattend matrix) inside a List. each list has diffrent data inside.

since blobs dont support hashmaps id have no idea how to do it.

[karmic basin]

Oh yeah I don't remember, can't go more complex than a blob array ?

[solemn hollow]

yes last time i checked array was the only available option

[honest plinth]

Can you not do one global array per type you need to provide data for, and let the content reference each other by index/count? It's not the nicest or safest way but it can help flattening the data a lot

Disclaimer, I know next to nothing about blob assets but that is how I have encountered things like this being solved previously. Don't serialize deep hierarchies. Try to do one layer only and indirect using indices

[timber ivy]

Not a fan of them at all.

I have more control over versioning etc this way

[solemn hollow]

hmm if i understand correctly id still have a List inside a List though.
a list of type1 still needs a list of references of type2. a list of type2 needs a list of references to type3. So while it's flattened its not by much and it sounds like more work. this might become much more useful for deeper hierarchies than 3 though?

[honest plinth]

Something like this was what I was thinking about. But writing code that accesses the data is a bit more messy since you need to keep the reference to container around and access things in the correct array.
You can potentially write some sugar around it though, or translate to a different data structure when loading (at the cost of CPU time of course)

A big upside by storing things like this (AOS style) is serialization does not have to work as hard. Most of the things becomes just plain data that needs no patching

Well, Container should have T1, T2 and T3 in the list of course (stupid typo)

[solemn hollow]

thank you! ill play around with it. im not sure it makes my case much easier but its worth a try.

[honest plinth]

Yeah it's just one more tool 🙂

[solemn hollow]

more tools was exactly what i was looking for 🙂

[north bay]

I use some nested blob arrays, no more than 2 levels tho'
Why would you have to recreate the structure to work with it?

You can always use Unsafe* collections and embrace the segfaults 😍

[winged bane]

You can used a FixedListByte if your nested list object count is relatively stable or small

[solemn hollow]

i realize this is really hard to understand but i am not able to explain my problem any better :

i got a list of matrices where each entry in that matrix is a list again.
its a datastructure that contains all rules for a WaveFunctionCollapse. now at runtime each tile needs to get out multiple matrices based on which possibilities that tile still has. each list in each slot in each matrix is intersected with one another to build 1 matrix that contains the combined rules that are needed to propagate to all neighbors.
So all i want to do is build the same datastructure(that 1 matrix that is an intersection of all Matrices in the blob) but not as a blob. To do that i need the same structure but replace Blobarrays with arrays. and that means i cannot just say blobmatrix = runtimeMatrix but instead have to build that runtimeMatrix from the ground up 😦

[solemn hollow]

can i somehow do sth like this : NativeParallelHashMap<Entity, BlobArray> ?
when trying to add something to that HashMap i get the error that the blobarray can only live inside of BlobStorage.
Access it by (non-readonly) ref instead: ref BlobCollapseKernel yourVariable = ref ...
I do access it by ref though.

[safe lintel]

cant you make a blobasset for each blobarray?

[viral sonnet]

you could cast tge blobarray manually to a nativearray

blobarray is just ptr and length

[solemn hollow]

hmm yes that could work. storing blobassetrefs inside blobassets is legal right?

[solemn hollow]

i dont know how that works.
NativeArray<CollapseKernel> collapseKernels = (NativeArray<CollapseKernel>)wfcData.WFCSettingsRef.Value.collapseKernels; is all i know about casting lol. do i need to do some unsafe stuff?

[viral sonnet]

var ptr = blobArray.GetUnsafePtr();
var length = blobArray.Length;
NativeArrayUnsafeUtility.ConvertExistingDataToNativeArray<T>(ptr, length, Allocator.None);```

[solemn hollow]

omg thanks. i feel very unsafe now but that should work without much restructuring 😄 i wont be able to make it work today anymore. ill probably need to ask follow up questions tomorrow.

[viral sonnet]

sooner or later you'll be using unsafe stuff 🙂 but it's actually very safe, unless the blob is disposed

[solemn hollow]

its time i learn how to handle unsafe stuff. i just dont know where to start.

i can cast lists to unsafe lists in the same way right?

[viral sonnet]

yep

[solemn hollow]

all my other lists inside this nativearray need to be unsafelists now

[viral sonnet]

pointers are so powerful, lots of crazy stuff can be done that's otherwise not possible

[solemn hollow]

yeah like shooting my foot off

[viral sonnet]

haha, that's bound to happen 😄

[solemn hollow]

ill try to stay in safe land anyways but for static data i guess ill somehow manage to work with pointers

[viral sonnet]

the more comfortable you get, the easier it will be. i learned pointers and advanced stuff in c++ but it was a long time ago. when i started to use them in advanced ways again i had a lot of crashes. good thing i got that under control again 😄 now i have a funky setup with ** that nearly fried my brain but then worked on the first compile

yeah, only use them when necessary. although i must admit, most of my jobs are unsafe now because i need some feature from pointers

like the manual component type version changing

[solemn hollow]

i feel like i havent seen any non unsafe code from you ^^

since its been a while since you learned it you probably dont have any good resources for learning unsafe c# ?

[viral sonnet]

i actually never used or required pointers or unsafe space in c# till dots. most if the stuff i knkw is still from the c++ era and it's nearly the same. so any good resource on c++ pointers shpuld help. just avoid the stl library stuff

[solemn hollow]

yah im just not used to read c++ . makes is a little more cumbersome

hmm how to cast from blobarray to unsafelist. i dont know how to navigate the unsafe apis yet.

i cant find the equivalent of "NativeArrayUnsafeUtility.ConvertExistingDataToNativeArray" for unsafe list

[coarse turtle]

For an unsafe list you can pass the root pointer to a constructor

[solemn hollow]

                var length = blobTiles.tiles.Length;
                var tiles = new UnsafeList<Entity>(ptr, length);```

something like this but not with void ptr

[solemn hollow]

im so new to pointers i dont know how to have a entity pointer

[coarse turtle]

I imagine
var ptr = blobTiles.tiles.GetUnsafePtr(); stores Entity?

[solemn hollow]

rider says its a void*

[coarse turtle]

sorry - tiles not the GetUnsafePtr()

cause tiles is just a collection of something

[solemn hollow]

tiles is a BlobArray<Entity>

[coarse turtle]

cool

You can just cast var ptr to be Entity* entityPtr = (Entity*)ptr;

[solemn hollow]

ah its like a normal type?

[coarse turtle]

Yea pretty much

the type will define how much the address should increment by if you want to get to the next address. So if you want to get the address of the next entity it would be like this - Entity* nextEntity = entityPtr++;

[solemn hollow]

i see. so void pointers are the ones that reference the head of a object and iteration is then done with typed pointers?

[viral sonnet]

yep, with a void* the type is undefined so you'd have to increment by the actual bytes the struct has. like in an iterator var element = ptr + i * UnsafeUtility.SizeOf<Entity>();

avoid using void* and cast it like itsjustblank said. makes your life easier 🙂

[solemn hollow]

alright easy enough! will just take a while to get to know all those unsafe apis

and when deallocating memory i manually have to make sure all pointers to it are somehow cleaned up too right?

[coarse turtle]

if you have a struct/class containing a pointer that you freed up, then yes you want to make sure that also gets properly cleaned up too

[solemn hollow]

but casting to a unsafe list just requires the disposal of the list right?

i get this error : ArgumentException: Allocator Unity.Collections.AllocatorManager+AllocatorHandle must not be None or Invalid
for that line : var tiles = new UnsafeList<Entity>(ptr, length);

the constructor takes no allocator :S

[coarse turtle]

ah yea - because the UnsafeList is pretty much an alias to ptr so - the constructor doesn't actually allocate anything new - it's just holding the ptr

It's kind of like if you do NativeArrayUnsafeUtility.ConvertExistingDataToNativeArray - there's no AllocatorHandle provided since the NativeArray is just an alias to already existing data

you wouldn't need to dispose the UnsafeList/NativeArray, but you would need to manage ptr since their implementation of Dispose requires a valid AllocatorHandle

[solemn hollow]

hmm but why does the compiler complain then. the blobarray is allocated. im casting that blobarray to a unsafe list. where is the mistake?

[coarse turtle]

🤔 maybe the safety system is complaining about that, fwiw this is all new UnsafeList<T>(T* ptr, int length) does

    public unsafe UnsafeList(T* ptr, int length)
    {
        this = default(UnsafeList<T>);
        Ptr = ptr;
        m_length = length;
        m_capacity = 0;
        Allocator = AllocatorManager.None;
    }

i figure it would complain in a job more so than during the constructor call there

[solemn hollow]

ah im doing it inside a job

[coarse turtle]

Are you calling the constructor in the job or is the UnsafeList<T> passed to a member field?

[solemn hollow]

i call it inside the job. but i might as well convert the blobasset to a nested struct with unsafe lists outside of the job once and pass that in. would be better anyways. i just wanted to test around with what i had cause you guys where helping me right now. guess i have to restructure it tomorrow.

[coarse turtle]

ah yea, might just be the safety system then (and sorry I didn't catch the conversion up there so I have little context 😅 )

[viral sonnet]

that's an odd error msg

[solemn hollow]

i find it funny that the first thing i experience with unsafe code is problems with the safety system 😄

[viral sonnet]

the allocator must be None in that struct

and the safety system usually runs on parameters from the job

is it a private variable?

[solemn hollow]

no but i did this : ref var blobTiles = ref blobKernel.kernelSlots[slotIndex]; Entity* ptr = (Entity*) blobTiles.tiles.GetUnsafePtr(); var length = blobTiles.tiles.Length; var tiles = new UnsafeList<Entity>(ptr, length);

first time i used that ref var cause the compiler complained without it

might that be the problem? ill get rid of it

[viral sonnet]

the ref can't interfere

that is pretty dangerous btw. why a list? does it need to resize? you'll overwrite data in the blob

[solemn hollow]

oh damn. i only wanted to read from it. i just did it to make the code more readable. so by casting it to a unsafe list im actually writing to the blob... i need to copy the list in the blob to a new unsafe list and work with that.

see i already lost a foot

[viral sonnet]

if you don't do anything nothing will happen. just when you add you'd corrupt the blob memory

actually, you don't even need an unsafelist. you just want to store the entry into a nhm right?

[solemn hollow]

sadly i got a NHM with key entity and value unsafelist<kernel> containing another unsafelist<entity>.

i need to sleep over all the new information i got from you guys. tomorrow ill now my problem better xD

goodnight! thanks alot

[viral sonnet]

have a good night o/

[viral sonnet]

i'm writing a performance test and i need to single call an ISystem. public void OnUpdate(ref SystemState state) no idea how to get the SystemState though

[coarse turtle]

maybe through WorldUnmanaged 🤔

[viral sonnet]

yeah, just found out that the SystemRef<T> has another update

what's going on here ... no errors. hm

the min value is what i get in-game. in the performance test there are HUGE spikes

this is less useful than i hoped :/

[rotund token]

That is some big variation

Got something running in background?

[viral sonnet]

not really something that it could spike this hard. and it's not spiking in any capacity when i run the game

i can run a bunch of tests and it's always spiking in the same manner

actually, i just realized the server from work was still running. still the same results. hm

wasn't doing anything

i've debugged the editor now to see the timeline. sometimes the job just take x10 of its normal time

haha, found the problem. the job was writing to a list and that list is only cleared by another system in runtime. now that i run the clear, everything's fine! yay!

same numbers as in-game now

[supple raft]

Hello. I'm trying to spawn foliage in my world with positions stored in a collection. but my first error that gets displayed is:** 'FoliageManager' is missing the class attribute 'ExtensionOfNativeClass'!** I'm a bit new to ecs and dots. Any thoughts? Am I even on the right track?

using UnityEngine;
using Unity.Entities;
using Unity.Transforms;

public partial class FoliageManager : SystemBase {
    Entity foliagePrefab;
    protected override void OnStartRunning() {
        foliagePrefab = GetSingleton<FoliageTag>().Value;
    }

    protected override void OnUpdate() {
        if (GameManager.instance.foilagePositionQueue.Count != 0) {
            Vector3 position = GameManager.instance.foliagePositionQueue.Dequeue();
            var foliage = EntityManager.Instantiate(foliagePrefab);
            var newTranslation = new Translation{ Value = position };
            EntityManager.SetComponentData(foliage, newTranslation);
        }
    }
}

Thank you for your time😊

[viral sonnet]

hm, only some leads. have you these all enabled?

[supple raft]

Neither of them

[viral sonnet]

you have to enable them with static singletons like the GameManager.

at least the domain

so good opportunity to get rid if it actually 🙂 there are a bunch of ways to create that queue without needing a singleton

you can move the queue to the FoliageManager. accessing it from there is even easier. getting a SystemBase is just a call to World.GetExisistingSystem<FoliageManager>()

[supple raft]

Hmm. The change didn't seem to make any difference.

[supple raft]

Awesome ill try that

[viral sonnet]

hm, i don't see anything wrong in the code that would lead to an error. tbh, i never had that one. it's an odd one for sure.

[rotund token]

is missing the class attribute 'ExtensionOfNativeClass'

we've seen this randomly

never found the cause

it doesn't seem to cause any issues though

[supple raft]

yeah. i tried some searching on the web but it didn't lead to anything. The error seemed to be older that dots so stange

I still get the same errors now when i have moved the queue to the FoliageManager System and accessing it like this @viral sonnet

World.DefaultGameObjectInjectionWorld.GetExistingSystem<FoliageManager>().foliagePositionQueue.Enqueue("The Position")

[viral sonnet]

i don't think this error has to do anything with the code. try to restart unity and see if it goes away. moving the queue is more of a suggestion to get better system code

[supple raft]

okay, thank you for the improvement!:)

[viral sonnet]

and as I said, keep in mind to enable domain reload whenever you are using statics

[supple raft]

I've turned those on;)

[viral sonnet]

if you can't get rid of them and still want to use singletons with fast reload (domain/scene reload disabled). this will help: https://github.com/joshcamas/unity-domain-reload-helper

[viral sonnet]

great, now i got the other test job ready and now i have to figure out why the test runs at 3.5ms and the in-game one with ~5ms 😄

huh, seems the big difference is, my test setup just writes to 1 parallel list because it has no idea about the threadIndex. iiiiinteresting. seems i lose a lot of read speed due to this

[viral sonnet]

the good thing, this doesn't affect performance. the bad thing, i have no idea where i lose 1.5ms. this whole testing setup is already bearing fruits, hehe. so much faster than to always run the game and look for timeline values 😄

i really wish for some better profiling. every burst job is like a blackbox.

[robust scaffold]

You and me both. I just look at the inspector outputs and guess based off the commands I see

[rotund token]

just need to get that superluminal license

[robust scaffold]

God, local to world is so big. But rolling my own local to world means rewriting 2d rendering

[viral sonnet]

can that profile the editor?

[robust scaffold]

One time 60 bucks. I like performance but I'm not that into performance.

[viral sonnet]

never used it QQ

i'd insta buy it if i can profile the editor 😄 and right now, for the live of me, i can't find out what's going on with this damn job that's running much slower in-game

[rotund token]

https://superluminal.eu/unity/
I believe it does
"built-in Unity support"

[robust scaffold]

Add profile markers to sections of the code. Then check the profiler to isolate the second of the code that is taking the longest

[rotund token]

Unity support
Unity applications make use of the Mono runtime to provide scripting capabilities through managed code (C#). Superluminal has full support for profiling Unity applications (both the Editor and Player builds) and will show you both managed and native code in all views, similar to when you're profiling any .NET application.

Supported versions
In order to resolve managed code, Superluminal requires functionality that was introduced in specific Unity versions. The list of supported Unity versions is as follows:

2019.4.23 and higher 2019 versions
2020.3.2f1 and higher 2020 versions
2021.2.1f1 and higher 2021 versions
2022.1.0a15 and higher 2022 versions

You'll still be able to profile any Unity version other than the ones described above, but in unsupported versions only native code will be resolved.

[robust scaffold]

Or that. Seems pretty nice.

[coarse turtle]

Ah i've mainly used superluminal in builds and found it hella easy to sample/profile

i haven't really looked at the editor integration 😅

[robust scaffold]

recreate arrays every frame using world allocator or cache the array in a persistent?

[viral sonnet]

persistent 🙂

in jobs caching them per-thread is nice too

thanks tertle, i'ved missed that paragraph. seems great then. instabuy 😄

[rotund token]

i look forward to the reprofiling and of your entire project

and the discussions around that

[viral sonnet]

i was using vtune before so i'm pretty interested about the results

[robust scaffold]

Huh, I didnt realize they made the NativeX containers burst compatible.

Yea, they're all structs. Wow, that's big.

The safety system is unmanaged. Why would anyone use UnsafeX then (outside of component)?

[rotund token]

what do you mean?

native containers have always been burst compatible

[robust scaffold]

Hrm... yea... I dont remember what I was thinking exactly.

Does anyone know if entity query's ToXArrayAsync guarantees corresponding order of entity and components?

wellll, i can just make a job that does it

[rotund token]

im not sure what the order will be (the arrays will match, index 1 will match the entity 1 etc)

but it will be deterministic

it'll be the same across every frame until structural change

and will match entityinqueryindex

[robust scaffold]

Hrm, alright.

[viral sonnet]

hm, seems i have the same problem in every profiler tool. some symbols can't be resolved and no or the wrong source code is shown -.-

[robust scaffold]

int indexOfFirstEntityInQuery from IJobEntityBatchWithIndex?

[robust scaffold]

You paid for it, thats where you start ringing up their support and demanding answers.

[rotund token]

entityinqueryindex = indexOfFirstEntityInQuery + index in chunk

[robust scaffold]

Of course, good good.

Unlike unity where they're not obligated to give a shit about whatever the free plebs say on the forums about DOTS.

[viral sonnet]

needs "Native Debug Mode Compilation". haven't used it very long now, seems to be pretty much the same tool as vtune/vs profiler. the experience is much better though and there's an assembly switch

one thing that sticks out, i know exactly which burst assembly instructions are called without scrolling endlessly 😄

[rotund token]

if you don't want to slow your entire project down you can do [BurstCompile(Debug = true)] to enable it only for specific jobs

just remember to get rid of it!

[viral sonnet]

thanks for reminding me, that could come in handy. 170ms per frame hehe

i habe to find out if the debug mode is actually needed. there is a burst temp folder with symbols but that wasn't working right

hm, most of the time is spent in burst.memset.inline.AVX2.i64

[robust scaffold]

memset should be really fast

[viral sonnet]

it is, doesn't seem i can do much here. it's just a return new struct

[viral sonnet]

looks like the debug assembly is not the same

the oddest thing, i have a call to a method which is just 1line and also marked with aggressive inlining. not visible in the burst inspector

[gusty comet]

Anyone know how I fix this Burst package error?

[rotund token]

seems like windows or another app (or a crashed app) has a file locked

[robust scaffold]

Hrm, all these systems are burst compiled but no green, unity fix

22 components on a single entity and I'm not even a quarter way through a rudimentary physics engine. I'm scrolling and scrolling and scrolling on the inspector to find the component I want. Unity plz

[rotund token]

just use the search bar?

sure as hell ain't searching through 70 components to find this ^_^'

also that seems like a lot of components

[drowsy pagoda]

how come NativeParallel hash maps cannot be deallocated via DeallocateOnJobCompletion attributes? Is this a limitation or a feature yet to be added?

[rotund token]

only native arrays can be

this has always been the case

i've been hoping they deprecate DeallocateOnJobCompletion for this in favour of dispose(handle) - it's inconsistent and magic

i reckon this only works on native arrays because native arrays are the only thing that exists in unityengine

everything else is provided by the collections package

[drowsy pagoda]

ah I see. I agree with inconsistency then.

[rustic rain]

I'd assume that you don't need to assign jobHandle for disposing to system's Dependency

[rustic rain]

Is there a way to get all components from 1 entity and copy them to other entity?

copy them to exact entity I mean

for example I have entity 10:1
And I have entity 11:1
I want to copy all components from 11:1 to 10:1, basically making a clone, with different entity value

is it possible?

[rotund token]

easy but not easily

but ah

can't you just use instantiate

then set the values

[rustic rain]

I don't know what components need to be copied

[rotund token]

or do you need to do this on an existing entity

well you said all components?

and instantiate would create an entity with all the same components

[rustic rain]

yeah, I need a literal clone

on exact entity reference

[rotund token]

that case you basically have to go deep

with chunk archetypes

dynamictypehandles

[rustic rain]

well, first thing - I need to set Archetype

[rotund token]

you can get all components

easy enough

[rustic rain]

then I need to do memcpy for component values?

[rotund token]

EntityManager.GetChunk(entity).Archetype.GetComponentTypes()

[rustic rain]

what if there are managed components involved?
For example RenderMesh

[rotund token]

but actually you can do it even easier

EntityManager.SetArchetype(entity2, archetype);```

[rustic rain]

oh, this is how you can get archetype easily?

just getting chunk?

[rotund token]

yes

[rustic rain]

huh

[rotund token]

archetype lives in chunk

so that will make 2 entities match same archetype

copying values is a little more effort

[rustic rain]

yeah, now values...

component types is not a problem from what I see

I mean

unmanaged types*

but managed

how are they even stored?

[rotund token]

in a List

    {
        readonly ManagedObjectClone m_ManagedObjectClone = new ManagedObjectClone();
        readonly ManagedObjectRemap m_ManagedObjectRemap = new ManagedObjectRemap();

        UnsafeMultiHashMap<int, int> m_HashLookup = new UnsafeMultiHashMap<int, int>(128, Allocator.Persistent);

        List<object> m_SharedComponentData = new List<object>();```

quite literally

List<object> m_SharedComponentData = new List<object>();

all shared components are just stored in a single list

[rustic rain]

shared is yeah

[rotund token]

oh managed components?

now you're getting really complicated

[rustic rain]

thing is

What I want

is to be able to call GetOrCreateSingleton in OnCreate in any system

and then once entity is actually loaded async

[rotund token]

they are again all stored in a list

[rustic rain]

it's assinged to that exact entity

[rotund token]

internal unsafe class ManagedComponentStore
{
List<object> m_SharedComponentData = new List<object>();
internal object[] m_ManagedComponentData = new object[64];

[rustic rain]

hm

I'd guess, I'll simple have to get index of component

and then just copy it

[rotund token]

that seems like a lot of effort for no reason?

[rustic rain]

it's not really no reason

it'll make system creation flow way easier

[rotund token]

i suspect you're solving an issue i've run into previously

[rustic rain]

and system itself will be fast

[rotund token]

where you want data a system needs that is stored in a subscene

but you dont want to add the dependency to every system

[rustic rain]

as no getting entity reference will be called during runtime

[rotund token]

but there's 2 easy ways to solve this

[rustic rain]

hm, not really

it's exactly what I menioned: some singletons I want to be loaded by conversion

not by runtime creation

[rotund token]

either just don't update the world until the subscene/s (or async data) are loaded

or just don't create the systems until the data is loaded

create world
load data
create systems

[rustic rain]

not creating systems is somewhat sketchy...

[rotund token]

this pattern is very specifically supported

you can even use a separate world

[rustic rain]

I still haven't figured how it's done, hehe

[rotund token]

CreateLoadingWorld
AddSubScene systems to LoadingWorld
Load SubScenes
Wait for SubScenes to load

Create your world
CopyEntities from loading world
CreateSystems
or
Create your world
AddSubScene systems to your world
Load SubScenes
Wait for SubScenes to load
Add rest of systems

[pulsar jay]

Do you guys use Assemblies? How do you handle Group sorting with Assemblies? I figured they cause a lot of unnecessary dependencies because you have to reference an assembly just to tell your assembly to run after this assemblies group or system

[rustic rain]

a tiny bit

I put them in places, where everything is enclosed

so other parts of game reference this assembly

makes compilation faster

[pulsar jay]

at least sometimes 😏

[rustic rain]

always, if you do it correctly

if instead you just makes dozens of assemblies that all reference each other

then you just make compilation slower as it will have to additionally resolve dependencies

[pulsar jay]

there are quite a few forum post complaining about them compiling even slower

[rustic rain]

sir, you can test it yourself

compile a dll

import it

and see how fast will your project compile with it

compared to having all sources

[pulsar jay]

yeah pbolem is Unity has to determine what needs to be recompiled which takes time and sometimes it fails to do so correctly

[rustic rain]

that's why I mentioned

that it has to be correctly

[pulsar jay]

for me it did not get noticably faster yet

but thats not the point

[rustic rain]

you can't just declare dozens of assemblies that all reference each other and call it a day

[pulsar jay]

I need them for testability

[rustic rain]

it's supposed to be a some kind of hierarchy that does not rely on each other

[pulsar jay]

the problem is as soon as you start using system groups you get additional dependencies

[rustic rain]

not really

you just declare groups as seprate assembly

🙂

and then all other assmblies that rely on it can reference it

[pulsar jay]

thats the kind of solution I was looking for

[rustic rain]

it's really hard to refactor your project into this kind of thing

way better if you just use it as pattern beforehand

[pulsar jay]

although it isnt perfect. If I want to extract one of the assemblies as a package it would always have a dependency on the project wide Groups assembly 🤔

[rustic rain]

yeah, managing it is kind of annoying

[pulsar jay]

I know. But now its too late 😬

[rustic rain]

never too late

especially if you're in ECS zone

[pulsar jay]

it would make a lot of sense to declare each group in its assembly and the have another assembly reference and sort all the groups

Like this:
MovementAssembly -> MovementGroup
AnimationAssembly -> AnimationGroup
GroupSortingAssembly -> Sort AnimationGroup to run after MovementGroup

maybe this might be possible using partial System classes 🤔

[rustic rain]

well kind of

welp, I just created a large assembly, kek

didn't realise it was ready to be separated as assembly

until this convo

[pulsar jay]

yeah the problem is the last step with GroupSortingAssembly might not be possible

afaik it only works with attributes or a complete custom bootstrap

[rustic rain]

hmmmm

I am trying to figure out a way to implementing debugging tools

in top down view game

for example, you click tool and you can remove entity once you clicked it

with that tool

I want a way so I and modders can easily create their own debugging tools too, by simply inheriting from some interface or class

biggest challenge, it has to be async (some things are done in before simulation, some are during)

[misty wedge]

For me the majority of time seems to be spent on assembly reloading and importing assets, or I'm measuring / doing something wrong

Although I never measured before splitting everything into assemblies

[rustic rain]

well, compilation time is kind of critical in Editor

[misty wedge]

?

[rustic rain]

imagine if your editor compiles assembly on every change for 30 seconds 😅

[misty wedge]

You don't say

I'd love it to be faster too, but it takes at least like 60 seconds each time

I have no idea why

[rustic rain]

oh

xD

Idk, my project is relatively young

compilation is fast for me

but I also use assembly defs as much as I can

[misty wedge]

I split everything into assemblies best I can so I don't know what else I can improve

[rustic rain]

takes about 11 seconds from ctrl + r til Editor is ready

[misty wedge]

The dream

[rustic rain]

this is when ECS dzen hit me kek

[misty wedge]

Like I said, it doesn't seem to be the actual compilation for me, since ``CompilationPipeline` callbacks have pretty low numbers

[rustic rain]

I realized that I can make systems and components in separate assemblies...

[misty wedge]

I mean you can do that with OOP code too

[rustic rain]

like fully independent from anything else

and whenver you want to react with that system

you just reference it's assembly and manipulate it's component

OR

you can even go deeper

and declare "neutral" assembly for components

on which both new and old systems will depend

key point

unless you change this neutral one

only new assemblies will be recompiled

[misty wedge]

Well yeah, that's the entire point

Which is why a lot of new stuff from the asset store switched to assembly definitions, their code rarely changes

[rustic rain]

so far I did it with Utility, Input and some UI code

[misty wedge]

I did it with everything, but reloading still takes ages

each folder is a separate assembly

[rustic rain]

that's not the goal

goal is to make assemblies not depend on each other

[misty wedge]

They depend on what they need to depend

[rustic rain]

I doubt if you split literally everything into assemblies

you get that

you probably have a large net of interconnections

[rustic rain]

yeah, and it causes sync point

[misty wedge]

?

[rustic rain]

where dependencies come to the point, everything has to be recompiled

[misty wedge]

Did you look at the screenshot above?

Script compilation takes 2 milliseconds

According to unity

[rustic rain]

says 67 seconds to me, kek

69*

[misty wedge]

That's the total amount of time until reload

Includes importing assets and assembly reloading

There's probably a way to debug what's taking so long, but I'm not sure how

[rustic rain]

hmm

I don't think assets are reimported during reimport

unless they changed

are you on 2021 or 2020?

[misty wedge]

It literally says "Importing Assets" in the popup window

2021.3.4f1 I think

[rustic rain]

🤔

[misty wedge]

So pretty close to LTS

Maybe my PC is just too old 😢

Also not sure if this is normal:

[misty wedge]

Just for reference, this is what happens if I also hook into AssemblyReloadEvents

Not sure where the extra 50 seconds are coming from before the editor is usable again though 🤷

I guess it's the editor reloading or something

[misty wedge]

Welp, not much I can do about this I'm guessing:

[robust scaffold]

Jesus. How large is your project? I would give up if I had to wait half a minute every time I changed a bit of code.

[rotund token]

Project at work takes me 30s seconds

Takes some people over a minute

[robust scaffold]

I can see how reducing iteration time is #1 priority for Unity. Because that is unusable.

[misty wedge]

I'm not really sure what the issue is, the project isn't very large tbh

[robust scaffold]

Maybe try splitting up your project into even more assembly references?

[misty wedge]

It already uses like 15

I split all logical pieces of code into assemblies, like in the screenshot I posted earlier

The issue isn't compilation, that takes 2 ms according to unity

[robust scaffold]

Have sections of code that are completed pre-compiled and add them as DLLs. Use extern hooks to get API?

Turn off domain reload? You have static variables?

[misty wedge]

I posed a screenshot earlier that showed that an [OnInitializeLoad] in burst takes 20 seconds for some reason, no idea whats going on there

[misty wedge]

Barely any, I could refactor those away. But it wouldn't save me time on code changes, that requires a domain reload regardless afaik

I didn't know you could natively debug domain reloads so extensively in unity, I only looked into it earlier. Ill try and examine the pieces a bit more to see whats going on

[robust scaffold]

https://docs.unity3d.com/Manual/DomainReloading.html

If you have no events and no static variables, you're safe to turn it off in editor.

With DOTS, that should be the case.

[misty wedge]

The domain reloading on code changes? I thought it's only for entering play mode

[robust scaffold]

Dont know. Try it?

[viral sonnet]

hm good question. i actually dont know if it also affects recompile time

[misty wedge]

I didn't think it does

🤷

Anyways that hook in burst code taking 20 seconds is really strange to me

[viral sonnet]

but yeah 30-60 seconds would make me tilt. especially with the auto save from rider. lol

[misty wedge]

You can turn the auto save off btw

[viral sonnet]

i know, kind of handy though. can switch to unity and it's already done :)

[misty wedge]

😢

[viral sonnet]

but there must be some dependency chain thats triggering. otherwise i wouldn't know why it takes so long

[misty wedge]

Ill post my entire log, sec

https://hastebin.com/keximequfa.yaml

Haven't looked at it yet, besides the 20 seconds burst thing

[rustic rain]

Dependency chain yeah

Can't you actually reduce amount of assemblies even more?

[misty wedge]

Where do you see that that's the issue?

[robust scaffold]

Unity.Burst.Editor.BurstLoader, Unity.Burst, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null (total: 19132ms, avg: 19132ms, min: 19132ms, max: 19132ms, count: 1)
Hrm. Yea. Burst is a problem. Quick google search doesnt lead to anything concrete.

[rustic rain]

I noticed that during reload it mentions a lot of your own assemblies that aren't supposedly related to each other

But I can't be certain on that

[misty wedge]

Well do they take up a significant portion of the compilation time?

[rustic rain]

The fact that they are even reloaded concerns me

[misty wedge]

Its probably a dependency of the assembly that I changed code in

[rotund token]

Auto refresh is turned off in unity

Nothing worse than tabbing in to read a config and having to wait for a recompile

Pretty much my entire team has turned off auto refresh at this point

[misty wedge]

How do you trigger a refresh then?

[rustic rain]

How to turn it off?

[misty wedge]

Never turned it off

It's in the editor settings iirc

[rustic rain]

Ctrl r

[misty wedge]

Ah

[rotund token]

What he said

[misty wedge]

Does it do an automatic reload if you enter play mode?

[rotund token]

It's just in preferences

[rotund token]

No

[rustic rain]

Huh, never found it

[rotund token]

Take a few days to get used to forgetting to refresh and wondering why you're changes haven't been made

But you get used to it

[rustic rain]

Does it affect Ui reloads? Caused by uxml edits?

[rotund token]

All it stops is asset reloads from external sources

If you make a change in editor it will be normal

[rustic rain]

Ah

[misty wedge]

Yeah but the long reload times are really bumming me out, especially since the project isn't that big

very annoying

[rotund token]

I'm used to it but yeah project is big

That said its not really scripts for us either

Takes 0.5-8s to compile the actual scripts and we have very poor assembly definition usage as project predates them

[rustic rain]

Unity says it was their focus in 2022

To improve loading speed

[safe lintel]

do assembly defs actually improve compile time?

[rustic rain]

1.0 should bring a lot of goods

[rotund token]

It's more they make it so you don't have to compile stuff

In my own project I have a lot of asmdef and it usually only compiles 2-4 of the assemblies on code change

[misty wedge]

It takes 2 seconds to compile for me, and then 70 seconds to reload the editor

[rustic rain]

Do you have burst sync compilation ticked?

[misty wedge]

Nope

[rotund token]

I suggest you bring all your code but nothing else to an empty project

And see how long it takes

[misty wedge]

To load a change?

[rotund token]

Yeah make a change

Compare

[rustic rain]

I wonder how assets can even affect that

[misty wedge]

I don't really have a ton of assets either

But It's a good idea anyways, I'll try it out, thanks

[rustic rain]

I'm way too used to dynamic assets loading from scripts

[misty wedge]

Unity still runs the asset importer

[rotund token]

It sounds like slow domain reload or some script triggering asset imports

[robust scaffold]

Imagine having assets. I believe in the square white rectangle life.

[rustic rain]

It is

[misty wedge]

This is in an empty project, just code

still takes 54 seconds until the editor becomes usable again

[robust scaffold]

Why did the compilation take longer?

Didnt it only take 4 seconds?

[misty wedge]

The text is badly named, the last number is the time it takes until the editor is usable again

It checks EditorApplication.isCompiling

As you can see the actual compilation time was 0 ms

Schnozzle.Pathfinding was the assembly I made a change in

Nothing references it so it only rebuilt Assembly-CSharp

[robust scaffold]

Have you been using:

[misty wedge]

I did earlier

[misty wedge]

^

[robust scaffold]

Burst still taking forever with this empty project?

[misty wedge]

^

I haven't checked that yet

But I assume that's part of it

[drowsy pagoda]

Is the following a valid approach: First job is a parallel scheduled job that uses ecb.pw to instantiate some entities. Second job is a single scheduled job that depends on First job. Can the second job, GetBufferFromEntity<T>(false) and use the deferred entities created in first job's ecb.pw to access those buffers provided that the command buffer system adds second job as dependency?

[robust scaffold]

No. Not unless you manually rewind (replay?) the ECB and insert a sync point.

Use a NativeMultiHashMap instead and then create the entities in the second job after you generate all the data.

[drowsy pagoda]

I see. Thanks for that.

[rustic rain]

So it is related to compilation after all

[misty wedge]

How do you guess that

[rustic rain]

I mean, if it's only code

What else can it. 3

Be*

[misty wedge]

I don't know, but unity is telling me compilation takes like 2 seconds

Not 54

[rustic rain]

So

Can you try and remove all assembly defs

But actually pure

That only rely on unity code

So you'll be left with like a couple of defs

[rotund token]

It's not going to be the defs

I have way more

[rustic rain]

But only ones that are certain not to be reloaded

[rotund token]

And it takes me 2-5s to reload

[rustic rain]

But your defs might be less interconnected between each other

[rotund token]

No they're not

My asmdef builder auto adds everything

[rustic rain]

Well we wouldn't know

How different is it anyway

[rotund token]

What version of burst are you using

[misty wedge]

1.7.3 I believe

[rotund token]

Hmm bugger that was meant to speed this up

[rotund token]

I reckon just remove 1 at a time

See what one is causing issues

[rustic rain]

Goal is to have independent assemblies anyway

What's the point to have assembly for every folder if each has to reference every other folder?

[misty wedge]

Bruh

They don't

They are separate systems

Some reference parts

Others don't

[rotund token]

Remember that's not possible

[misty wedge]

^

[rotund token]

You can't have circular dependencies

[coarse turtle]

well eventually iirc, Unity is on the path to migrate asmdefs to csprojs (once they reach .net6)

[misty wedge]

You can't reference circularly

[rustic rain]

Why not?

[rotund token]

Because assemblies can't reference each other

[rustic rain]

Can't it be done through a 3rd assembly?

[rotund token]

No

That'd still be circular

It has to be a hierarchy

A ref b ref c ref a is not allowed

[misty wedge]

Why is why they're such a nightmare to set up on established projects

[rotund token]

Yep

But enforces a great structure on new ones

[misty wedge]

Yeah, I really like that tbh

forces you to think about it more

[misty wedge]

Removing them made no difference in reload time

Since the majority of it is domain reloading

[rotund token]

Have you turned on burst timings?

Since that was 20s right?

[misty wedge]

Ok so it seems a significant portion of it was debug mode, I didn't know it tanked iteration time that badly. It's still kind of high though, maybe around 25 seconds until the editor becomes usable, 13 seconds until compilation and domain reload finish

https://hastebin.com/evokumatuw.yaml

[misty wedge]

I've never used burst timings I don't think

[drowsy pagoda]

I notice that Debug.Log is supported in jobs. Is that specifically to develop mode or will this work in production release mode?

[rotund token]

works in release

[misty wedge]

It's now down to like 4 seconds, not sure if that's normal

[misty wedge]

Yeah, I meant it's off, I've never turned it on 😅

[rotund token]

it will tell you exactly what is slow in burst

[misty wedge]

Where does it show this information? The profiler?

[rotund token]

just the log

[misty wedge]

What does it show, the milliseconds each burst function took that frame?

[rotund token]

it shows how long each job took for burst to compile

[misty wedge]

Ah

[rotund token]

 - All                                      :    4078ms
   - Discover Methods to Compile            :      40ms
     - Calculate Hash of C# IL              :      40ms
   - C# IL -> LLVM IR module                :    2637ms
     - Parse C# IL from assemblies          :     585ms
     - Transform C# IL                      :     148ms
     - Produce LLVM IR module               :    1904ms
   - LLVM IR module transformations         :     845ms
     - Add target CPU info to module        :       0ms
     - Optimizations on module              :     831ms
     - Cleanup module                       :       0ms
     - Verify module is valid               :      13ms
   - Extract used static readonly variables :       1ms
   - LLVM IR module -> DLL                  :     534ms
     - Splice into per-entry-point modules  :      27ms
     - LLVM IR modules -> object files      :     429ms
     - Get objects from object cache        :       0ms
     - Link object files -> DLL             :      59ms
     - Add objects to object cache          :      12ms
   - Dump                                   :       0ms

Counters:
- Number of Functions                      :       248
  - Entry Points (job or function pointer) :         3
  - Declaring Type or Function is Generic  :       102
- Number of Instructions                   :      9852```

and what part is slow

it's very interesting comparing this to previous burst versions and seeing how much improvement they've actually made

[misty wedge]

https://hastebin.com/ujipohapoz.yaml

So I'm guessing this whole part takes ~5x as long if you are in debug mode

Since it was at 20s before

[rotund token]

remember these are the burst compiling

so they are per thread

and they are async

if you really want to get deep burst debugging info

add this hidden debug option
UNITY_BURST_DEBUG=1
to your startup

[misty wedge]

3.5s seems fine I guess

The domain reload / editor reload is what's bothering me

[rotund token]

yeah i was hoping this would dump info about that init

but it doesn't

i think you'd need to use the hidden debug above

which will basically make burst compiler dump a lot of info in the Logs folder

it's probably not that useful for you, but if you wanted to get it fixed you can shoot it over to the burst team to look at

[misty wedge]

I'm also starting to think unity has a memory leak on linux

It starts pretty low and goes up to like 50% of my RAM at 32 gb

[rotund token]

i am finding that i can easily push 10-15gb of unity memory usage in my work project

though i don't really have the same issue in my personal project

but they are different unity versions (2020.3 vs 2021.3)

[robust scaffold]

Jetbrains is swallowing 6GB of memory. 3x more than unity in my case.

[misty wedge]

yeah rider also uses a huge amount on my machine

but unity uses more

sometimes unity will crash and my memory usage will go from 95% to sub 50

[robust scaffold]

Hrm, restarted rider and the memory usage halved.

[misty wedge]

I also feel like unity crashes more on linux, but it crashed a lot on windows too for me, so I'm not sure if it's just a feeling

[robust scaffold]

Moving to 2021, I actually dont experience that many "unexpected" crashes.

[misty wedge]

What's an expected crash? 😅

[robust scaffold]

Expected as in me fucking around with graphics draw calls stalls the GPU and forces unity to restart. But thats my fault.

[misty wedge]

Ah

[rotund token]

oh yeah i get a lot of expected crashes 😄

[misty wedge]

Unity seems a lot more unstable when I'm in debug mode

It also uses vastly more memory

[rotund token]

i dont get many unexpected crashes from unity these days

at least not compared to a year ago

[robust scaffold]

But actual unity crashes? I dont remember the last one where unity just up and died on me. I always post them here to complain and I havent once since 2021 and 0.50.

0.17 was quite buggy on that front to be fair.

[misty wedge]

The odd thing is when unity crashes for me on linux there's nothing really interesting in the editor log

[robust scaffold]

Using an IDE debugger? Yea, I never turn those on unless I'm really desperate to figure out whats going on.

Usually, debug.log is everything I need. Or not available at all given shaders.

[misty wedge]

I'm still contemplating whether I should just develop on a VM

[robust scaffold]

Or buy a windows?

[misty wedge]

I hate dual booting

[robust scaffold]

We're programming in C#. Gates demands sacrifice.

[misty wedge]

C# works on linux 😅

also probably what saved C# in part, nobody wants to run ASP on a windows server

[robust scaffold]

Also @rotund token do you have any stats on using ToComponentDataArrayAsync vs just a custom IJobEntityBatch writing to NativeArrays?

I got 4 and possibly more as I flesh this system out and the job handles are kinda getting long.

[rotund token]

it's just done in a IJobEntityBatch

    unsafe struct GatherComponentDataJob : IJobEntityBatchWithIndex
    {
        [NativeDisableUnsafePtrRestriction] public byte* ComponentData;
        public int TypeIndex;

        public void Execute(ArchetypeChunk batchInChunk, int batchIndex, int indexOfFirstEntityInQuery)
        {
            var archetype = batchInChunk.Archetype.Archetype;
            var indexInTypeArray = ChunkDataUtility.GetIndexInTypeArray(archetype, TypeIndex);
            var typeOffset = archetype->Offsets[indexInTypeArray];
            var typeSize = archetype->SizeOfs[indexInTypeArray];

            var src = batchInChunk.m_Chunk->Buffer + typeOffset;
            var dst = ComponentData + (indexOfFirstEntityInQuery * typeSize);
            var copySize = typeSize * batchInChunk.Count;

            UnsafeUtility.MemCpy(dst, src, copySize);
        }
    }```

[robust scaffold]

Huh. Might as well implement my own then. Alright.

I've gotta say, while Unity cant change anything about C#, their struct based job system makes the update method look pretty smooth:

[misty wedge]

What would you like them to change?

[robust scaffold]

state.WorldUnmanaged.UpdateAllocator.AllocateNativeArray<> is so long. They should make a method in root SystemState that has AllocateNativeArray(). yes, I know I can have my own local ref variable to do it but still.

[rotund token]

just write your own extension method on systemstate?

[misty wedge]

I meant what would you like them to change about C# if they could

[rotund token]

(but yeah that line length is hilariously long)

[robust scaffold]

some inline operator instead of ComponentType.ReadOnly<>() for designating or flagging a type.

Might find 0 use outside of DOTS but it's so painful. Compared to typeof() which is nice and short.

[misty wedge]

How does allocation work for SystemState? Can't you do new NativeArray()?

[rotund token]

UpdateAllocator is meant to replace TempJob

it doesn't need disposing (auto every 2 frames since there are 2 and they alternate) and most of the time uses existing memory instead of allocating new

[robust scaffold]

The 4 NAs in that screenshot is using UpdateAllocator. An "automatic" way to dispose of arrays every frame.

[misty wedge]

I see

What happens if you request a native collection with TempJob?

[robust scaffold]

Same as usual old way

[safe lintel]

whats that image from? training samples repo?

[robust scaffold]

My code.

Custom 2D physics. Slow progress.

That's the AABB separating axis check (simple broadspace).

[safe lintel]

all that state.WorldUnmanaged.UpdateAllocator.etc is the official api?

[robust scaffold]

Yep. It's the new in system automatic allocator.

In SystemBase, it's World.UpdateAllocator

Unmanaged for burst compilation. Since update in ISystem is completely bursted.

[safe lintel]

so they havent gotten rid of their love of gratuitously long api names

[robust scaffold]

You can simplify it, it's a ref return property. But it's still megalong.

[misty wedge]

What's the advantage over allocating tempjob inside a job? Being able to use the collection outside of the job?

[robust scaffold]

Inside a job? You cant allocate TempJobs inside one. I think.

You can allocate Temps but those cant cross back into main thread.

[misty wedge]

I meant Temp, my bad

So it's just about the auto deallocation?

[robust scaffold]

Yea, these arrays are collecting the all entity's AABB and so forth for iteration in the next job.

Just the logic required.

[robust scaffold]

Kinda. Thats why I'm using UpdateAllocator. But I need the native arrays in the first place because the next job after needs to iterate through all AABBs for example, not just the ones inside the chunk. So a Native array needs to first gather all AABB components and then be passed into the Overlap job.

[robust scaffold]

First time using the UnsafeUtils for manual mem operations. This works right?

[viral sonnet]

yes but i don't think you need to add the index

[robust scaffold]

These out-arrays collect all component values across chunks so it needs to be shifted.

[viral sonnet]

i see

[robust scaffold]

Well, it seems to work. Nothing exploded.

[misty wedge]

What hardware are all of you running for your development machines?

Maybe I should upgrade

[robust scaffold]

Just a small laptop. 16GB ram

[misty wedge]

I really wonder how you get so little time spent on domain reloads

[robust scaffold]

My project just isnt that big

[misty wedge]

Mine isn't either 😅

[robust scaffold]

This is just physics but most of these are 10 line single struct IComponentDatas

[misty wedge]

I don't have huge components either

They're all pretty small since they usually just reference blobs

[robust scaffold]

600 colliders. O(n^2) brute force check in 0.14 ms. Dont know if thats good.

O(n^2) definitely isnt good but is it worth it to implement one of the O(n log n) spatial partition algos?

Also of note, a custom explicitly typed job to copy component data to a NativeArray is far superior to Unity's GetComponentDataArrayAsync. Theirs was multithreaded and all over the place. Singlethreaded with explicit typing results in literally 3ns operation. Little variation between frames.

[viral sonnet]

how long does it need multithreaded? the arrays that are copied to are not aliased so should be pretty good in parallel

might only show some gains in the 10k range though. and that aside, i also had some jobs that were superior in 1 thread and horrible MTed

[robust scaffold]

ughhhhhh, i have to implement spatial partitioning. Making a system halfassed is not what I do.

[balmy thistle]

Fascinating - is this in the context of async streaming, or a more level-oriented model

[robust scaffold]

Physics engine. 2D. I'm basically gonna re-invent / copy the DOTS physics engine as they have apparently an amazing spatial partitioning system.

I'm gonna copy it but with my own components. So slice off a float from every component.

Possibly be able to integrate the results with my lighting engine for small object shadow casting.

Oh my god, im just gonna be writing a Box2D. If unity ever realizes the existance of 2d, all my work will be completely wasted.

[balmy thistle]

I know the feeling, I spent an inordinate amount of time writing a 2d planar shader for a unity game in the 5.2 era

I think they have real 2d lighting now

[robust scaffold]

That I know for certain they dont have. So I'm quite happy about having to roll my own.

[balmy thistle]

holy snap time to package up an asset store plugin

[robust scaffold]

They're still beta-ing cone lighting and such.

Nah, it's extremely custom and niche tilemap2d lighting.

https://media.discordapp.net/attachments/300454903250419712/1002783371065307136/unknown.png

[balmy thistle]

Oh I was talking about myself, not that I could, code belongs to my old employers

yeah I had pixel perfect shadows

swept via a ray march in the shader

[robust scaffold]

Yea, mine's similar.

[viral sonnet]

why not just use unity dots physics for spatial queries?

their bvh is pretty good

biggest problem is the stateless nature. wish they would invest into caching instead of just saying use havok

[rotund token]

what's wrong with havok?

i will say it's kind of nice the physics was built in a way you can literally just swap out physics engines if you wanted to roll your own

and re-use components

[robust scaffold]

Yea. And the ability to integrate alternative or custom physics solutions into the greater unity engine. Both DOTS and mono.

[robust scaffold]

Honestly, the sole improvement I might be able to bring to my own physics system is complete integration with entities. I see why Unity has isolated their intermediate physics data to a giant pile of NativeArrays and makes complete sense from a modular package perspective. But as a hobbyist programmer, I am free to make terrible decisions like bloating my entities with 30+ intermediate physics data for easy debugging. Easy if I unlock the scroll wheel on my mouse.