archived-dots

[safe lintel]

shouldnt need to add burstcompile if the job has it

[dense crypt]

Might just be profiler overhead too

[viral sonnet]

2 very simple IComps

[dense crypt]

Prob profiler overhead then, is this in a build?

[west lichen]

I could really use some help with this.
I have a job system that calculates a bunch of stuff per frame. I have used jobs and burst before, but I've never run into this. I can't even find anything online mentioning this. called JobDebuggerFrameMaintenance is TANKING my performance.
Anyone know what this is?

Also, sorry if wrong chat.

[viral sonnet]

I'll test it in a build but from lots of build testing, this shouldn't be something related to profiler tbh

the job overall takes around 6ms and I've picked it apart to 2.11ms and 3.00ms with these 2 lines

[dense crypt]

Hang on a minute, you prob have casting overhead

Unless I interpretate your code wrong, you are assigning an int to an enum no?

[viral sonnet]

no, the int spellId isn't really used

I might have another lead now to what's going on. Might be even stupider if true lol

[dense crypt]

Do you have some UI window open related to Job Debugging?

[west lichen]

I'm almost certain I do not.

[dense crypt]

Try disabling the Rendering profiler module

[west lichen]

It didn't look like it, but I think I did. I restarted unity and its gone. brb, ripping up my resume.

[viral sonnet]

Guys, found the culprit. spellCasterSpell was a ref, so I wrote back to the chunk. removed the ref and activated my old method of writing back. Still, not too happy about how much time I'm losing. writing back 250k times and needing 0.9ms is a little much

[rotund token]

how are you profiling these 2 lines?

[viral sonnet]

with commenting out 🙂

[rotund token]

that is not a good way of profiling because burst often realizes ok half the stuff you did above does nothing now

strip it out

it's like the guy on forum who had this loop and was complaining about nativearray write performance

when he commented out the write to nativearray it was 60ms faster!

well turns out the entire job and the costly calculations were just stripped when he commented out that one line writing

[viral sonnet]

i'm aware of that 🙂 burst strips exceptionally well. I found the reason why it took so long a few minutes ago.

[rotund token]

yeah i noticed after i commented

[viral sonnet]

I'm still not in the clear why this particular write back to the chunk takes longer than others. the data amount is small

[rotund token]

anyway if anyone was interesting in what i'm talking about forum guy https://forum.unity.com/threads/writing-to-nativearray-in-ijobparallelfor-is-extremely-slow.1254027/#post-7970928 (no haters, just pointing out common error)

[viral sonnet]

the variable is not used anywhere else so I can conclude that this is really the only thing that's happening. just checked if anything could interfere that gets striped when I take out the line

feels like cheating when you write back to a ref variable of the array var spellCasterSpells = (SpellCasterCastedSpell*)chunk.GetComponentDataPtrRO(ref SpellCasterCastedSpell_ReadHandle); next level dirtiness haha

I don't do it actually! just out of mistake and some (dumb) tests I did with ref a while ago which bites me now in the ass. I'm also not too sure about if the usage of all the refs destroys memory locality in the loop

[rotund token]

what advantage does that have over just getting the regular native array and using UnsafeUtility.ArrayElementAsRef<T>

optionally wrapping it with a nice little ElementAt method to hide unsafe code

            where T : struct
        {
            return ref UnsafeUtility.ArrayElementAsRef<T>(array.GetUnsafePtr(), index);
        }```

[viral sonnet]

none at all, turns out the usage of refs does indeed destroy memory locality

from 3.1ms to 2.6ms. pretty crazy

[rotund token]

btw i look forward to seeing your final demo

you've put a lot of work into it

i like to see crazy things

[viral sonnet]

thanks! I appreciate it! 🙂 it turned crazier than it should be. haha, could've thrown the thing out a long time ago compared to how other packages are performing or MBs in general

For the final demo I plan to have lots of entities trying to destroy a boss. Rendering will be a huge bottleneck so maybe it'll be just 2d or something

OR - wait for it - lots of capsules.

[rotund token]

i was about to say

im rendering 50k capsules, 50k spheres, 50k cubes atm in my save demo

and its only taking 0.95ms

[viral sonnet]

great, good to know!

[viral sonnet]

sometimes 1 line of code creates the perfect storm of confusion. instant spells are not meant to be written back to the spellcaster, only spells that are cast over a period of time. but as I was using ref, the spell was written back which activated my "a spell is casting" routine in the next frame. and as I'm testing with a global cooldown of 0 this was going on for every frame then. So that explains the huge increase in time with that line. glad I found that out ... note to myself, be more careful with refs and don't add refs and leave the project for 3 months

funny enough, I was testing with 500k spells instead of 250k then.

[rotund token]

new goal, 1million spells

[viral sonnet]

now that would be something!

[dense crypt]

not just something, tertle assigned the task to you, sooo.... chop chop!

[viral sonnet]

challenge accepted! 😏

[west lichen]

"It is not recommended to Complete a job immediately"
How are you supposed to avoid calling .Complete() directly after creating job?
I'd understand if there was some sort of callback you could add, but that's not the case.
I'm using BURST so I can't access the objects directly from the job, right?
You can't just schedule because you very likely need to Dispose() of some stuff.

[rotund token]

all native containers have Dispose(JobHandle)

to dispose them after the job is complete

[west lichen]

I'm sorry. Could you elaborate?
The documentation on this is not great. When would you use the overloaded Dispose()?

[dense crypt]

Whenever you Schedule() a job you get a JobHandle back. Immediately call YourCollection.Dispose(ThatJobHandle)

Then it will automatically dispose the collection when the job is completed

[west lichen]

Oh. Well dang. I totally managed to miss that functionality.

[dense crypt]

If you're using your collection on two jobs or more then you gotta do it with the last handle

[rotund token]

but yeah to answer your original question, ideally you should never call Complete

pretty much the only acceptable exception is if you have to write something to a managed object like a Mesh or Animator etc

actually i guess this depends, are you using entities or are you just writing jobs to optimize game object world?

if you're not using entities and just using jobs to optimize your existing game object world then you'll have to call complete at some point

[west lichen]

That was suuuper helpful and totally answered it.

If I say, want to build a list of jobs and, once they are done, get the values out of the IJob, do I need to maintain a reference to both the handler and the job?
Or is there a simpler way to do this?
For example:

List<IJob> jobs...
jobs.add(JobMakingMethod());// JobMakingMethod creates and disposes some deps, so it needs to create the handler

Do I need to just return the IJob for the value later and teh JobHandler in order to wait for the jobs to complete?

[rotund token]

you should never need to hold a reference to a job

[west lichen]

Okay. How would I get the values out from it later, then?

Once completed.

Does JobHandle somehow have that functionality?

[rotund token]

just to confirm, not using entities and just using jobs for optimization?

[west lichen]

Yeah.

[rotund token]

you would pass in native containers, write to these native containers, read the result form the native containers

if you write to just a local field in the job you wont be able to read it

[west lichen]

Ah!

[rotund token]

you'll only be reading the original job copy

jobs are structs

[west lichen]

Fantastic!

That totally makes sense.

Err, wait.

If I use .Dispose(JobHandle), which disposes on job completion, then the native container I've written to will not have the value I'm looking for. Do I just need to dispose of it manually after reading from it?

I just want to make sure that's the intended way to use it.

[rotund token]

yep if you need to read from it

then dont dispose of it after the job until after you've read it

[dense crypt]

You can just permanently allocate (Allocator.Persistent) a collection and keep it around until OnDestroy, where you would dispose it

[west lichen]

Gotchya. Okay. Thank you.

[west lichen]

I was under the impression this was not a good idea.

[dense crypt]

Perfectly fine to do if you want to save some output, pretty much required when working with MonoBehaviour + Jobs

[rotund token]

really depends how much memory you are reserving

and how often you allocate it

if you do it every frame, may as well just keep it around

if you are allocating 10MB once in a blue moon seems a bit wasteful to keep it around

[west lichen]

Great point. This has been super helpful. Thank you guys.

[rotund token]

there is also a performance cost of allocating

especially if you are allocating large arrays

so yeah, really depends on the situation

[viral sonnet]

tertle, do you know if static classes with extension methods without a burst compile tag are burst compiled when used form a burst compiled job?

[rotund token]

100%

pretty sure burst can't call non-bursted code except native calls

[worthy rampart]

^^

U are 100% correct

You're either bursted or you aren't

[viral sonnet]

huh, okay, a test I did confused me. using the static extension method I had 0.40ms timings and with a local method it went down to 0.00ms - it made no difference in my main code though. reason I'm asking is because I can't throw a burstcompile tag on these methods without burst complaining about data types. so why can it be compiled in a job but not otherwise?

[rotund token]

if you put a burst compile tag on a method you are also compiling it as a function pointer

function pointers can't use structs (except helper structs wrapping a single pointer)

you can only pass in primitives and pointers

[viral sonnet]

ok, that clears things up then. thanks!

ok, it's getting late. have a happy easter everyone! o/

[gusty comet]

How can I include GameObjects instantiated during the conversion process to the conversion process? For example, I would perform the following steps:

1. Have a SpawnerAuthoring that references some prefab GameObjects
2. In a GameObjectConversionSystem, a ForEach would have each SpawnerAuthoring call GameObject.Instantiate on the prefabs, creating new GameObjects
3. Another GameObjectConversionSystem that runs in a later GameObjectConversionGroup queries on the prefabs
   Right now, the later GOCS only recognizes the prefabs. The instances of the prefabs were never added to the conversion process despite having ConvertToEntity and all that, and in the DOTS Hierarchy view, they're missing (but they're present in the Scene Hierarchy)

[rotund token]

you probably don't

while i've never tried to do this, as far as i'm aware the first step of conversion is to gather all potential gameobjects for conversion. by the time you're into the conversion systems it's too late to add new gameobjects to conversion

the only way around this would probably be to manually convert the gameobject into the destination world yourself using GameObjectConversionUtility

that said, this all sounds wrong to me

you should be creating entities not gameobjects during conversion

[calm edge]

are there any good patterns for loading settings into systems?

[worthy rampart]

As someone that just did this rabbit hole last week, just use an editor script to generate a bunch of gameobjects

And let them be converted as normal

[rotund token]

anything in particular you're looking for?

i just convert them all in a subscene

[calm edge]

I just want to get a little data into the system

[rotund token]

i just have a very basic base scriptable object

    public abstract class Settings : ScriptableObject, ISettings
    {
        public virtual void DeclareReferencedPrefabs(List<GameObject> referencedPrefabs) { }
        public abstract void Convert(Entity entity, EntityManager dstManager, GameObjectConversionSystem conversionSystem);
    }```

and a few convenient implementations, such as

        where T : struct, IComponentData```
if my settings are a single component for example

with a nice little window that finds them all in my project to display in 1 place with search and a few other things

and they're all referenced by a script in a subscene and just converted as normal

added a convenient button (Apply) to auto reimport the settings subscene(s) for when I make changes [plural because client, server and client/server can have different settings]

and thats basically my settings workflow. pretty simple/basic

[rotund token]

one thing i'll add is there would definitely be an advantage of using gameobjects over scriptable objects for your containers as the subscene would detect changes automatically

[ember mountain]

Oh, I almost forgot that since 2021 is in LTS now, it is possible to use colliderInstanceID in physics jobs. 🥳

[haughty rampart]

entities is not yet available for 2021

[celest roost]

what are dots?

[amber flicker]

not sure if it’s helpful - in case you weren’t aware of it, if you have an SO you can use this so long as you have a GO somewhere in your conversion https://docs.unity3d.com/Packages/com.unity.entities@0.50/api/Global-Namespace.GameObjectConversionSystem.DeclareAssetDependency.html

[calm edge]

why do you need to convert the settings? to a singleton entity?

Is it possible to reference a DynamicBuffer from a job? I seem to have made it angry

oh nvm, I was looking at older docs, the latest ones explain why you can't

[gusty comet]

That sounds like it would work for most GameObjects, but mine are too large and contains a lot of rendering components to instantiate in the Editor without causing huge slowdowns. I also prefer wiring together GameObjects instead of Entities due to the abstraction that Authoring offer

[gusty comet]

I'll do this then. This means though that the spawner system will be manipulating the IComponentData of the Officers, Formations, and Soldiers, but I guess this breaking of encapsulation is inevitable

[rotund token]

I'm aware of this but I've never been able to get this to react to scriptableobject changes. If you have a working example of making changes to a SO and it triggering a rebuild of a subscene I'd love to see it.

[gusty comet]

If I were to have a SystemBase update under GameObjectAfterConversionGroup, would it operate on the Entities in the Conversion World, or the Destination World?

Also, would it only update once?

[north bay]

Once you save the asset the subscene will react to the changes of the scriptable object

if (GUILayout.Button( "Save" ))
{
    UnityEditor.AssetDatabase.SaveAssetIfDirty( this.serializedObject.targetObject );
}

[amber flicker]

Yea as Script says - so long as the SO is considered dirty then when saved it updates for me.

[worthy rampart]

you generate all the game objects into a subscene

and then close the subscene

so it only happens once

[gusty comet]

I was just about to suggest that., but you beat me to it

Could I use the subscene like a prefab, and scatter it around the main scene?

The entire subscene will contain a Formation, an Officer, and dozens if not hundreds of Soldiers. Having them all in the main scene would slow the Editor to a crawl, but if I have them in a subscene, would they technically not "exist" until runtime?

[worthy rampart]

They can exist as entities in the closed subscene

[gusty comet]

And in the main scene I can have a simple editor script that draws a rectangle or triangle representing the formation's soldiers

[worthy rampart]

They'll be in the inspector and stuff

I would probably do something where you have a single subscene of them

And registers them as entity prefabs

That you then spawn via a spawning system or something

[gusty comet]

Thanks a ton! I really appreciate your insight. This might just be the simplest and most effective solution

So, to reiterate:

1. Create a subscene
2. Populate the Subscene with Formation, Officer, Soldiers centered around origin (0,0,0)
3. Drop multiple instances of the Subscene in the main scene where appropriate

Some caveats are: is it possible to pass in data from the Scene into the Subscene? Like in the Scene, configure the number of soldiers in the formation, and the Subscene spawns and converts the right number

[worthy rampart]

So I have never done multiple instances of a subscene before

if that works then great

What I probably would do is something more like

subscene of authored squad/units that register themselves

so that you can then create a SquadSpawnSystem

and place Spawners

which have things like how many of each type of unit they will create

and that would probably be a runtime system

[gusty comet]

Thanks! Before I started implementing your suggestion, I tried moving the spawning logic from a GameObjectConversionSystem to the Awake() callback of the spawner script. It spawned the new GameObjects, but they were also picked up by the conversion process, and wired up correctly. However, I've never seen examples of people spawning Authorings in the Awake() callback of Monobehaviors. Is this a valid pattern?

[worthy rampart]

Ooh

That's an interesting idea

I like that alot

[ember mountain]

Can I write at NativeArray index other than the one currently passed into JobFor scheduled in parallel?
I allocated a native array and passed half of it's length as an argument for the parallel job. Yet when I try to assign value at array[index+1] I get System.IndexOutOfRangeException: Index {0} is out of restricted IJobParallelFor range [{1}...{2}] in ReadWriteBuffer. Yet everything works fine when writing in array[index].

[worthy rampart]

Yes you have to disable the safety for that array

[ember mountain]

NativeDisableParallelForRestriction attribute?

And what is a "proper" way to write multiple values in NativeArray for each index if there is any?

[left oak]

You're not trying to write to the same index from multiple threads, right? That would not be good.

[ember mountain]

No, I write several commands for each index.

    Node node = nodes[index]; //from [ReadOnly] public NativeArray<Node> nodes;
    int commandIndex = index + 1;

    //into [WriteOnly] public NativeArray<CapsulecastCommand> commands;
    commands[( commandIndex * 4 ) - 4] = CreateCapsulecastCommand( node, math.forward() );
    commands[( commandIndex * 4 ) - 3] = CreateCapsulecastCommand( node, math.right() );
    commands[( commandIndex * 4 ) - 2] = CreateCapsulecastCommand( node, math.back() );
    commands[( commandIndex * 4 ) - 1] = CreateCapsulecastCommand( node, math.left() );
}```

[left oak]

So... is this supposed to be preparing the arguments that you're gonna pass in to CapsulecastCommand.ScheduleBatch, or?

[west lichen]

Is it reasonably fast to use access a NativeArray? Or should I convert it to a normal before accessing it thousands of times?

[haughty rampart]

native array is fine

[ember mountain]

Yes, I do ScheduleParallel for this job, and then pass the resulting array in ScheduleBatch.

[left oak]

Ok, well, in general, the best way to write multiple values to a native array would be to vectorize it, but I don't know how well that works here

Since you're always writing out the four cardinal directions for every node, you should be able to reduce this to a single instruction, I think

[ember mountain]

Not sure what you mean by "vectorizing".

[haughty rampart]

code vectorization

[left oak]

for simd

[haughty rampart]

aka write multiple values at the same time

best use intrinsics for that

[left oak]

If I've learned anything from our performance-oriented friends here, I'd profile this first, haha

[haughty rampart]

no need. just look at the burst output

[left oak]

Even better!

[grave zenith]

Hello

[blissful sage]

Is it possible that if I use EntityManager.Instantiate on prefab with child converted with GameObjectConversionUtility.ConvertGameObjectHierarchy it spawns that child in gameobject hierarchy for some reason? It seems like for how many times I call EntityManager.Instantiate I have that many leftover gameobjects looking like child of that original prefab ... did somebody encountered something like this? It seems it does this only in build . Maybe Im working on it too long and dont see something obvious ... its weird.

[ember mountain]

So... Huh, I can just add Unity.Burst.CompilerServices.Loop.ExpectVectorized(); according to https://docs.unity3d.com/Packages/com.unity.burst@1.7/manual/docs/OptimizationGuidelines.html. 👀

[left oak]

Yes, that should alert you if something's not being vectorized, but that's it

[ember mountain]

Amazing...

[left oak]

Actually getting it to vectorize is an entirely different story altogether

[haughty rampart]

i mean burst tries to as much as possible already, but better to add the appropriate attributes, etc, or go to intrinsics

[left oak]

Welcome to dots! 👋

[ember mountain]

To be honest, I don't know what I'm doing I'm new to DOTS and the need for NativeDisableParallelForRestriction to write in multiple array indexes was already unexpected.

[left oak]

Oh I understand. I'm definitely not a vectorization expert by any means. Just know that there's a higher performance ceiling there, in case that you need it someday.

[ember mountain]

Given the fact that the system already raycasts, boxcasts and capsulcasts across the entire level before the end of a pressed button animation is kind... unbelievable.

[robust scaffold]

Intrinsics is honestly a last resort. Ideally, you just write very burst friendly loop code and just check if burst vectorizes as expected. Sometimes it just may surprise you.

[haughty rampart]

yeah of course. just said it's an option.

nice seeing you around here again though

[robust scaffold]

Oh yea, I'm back to my coding dungeon. The video game I usually play (too much to be fair) has finally burnt out and I've looped back into unity coding.

[haughty rampart]

XD

[left oak]

I was hoping one of our vectorization experts would show up! 👍

[robust scaffold]

Oh boy, what ya need?

[left oak]

Oh no, not me, just to help confirm what we were discussing above!

[robust scaffold]

The physics discussion?

[left oak]

Yeah, silence coder was trying to write multiple data in a native array

[robust scaffold]

ExpectVectorized() is very rudimentary. Basically a learning tool to see if a simple for loop adding a value from one array to another or itself produces good purple packed values.

If you go beyond addition and subtraction (multiplication maybe), it basically will always return false. It's not because your code isnt vectorized but because ExpectVectorized() is not that advanced.

It's better to understand the burst inspector and watch for the glorious purple VXXXPS and so on. Vectorized command operating on a packed single (or int) value.

If it ever drops to non-purple (and it's not just a memcopy call) or if it suffix's with SS (scalar single), that drops out of vectorization and you need to make sure you know why and if its intended.

Now for multiple parallel writing to a native array, it's actually far better to first make everything singlethreaded.

Because vectorization operates on a single thread. Single command, multiple operations. Vectorization can not cross thread boundaries.

[robust scaffold]

Are you trying to vectorize this?

[ember mountain]

No, I didn't even know what it means prior to this day. I tried to figure out why I get System.IndexOutOfRangeException: Index {0} is out of restricted IJobParallelFor range [{1}...{2}] in ReadWriteBuffer upon writing into anything but commands[index].

And what is "proper" way to work with native arrays of different length in parallel jobs.

[robust scaffold]

Whenever you get an error like that, disable burst. Run again, and the string interpolation will work.

[robust scaffold]

"Proper" is a very strong word. You're gonna get a lot of people jumping in describing their own usage and since Unity is mute on the topic, well there's a lot of functional ways.

My first question, do you need to do it in parallel?

I've come full circle on the topic of parallelization. Singlethreaded jobs are actually superior to multithreaded in a lot of cases.

[ember mountain]

Huh, to be honest I don't know. It seems I've should used [WriteOnly, NativeDisableParallelForRestriction] instead of [WriteOnly] for the output array. At least this fixed the issue.

[robust scaffold]

Especially in the cases of differing lengths. Singlethreaded jobs is simulatiously "free-er" to do whatever you want and, using vectorization, faster.

[left oak]

That's interesting, so I'm really only vectorizing the work on each individual thread

[robust scaffold]

yep. And unless you got multiples of 8 values per thread, which isnt a guarantee unless you pad your chunks using shared component data, you're just wasting vectorization operations

[left oak]

Hmm, I took a look at my burst inspector, and my executes are very purple, probably because I try to operate on 4-wide structs

[robust scaffold]

That also works but be careful about wasted memory space.

[rotund token]

'purple' love it

[left oak]

Where is the waste being introduced?

[rotund token]

are they ps or ss instructions

[left oak]

vps thank glob... I really need to learn what all these symbols mean

[robust scaffold]

If you're using structs that are 32 or 64 bit wide and not all the values are used, you're introducing padding which is wasted memory. If you're using the full 8 floats/ints, good. No waste. If you're using more or less and not a multiple of 8 ints/floats (or 4 longs/doubles), you're dropping out of vectorized code in order to operate on those values.

[left oak]

I see! I guess I've been accidentally following good practice... I really want to understand the burst compiler better, as it feels like a good friend whom I just don't understand, haha

[ember mountain]

Wait, I need to write structs in a way that the sum of all attribute types fits in a specific length? 👀 💦

[rotund token]

the issue i have is i already have the asset dependency declaration

                conversionSystem.DeclareAssetDependency(this.gameObject, setting);```
but when i stepped through the DeclareAssetDependency it always returned here
```            if (dependsOn != null && !dependsOn.IsAsset() && !dependsOn.IsPrefab())
            {
                return;
            }```
that said, I haven't tested this since 0.50
-edit- well it appears to add to the dependency list now but making changes to the scriptable object still isn't triggering a subscene rebuild so i don't know

[robust scaffold]

Ideally, yes. Practically, no.

[robust scaffold]

Get your code nailed down first. Get used to DOTS, how it functions, how data is arranged, and how data should be accessed linearly. Then dive into burst and into the madness that is 64 bit wide operations.

[north bay]

I've been using DeclareAssetDependency for quite a while without issues
Hitting 'Save Project' should trigger the subscene rebuild

[rotund token]

nope, i realyl don't know

i'm using the default inspector instead of my editor window for testing

to ensure i'm not screwing up dirty

oh wait ok i got it

had to rebuild all the subscenes once

[north bay]

Huh, did you maybe not increment your ConverterVersion?

(after adding the DeclareAssetDependency thing)

[rotund token]

i dont bother with versioning in my own library (as in i don't increment it)

since im only person using it

and i have a button in toolbar to just reimport them all after i make a change

i just happened to be testing right then on the only settings file that hadn't been added as a dependency

anyway all good cheers, good to know this works

one less thing i have to make designers remember

[robust scaffold]

I just gave up on subscenes. No clue how any of that works.

[rotund token]

going to be real sad after you wait all this time for 1.0 to realize none of the new features work without subscenes 😦

[robust scaffold]

Imagine rendering entities. Ha. I have 1 sphere actually rendered and the rest is UI elements. Subscenes need not apply.

[rotund token]

imagine the possibilities for that 1 sphere!

[robust scaffold]

Just stack more and more post processing on that one sphere.

Basically what I'm doing

Color pallet swaps in post processing is how I'm rendering data changes. Entities side generates a set of arrays with color swaps and indices to change and passes it over to the shader which alters a texture that is then plastered onto that sphere.

[robust scaffold]

@rotund token Also, do you know what's up with the new AllocatorHandle business? It's messing with my autocomplete as Allocator.X doesnt register despite being implicitly converted.

[rotund token]

haha yeah the breaking of auto complete is a bit annoying

but this is their new allocator that they haven't converted everything across to yet, currently only implemented via World.UpdateAllocator

you can use that instead of tempjob and not worry about disposing - it's cleared every 2 world updates (there are 2 allocators and they alternate every frame)

from memory only been setup to work with NativeMultiHashMap and NativeArray atm

via
CollectionHelper.CreateArray
CollectionHelper.CreateNativeMultiHashMap

good chance Allocator.X doesn't exist in 1.0 if they get around to switching everything

so yeah World.UpdateAllocator is now tied to World update loop instead of frame rate

fixes the leak complaints when fps is 4x your fixed update rate

unfortunately though, the patch notes say that ECBS was converted to this new allocator, in reality it's still using TempJob so it still leaks. i assume this was a last minute revert.

[robust scaffold]

Issue is, I'm not using any of the entities package. Just Burst, Jobs, and Collections. Will World update then be tied to monobehavior update?

[rotund token]

WorldUpdate is just the allocator they implemented in the entities world

you could write your own allocator for monobehaviour update

or whatever update loop you want

or just implement the allocator

it exists in collection package

called RewindableAllocator

[robust scaffold]

hrm, I'll stick with temp, tempjob, and persistent until they pull it from my cold dead hands...

RewindableAllocator, huh. Hrm

[rotund token]

    /// An allocator that is fast like a linear allocator, is threadsafe, and automatically invalidates
    /// all allocations made from it, when "rewound" by the user.
    /// </summary>
    [BurstCompile]
    public struct RewindableAllocator : AllocatorManager.IAllocator```

[robust scaffold]

Wait, that sounds useful

[rotund token]

yeah that's what UpdateAllocator is

public ref RewindableAllocator UpdateAllocator => ref Unmanaged.UpdateAllocator;

[robust scaffold]

If I assign all my persistent native arrays to this new RewindableAllocator, I just need to "rewound" it at game exit and it'll automatically clear the dozens and dozens of persistent arrays?

[rotund token]

sure

[robust scaffold]

And I dont need to update all these IDisposable patterns? Oh wow

[rotund token]

well you don't need to rewind it

you just need to dispose it!

unless you want to re-use memory

[robust scaffold]

Well, I just need to dispose of now this one allocator and everything else allocated using this specific RewindableAllocator will do so automatically?

Where was this in the docs?

• Dispose() - Dispose the allocator. This must be called to free the memory blocks that were allocated from the system.
• Rewind() - Rewind the allocator; invalidate all allocations made from it, and potentially also free memory blocks it has allocated from the system.

Does rewind need to be called before dispose?

[rotund token]

i dont think so

so from my limited looking at this from what i understand rewind allocator allocates blocks of memories as required
when you call rewind,

• any block of memory that was used this update are put back in the pool for next frame
• any blocks of memory that weren't used this update are freed

so it avoids a lot of allocations per frame by re-using memory

but also avoids hanging onto memory forever if you have abnormal frames with a lot of allocations

[robust scaffold]

• Initialize(Int32) - Initializes the allocator. Must be called before first use.
  Is this the maximum number of allocations (native arrays for example) that will be associated with this allocator?

[rotund token]

nah just initial size

it can grow as much as it wants i believe

[robust scaffold]

Makes sense. Seems like a temp implementation.

[rotund token]

there are other allocators included such as ScratchpadAllocator

though i havent looked at them at all

[robust scaffold]

/// <summary>
/// Dispose the allocator. This must be called to free the memory blocks that were allocated from the system.
/// </summary>
public void Dispose()
{
    if (JobsUtility.IsExecutingJob)
        throw new InvalidOperationException("You cannot Dispose a RewindableAllocator from a Job.");
    m_used = 0; // so that we delete all blocks in Rewind() on the next line
    Rewind();
    m_block[0].Dispose();
    m_block.Dispose();
    m_last = m_used = m_best = 0;
}```

Already calls rewind anyways in the dispose

It seems like the parameter sent to Initialize() of RewindAllocator is the size of the memory the allocator is in charge of

[robust scaffold]

It doesnt seem to be able to grow beyond the initial byte size

[rotund token]

isnt that each individual block

but it can create new blocks

[robust scaffold]

Nvm, found the growth section. It seems that if the initial value is set to a dummy value of 1, then the first block is wasted.

And there is a maximum of 64 associated memory allocations per rewind allocator

m_block = new UnmanagedArray<MemoryBlock>(64, Allocator.Persistent); And they're using persistent anyways

And 64 seems a bit small for a constant max number of allocations per rewind allocator. Especially since a global World allocator is provided to use across all systems in that world.

[rotund token]

yeah MemoryBlock is 128KB in entities

UpdateAllocatorHelper0.Allocator.Initialize(128 * 1024);

so that's 8MB by default but each of these blocks can grow as much as they need

[robust scaffold]

hrm, interesting

[robust scaffold]

RIP. No rewindable allocator for native arrays then

[rotund token]

i told you!

CollectionHelper.CreateNativeArray

NativeArray<int> myArray = CollectionHelper.CreateNativeArray<int, RewindableAllocator>(1234, ref this.system.World.UpdateAllocator);

[robust scaffold]

I know, but I couldn't just ctrl-h replace all references the easy way...

[rotund token]

oh yeah

bit more work than that

[robust scaffold]

grrrrr

work

I have all my native arrays organized like this because it's easier to refactor something but damn, now I have to write out the types.

[rotund token]

2021(2?) 🤤

[robust scaffold]

curse you unity

[robust scaffold]

2021 beta. Havent yet downloaded 2022 as I'm just now getting back into the mindset of coding again and dont want to fuck something up major moving to the next major

As mentioned earlier, my post processing stack on the single sphere is a very very tall house of cards. If something in the middle breaks, the whole tower collapses in upon itself.

[rotund token]

ok do you mean 2022.1 beta or 2021 because 2021 is in lts now o_O

[robust scaffold]

Oh yea, 2022 beta. Gonna move onto 2022 alpha soon because gotta chase them bleeding edge

[rotund token]

was so sad downgrading my project from 2021 to 2020

and having to revert all the nice syntax

[robust scaffold]

Fuck unity, im not changing my syntax. Where's muh overloader

[left oak]

I don't use Hybrid Renderer, so I'm tempted to try 2022, but I'm worried that the rug is going to be pulled out from under me by 0.51...

[rotund token]

i wouldnt expect too much to change in 0.51

the primary reason entities cant run in 2021 isnt to do with the entities package

unity engine is missing methods from jobutility

that exist in 2022 and were ported back to 2020

so primarily they just need porting to 2021

obviously no guarantee though

[viral sonnet]

hey everyone o/

[robust scaffold]

hi

[viral sonnet]

I'm thinking about writing a better blob storage. Can anyone confirm that BlobArrays and BlobPtr are actually living outside the locality of the blob struct? From the source code I think that's the case

or is it just one linear stream of memory?

[safe lintel]

better than unity's current blobs?

[viral sonnet]

yes, I really need a way to get the BlobRoot and have all the data locally. Reason is, I have several BlobPtrs and BlobArrays inside the root and if those are stored somewhere else it's not very helpful for what I need

but, I can't confirm if Unity's blob are already working like I want. I tried watching the memory in cheat engine but I failed to read anything useful

@robust scaffold how's your project going? Just read you're going back to coding. same as me, took a 3 month break

[robust scaffold]

Yep. Just started back up my IDE today. Slowly making my way through my old code and what I left undone. Everything still "works" though. No error on running. Thing is, nothing is actually running. Still remaking initialization code and thinking about my custom entities implementation (piles of native arrays).

[viral sonnet]

had enough of entities or what's the reasoning for your own custom entities implementation?

[robust scaffold]

I needed to separate my entities by geographic location for specific region based inter-entity logic and also a specific global inter-entity logic pass. Inter-chunk logic just doesnt seem to exist.

For example, I had a job that iterated within a specific region, which is represented as a chunk. Perfectly fine in Entities. But I also had another job that needed to iterate between entities located in regions 1, 3, and 4. Passing the same data in between all three data. Sure I can do that using intermediary native arrays or appropriate storage but then parallel implementation straight up doesnt exist.

I needed a thread that operates a job within regions / chunks 1, 3, and 5 and another parallel thread to operate within chunks 2 and 6 seamlessly. That isnt possible due to Entities segmenting chunks and there is no selective cross-chunk iteration. You either iterate within a chunk or iterate along all chunks.

With one native array, I solve that issue by using custom indexing.

[safe lintel]

@rotund token how do you use RemapEntityFields in your example? I assume hashmap is old and new entity but not sure what the other inputs come from? also you are deserializing to a separate world first?

[viral sonnet]

someone on the forum wrote about doing a custom iteration of chunks. as we have pointer to chunks, this sounds similar to yours. is that close?

[robust scaffold]

Really? Got a link?

[viral sonnet]

he just said, he did, not really more about implementation. give me a minute

that's the thread: https://forum.unity.com/threads/sorting-by-chunk.1261010/#post-8016257

[robust scaffold]

Also, I needed off main thread structural changes. With native arrays, I can do whatever I want, however I want. With Entities, using ExclusiveEntityTransaction is literal pain.

My current rudimentary entity storage is a pre-allocated native array of region size * 256, for a hardcoded max 256 entities per region. I can always just bump it up if my testing shows that it hits the limit too many times.

[ionic sierra]

Do I need exactly 2020.3,30 as it says here...or can i use 2020.3.33, which is the latest?

[robust scaffold]

.33 will work. Just stay within 2020.3.

[ionic sierra]

thanks

[viral sonnet]

Ha, yeah, I'm also on the verge of just working on preallocated blocks of memory that just have to be "big" enough.

damn, unity dots forum problem is derailing. some seem to revel in it

[robust scaffold]

The main issue is the lack of any in editor inspector ability. And Rider's debugger is unreliable due to being purely unmanaged and cant view fixed sized buffers, which is everything in my program

[robust scaffold]

DOTS forum posts derailing? Say it isnt so.

[viral sonnet]

I sometimes use Cheatengine to view raw memory 🥶

of course that's not really useful lol

is the VS debugger any better there?

[ionic sierra]

DO I need VS 2022, or is 19 fine?

[viral sonnet]

19 should be fine but VS 2022 is so much better

[ionic sierra]

Thanks

[viral sonnet]

is there an option for burst so code isn't culled? I know 95% of my timings are data acquisition and it's highly annoying when data acquisition is culled because it's not used. I often can't make reasonable decisions where the heavy operations are - or I'd have to believe that 250k calls of flag Contains method now takes 1ms

I don't know - I can shave off 0.20ms just by adding "in" to my spellBase parameter in 4 methods

that's a 4% performance increase, from 5.0ms to 5.2ms. little too much for something like that. memory locality is getting very unclear to me how it performs

I felt I had a very good grasp in C++ and Mono but with Entities and Burst this has turned into a guessing game.

[worthy rampart]

I add in almost everywhere I can

[viral sonnet]

I'm like, where's my damn fast STACK?

[rotund token]

Random question enzi, do you tell burst your pointers aren't aliased?

[viral sonnet]

I haven't gone much into the NoAlias tag. just for some very specific use cases

[rotund token]

Also hint assume is extremely powerful under certain circumstances hitting double digit gains

Also extremely risky

I wrote a little wrapper for it to do a assert check before the hint assume when safety is on to avoid issues

Then without the risky downside I feel like it's just free performance

[viral sonnet]

can you share this wrapper?

[rotund token]

On tram heading home will do when I'm back

[viral sonnet]

i'm honestly too unsure most of the time where NoAlias applies. like which context of locality does it assume? It definitely should be useful for many parameters I use

[rotund token]

My only real benchmark of this was my recast implementation where I swapped my asserts with hint assume and got 11% performance improvement

[viral sonnet]

thanks tertle

[rotund token]

Honestly no idea what burst changed

But it was a happy improvement

So these days I just chuck it everywhere

[viral sonnet]

currently on this method, the middle block are all structs from the chunk. would a NoAlias apply here?

[rotund token]

Probably not

[viral sonnet]

hm, not really. I think I'm misunderstanding NoAlias still

[rotund token]

Just look out anywhere you use NativeDisableContainerSafetyRestriction

[viral sonnet]

hm, I only have some on write handles because I also have read handles in the same job. not applicable there I think

[rotund token]

Anything with safety disabled with be considered to alias with all other native containers /pointers in the job

[viral sonnet]

oh, that's huge

[rotund token]

Actually all those ref above might benefit from no alias probably not because different types

If they don't actually alias

I kind of forgot about that

https://docs.unity3d.com/Packages/com.unity.burst@1.7/manual/docs/OptimizationGuidelines-Aliasing.html#noalias-function-parameter

Look at that particular example

I'm only really bringing this up because of how much you like to over optimise

Normally I would not suggest for people to get this pedantic 💩

[viral sonnet]

I appreciate it 🙂

[rotund token]

{
    using System;
    using System.Diagnostics;
    using Unity.Burst.CompilerServices;
    using Debug = UnityEngine.Debug;

    [DebuggerStepThrough]
    public static class Check
    {
        public static void Assume(bool assumption)
        {
            IsTrue(assumption);
            Hint.Assume(assumption);
        }

        [Conditional("ENABLE_UNITY_COLLECTIONS_CHECKS")]
        public static void IsTrue(bool condition)
        {
            Debug.Assert(condition);
        }
    }
}```

that's basically it for my assume wrapper

nothing special just adds safety to Hint.Assume which makes the compiler just just assume something without actually verifying it

[viral sonnet]

thanks, gonna look into it

[rotund token]

to quote the manual

The power of the assume intrinsic is that it allows you to arbitrarily tell the compiler that something is true. A developer could tell the compiler to assume that a loop end is always a multiple of 16, meaning that it can provide perfect vectorization without any scalar spilling for that loop. A developer could tell the compiler that a value isn't NaN, is negative, etc - the sky is really the limit here.

The danger with the intrinsic though is that the compiler will assume the value is true without checking that it really was true - you as the developer have promised to the compiler that it must be true, and Burst is a trusting compiler - it entrusts that the promise is kept! As a result, this intrinsic should be one of the last tools left on the shelf - it is useful and powerful, but care must be taken.

so my theory was just, why not just make it safe when safety is enabled to avoid issues

[viral sonnet]

this line crops up the second time now, it takes so much cpu time. nearly up to 1.5ms which can't be right. the if isn't hit and elses into an empty block

flag is a byte, maybe that's my problem?

[rotund token]

can you show me what contains if/else block does

[viral sonnet]

maybe the spellBase gets culled when not using the if. either way, the data acquisition + the contains method takes way too long

sure

i've another place where I checked the baseFlags and it took rougly equally as long. the operations are minimal.

[rotund token]

on a completely random note
(keys & flag) == flag
is the same as
(keys & flag) != 0
right?

always just wonder about styling preferences

[viral sonnet]

I can chain some flags and I want every flag to be active then

I use > 0 in a ContainsAny method

ok, I at least can conclude that it has nothing to with Contains itself. even with something like spellBase.castTime > 0 on the if it performs the same. so, the problem is really in the SpellBase struct itself.

[viral sonnet]

I'm starting to think I'm running into very weird problems. fully commented out, just a debug log that doesn't get hit ~4.0ms, 2nd version, doesn't get hit ~5.0ms

projectileArchetype and spellStats would be the only 2 variables that could get culled

that's why bursts culling is annoying me. way too aggressive in debug mode

[rotund token]

what is happening in burst inspector

successful weekend, subscene save/load working while playing

final feature on my todo list for this save library

I realised yesterday this could be used for a decent partial rollback

[viral sonnet]

a lot of differences, i'm not sure I can make anything out of it. the assembly copied to text is 600kb

[rotund token]

All you really care about is the few lines matching the above

[viral sonnet]

this is insanely hard to read for me. the biggest thing I could find is that the actual method call isn't present which makes zero sense because other stuff is happening in this method. It can't just get culled away

[west lichen]

So is burst compile just always enabled? I don't see it as an option under "Jobs" tab and I see the job in the burst inspector.

[viral sonnet]

other things seems to be in order. the Contains line in particular. nothing seems out of the ordinary

You can turn it off with unticking "Enable compilation" in Jobs->Burst

this also disables compiling if you have code changes

for developing it's useful to enable "Synchronous compilation"

otherwise burst gets compiled when starting the game which can result in hitches, just a sidenote

[west lichen]

Does anyone know what:

EditorOnly [DisposeSentinel.Create]

Is from? I'm doing a TON of calculations for a gravity simulation and the profiler is stuffed with these. The jobs themselves are BURSTed and IJobParallelFor, if that helps any.

[viral sonnet]

Where is this shown? It's usually from Leak detection

[haughty rampart]

why are you not using Enum.HasFlag() anyway?

[rotund token]

does burst have a specific implementation of that?

otherwise HasFlag is managed last i checked

it does a GetType and a bunch of stuff

    {
      if (flag == null)
        throw new ArgumentNullException(nameof (flag));
      return this.GetType().IsEquivalentTo(flag.GetType()) ? this.InternalHasFlag(flag) : throw new ArgumentException(Environment.GetResourceString("The argument type, '{0}', is not the same as the enum type '{1}'.", (object) flag.GetType(), (object) this.GetType()));
    }```

yeah...

[haughty rampart]

oh right. unity still has the OLD implementation of HasFlag()
with .net core it'll just literally be ~ your code

[rotund token]

which will be nice... one day

[viral sonnet]

I couldn't make anything out of it. At this point Burst is screwing with me lol. Maybe some of you can find something more. The method in question is FinishCasting

I need to sleep, so good night everyone o/

[devout prairie]

BovineLabs??

[rotund token]

The name I release all my personal code under

[worthy rampart]

I'm pretty sure this was due to LLVM improvements

NoAlias is used everywhere by Rust automatically so there's been more effort into improving optimizations based on it in LLVM since it started growing in popularity

[rotund token]

i was talking about hint.assume

[devout prairie]

Hehe ahh

[rotund token]

There's a bit of backstory behind the name

20+ years ago there was a cheating community for coding bots for runescape that existed on bovinelabs.com
after it was shutdown scammers took over the domain but i managed to hijack it from them a few years later and have used it ever since as a name to just release code under

[devout prairie]

Haha brilliant.. the 'bovine' caught my eye as for no apparent reason i decided to start using the alias MidnightCow a while back.. i'd love to know how in the hell you hijack a domain!

[worthy rampart]

same response

LLVM did a bunch of work on the hint assume stuff

it wasn't really being used much until recently

[austere pebble]

Its no way latest unity dots will work on unity 2021.3? without using any fork

[rotund token]

i'm pretty sure even if you forked it you couldn't get it to work, it's not really the package that's the issue - it's the engine

the issue is 2021 is currently missing a bunch of jobutility features

this is why you can use entities 0.50 on 2022 (with some minor modifications for profiler/ui) because the job stuff exists

[calm edge]

Should hybrid renderer handle 100k (unique) objects easily? It's taking ages to do culling and batching

When they're not even on the screen

[zinc egret]

Hello, how can i light the bulb (example) using dots, i have entity only without reference to sprite

[safe lintel]

@calm edge hybrid works best when entity render data is shared among many, my understanding is it will give zero benefits if its that much unique data. but it might be a good case for the hybrid team to understand so id post about it in the dots rendering subforum

[ember mountain]

Sorry for the stupid question, but why FixedListInt32 called "an unmanaged, resizable list of int" is it has fixed capacity and Add() does the same as AddNoResize()?

[coarse turtle]

yes

although FixedListInt32 is deprecated now in the latest release of collections and should be replaced with FixedList32Bytes<int>

[ember mountain]

Alright, FixedList32Bytes<T> is also called a resizable list with no actual resizing as far as I can tell. Is it just a perpetual typo in the documentation?

[robust scaffold]

The "size" in this case is the quantity of elements in the list. Not actual capacity of elements which is fixed.

[robust scaffold]

FixedList is "resizeable" as opposed to a C# fixed buffer which is just an array.

[ember mountain]

Ah, now this makes sense. Thanks!

[viral sonnet]

I'm continuing my adventure from yesterday, finding out why a branch that's not hit adds 1ms of overall timing

commenting out the inside of the branch and the cost of 1ms is gone

well, I leave a debug.log in so it doesn't get culled completely

the only reasoning I'm left with is that Burst optimizes unused parameters away

[void girder]

An easy check would be to check the burst inspector right?

[viral sonnet]

yes and no, the assembly is ~600k and it confused me even more because when I comment this block out, I can't even find the method call in the assembly. which is odd because there's other code still in this method.

and when I really comment the method out, the whole thing runs faster. so that's another oddity

[void girder]

Right I've had that same confusion as well

[viral sonnet]

I find more questions instead of getting any answers is where I'm at lol

But honestly, it mostly comes down to a problem I'm having for a longer time. I essentially have a "database" of blobs which are used in the job and this data is highly inefficient. Every pull of data from this blob adds a lot to the job overall time

One solution to this problem would be to set the blobRoot struct in the initial job parameters. At least for this one job, there are many BlobArray and BlobPtr in this blobRoot which would lead to the same problem in another job I'll optimize after this one.

I don't have a conclusive answer if a blob is one stream of linear data even when BlobArray and BlobPtrs are in it

from the source code of the blob I think everything regarding a pointer lives in another space of memory

[viral sonnet]

got some answers, when commenting out, the whole method got inlined, so that answers why I couldn't find the method call in the "faster" code variant. [MethodImpl(MethodImplOptions.NoInlining)] helps here. Then the rest was as expected, the "slower" version has to pull in lots more data which quickly adds up to the 1ms. Now I've to find out if most of the time of the 1ms is in chunk data or the blob data. If I'd have to bet, I'd say most of the time is needed from pulling the blob data

[ember mountain]

Probably my answer should have been as "Because my other jobs are parallel. And I yet to figure out a way to combine dependencies between Schedule(), ScheduleBatch() and ScheduleParallel() without explicitly waiting for Complete()". 😔

[worthy rampart]

have you tried printing out the addresses

[viral sonnet]

I did for the base ptr once. Good idea, I should extend this to the other variables too

0x249A960ECA0 spell size: 12                              -> 2 515 397 569 696 + 12
0x249A960ECAC spellBase 40                                -> 2 515 397 569 708 + 40
0x249A960ECD4 requirements 24                             -> 2 515 397 569 748 + 24
0x249A960ECEC SpellRequirementFlagsBasic 4 times: 1       -> 2 515 397 569 772 + 8
0x249A960ECF4 SpellRequirementFlags 4 times: 2            -> 2 515 397 569 780 + 8
0x249A960ECFC targeting 16                                -> 2 515 397 569 788 + 16
0x249A960ED0C AbsorbShield 8                              -> 2 515 397 569 804 + 4
0x249A960ED10 Charge 1                                    -> 2 515 397 569 808 ```

seems to be the case. well, that's great to have figured that out

not full yet. some things that stick out here, flagsBasic is size of 4 and 1 length but has 8 bytes. flags adds up to 4*2 = 8. AbsorbShield is a BlobPtr and not present, still reserves 4 bytes which seems to be a null int32 ptr

BlobArray and BlobPtr are allocated right after the blob root

You guys surely have read the events thread from PhilSA by now. I must say, my ultimate conclusion is that all ECS and mixed NativeContainer solutions are kind of pathetic in terms of speed. Not even one comes close to the burst compiled NativeArray solution with pointers.

and that solution is single threaded ...

[rotund token]

In the end though, most people reading and debating that thread are only going to be sending 100s of events a frame max so they should just use whatever is just maintainable and flexible

[viral sonnet]

Yeah, sure but then you could also still program in MBs. Threads like these are cannon fodder for big critics like AcidArrow and that other guy.

My point is, we can program, data oriented without Entities

Personally I'm attracted to the fastest solutions, no matter what's currently hip. On the way we make mistakes anyway in some form and with sticking to the fastest, we probably still end on top. Entities doesn't give me the feeling of being on top. Like a framework that doesn't fit the purpose.

For many use cases it's wonderful. For game mechanics. Hmmm, still not sure about it. And the problem with the use cases where it's wonderful. 99% is burst and unmanaged memory, not entities

Eh, I'm just rambling somewhat. What I'm really interested is where it actually loses that much performance.

And where we can improve it. I still think Entities is dope AF

[viral sonnet]

58ms single threaded -> 122ms multi threaded over 8 threads -> 15.33ms per thread - not too great

[rotund token]

well that's 66fps instead of 17fps

[viral sonnet]

in that regard, it's good of course 🙂 58/15 = 3 so the tread utilisation suffers hard

[rotund token]

ive profiled a few entities projects

and in the end generally none really get close to maxing thread usage

seen ranges between 40-85% idle time

depending on platform

so in general my advice to teams is just take whatever you can off main thread even if it's less efficient

[viral sonnet]

yeah, I think that's a hard problem in multi threaded code in general. the dream of single threaded ms divided by thread count is a pipe dream

[rotund token]

big issue is often different hardware

if you were targeting just XYZ console then you know exactly how much work you can do per frame before you get blocked by workers

but on PC / mobile you often have to work within min hardware specs you're willing to use

so you max the hell out of this 4 core low end chip

but throw it on a 5950X and it's mostly idle 😦

[pliant pike]

to be honest I don't really understand that event thread, I use event entities and requiresingletonforupdate() all the time

I'm stupid I just prefer the simplest methods, one frame of delay doesn't seem to bad to me

[rotund token]

event entities fall apart in the 100s/frame

they're great for high level state changes or uncommon events but if you have a constant stream of them per frame the ecb playback will affect your fps

so it really just depends on your use case/needs

if you don't need 1000-10000s+ of interactions then it's fine

[pliant pike]

yeah I guess, I mean I would use that to activate the event to start

like I did my own attack system stat thing where I have 20 entities attacking 10,000 entities each

its one event really

or two one to record all the attacks and the second to instigate the attacks

[viral sonnet]

The fundamentals of the "event system" are, how to interact with entities outside your current chunk and random access in general. It's where the nature of Entities just fall apart because it's not really built or designed for that but the problem is very integral to many game mechanics

[pliant pike]

I guess I'm not sure where you would need thousands of events

[rotund token]

most people in that thread are discussing the other side

imagine a giant aoe attack dealing damage to a swarm of creatures

[viral sonnet]

This has cropped up time and time again and nobody has a really good answer to the problem

[rotund token]

the 'damage event' is what the discussion is mostly about

[pliant pike]

yeah thats what I was doing figuring out how fast I could do it

[rotund token]

a lot of people also don't consider the broader picture. applying your damage is only half the picture

but in a real game you want to do a bunch of other things

you want to record damage, interactions etc for telemetry

you want to record damage/kills etc for stat tracking and achievements

[pliant pike]

yeah there are limits to how much you can do

[viral sonnet]

right, if the base implementation isn't good this will lead to many issues

[pliant pike]

but anyway it seems what you can do with dots is way beyond monobehaviour just by default

[rotund token]

of course, people always just want more

and simply having the ability to do new things opens up ideas

and pushes new boundaries

[viral sonnet]

devs on the bleeding edge should never be content 🙂

[rotund token]

no longer thinking in the 100s of unit simulation and now think in the 1000s++++

[pliant pike]

yeah exactly I don't really want 10,000 attacks I was just seeing how far I could push it

[viral sonnet]

ha, 10k. here I'm doing 250k only. which is not healthy!

my spellsystem with 10k performs at 0.83ms \o/

[rotund token]

the one thing that stuck with from a talk about entities/burst a long time ago was that even if your game isn't that intensive you can still benefit greatly from the tech as it opens new possibilities.

imagine a fps exploration game where you're just running around a planet exploring. nothing is intensive about this. however imagine if you could now simulation a giant space firefight in the sky above the player. how cool of an experience would that be.

[pliant pike]

lol just imagine playing a game with 250k characters

I don't know what is happening but it looks cool

no mans sky basically does that currently

[viral sonnet]

yeah, even a 1000 characters on screen are total madness

[rotund token]

yeah but multiplayer games

could have multiple parts of the world simulating at once with lots of actors

[pliant pike]

ok yeah we might need it for the metaverse

[rotund token]

🤢

[viral sonnet]

there is no we in metaverse. only the suck

[viral sonnet]

huh, I actually don't know. Are DynamicBuffers thread safe?

[pliant pike]

I mean not if your looking through a single one

[viral sonnet]

so you can't write to a single one from multiple threads is what I've still in my head

[pliant pike]

you can do it, if you use disableparrarellforrestrictions or whatever its called

I've done it before when I know each thread isn't going to access the same elements

[viral sonnet]

Ok, I see. I can't ensure that. Parallel writing is such a pain 😄

[pliant pike]

yep it is, that's why I'm glad they have all the restrictions because it would be a mess if they didn't

[viral sonnet]

pretty much 😄

[viral sonnet]

How are you saving and getting jobHandles that are dependencies for other systems?

my method seems outdated with ISystem structs

[pliant pike]

personally I don't do that at all anymore

I just leave it to all automated

[viral sonnet]

hm, so how do you handle when a system writes to an IComp and another system later reads from it?

very simple example, I clear some trigger buffers in a job which is a dependency for another system that later writes to it

[pliant pike]

well I might sometimes use [UpdateAfter()]

[viral sonnet]

Do you schedule or Run the jobs? Because with scheduling, system OnUpdates don't care about job completion

[pliant pike]

and if I have say a chain of systems then I might use events to activate them one after the other using RequireSingletonforUpdate()

that's probably not the best way though

[pliant pike]

oh that is true, I did not think about that 🤔

[viral sonnet]

with SystemBase classes we can share the jobHandle as property but with an ISystem struct I'm actually pretty puzzled how that should work

[pliant pike]

I don't have that complex system dependencies yet I guess, but I would probably just let them run as and when they are scheduled and take the data that is there when they get it whatever it is

I'm just going with the view I want to keep dependencies as low as possible

[viral sonnet]

jobs have dependencies anyway, look at it this way. the better you setup dependencies, the more the unity worker process can figure out to run the jobs as tight as possible. otherwise waiting can occur on jobs that don't even need to wait

[pliant pike]

well I mean what's the advantage to doing it manually, If had lots of dependencies I would probably put them all in the same system anyway

[wraith urchin]

Hi im studying DOTS 0.50, and want to make a tower defense find target system, here is my code

protected override void OnUpdate()
    {
        Entities
            .WithAll<TurretTag>()
            .ForEach((Entity entity, ref TargetData target ,in Translation translation) => {
                float3 _position = translation.Value;
                Entity closestTargetEntity = Entity.Null;
                float3 closestTargetPosition = float3.zero;

                Entities
                .WithAll<DroneTag>()
                .ForEach((Entity droneEntity, ref Translation droneTranslation) => {
                    if(closestTargetEntity == Entity.Null)
                    {
                        closestTargetEntity = droneEntity;
                        closestTargetPosition = droneTranslation.Value;
                    }
                    else if(math.distance(_position, droneTranslation.Value) < 
                        math.distance(_position, closestTargetPosition))
                    {
                        closestTargetEntity = droneEntity;
                        closestTargetPosition = droneTranslation.Value;
                    }
                }).Run();
                target.entity = closestTargetEntity;
            }).Run();
    }

and i have this error

 error DC0029: Entities.ForEach Lambda expression has a nested Entities.ForEach Lambda expression. Only a single Entities.ForEach Lambda expression is currently supported.

I'm reading that Entities nested Entities querys its not allowed anymore, where can i read how to achieve this logic with ecs ? 😦

[safe lintel]

you can use ComponentDataFromEntity/GetComponent and get the query as a nativearray to iterate through

[viral sonnet]

I'm not pretending to give the best solution. Just a solution. First, make an EntityQuery so you can get all the entities with a DroneTag. use the EntityQuery.ToEntityArray to get this array. Then ForEach over all the TurrentTag and then iterate over the drones entity array utilizing GetComponent<Translation>

It'll be essentially the same thing you're doing. There are better solutions but if you're not running into performance problems, it's okay to do so

[wraith urchin]

i wil try

[ebon glade]

hello, i was wondering if anyone has advice on the best way to reference a system from a monobehaviour

i want to call a method i have made in the system from the mono

i can't use FindObjectOfType though because a system is not an object

[wraith urchin]

@safe lintel @viral sonnet Thanks ! Its works now

protected override void OnUpdate()
    {
        EntityQuery droneQuery = GetEntityQuery(ComponentType.ReadOnly<DroneTag>());
        
        var allTranslations = GetComponentDataFromEntity<Translation>();

        NativeArray<Entity> drones = droneQuery.ToEntityArray(Allocator.Temp);

        Entities
            .WithAll<TurretTag>()
            .ForEach((Entity entity, ref TargetData target ,in Translation translation) => {
                float3 position = translation.Value;
                Entity closestTargetEntity = Entity.Null;
                float3 closestTargetPosition = float3.zero;

                foreach (Entity drone in drones)
                {
                    if (closestTargetEntity == Entity.Null)
                    {
                        closestTargetEntity = drone;
                        closestTargetPosition = allTranslations[drone].Value;
                    }
                    else if (math.distance(position, allTranslations[drone].Value) <
                        math.distance(position, closestTargetPosition))
                    {
                        closestTargetEntity = drone;
                        closestTargetPosition = allTranslations[drone].Value;
                    }
                }
                target.entity = closestTargetEntity;
            }).Run();
    }

But now how can i achieve this with a more ScheduleParallel way ? Need I use jobs ?

[viral sonnet]

use ScheduleParallel instead of Run. i see no issues

you can cache the entityQuery in OnCreate btw and dispose the drones array after you're done

otherwise you have a memory leak

drones.Dispose(Dependency) will dispose it after the job is done

[safe lintel]

use TempJob allocator and WithDisposeUponCompletion and id probably schedule it as a job

should just be able to use ScheduleParallel

[cerulean pulsar]

Does anyone know how to cast a ray against a particular collider without using CollisionWorld?

I tried the following and I'm not getting a hit: physicsCollider.Value.Value.CastRay(raycastInput)

Using collisionWorld.CastRay(raycastInput) works but seems to be doing extra work, even if I use a CollisionFilter that filters out everything but the collider of interest.

Interestingly, physicsCollider.Value.Value.CastCollider(colliderCastInput) does work

[safe lintel]

you need to use an ICollector

oh without using collisionworld

[cerulean pulsar]

I only need a yes/no result

[safe lintel]

never not used the collision world

[cerulean pulsar]

It's weird to me because CastCollider works

But CastRay doesn't work, unless I'm doing a user error

[safe lintel]

Why can’t you use the collisionworld?

[rotund token]

are you accounting for the collider being in local space?

as in it's transform is likely around 0,0,0

[cerulean pulsar]

That's a really good point. I will check this tomorrow

[viral sonnet]

does anyone have an efficient 64bit bitmask?

[rotund token]

i have so many bitmask structs

[cerulean pulsar]

It worked. Thank you

[rotund token]

BitArray8, BitArray16, BitArray32, BitArray64, BitArray128, BitArray256

[viral sonnet]

hehe, I just want to see if my bitmask implementation is good enough

[rotund token]

FixedBitMask of any size you want, maybe you want 1000 length bit mask? this is the one!

you just implement your own struct

[StructLayout(LayoutKind.Explicit, Size = 32768)]
public struct BitMaskSize32768 : IFixedSize {}

public FixedBitMask<BitMaskSize32768> GiantBitMask;

[viral sonnet]

very cool, thanks

[rotund token]

and bam 32768 long bitmask 😄

As for the BitArray, can't guarantee optimal performance. as summary says i just took the one they used and made it work i burst/generic

Originally based off com.unity.render-pipelines.core@12.0.0\Runtime\Utilities\BitArray but made generic and burst friendly.

[viral sonnet]

64 flags is enough right now 😄 how can I look for 2 flags in a BitArray64? is there already a method for it?

[rotund token]

from memory you just create a new BitArray64 with the 2 flags set then compare

under the hood it's just a single ulong field so all the instantiation stuff gets compiled away

{

}```

for example

been a while since i've used it though

{

}```

if you were just checking flags

hmm looking at this should really add a ulong implicit conversion

[viral sonnet]

previously I just extended my 32bit flag with another int which was pretty lazy because I couldn't figure out the conversion. took another look at it and forgot that something like 1uL exists so no more double conversions. I'll probably end up using your library, HumanizedData is nice and all the operators. mine is the most basic form possible 😄

getting late, so heading to bed. I've reached a little under 16ms now from 20ms. Actually quite good but the room for improvement gets smaller and smaller. The biggest room for improvement is the HandleCombatEvents method. Setting the triggermap on source/target takes really long. No idea how to improve it though. Another one is just a HasComponent<SpellCaster> where I add pushback to casting spellcasters and the last one is the combatFlag which gets set multiple times

I still have some race conditions left in this method. Maybe the access from multiple threads to one pointer slows something down? I've never witnessed this slowing down. Just weird data output

[dusk crystal]

okay so I'm a noob to DOTS and I'm trying to set up an empty project with it so I can try making something using it and stick it on my portfolio.
https://docs.unity3d.com/Packages/com.unity.entities@0.17/manual/install_setup.html
I'm using this page to try and set it up. I installed the three packages it mentioned but I'm getting a bunch of errors (16 to be precise). Most of them revolve around not being able to find the namespaces Editor and ProfilerModuleViewController, abstract methods not being overridden, or overrides on methods that don't exist. I'm not sure what packages I'm missing, so I figured I'd check here since I feel like this is probably a complete noob error.

[shell berry]

You can run a job that collects your Drone data Burst compiled and in parallel. This is how I would do it (it's in chunk iteration though because it's what I know on top of my head):

This is DroneStash:
public readonly struct DroneStash {
public readonly Entity entity;
public readonly float3 position;

public DroneStash(Entity entity, float3 position) {
    this.entity = entity;
    this.position = position;
}

}

[white island]

Question: what’s the easiest way I can sync a component’s data with a GameObject’s?
My situation is complicated: I have NPCs that I want to move around even when not loaded in. When they are loaded in, the GameObject version of themselves takes control for the player to see and interact with. But when I unload the GameObject, I want an Entity to take over to pathfind between unloaded cells to simulate the NPC moving around off screen. To do this, I need to make sure that the entity keeps track of where the GameObject is, so that when it needs to take over it starts at the right spot.

[rotund token]

it's definitely odd to swap ownership but there's nothing stopping you swapping the CopyTransformToGameObject and CopyTransfromFromGameObject components

[shut pewter]

I'm wondering what's a good way to keep tabs on entities

Aka I have an ID for units that my client has, and I want to just transit that ID over the network

I used to keep a dict with references but I'm guessing ECS wouldn't like that

[rotund token]

from memory that's how netcode does it (nativehashmap not dictionary though). at least for mapping fields.

[dusk crystal]

okay so I think I figured out everything necessary to create a DOTS/ECS project, now I'm trying to figure out the actual workings of it all. So I know that an Entity class solely contains data about an object, and that functions are ran from elsewhere, but I'm figuring out how to actually create and implement those functions in an entity

[shut pewter]

I see, maybe that's a better way. I have another question: I have rather big tables (up to 150 fields) that define templates for Skills, Buffs and other.
Currently I load these from a SQLite database at server startup. My skill system just has the ID of said skill, and grabs the template as needed, but that's a call to a Dictionary outside of the system. I wonder what's a better way of doing it

The templates are probably too big for components. (and before you say I should make the templates smaller, I have tens of thousands of records for each, it makes it hard)

[shell berry]

Can you divide those 150 fields into their own groupings like if they go together? Each group would then be its own component. I wouldn't suppose that all skills and buffs need those 150 fields.

ECS is really good for modelling objects with similarity and differences.

[shut pewter]

Probably can, but most systems would need all of them

Where the actual differences are, I have other tables

I wish there was a guidebook of good practices for common problems 😄

[shell berry]

There was one. Just forgot the link.

[shut pewter]

I have a buff system which applies a list of tags (one example: I have 3 different buffs which have a freeze-like effect. They apply a "cold" tag which boosts fire damage).
What's a good way of knowing if an Entity has a tag, for example. I know there are empty components that can be used as tags, but what if I want a "valued" tag ?

I'll try to look for one but my project currently feels like it's a bad translation of my existing one rather than proper ECS because I probably use a lot of antipatterns

[shell berry]

What's a good way of knowing if an Entity has a tag -> One is by using the type in EntityQuery. If not, there's ComponentDataFromEntity.HasComponent().

[shut pewter]

Well, that's to get the tag as a component, not get the value, is it ?

Let's say my Entity has the tag 381, I know I can get every entity that has a tag, but not only those with a specific value

[shell berry]

If you need it frequently, you can store them in a temporary NativeHashMap or NativeMultiHashMap using the value as the key. This is pretty fast to populate as it can be run in parallel.

Eh, I can't paste code. Would like to show you something.

[BurstCompile]
public struct CollectHouseholdsByIncomeLevelJob : IJobEntityBatch {
    [ReadOnly]
    public EntityTypeHandle entityType;
         
    [ReadOnly]
    public ComponentTypeHandle<Household> householdType;
         
    [ReadOnly]
    public IncomeLevel.Converter converter;
         
    public NativeMultiHashMap<IncomeLevel, Entity>.ParallelWriter resultMap;
         
    public void Execute(ArchetypeChunk batchInChunk, int batchIndex) {
        NativeArray<Entity> entities = batchInChunk.GetNativeArray(this.entityType);
        NativeArray<Household> households = batchInChunk.GetNativeArray(this.householdType);
 
        for (int i = 0; i < batchInChunk.Count; ++i) {
            Household household = households[i];
            IncomeLevel incomeLevel = this.converter.ConvertFromIncomeValue(household.incomeValue);
            this.resultMap.Add(incomeLevel, entities[i]);
        }
    }
}

There, something like that.

[karmic basin]

Short answer is you can use the entities packages from MB so nothing's stopping you from exposing a public method from your system and then calling GetOrCreateSystem<MySystemContainingTheMethodIWantToCall>.MyMethod()
Long answer includes warnings about order of execution between systems and monobehaviours, but I'll let you enjoy that on your own 🙂

[karmic basin]

That looks nice 👍 Do you have it on a repo ? Bitmasks are like regexps for me. I love them but using them only once in a while so always loose time to refresh my memory beforehand 😅
A Facade like this would definitely help a lot of people

[rotund token]

i intend to release my core library soon~ but for now its not public

but you're welcome to use it, consider it unity license since it's based off some of their work

[karmic basin]

This one maybe ? https://learn.unity.com/course/dots-best-practices

[karmic basin]

Ok I'll yoink it but use it as a cheatsheet so it's not 100% magic for my brain 😄 Thanks again

[shell berry]

Yeah, that one. Thanks!

[ebon glade]

Thank you!

[pine plaza]

hi everyone

i wanna make a ai in dots, where should i start

[patent stag]

What's the current status of DOTS? Is it being actively developed? How ready is it?

[karmic basin]

Last month status announcement: https://forum.unity.com/threads/dots-development-status-and-next-milestones-march-2022.1253355/

But to quickly answer:

• current status is... experimental
• actively developed, yeah, though communication often lacking
• Collections, Jobs, Burst are 100% ready for production. ECS not yet in v1.0 but more and more stable.
  For other packages of the DOTS stack, it depends on your tolerance to handle experimental tech

Hope that helps draw the big picture

[patent stag]

Thanks that's very helpful

[worn valley]

I messed around a lot with 0.4 (long time ago version) and .17 and I never had any bugs that were Unity's fault. It's just annoying when they change stuff in future versions and it breaks your version due to relying on some specific methods.

[patent stag]

Good info, thanks. Where would you recommend start learning DOTS

It's not like there are complete tutorial series or books to buy for now

So I guess just the documentation?

[worn valley]

There is a complete tutorial based on 0.17 but I can't remember the website. I would just watch a few intros to it by YouTubers. It takes some time to get your head around DOTS and ECS but after you "get it" it makes it way easier to maintain good code.

[karmic basin]

Talking about this one I assume https://dots-tutorial.moetsi.com/

[viral sonnet]

If you heavy heavy usage of HasComponent, check out this thread: https://forum.unity.com/threads/componentdatafromentity-hascomponent-doesnt-use-m_cache.1270037/

[wraith urchin]

thanks a lot i will try it
But guys what do u think about this aproach ? My games its going to be like 20 turrets but 3k drones :p maybe i need to switch on dots.physics ?

public partial class TargetClosestDroneSystem : SystemBase
{
    protected override void OnUpdate()
    {
        EntityQuery droneQuery = GetEntityQuery(ComponentType.ReadOnly<DroneTag>());
        
        var allTranslations = GetComponentDataFromEntity<Translation>();

        NativeArray<Entity> drones = droneQuery.ToEntityArray(Allocator.TempJob);

        Entities
            .WithAll<TurretTag>()
            .WithReadOnly(allTranslations)
            .WithReadOnly(drones)
            .ForEach((Entity entity, ref TargetData target ,in Translation translation) => {
                float3 position = translation.Value;
                Entity closestTargetEntity = Entity.Null;
                float3 closestTargetPosition = float3.zero;

                foreach (Entity drone in drones)
                {
                    if (closestTargetEntity == Entity.Null)
                    {
                        closestTargetEntity = drone;
                        closestTargetPosition = allTranslations[drone].Value;
                    }
                    else if (math.distance(position, allTranslations[drone].Value) <
                        math.distance(position, closestTargetPosition))
                    {
                        closestTargetEntity = drone;
                        closestTargetPosition = allTranslations[drone].Value;
                    }
                }

                target.entity = closestTargetEntity;
            }).ScheduleParallel();

        drones.Dispose(Dependency);
    }
}

[viral sonnet]

makes sense to switch to a fast spatial system

you'd still need to iterate over a few ones but much better than every drone

if you don't want full physics, although, why not - there's the boid sample which does quantitzation and cell merging to reduce entities to check for distance: https://github.com/Unity-Technologies/EntityComponentSystemSamples/blob/master/ECSSamples/Assets/Advanced/Boids/Scripts/BoidSystem.cs

[shut pewter]

I use a simple chunk system to manage spatial interest -- My world is divided in 64x64 regions, every unit entering/leaving a region will send a network message to every relevant player. This reduces spatial queries from ~120k entities to usually 20-30

[safe lintel]

anyone having problems using Random inside of components in 0.50?

got a dirt simple component public struct Rng : IComponentData{ Random random } but its not getting added during conversion

[viral sonnet]

namespace problem? Is it really the one from Unity.Mathematics?

[white island]

what does the Entities.ForEach Lambda expression captures a non-value type 'this'. This is only allowed with .WithoutBurst() and .Run() error mean? It's all very cryptic.
The context is this:

Entities.ForEach((ref NPCWhereabouts whereabouts) =>
            {
                //TODO: GameObject in scene

                //Check if in destination cell
                if (whereabouts.cell.Equals(whereabouts.destinationCell))
                {
                    return;
                }

                whereabouts.moveToNextCellTimer += Time.DeltaTime; //count up timer
                //if timer done
                if (whereabouts.moveToNextCellTimer >= whereabouts.walkSpeed * 500) //500 is cell size for now
                {
                    whereabouts.moveToNextCellTimer = 0f;
                    whereabouts.cell = CellNavigationSystem.CellNodeAStar.getNextNodeInPath(whereabouts.cell, whereabouts.destinationCell); //set current cell to next cell
                }

            }).Schedule();

hit enter too early whoops

[haughty rampart]

it's not at all cryptic. it tells you exactly what's wrong.
what i think the exact problem is: it's that you capture Time.DeltaTime. move that outside

[white island]

ah ok I thought that might be a problem

it is cryptic even if it does tell me whats wrong lmaop

[haughty rampart]

why is it cryptic?

[safe lintel]

@viral sonnet yeah its the one from mathematics. it all compiles, just doesnt get added during conversion. zero errors, press play, rng component not there. used to work in 0.17? I had a package based around it but disabled it(for another reason) so just now seeing its not working for whatever reason.

[viral sonnet]

Just a guess. The AggressiveInlining could be a problem

You could try the older Mathematics random version and see if it works.

I use Random as a job property. Never used it in comps.

I'd suggest you just move this. You want it in a job anyway, right?

[haughty rampart]

yeah i'd recommend against storing random in components

[viral sonnet]

Scrap, the inling. The only diff I found between the 2 Random.cs versions was the [Il2CppEagerStaticClassConstruction] tag. I don't think the tag is related though.

[white island]

How do I use CopyTransfromFromGameObject? There is very little documentation

[viral sonnet]

The new CDFE has an internal for IsComponentEnabled and SetComponentEnabled 😄

[safe lintel]

it literally says this though Designed for minimal state (32bits) to be easily embeddable into components.

anyway i made a new test project and its working so now I guess the problem is something in my project is preventing this 😩

[viral sonnet]

Wait, is this copy/pasted? There's no public

[safe lintel]

no it was just pseudocode

sorry

it is actually public in both projects though

[rotund token]

A lot of the enable component stuff is 0.50

Including the interface you need to implement

(Which I am quite thankful it is opt in)

[viral sonnet]

What do you mean?

[rotund token]

From the implementation to make a component support enabling you need to make your component inherit from IEnableComponent (name made up, on the tram not on pc)

So only components with the interface support enabling

Which is good. I had huge concerns about performance when it seemed like this would be enabled on every component

Even if 95% you'd never disable

Obviously none of this is final but it's what the implementation looks like atm

Post it when I get home in like 10min

[viral sonnet]

thanks, good to know. Why do you think they don't tell us? Have you tested it if it's busted?

[rotund token]

Oh you can't use it at all

Entity query has no support in it

Completely stripped or just not added yet

[viral sonnet]

heh, well then 😄

[rotund token]

There are just randomly pieces left in

[rotund token]

    /// An interface for a component type which allows the component to be Enabled and Disabled
    /// </summary>
    /// <remarks>
    /// This interface is marked as "internal" during development of the feature. It will be made public when the feature is complete.
    /// </remarks>
    internal interface IEnableableComponent
    {
    }```

can go look at
ComponentEnabledBitFeatureTests.cs and ComponentEnabledBitTests.cs

for their whole suite of unit tests on this feature

[safe lintel]

blargh library delete doesnt solve my issue

[rotund token]

is this storing random on a component?

[safe lintel]

yeah

[viral sonnet]

any ideas how I could improve my terrible trigger flags method? I think I should use the actual bit-shifted flag values instead of calculating them and setting them one by one

[viral sonnet]

Oh well, I should've profiled this before. Removing the bitshift stuff makes pretty much 0 difference

[safe lintel]

ok cancel the outrage, it was all user error. forgot I had custom conversion system that ignored any authoring monobehaviours(but produced an almost 1:1 copy of what that authoring component did but without the rng component) 🤡

[viral sonnet]

glad you found it 🙂

[safe lintel]

literally thought i was going insane

[viral sonnet]

hehe, moving back and forth, WHERE'S THE DIFFERENCE??

I'm currently in disbelief about:

6.41 -> no setflags method
12.90 -> just spellId
14.00 -> all

just setting an IComp that has an int and ulong

currently my measurement is with a singlethreaded 250k loop that setting one CDFE takes 3ms

haha, as posted on the forum I've a custom CDFE implementation where I can get the ref value directly. Just tried the "normal" way which puts it from 14.00ms to 20.00ms.

people are wasting so much cpu time with the unity CDFE implementation

[safe lintel]

could you optimize your if statements with math.select?

[viral sonnet]

probably but sadly not worth it currently. losing too much on other things.

this is the heavy hitter

just getting the data and writing something back.

all the ifs and setting flags adds up to 1.1ms

just getting the ref and writing back is around 6.5ms

With the power to get Refs I'm also somewhat tempted to just cheat. getting RW pointers of course takes longer than RO but I absolutely do not care about any version or DidChange as they are next to useless to use anyway

[viral sonnet]

@rotund token Do you have an updated version of NativeMultiHashMapExtensions.cs? The one you sent the last time uses some variables that are not there anymore

[rotund token]

im not sure anything has changed but you do need internal collection access

[viral sonnet]

for this?

[rotund token]

yes

[viral sonnet]

How do I get internal collection access? I'm not aware what that is

[rotund token]

create an Assembly Reference

then put an AssemblyInfo.cs inside that directory like this

[assembly: InternalsVisibleTo("BovineLabs.Core")]```

except for your assembly

[viral sonnet]

ok, tyvm 🙂

[rotund token]

this way you can give yourself internal access to packages without having to make changes to the packages

[viral sonnet]

that's neat

[viral sonnet]

that was helpful, got it working! thanks a lot

[viral sonnet]

well, shit. building the NMHM takes too long 😦

I'm on 2ms with adding to the NativeLists and then around 13ms when doing ClearAndAddBatch

ah, using a persistent NMHM now. much better, down to 5.67ms for everything

[viral sonnet]

NativeList parallel writing performance is my downfall. I have a budget of 6ms as baseline with this whole HandleCombatEvent stuff which involves the processing of the events and writing to an UnsafeParallelBlocklist (for combatlog and stuff). Guess how long it takes to not process any events and writing to a NativeList instead? 6ms ... I really get no damn leeway

[rotund token]

NativeList parallel writing performance is my downfall
yeah the context switching on Interlockeds really kill this

if you want to make it fast you need to batch add a chunk of elements at once

<as always pull up my library>

            where T : unmanaged
        {
            idx = Interlocked.Add(ref nativeList.ListData->m_length, length) - length;
            ptr = (T*)((byte*)nativeList.Ptr + (idx * UnsafeUtility.SizeOf<T>()));
        }```

let you reserve a chunk on the native list at once

so you directly write to without constant interlocks or even memcpy all the elements per thread at once if able

[viral sonnet]

I've that piece of code already sitting in a comment at the top of a file but forgot about it. thanks for the reminder! still need to figure out how to use that correctly.

[white island]

Is it possible to like, link an entity and a GameObject? Like have them be able to reference eachother? I need the entity to keep track of whether a GameObject exists and where it is

Unless… maybe I’m taking the wrong approach.

[rotund token]

you can add managed objects to entities

EntityManager.AddComponentObject

alternatively you can have managed component data
public class ManagedComponent : IComponentData
{
public GameObject MyObject;
}

note the class not struct

[vagrant lotus]

What is the most up to date way to write code to react to collisions in 0.50?

[rotund token]

are you looking for something outside of just ICollisionEventsJob?

[devout prairie]

add a shocked emoji to your twitter bio

[vagrant lotus]

just ICollisionEventsJob

[devout prairie]

really need to catch up with 0.5 😐

still a little annoyed over the removal of animation

[calm edge]

cough 0.50

[wraith urchin]

Guys how can i have the Parent rotation value ?
i have this turret that aim a target, with some gun on init as a child, the guns spawn the bullets entities, already resolve the position but i cant copy the rotation of the turret to the bullets :C

protected override void OnUpdate()
    {
        float deltaTime = Time.DeltaTime;
        EntityCommandBuffer.ParallelWriter commandBuffer = m_EntityCommandBufferSystem.CreateCommandBuffer().AsParallelWriter();

        Entities
            .WithBurst(FloatMode.Default, FloatPrecision.Standard, true)
            .ForEach((Entity entity, int entityInQueryIndex, ref FireData firedata, in Rotation rotation,
             in Translation translation,in LocalToWorld localtoworld) =>
            {
                if(firedata.time >= firedata.fireRate)
                { 
                    /// Instantiate
                    firedata.time = 0f;
                    var instance = commandBuffer.Instantiate(entityInQueryIndex, firedata.prefab);
                    
                    /// Set Rotation
                    quaternion rot = rotation.Value;

                    commandBuffer.SetComponent(entityInQueryIndex, instance, new Rotation { Value = rot });

                    /// Set Position
                    float3 pos = localtoworld.Position;
                    
                    commandBuffer.SetComponent(entityInQueryIndex, instance, new Translation { Value = pos });

                                    }
                else firedata.time += deltaTime;

            }).ScheduleParallel(); 

        m_EntityCommandBufferSystem.AddJobHandleForProducer(Dependency);

[viral sonnet]

easiest is to use a ComponentDataFromEntity<Rotation> lookup of the parent entity

[hot basin]

do jobs work in the editor?

[viral sonnet]

of course, what do you mean?

[wraith urchin]

its work with this

protected override void OnUpdate()
    {
        float deltaTime = Time.DeltaTime;
        EntityCommandBuffer.ParallelWriter commandBuffer = m_EntityCommandBufferSystem.CreateCommandBuffer().AsParallelWriter();
        
        ComponentDataFromEntity<Rotation> allRotations = GetComponentDataFromEntity<Rotation>();
        
        Entities
            .WithReadOnly(allRotations)
            .WithBurst(FloatMode.Default, FloatPrecision.Standard, true)
            .ForEach((Entity entity, int entityInQueryIndex, ref FireData firedata,
            in LocalToWorld localtoworld, in Parent parent) =>
            {
                if(firedata.time >= firedata.fireRate)
                { 
                    /// Instantiate
                    firedata.time = 0f;
                    var instance = commandBuffer.Instantiate(entityInQueryIndex, firedata.prefab);

                    /// Set Position
                    float3 pos = localtoworld.Position;
                    
                    commandBuffer.SetComponent(entityInQueryIndex, instance, new Translation { Value = pos });

                    /// Set Rotation
                    quaternion rot = allRotations[parent.Value].Value;
                    commandBuffer.SetComponent(entityInQueryIndex, instance, new Rotation { Value = rot });
                }
                else firedata.time += deltaTime;

            }).ScheduleParallel();

        m_EntityCommandBufferSystem.AddJobHandleForProducer(Dependency);
    }

[viral sonnet]

great! call it with GetComponentDataFromEntity<Rotation>(true) as you'll be just reading. default is false

[wraith urchin]

oh i forget that , thanks ❤️ , because true its says that is read only right ?

[viral sonnet]

correct

and as I'm seeing you calculate with time and deltaTime. changing this to ticks and calculating an endTick for the next fire to go off will be a little more straightforward when it comes to rounding errors and offsets

easier for debugging or when you ever want to have some kind of replay function for example

[wraith urchin]

oh i can never read about tick , i will investigate that, the replay functions sounds really great

[viral sonnet]

it actually got a lot more simple with unity having added the VariableRateSimulationGroup. Just add 1 to a uint in a system OnUpdate.

heh, just finished implementing a RefEnumerator for NativeMultiHashMap. Much faster iteration on values. 6.8ms->4.3ms with 250k iterations

[wraith urchin]

i want to be like you some day :3

[viral sonnet]

you're on a good track with how fast you get things working

[wraith urchin]

so after 4hs im stuck with some physics trigger event detection :c
guys anyone can help me scheduling mi ITriggerEventJob 😦 ?

[UpdateAfter(typeof(EndFramePhysicsSystem))]
public partial class DamageHealthSystem : SystemBase
{
    [BurstCompile]
    partial struct TriggerJob : ITriggerEventsJob
    {
        public ComponentDataFromEntity<DamageData> damageData;
        public ComponentDataFromEntity<HealthData> healthData;

        public void Execute(TriggerEvent triggerEvent)
        {
            if (healthData.HasComponent(triggerEvent.EntityA) &&
                damageData.HasComponent(triggerEvent.EntityB))
            {
                HealthData health = healthData[triggerEvent.EntityA];
                DamageData damage = damageData[triggerEvent.EntityB];

                health.Value -= damage.Value;
                healthData[triggerEvent.EntityA] = health;
            }

            if (healthData.HasComponent(triggerEvent.EntityB) &&
                damageData.HasComponent(triggerEvent.EntityA))
            {
                HealthData health = healthData[triggerEvent.EntityB];
                DamageData damage = damageData[triggerEvent.EntityA];

                health.Value -= damage.Value;
                healthData[triggerEvent.EntityB] = health;
            }
        }
    }

    EndInitializationEntityCommandBufferSystem m_EntityCommandBufferSystem;

    protected override void OnCreate()
    {
        m_EntityCommandBufferSystem = World.GetOrCreateSystem<EndInitializationEntityCommandBufferSystem>();
    }

    protected override void OnUpdate()
    {
        TriggerJob triggerJob = new TriggerJob
        {
            damageData = GetComponentDataFromEntity<DamageData>(true),
            healthData = GetComponentDataFromEntity<HealthData>(),
        };

        var jobHandle = triggerJob.Schedule();

        m_EntityCommandBufferSystem.AddJobHandleForProducer(jobHandle);
    }
}

I tryied with this triggerJob.Schedule(stepPhysicsWorld.Simulation, ref buildPhysicsWorld.PhysicsWorld, inputDependencies);
but unity says that is a depecrated method

[rotund token]

remove this

ref buildPhysicsWorld.PhysicsWorld,

you should also have your
this.RegisterPhysicsRuntimeSystemReadOnly()
in OnStartRunning

also i dont think
[UpdateAfter(typeof(EndFramePhysicsSystem))]
is doing much (you probably get a warning) as you aren't updating this system in the fixed simulation group

[viral sonnet]

hey tertle! currently trying to wrap my head around making ReserveNoResize work in a parallel thread. Have you ever used the method in parallel?

[rotund token]

i wrote it for parallel

it serves minimal use single threaded

you can just get the pointer and write to that

[viral sonnet]

ok cool. How did you figure out the length with which to increase? The overall size of the NativeList is unknown to me so I just increase by 20 for now. I have written a wrapper that keeps track of where it's at which gets instanced for every thread. Maybe I'm overcomplicating it

[safe lintel]

btw tertle, how does RemapEntityFields work? im unsure of the byte ptr, where I get the offsets and offset count from. do you deserialize into a separate world and then move them into the main one?

[rotund token]

nah i do it all on main world

it's fast enough

let me answer 1 question at a time quickly before i have to do this standup

[safe lintel]

np

[rotund token]

i know the length of the list beforehand because i precompute the size i need

as for each thread, generally when i use this i already know how much data each thread needs to write

for example stream.RemainingItemCount * UnsafeUtility.SizeOf<Event>()

so i just reserve the entire space I need once off per thread

[viral sonnet]

alright, thanks! I've some constraints that make it a little tricky. I'll get there, performance looks much better when allocating in blocks. Nearly on par with NativeStream/UnsafeBlockList

[rotund token]

ok are you talking about my RemapEntityFields method i think i posted at one point or other

[safe lintel]

yeah

[viral sonnet]

btw. I wrote a RefEnumerator for NMHM in case you're interested. Iteration is much faster

[rotund token]

TypeManager.EntityOffsetInfo* offsets = TypeManager.GetEntityOffsets(this.TypeIndex, out int offsetCount);

typemanager has all the info for you

which is why it's so easy to implement

[safe lintel]

neat, that part sounds easy enough. what about the ptr?

[rotund token]

sorry mine is a bit different now but i think i posted this one most recently in here from history

            private void RemapEntityFields(ref T data, TypeManager.EntityOffsetInfo* offsets, int offsetCount)
            {
                var ptr = (byte*)UnsafeUtility.AddressOf(ref data);
                for (var i = 0; i < offsetCount; i++)
                {
                    var entity = (Entity*)(ptr + offsets[i].Offset);
                    *entity = this.Remap.TryGetValue(*entity, out var newEntity) ? newEntity : Entity.Null;
                }
            }```

[safe lintel]

is the ptr the raw component data?

[rotund token]

it's just whatever component you're working on

(that T would have a struct etc restriction on it from its class)

            RemapEntityFields(ref tr, offsets, count)```

(where tr is actually your deserialized data instead of a default field)

(obviously translation has no Entity component on it)

[safe lintel]

really appreciate it. got serialization working(on the mainthread) thanks to you so I think entity remapping is the last part to get it fully working

one more question not related to serialization: did you modify the physics package at all for your custom debug display? was just browsing the builtin debug code and the mesh part of the collider was internal

[viral sonnet]

tertle learned me yesterday you can get internal access to packages. guess that's what he did to write the extensions

I can quickly move you through it if you want

[safe lintel]

oh the asmref thing, yeah will try that

[viral sonnet]

right, if you get stuck somewhere just ask

[viral sonnet]

When your Unity constantly crashes on every play, takes ages to start and cries about some temp lock, you know you are in low level territory. 🤣

[wraith urchin]

i made it thanks ! 😄 actually the changes was at the final of this post 😄 https://docs.unity3d.com/Packages/com.unity.entities@0.50/manual/upgrade-guide.html

now i can damage drones with bullets 🔥

[boreal forge]

I'm having some trouble with the conversion work flow using IConvertGameObjectToEntity and I was wondering if you guys have any thoughts. I'm trying to generate a mesh in MonoBehaviour "land" and assign it to the object in the convert method using RenderMeshUtility.AddComponents from the hybrid renderer. And when I run the program, my entity doesn't even look like it has a mesh component in the DOTS hierarchy.

public Mesh mesh;
public Material material;

public void Convert(Entity entity, EntityManager dstManager, GameObjectConversionSystem conversionSystem)
{
    Debug.Log("Converting");

    GenerateMesh();

    var desc = new RenderMeshDescription(
       mesh,
       material,
       shadowCastingMode: ShadowCastingMode.Off,
       receiveShadows: false
       );

    RenderMeshUtility.AddComponents(entity, dstManager, desc);
}```

Is there something I'm missing to set up a mesh renderer for hybrid?

[white island]

iirc you have to set shared component for something

dont remember offhand

[boreal forge]

it looks like the docs say you only need RenderMeshUtility https://docs.unity3d.com/Packages/com.unity.rendering.hybrid@0.50/manual/runtime-entity-creation.html

but I could be reading it wrong

[boreal forge]

I put the same code on a new object and it seems to be working? I'm not sure what the difference is between the 2 objects though.

[white island]

my knowledge could be outdated

[white island]

Wait. Can I run the Foreach multithreaded??

that was the entire point wasnt it

[white island]

Ok so. How would I go about writing a burst-able/blittable A* implementation. a bit of a tall order but

[rotund token]

it's actually not that hard

i guess my question is, what part of it needs to be managed?

[hot basin]

probably nothing NEEDS to be managed, but often it has a lot of references

[hot basin]

I probably would make array/dynamic buffer of nodes and use index as "reference" to another node

[rotund token]

an a* implementation just happened to be one of the first major jobs i ever wrote

looking back at the code now it's pretty gross

however it was still a relatively easy thing to implement in jobs

[rotund token]

i would argue you've maybe already gone down the wrong path with this. not because your idea is wrong but because your 'world' doesn't need to be tied to the A* implementation at all

they're different problems

there's nothing stopping you making

    where T : struct, IWorldGrid
{```

with IWorldGrid having methods like
NativeArray<Node> GetNeighbours(Node);
etc

then you can re-use your implementation regardless of your world

freely switch between grid based, point grid, navmesh etc

(obviously if you stumble upon my first implementation i mentioned it does nothing like this and it was hard tied to a world grid. this is just how I would tackle the problem now with a lot more experience)

[viral sonnet]

We should bring the fast parallel NativeList method out into the world 😄 It's so much more convenient than having to rely on NativeStream which has slow iteration. I have lazily used tertles namespace 😄

With this you can have NativeStream write speed in parallel threads and not have to do any data transformation. NativeStream of course has still it's uses

[cerulean pulsar]

How did you get past the RetainBlobAssetSystem.cs This method should have been replaced in codegen error? The system has the partial keyword already. Got past the profiler errors. (Assuming you had this error when you tried it back then)

[rotund token]

Oh for the record I have allocated large methods for native stream as well to greatly speed it up 💩

[rotund token]

are you using the 2022 branch someone made?
https://github.com/Extrys/UNITY-DOTS-0.50-2022-beta-compatibility

[cerulean pulsar]

I tried that and got this error. Also got it manually. Diff checked that .cs and it was the same

[rotund token]

not sure

[cerulean pulsar]

Interesting. Thanks for that repo by the way

[rotund token]

i tested it when it first came out and it worked fine for me but i'm not using it so haven't tested on newer versions

[cerulean pulsar]

Gotcha

[rotund token]

(it's not my repo)

[cerulean pulsar]

Yeah

[rotund token]

totally possible any update to 2022 could break it

[cerulean pulsar]

Yeah that's fair

My chasing of 2022 isn't even justified by my project. Gonna concede and do 2020

[rotund token]

i hope 0.51 comes sooner than later

but still got the whole patch first probably so not holding my breath

i have to reimport 2 files every time i load my project atm due to dodgy ui toolkit in 2020

[viral sonnet]

I've been using 2020.3 for so long now I already developed Stockholm Syndrome

[hot basin]

I would like to read some of your code 😄 I think I'm missing some fundamentals here

[hot basin]

that's why I'm on 2021.2 and waiting for 0.51

as 0.50 has nothing huge really

[viral sonnet]

sadly true

[rustic rain]

wdym nothing huge?

aren't all those debugging tools are big upgrade?

[pliant pike]

yep they are to me

[hot basin]

they are definitely helpful but are they huge?

i think enabled/disabled component would be huge

and maybe worth staying on 2020.3

but beside ui toolkit I have dependency on my animation gpu rendering which is using 2021.1+

so yeah, I will wait for 0.51

[misty wedge]

Is something like the result of a pathfinding search better stored in e.g. a DynamicBuffer on the entity that requires the path, or inside a system somehow?

[pulsar jay]

Always better to store your data in entities. So yes DynamicBuffer would make sense

How is it possible that query.ToComponentDataArray causes an InvalidOperationException because of a previous job? It isnt even a job itself so why does it conflict with one?

[worthy rampart]

if you are going be processing it with other data from the entity it probably makes the most sense living in a dynamic buffer

you didn't really give enough info tho

[safe lintel]

@pulsar jay whats the full error?

[pulsar jay]

InvalidOperationException: The previously scheduled job PlaceTowerSystem:PlaceTowerSystem_LambdaJob_1_Job writes to the ComponentTypeHandle<CellPosition> PlaceTowerSystem_LambdaJob_1_Job.JobData.__cellPositionTypeHandle. You must call JobHandle.Complete() on the job PlaceTowerSystem:PlaceTowerSystem_LambdaJob_1_Job, before you can read from the ComponentTypeHandle<CellPosition> safely.

[safe lintel]

I think you need to use ToComponentDataArrayAsync to safely use the array

[pulsar jay]

But the error is thrown in a LateUpdate call from a MonoBehaviour

[white island]

In theory, nothing?

I’ve also never written an A* implementation before that worked hahah

Can jobs use pointers? It’s unsafe already

Sort of like one of those fancy linked lists

[coarse turtle]

Yea jobs can use pointers but you'll need to add:

[NativeDisableUnsafePtrRestriction]
public T* Field; // Where T is unmanaged

If you're thinking of using a linked list you can, but I find that you can use create a linked list with 2 arrays. 1 array stores a compact version of the node information, and the other array stores how each node links to each other (storing a pair of indices). You can turn the index pair into a single index if you only care about a singly linked list.

[ NodeInfo, NodeInfo, NodeInfo, NodeInfo ]
[ (-1, 2),  (2, 3),   (0, 1),   (1, -1)  ] // (previous index, next index)

[white island]

Why use indexes at all?

[coarse turtle]

Avoids using pointers in case you need to realloc. If you keep your indices to 2 bytes (short), you also store less information so the information for the next and previous node in total is 4 bytes, compared to a pointer which can be 4/8 bytes wide depending on the machine's architecture. Also the indices tell you the order of how the linked list will flow if you want to read forward/backwards.

imo, it's easier to manage compared to pointers (because you can use something like 2 NativeLists). It's still doable with pointers, but I also I don't want to manage a series of pointers if you have to manually allocate the address to have that content.

If you allocate a large space at once because you want to compact the information together and then find out you need to allocate more, you now need to allocate an additional node which will live in a separate memory space. You can take an extra step to compact the memory together, but that's more hassle as you need to relink your nodes with the correct pointers. 🤷

[white island]

I see. So for a node’s connections, I would have
-short for start index
-short for end index
-byte for cost
And then for a working path I would just have a queue of indexes to the list of connections

[coarse turtle]

Yea

theoretically you can make the start & end index smaller using an sbyte, but that depends on the total number of nodes you want to have, since sbyte can only support between [-128, 127]. If you're going to have well more than 127 nodes, then short would work. (Otherwise keep increasing the datatype or feel free to use unsigned versions and treat the MaxValue as the beginning/end of the linked list).

[white island]

Yeah I already considered using a byte but I already know I’m going to have more than 256 connections

Nodes, rather

[pulsar jay]

Is there a way to early out of a Entities.Foreach? In a forum thread Joachim suggested to use return but return only ends the execution on the current entity it does not stop the iteration

Entities
  .WithSharedComponentFilter(map)
  .ForEach((in CellPosition cellPosition, in CellHeight cellHeight) =>
  {
    if (cellPosition.Value.Equals(cell))
    {
      height = cellHeight.Value;
      return; //Does not stop the foreach
    }
}).Run();

[viral sonnet]

You'd need to manage with an outside variable to break the whole ForEach

[safe lintel]

would that work? wouldnt the foreach still just run to completion?

[pulsar jay]

height is a captured variable. but that does not stop the iteration

[viral sonnet]

no, you'd need like a bool iterate = true; and set it to false in the if

it'll still run though but never executed then

for anything special use IJobChunk/IJobEntityBatch

[pulsar jay]

oh you mean like if(found) return;?

mhh would work I guess although a bit stupid that it would still need to check that condition n times before ending the iteration 🙄

[viral sonnet]

there's not really an early out, like break in a loop

[pulsar jay]

Wierd that Joachim proposed that return would work: https://forum.unity.com/threads/query-a-value.801915/#post-5340177

[viral sonnet]

technically it is an early out 🙂

your job code is lightweight, with something more complex this makes sense

very similar: https://forum.unity.com/threads/how-to-break-out-of-entities-foreach.890284/

I must say, it's kind of an oversight for break to not work in Entities.ForEach but that's the trade-off with auto-magic code

otherwise the outside loop would have to check a condition everytime even when not needed

[safe lintel]

is there a reason you are using run? think id agree that just return on a burst job would be really fast and more or less the same as just breaking the loop unless you are working with a truly exorbitant amount of entities

[pulsar jay]

It is a utility method. And I would not be able to wrap it in a method with a return value when using Schedule would I?

The thing is I still seem to run into a race condition with another job writing to the CellPositions which should not happen when this code is running on main thread afaik?!

[viral sonnet]

Not with Run, no.

[pulsar jay]

How is this even possible then? #archived-dots message

The error is thrown in the Foreach Run 🤔

[viral sonnet]

The other job has Schedule, ain't it?

[pulsar jay]

yes

[viral sonnet]

Schedule moves the job to the worker process and doesn't block the main thread. It's not finished when you Run the other job on the main thread.

So either you use the jobHandle from the first job as dependency for the second or Run both

or you just call Complete() on the first job handle. Run doesn't have a dependency as input

So scheduling one and then Run another when they have a dependency doesn't work

[pulsar jay]

Wow that would make Run pretty dangerous or maybe even useless 🤔 Always thought that all the jobs will be finished before the MainThread jobs are being run

Maybe this is a special case as it seems to be run from LateUpdate?

[viral sonnet]

nope, worker thread jobs can run for at least 1 and even 2 frames.

they are truly async

Run is not useless, it just expects everything to be finished. Try Dependency.Complete() before the Run task.

[pulsar jay]

That would mean I would have to get the dependency handle from the other system?

Because I cannot possibly know which systems will write to the compnents used in the query. Could be many

[viral sonnet]

Dependency = new JobStruct().Schedule(Dependency); will chain that for you

[safe lintel]

you said its a utility method? I think you could schedule it

[pulsar jay]

but how do I get the return value then? And isnt Dependency managed System class wide? So it would merge the dependencies of all queries together?

[safe lintel]

what are you returning in the original utility?

[pulsar jay]

The height

    public float GetCellHeight(MapComponent map, int2 cell)
    {
        float height = 0f;
        Entities
            .WithSharedComponentFilter(map)
            .ForEach((in CellPosition cellPosition, in CellHeight cellHeight) =>
            {
                if (cellPosition.Value.Equals(cell)) height = cellHeight.Value;
            }).Run();
        return height;
    }

[safe lintel]

hm maybe not then. suppose its how its used, I have done something there I get the result from a job as a nativearray, and use the result on the main thread in the next frame, then clear nativearray and schedule.

[pulsar jay]

Seems like my design with a System containing multiple utility methods is pretty flawed as "magic" dependency management seems to heavily rely on systems 😬

[pulsar jay]

Found in the docs that Run actually waits for all dependencies to complete: "When you call Entities.ForEach.Run() the job scheduler completes all scheduled jobs that the system depends on before starting the ForEach iteration"
So this should not happen (normally) but I suppose the dependency management just breaks when calling a Systems public method from somewhere else than its own OnUpdate

[viral sonnet]

Ah alright, good to know. I've never mixed them or used them outside of SystemBase. Did you solve it?

[white island]

Alright, so I'm trying to make an AStar algorithm that's able to be done through jobs. So far I've gotten this:

    struct Node
    {
        public FixedString32Bytes cellID;
        public int3 coords;
    }

    struct Connection
    {
        public ushort start; //index of start
        public ushort end; //index of end
        public byte cost; //cost of traversal
    }

    public static class CellNodeAStar
    {
        static NativeArray<Node> nodes; //Nodes in graph
        static NativeArray<Connection> connections; //Connections

        public static void generateNodeGraph()
        {
            //TODO: Deserialize graph
        }

        public static void pathFindThroughGraph(FixedString32Bytes startID)
        {
            //Queues
            NativeQueue<ushort> openList = new NativeQueue<ushort>(Allocator.Temp);
            NativeQueue<ushort> closeList = new NativeQueue<ushort>(Allocator.Temp);
            NativeQueue<Connection> workingPath = new NativeQueue<Connection>(Allocator.Temp);

            //AStar


            //Dispose
            openList.Dispose();
            closeList.Dispose();
            workingPath.Dispose();
        }
    }

Where do I go from here? I understand the algorithm conceptually, I'm just not entirely sure where to begin.

[white island]

Based off of this approach

And I just realized workingPath should be indexes to the connections rather than a duplicate of the connections

[wraith urchin]

Hello again guys 😄 i'm trying to upgrade some repo of dots boids simulation
https://github.com/BadGraphixD/How-many-Boids-can-Unity-handle
So i think that i can achieve the task but
How can i upgrade IJobNativeMultiHashMapMergedSharedKeyIndices a job with this interface jajaja where can i start looking for

[coarse turtle]

What's your input? I don't remember the exact details on how A* is implemented, but typically I've seen the initial set of nodes implemented with a priority queue.
You'd check neighbors, keep a history of where you came from, and calculate the cost so far as long as you have neighbors to keep checking.
(I'd say I'm very rusty with A* since it's been a while since I had to implement it from scratch 😅 )

[devout prairie]

I guess using pointers would in some way negate the benefits of using a typical dots data layout right?

As it becomes random access..

[rotund token]

This job interface doesn't exist anymore. You need to make a copy of it from the old version if you want to use it.

[wraith urchin]

But i mean there is a way to upgrade it in a new version way right ?

[rotund token]

No

[coarse turtle]

if the pointers are just malloced on the fly, yea. If you can guarantee that the next pointer is the next offset from the previous pointer, then i'd say it's more of a typical dots layout. But that's rarely the case unless you realloc a larger buffer size and cpy from src -> new pointer. 🥴

[rotund token]

Skyrbunny is your world grid based or do your nodes have unknown number of connections

[devout prairie]

yeah i was curious how that might work and thinking the same thing, it might be possible to line up the pointers but then maybe as well just using a native container for the data in that case anyway

[rotund token]

how do people feel about something like this

    {
        public Action<int, NativeArray<byte>> Write;
        public Func<int, Task<NativeArray<byte>>> Read;
    }```

i look at this and instantly think gross

however except for reflection looking for an interface implementation i can't think of another good solution atm

context : been working on a universal saving solution (that has turned out great) but i need to be able to allow file writing/reading methods to be supplied so a user could handle it differently on different platforms