archived-dots

[robust scaffold]

Sure, I could have a citizenship component located on every single population entity but that will waste a fuck ton of memory and wont be easily vectorized

Because the citizenship random access the national interest float, which I then vectorized apply to all bank accounts within the population of that chunk

[sinful cipher]

So wait...
Whenever you create a new Citizen entity, do you add the Citizenship chunk component immediately, or at a later stage?

[sinful cipher]

related: do you create new Citizen entities within a job?

[sinful cipher]

This kind of sounds like what I'm attempting to do: minimizing random access

[robust scaffold]

1. I create a citizen entity then add a <SharedCitzenship> component.
2. I set that <SharedCitizenship> component (which is just a single entity) to the country entity it is a citizen of.
3. Then in another IJobEntityBatch (not bursted and on the main thread), I read the <SharedCitizenship> of that chunk then set the chunk component <ChunkCitizenship> to that value.

[robust scaffold]

Yep. I spent 6 hours last night converting my entire loading sequence to a job thread.

[sinful cipher]

Hahaha I knew it! I was thinking of something similar, but wasnt sure if this was the dumb way of doing it.

SharedCitizen is just IComponentData though, not ISharedComponentData?

[robust scaffold]

It's a shared component data, hence the prefix

private struct MirrorStaticToChunk : IJobEntityBatch
{
    public EntityManager Em;
    [ReadOnly] public SharedComponentTypeHandle<CitizenshipShared> CitizenshipShared;
    public ComponentTypeHandle<CitizenshipChunk> CitizenshipChunk;
    [ReadOnly] public SharedComponentTypeHandle<AreaShared> AreaShared;
    public ComponentTypeHandle<AreaChunk> AreaChunk;
    public void Execute(ArchetypeChunk batchInChunk, int batchIndex)
    {
        var countryEntity = batchInChunk.GetSharedComponentData(CitizenshipShared, Em).Country;
        batchInChunk.SetChunkComponentData(CitizenshipChunk, new CitizenshipChunk(countryEntity));
        if (!batchInChunk.HasChunkComponent(AreaChunk))
            return;
        
        var areaIndex = batchInChunk.GetSharedComponentData(AreaShared, Em).Index;
        batchInChunk.SetChunkComponentData(AreaChunk, new AreaChunk(areaIndex));
    }
}```

[viral sonnet]

Like, archetype component? 🙂

[sinful cipher]

Oh wait I see -- it doesnt happen in a job

[robust scaffold]

what

[viral sonnet]

i made that suggestion a while ago. not that it exists, sorry

[robust scaffold]

thats not a thing... right? A component that can fracture chunks

oh, hahaha. Made me worried for a sec

[sinful cipher]

I mean burst job

So wait, I can access SharedComponentData in non-burst jobs?

[robust scaffold]

Nope. Just a "regular" threaded job

[robust scaffold]

Must be on the main thread. Basically a glorified for loop.

Not an actual "job".

[sinful cipher]

Haha yeah thats what I was thinking: this doesnt necessarily need to be a job at all

Hmmm....

[robust scaffold]

new MirrorStaticToChunk
{
    Em = EntityManager,
    CitizenshipShared = GetSharedComponentTypeHandle<CitizenshipShared>(),
    CitizenshipChunk = GetComponentTypeHandle<CitizenshipChunk>(),
    AreaShared = GetSharedComponentTypeHandle<AreaShared>(),
    AreaChunk = GetComponentTypeHandle<AreaChunk>()
}.Run(GetEntityQuery(_sharedChunkDesc));```

Well yea, you can do this using good old fashioned For() and ForEach() loops but eh

[sinful cipher]

Hey thanks a lot, this clears things up quite a bit.

Time to tear down some stuff again lol

[viral sonnet]

that's new

[robust scaffold]

yea, I've been working on this project of mine for.... 7 years now? Started out using javascript, went to GO unity, then blindly fumbling around in DOTS, and now I'm neck deep in vectorizing code

[viral sonnet]

can't schedule IJobEntityBatchWithIndex with a batchcount of more than 1

[robust scaffold]

Null reference from within DOTS? What? How

[viral sonnet]

lol, that's quite a ride

[sinful cipher]

I also had something nice going in Unity with regular C#, but performance was abysmal. Hence I'm now here counting bits and what not

[robust scaffold]

right, i need to figure out if bitwise operations is vectorized

if it is, that unlocks a whole new world of premature micro-optimization

[sinful cipher]

"premature" lmao

[robust scaffold]

Does the entity query you are operating on have entities within it? If .GetCount() is throwing an error, that typically means there are no entities. Well, the job shouldnt be running though.

[robust scaffold]

DOTS is a premature optimizer's wet dream

[viral sonnet]

i tried spellCasterQuery.CalculateEntityCount() and returns on 0

lol not sure what's going on. it's only doing that if I increase that batchcount

[robust scaffold]

huh, that should be an early return then. IJobEntityBatch was introduced with 0.17 and "fresh" out of the oven. IJobEntityBatchWithIndex even more so.

[sinful cipher]

Optimization can be kind of addicting. Especially if you have this excellent profiler that tells you exactly where time is spent.

[robust scaffold]

Yea. Solid green bars filling the screen. So beautiful

[viral sonnet]

damn, the algorithm needs the index :/

[sinful cipher]

10 ms becomes 1 ms
best feeling

[viral sonnet]

lol yeah

[robust scaffold]

batch index?

[viral sonnet]

the indexOfFirstEntityInQuery so I can safely write to the nativestream

[robust scaffold]

When I finally got the first loop vectorization to work and 99ms got cut down to 6.7ms, holy shit. Bottle that feeling and sell it instead of what I'm making

[robust scaffold]

Hrm, true. Well, you usually dont need more than 1 batch per chunk... usually.

[viral sonnet]

We run it on a loop through KornFlak's mind. And the chemical it makes his brain secrete goes into every KornFlak's Simple Wafer's Wafer Cookie. Come home to the impossible flavor of your own completion.

[sinful cipher]

When they stated 25x or 50x performance improvements in some of the first DOTS presentations, they werent kidding

[robust scaffold]

Yea but the codebase I'm "converting" is made in the most dense Object Oriented style I have ever seen (written in Java).

The inheritance of some of these classes are 8 classes deep

and now I'm trying to convert it into the most dense ECS format for performance. It is quite the challenge

[sinful cipher]

Kind of crazy how late i learned about the whole CPU cache relation with performance. It was a little before DOTS came out, so when it did, it was as if they knew I wanted this.

[sinful cipher]

lol

[robust scaffold]

https://github.com/uwol/computational-economy/blob/master/src/main/java/io/github/uwol/compecon/economy/sectors/financial/impl/BankImpl.java

I mean look at this nonsense

classes within classes within classes within classes. Relational values that relate to other relational values

[viral sonnet]

that's why devs hate OOP

[sinful cipher]

My approach is currently a bit more pragmatic perhaps. I have like a time budget for a particular set of systems (e.g. <1ms / update), and also try to favor ease of reading & maintainance instead of performance if possible

[sinful cipher]

I noticed especially with Java the class inheritance hierarchy can get very deep.
Usually not so bad with C#, although the feature set is pretty similar 🤔

[robust scaffold]

yep, and so many relational values. Bank account of one changes means a logging method and a dozen other values also change

and it's all delegate functions so they're dynamically called

And I'm gonna figure out how to parallel and maybe even vectorize a clearing house market exchange for millions of entities...

[sinful cipher]

Is this all old stuff you wrote yourself that you're converting now?

[robust scaffold]

Nope. This is some rando's online that I cant even get in touch with

I just like it and I wanna build a game on top of. It's the most feature complete entity based economic simulator out there

[sinful cipher]

I see 🤔

[robust scaffold]

When I first found this about... 3 years back. I thought: Entity based? Wow, that'll be easy to convert to Entity component system. hahaha. no

[sinful cipher]

Its like learning how to program all over again

For me, things were going well until I added 3 more such planes. I was already at 35 fps by then on a decent PC.

[robust scaffold]

VR? Nice. Reminds me of War Thunder kinda

[sinful cipher]

Not VR. The clouds are proper volumetric ones though. Spherical planet/world. Decent aero simulation (although still a really rough approximation)

Not quite sure where I'll take it but I couldnt experiment any further due to said performance issues

[robust scaffold]

Bitwise operations are vectorized.

This is revolutionary, kinda

[sinful cipher]

I cant read any of this haha. When did you learn to read this stuff?

[robust scaffold]

i have no clue what vmoveXXXXX or whatever are. But they're purple and not green and have v in front for vectorized operation

[sinful cipher]

left column is CPU instruction and right column is arguments?

[robust scaffold]

yea, it's assemby

but i am monkey. I see purple. I am happy

[sinful cipher]

I see

[sinful cipher]

haha 👍

[robust scaffold]

The goal is the least amount of operations and the most amount of purple

[robust scaffold]

Yep. Bitwise flag check operations are also vectorized. Woooo

This might help with that other guy, @viral sonnet . Have you considered bitwise flags?

[viral sonnet]

ah sick, not really but i'll plan to

[robust scaffold]

This is what my testing method call looks like

[robust scaffold]

And this is the "gold" standard of vectorization. 3 lines. Vectorized addition and set.

of course, it aint doing much but it's real fast

[sinful cipher]

damn i was about to ask, does this do the same as in the previous screenshot?

[robust scaffold]

Nope, that's the original code (before testing some other stuff)

[sinful cipher]

ohh right

[robust scaffold]

that will return true for ExpectVectorization(). Really simple code

The bitwise check and select code will not

[stiff skiff]

Why the unsafe pointer over the native array?

[robust scaffold]

As mentioned earlier with Enzi: "Burst's auto-vectorization works entirely on pointers. It doesnt seem to understand that ref NativeArray<float> is identical to float*. The pointer will get vectorized. The ref Nativearray will not."

I was unable to get any purple with ref Nativearray, give me a sec

[stiff skiff]

Wow, thats pretty bad

[robust scaffold]

Huuuuuh

it works

[stiff skiff]

Also, instead of in float doing a normal float would likely work better as well?

[sinful cipher]

I should remember this.

[stiff skiff]

Should remove a pointer deref in vaddss

[robust scaffold]

actually no

actually I have no fucking clue what is going on

[stiff skiff]

I can help there

It's doing loop unrolling as well now

[robust scaffold]

there seems to be two different addition commands

[robust scaffold]

good?

unrolling seems good

unrolling in shaders always good

[stiff skiff]

Yes in this case that would be good

[sinful cipher]

lol

[robust scaffold]

Original float* code

i dont think there's any difference....

[stiff skiff]

Ah right

It has an unrolled loop with vectorization here

and a bit to do the rest

[robust scaffold]

Removing the in float

Moving to ref NativeArray<float>

alright, no difference between the float* and ref NativeArray<float>

Good to know

alright, lesson today: remove the in parameter, it doesnt seem to do anything useful

[stiff skiff]

Since it's using LLVM to do these optimizations, it's pretty smart at optimizing code

[robust scaffold]

I'm still very new to vectorization. God I wish Unity would publish a list of all commands and operations vectorized.

[stiff skiff]

Though I wish the debug tools were a little better to help people understand what is happening, and what got optimized

[stiff skiff]

What do you mean with that?

[robust scaffold]

Some commands can be vectorized, some can not. Like bitwise operations (conditionals and movement) are vectorized (just learned a few minutes ago) but some other operations are not

[sinful cipher]

Wouldnt this be determined by a CPU's instruction set?

[stiff skiff]

Ah right, thats not "really" a Unity thing

Also conditions "can" be vectorized

just not automatically (I think..)

[robust scaffold]

Like setting the int value of a pointer to 0. Can not be vectorized.

Interlocked isnt vectorized either

[robust scaffold]

then there's got to be a list somewhere....

[stiff skiff]

Of all cpu instructions 😛 ?

[robust scaffold]

Of all v- prefixed operations yea

[sinful cipher]

idk you probably already went here https://en.wikipedia.org/wiki/Advanced_Vector_Extensions

[robust scaffold]

Yea I saw that but that can not be all of it.

[sinful cipher]

https://docs.oracle.com/cd/E36784_01/html/E36859/gntbd.html

[robust scaffold]

https://www.intel.com/content/www/us/en/docs/intrinsics-guide/index.html#

Great. Now I gotta figure out what they all do

oh wait, I can click on it

[sinful cipher]

(from my link)

[robust scaffold]

now I gotta figure out how to use all these packed instructions. There's a lot of them

[sinful cipher]

AMD's manual: https://www.amd.com/system/files/TechDocs/26568.pdf

seems more extensive?

[robust scaffold]

my god, how in the world am i gonna learn this. It's like staring at DOTS when I first began. Except with even less community support but a hell of a lot better documentation

[stiff skiff]

Assembly is not something you every fully learn

[robust scaffold]

Gotta figure out if this works with burst and what burst can recognize. I aint gonna start scribbling out my own assembly functions

[stiff skiff]

Just learn where to find it, and search every instruction you dont understand

[robust scaffold]

Tertle's famous

[sinful cipher]

Suggestion: Maybe '0' is too implicit? Worth a try to assign '(int) 0' or '0u' (unsigned?) instead?

[robust scaffold]

I'm pretty sure it's because the pointer itself is not aligned linearly to allow for packing of values

[sinful cipher]

What does the struct look like that you're operating on?

The Census struct

[robust scaffold]

Required for use in interlocked.add

Since I'm obtaining a temporary value from ComponentDataFromEntity

[sinful cipher]

Could it have something to do with StructLayout? As in, perhaps you could fix this with an explicit layout?

[robust scaffold]

The main requirement is that Interlocked requires a reference / pointer to the target integer value

[sinful cipher]

oh wait let me read your previous thing

[robust scaffold]

Using CDFE, it only returns a temporary value and unity does something else with setting it.

People on the forums are begging for a ref ElementAt() version for ComponentDataFromEntity that would make interlocked work directly. But it doesnt, so this "hack" is needed

[sinful cipher]

You'd think it just some number

[robust scaffold]

It is. It's a memory location though.

anyways, gotta convert all my pointer* to ref NativeArray and remove unsafe.

[sinful cipher]

Oh you're setting the int at the pointer location to zero, not the pointer itself?

[robust scaffold]

Yes. The pointer does not change. The value that the pointer is directing to is being set to 0

[sinful cipher]

Ooh.
So the same thing with "public int Ptr;" instead of "public int* Ptr;" would vectorize?

nvm I'm guessing too much lol

[robust scaffold]

Well, making the pointer no longer a memory pointer but instead a value int, it will vectorize

[sinful cipher]

I'll let you do your thing :p

[robust scaffold]

Wouldnt do anything though

I wont be able to interlock add it in a parallel job though.

which is why I need that pointer, to obtain a ref-able variable to shove into Interlocked.Add()

ComponetDataFromEntity returns a copy of the actual variable (for readonly safety). But that is annoying when I have a threadsafe operation called Interlocked.

Thing is, Unity CTO, the guy in the forums and overall command of DOTS, says to avoid using Interlocked. But he's wrong and I'm right.

[sinful cipher]

What was his reasoning?

What even is the point of Interlocked if thread-safety is already guaranteed by the job system?

[robust scaffold]

https://forum.unity.com/threads/could-componentdatafromentity-t-have-a-return-by-ref-version-of-indexer-for-atomic-operations.866524/#post-5705032

[robust scaffold]

When you want to step outside unity's thread safety and into more exotic threading mechanics.

I'm needing to sum up the total for all chunks depending on the citizenship of that chunk. And that's to be done in parallel

And yea, i measured my performance. Singlethreaded, 4ms. Multithreaded + Atomic Interlocked, 0.7ms. No shit unity.

[sinful cipher]

Oooh I see
You write to 1 summed value from multiple threads

[robust scaffold]

Yep.

[sinful cipher]

Although
you in effect have a map where the key is the country and the value is the sum

[viral sonnet]

I can't seem to get anything vectorized :/

[robust scaffold]

One "thread" per chunk that then sums to a single component on an entity. Multiple threads access that component. Requires use of an atomic addition to prevent race conditions

[robust scaffold]

Ah, we did some burst inspection. No need for pointers. You can use ref var.

I was apparently dumb earlier

[robust scaffold]

as for the rest, give me a sec\

[viral sonnet]

ah ok, seemed weird because ref var should be the same under the hood. well, just semantics.

sure, thanks! 🙂

[robust scaffold]

Oh yea, that conditional if statement will break vectorization

[robust scaffold]

Ah fuck, I got class in 8 minutes. I dont know if I can finish typing this out and checking the inspector

I'll be back in.... god 2 hours?

[viral sonnet]

no issue, thanks a lot for your help so far! 🙂

[robust scaffold]

Huh, this is not vectorized and I dont quite have enough time to trial and error to figure out why

[robust scaffold]

That however is vectorized. And yet the one I just hammered together is not. No clue why

i'll think about it when i get back in 2 hours

[viral sonnet]

hm, i guess writing to a single value is the problem.

[robust scaffold]

no, single value does work

[viral sonnet]

then no clue 😄

other than it's REALLY finicky

[robust scaffold]

actually yea. That might be the issue

well, it is writing to delta[0] which is a single value

and that is vectorized

do I need to set the output to be a single array?

either way, running late

[viral sonnet]

have fun o/ 😉

(as much as you can in class, lol)

[viral sonnet]

eh, that was a little of a goose chase for me. :/ only arithmetics can be vectorized. if you have any kind of complex code with branching it won't work. hoped that something changed to the 10 years ago where I wrote simd the last time. seems to be pretty much the same restrictive shit. haha

[robust scaffold]

Branching is possible. Only inline condition ? A : B.

Not very optimized though

[sinful cipher]

Is there a better or built-in way to determine 'maxChunkEntityCount' as I do here?

I'd like to avoid using EntityQuery's CreateArchetypeChunkArray() on the main thread

EntityQuery provides CalculateChunkCount() and CalculateEntityCount(), but not something like "CalculateMaxEntityCountPerChunk()"

Related: Are entities of the same archetype split into multiple chunks? As in; chunks always have a finite size?

[robust scaffold]

Maximum number of entities in a chunk can be calculated at compile time. I believe 16KB is chunk size, then you simply add the number of component size within an entity.

Its somewhere. Unity pulls that data for the entity debugger.

[sinful cipher]

Although I can avoid having to deal with this if I knew how to use NativeMultiHashMap lol,

Maybe its about time I try using NativeMultiHashMap properly

@near mesa do you happen to have an example of iterating through NativeMultiHashMap values for a given key? 😋

The official ECS Boids sample is about the only one I can find so far

[robust scaffold]

I know how to do it, done it plenty of times before a big overhaul of my code. But none on hand. Will need to write up some code manually.

[north bay]

Are you looking for the max entity capacity that chunks which match the query can have?

[sinful cipher]

Some post on the internet suggests "NativeHashMap<TKey, NativeList<TValue>>". Feels a bit dubious to nest NativeCollections like that.

[sinful cipher]

Yeah exactly

[north bay]

Do you know which component your query will have? If so you can use EntityManager.CreateArchetype(components).ChunkCapacity

[sinful cipher]

Oh so I can request it from the archetype?

Wait lets see

Oh right there is such a thing

Although this won't get me anywhere as I would need to know what Archetypes can occur beforehand

I guess I'm better off going for another approach

Oh i now found plenty of NativeMultiHashMap examples: https://github.com/search?q=NativeMultiHashMap&type=code

I found something I can try

Another example with an Enumerator instead of Iterator... Hmm

This should be sufficient for now.
Thanks everyone

[viral sonnet]

In my head canon Arowx is the shit post account from Joachim Ante

that would be so funny 🤣

[north bay]

lmao

[robust scaffold]

Enumerator is easier to loop through personally.

[sinful cipher]

Lol NativeHashSet<T> is just a wrapper for NativeHashMap<T, bool>

[robust scaffold]

Yep

[robust scaffold]

Burst 1.7.0 is apparently on changelog with some really nice improvements to the inspector yet it's not in the package list yet. What version of unity are you using?

[coarse turtle]

i wonder how the asm for readonly span gets optimized in burst 1.7.0

[robust scaffold]

Added support for System.Span<T> and System.ReadOnlySpan<T> within Bursted code. These types are not allowed as entry-point arguments.

AH HA, GOT IT. Just had to manually type out the fucking burst package version into the package manifest file. Unity, you will not deny me my cutting edge burst

[sinful cipher]

2020.1.9f

[coarse turtle]

lol, I probably won't update to burst 1.7 immediately, but I imagine the no entry point arguments means they can't be used as function parameters?

[robust scaffold]

The package is public but unlisted. Now lets see these improvements to the inspector

[robust scaffold]

Cant be used to pass between mainthread and jobs. So no job struct parameters but probably can be used if defined within the Execute() or other local functions

The cross boundary between mainthread and bursted jobs is so restrictive.

[coarse turtle]

Cool, makes sense. You would not want to pass Span & ReadOnly Span as a member in a struct when scheduling a job.

[robust scaffold]

Thats the thing, you. Yea.

Holy shit, it's so fancy now

[coarse turtle]

ok, I'm updating it

that's easier to read lol

[robust scaffold]

i dont know what any of these things are doing but apparently it collapses the default burst code leaving behind only those derived from C# code

and then those arrows are.... something?

That's the archetype scheduling code

What I really wish is for custom comments from C# code that appear inside burst inspector. So I know where my code is actually located. Although these arrows help a lot

[sinful cipher]

Does anyone here spend any effort in ensuring their code remains Unity editor 'hot reload-proof'? (Or is it not really worth it)

[robust scaffold]

hot reload?

you mean the domain reload option?

[coarse turtle]

like you want it to not recompile after you edit your source while your game is live?

[sinful cipher]

Do a code change while the editor is in play mode and have it seamlessly continue running after recompilation without any state loss or corruption

[sinful cipher]

I liked the feature when I initially started working with Unity. Although over time it has become harder and harder to properly maintain compatibility with this

[coarse turtle]

well, afaik hot reload in .net only became an official feature with .NET 6. I think with burst it becomes a bit more different (I'm not entirely sure how the generated burst dll is linked to the mono runtime.)

[robust scaffold]

No clue how that would work. If it could work with DOTS / burst in the first place so eh

[coarse turtle]

there is live link. But I have yet to try it.

https://unity.com/releases/2019-3/dots-sample#unity-live-link

[sinful cipher]

Live link? Well the feature Im talking about has been in unity for years

all it does is serialize whatever is in runtime at the moment

recompile

then deserialize all state again and continue the simulation from there

Although of course you need to make sure all state is serializable

• deserializable

[coarse turtle]

Yea I get what you're saying. TBF, I don't really know how well it works with DOTS. I usually just leave my editor at Recompile After Finish Playing

[sinful cipher]

Usually anything you put in jobs will already be serializable... although... pointers may not be

[coarse turtle]

Yea, so not sure how well it works with native containers.

[viral sonnet]

any gist of how Span is useful for us? Does it mean anything when we can use NativeArray?

[sinful cipher]

IMO is more trouble than its worth but I was wondering if some are actually doing things "right" in this regard (hot-reload compatibility).

[coarse turtle]

Might be relevant to your needs: https://forum.unity.com/threads/improving-iteration-time-on-c-script-changes.1184446/
xfoofx also worked on the burst compiler team.

[robust scaffold]

increased waiting times (from 2s previously to 4s/6s)
Wait, it's not normal to wait 4s/6s for recompiling?

[coarse turtle]

typically no

[sinful cipher]

Oh no compilation times are fine for me. I separate my stuff into several assemblies. I was just wondering if this is something that is actually valued a lot in the "real world".

[coarse turtle]

oh hot reloading is very convenient

[robust scaffold]

https://docs.microsoft.com/en-us/archive/msdn-magazine/2018/january/csharp-all-about-span-exploring-a-new-net-mainstay

[sinful cipher]

but it breaks easily right?

[viral sonnet]

we don't use microsoft c# 🙂

[coarse turtle]

I've only done it in C, never with Unity directly. I actually thought livelink was suppose to be the "hot reload" solution for Unity DOTS

[sinful cipher]

any "static Dictionary<GameObject, ExtraProperties>" you have lying around will have to be accounted for and dealt with properly for (de-)seerialization

[robust scaffold]

what do you mean? Our code is in C#. Microsoft makes C#

[robust scaffold]

struct MutableStruct { public int Value; }
...
Span<MutableStruct> spanOfStructs = new MutableStruct[1];
spanOfStructs[0].Value = 42;
Assert.Equal(42, spanOfStructs[0].Value);
var listOfStructs = new List<MutableStruct> { new MutableStruct() };
listOfStructs[0].Value = 42; // Error CS1612: the return value is not a variable

[viral sonnet]

Technically we have Mono. Anyway, I've already read that article. What's the difference to NativeArraySpan and NativeSlice

[coarse turtle]

https://github.com/Unity-Technologies/mono this is the version of mono that unity maintains.

[sinful cipher]

Live Link sounds like it does not imply it will employ actual hot reloading. Just that it will received the updated bundles and - I guess - relaunch the game with the new stuff.

[robust scaffold]

Baked into the language? No need for class declarations and such?

Basically span seems to be able to return ref values to elements of the array (removing need to create temp vars then reset the array element). Nothing particularly fancy.

[viral sonnet]

That's useful if they have implemented it that way. NativeArray lacks a ref

[robust scaffold]

var t1 = (int*) new NativeArray<int>().GetUnsafePtr();

Well there is a pointer sure

[viral sonnet]

didn't mean that we can't do it. it's just easier to write

[sinful cipher]

If you need to understand NativeSlice; I worked with it before

[robust scaffold]

I wonder where ref ElementAt() comes from. Which collection has it?

[viral sonnet]

NativeList has it

[robust scaffold]

Ah, because nativelist is a pointer to a buffer. Not the buffer itself

[coarse turtle]

FixedList has ref ElementAt

UnsafeUtility.ArrayElementAsRef<T> might be able to use this to write a NativeArray ref ElementAt

[robust scaffold]

Right, people have said to roll your own extension method using that

[viral sonnet]

yes, this works too

[coarse turtle]

yea, I actually only just found out about that. I've been rolling with

return ref *(((T*)NativeArray.GetUnsafePtr()) + i);

[robust scaffold]

Does that work with non-continuous collections? NHM/NMHM?

[coarse turtle]

Maybe on a NHM bucket.

haven't tried

[viral sonnet]

NHM just returns the struct. all the collections leave a lot of flexibility out

IMO all should be able to return refs/pointers

[robust scaffold]

At least the unsafe versions should

Unity cares too much about handholding the programmer. Let us shoot ourselves in the foot.

[viral sonnet]

Change the optimization pipeline to run the loop unroller exclusively after the loop vectorizer. This improves codegen in a lot of cases (mostly because the SLP vectorizer is unable to vectorize all the code that the loop unroller could have). -> wonder if this would improve my code at some places 🤔

[viral sonnet]

yeah, i really dislike this in the NativeX space. handholding equals bad performance and that's not why I'm using these

beautiful rainbow

it's like playing factorio again

[robust scaffold]

Unity really is a superior factorio

I literally can not play that game anymore without thinking: "I could be working on my project right now."

[sinful cipher]

What happens to NativeCollections allocated with Allocator.Persistent after exiting play mode?

[robust scaffold]

memory leak

[sinful cipher]

lol

[robust scaffold]

you gotta close the editor and then reopen

really annoying

[sinful cipher]

well that explains some things

Oh right I wasnt sure if the leak detection would pick this up but it does. The error will come when attempting to start play mode for the second time.

[robust scaffold]

Hrm

New Burst 1.7.0 seems to have broken something with EntityManager.MoveEntitiesFrom()

nevermind, not burst, just my shitty code

[sinful cipher]

I dont have to fix/pin anything with when working with UnsafeUtility.MemCpy if working with NativeCollections right?

(Native to Managed)

(Native to Native)

I mean it works but maybe I introduced some safety issue here

[robust scaffold]

You know that Unity has an extention method CopyFrom and CopyTo

And the managed version

[sinful cipher]

Yeah but I'm also converting the struct type during the copy

e.g. this allows fast conversion from Vector3[] to NativeArray<float3>

[robust scaffold]

.Reinterpret<>() the array before copying? Ah, from managed to native

[sinful cipher]

Yeah Reinterpret basically

[robust scaffold]

well, you can copy to a NativArray<Vector3>() then .Reinterpret<float3>() to get it in the end

without having to make your own version memcopy

[sinful cipher]

Is my version correct, though? :p

[robust scaffold]

maybe? Unity has a lot more safety checks for read access

[sinful cipher]

I'm primarily concerned about the fixed {} block

I had to do it for the managed array, but not for native

hmm I'll check what Unity has then

This was from before Unity had all those extensions I believe

[robust scaffold]

(void*) ((IntPtr) (void*) gcHandle.AddrOfPinnedObject() + dstIndex * UnsafeUtility.SizeOf<T>())

Unity has some really strange casting here

[coarse turtle]

yea looks right

[sinful cipher]

Unity's reinterpret does an allocation here that I dont have to do :/

[robust scaffold]

Also, this just in: EntityManager.MoveEntitiesFrom() can not handle chunk components

will throw null reference exceptions

[sinful cipher]

Hmm alright. So far I havent seen strange things happen so I suppose I'll just go with it for now.

[coarse turtle]

also you can just do

fixed (T* ptr = array) {
}

[sinful cipher]

where did you find this?

[robust scaffold]

Unity's CopyTo / CopyFrom implementation

found in NativeArray

[sinful cipher]

Ooh right.

Looks like it basically does what I'm doing

without the fixed { } block somehow

Although now that I think of it, the fixed { } block probably just compiles to what Unity has in their implementation

[viral sonnet]

i'm at a point where I'd like to start a job from a job 😄

[robust scaffold]

hahaha, yep

[sinful cipher]

Its like the "I want to put a prefab in a prefab" for people working with DOTS

at some point everyone gets there

then disappointment hits

[viral sonnet]

or figure out a better way to create data in a job that is used to start the actual job then. NativeStream is the best solution I've found but even that is not async in that sense, takes a lot of time and blocks the main job

[frigid wigeon]

hello guys do you know any best tutorial for dots 2021 😦

[sinful cipher]

Yeah I often chain Jobs for this reason

[frigid wigeon]

iam begging to understand ECS system

[sinful cipher]

hey me too

[robust scaffold]

Oh boy. So are we.

[frigid wigeon]

every version has differnt defenition

like

World.Active.EntityManager

1.8 dosent exist

WTH ??

its keep changing

[sinful cipher]

lol

jep

[robust scaffold]

0.18 is dead. Long live 0.21 (eta 203X)

[frigid wigeon]

😭

how do i optimize game 😦

[sinful cipher]

I just stick with Entities 0.17 because I understand it and it works

[frigid wigeon]

0.17 ??

iam using 0.18

[robust scaffold]

loop vectorization and pain

[viral sonnet]

it's World.DefaultGameObjectInjectionWorld.EntityManager for a long time now

[robust scaffold]

impossible

[frigid wigeon]

can i punch my face

[robust scaffold]

wait, are you using hybrid renderer?

[frigid wigeon]

i am been struggling for 2 days

straight

YES

YES YES

YES

[robust scaffold]

Ah, yea. Dont.

[sinful cipher]

lol

[frigid wigeon]

WHAT DO YOU SUGGEST

i have so many instantiated objects

around

10,000 to 50,000

units

runs 0.1fps

[robust scaffold]

Use com.unity.entities. Documentation is here: https://docs.unity3d.com/Packages/com.unity.entities@latest

[viral sonnet]

good lord, can you type sentences?

[frigid wigeon]

nope

iam specifically

looking at

Hybric ECS

that means

[low hazel]

@frigid wigeon Type properly please

[frigid wigeon]

i have to figure myself right

😦

okay

[robust scaffold]

Basically understand what is Entities first (without even displaying anything). Then try to use Hybrid Renderer to try and render things

We're all here trying to understand what Entities is so, yea.

[frigid wigeon]

😦

[sinful cipher]

I still use Graphics.DrawMesh instead of the hybrid renderer because at least I know it will work

even though I'm leaving a lot of performance on the table

Also still Unity 2020.1.9f. Unless there is a compelling reason to update, I usually dont.

[robust scaffold]

Burst I think has minimum required version of 2020.2+

[sinful cipher]

Huh, it works over here for me

Maybe its a newer version you're thinking about

[robust scaffold]

Here's my journey: Understand how to schedule an IJob -> Understand how to schedule an IJobParallel -> Understand what Burst is -> Install Entities

[viral sonnet]

HybridRenderer is good for static/non-moving objects out of the box. For many moving ones it gets a lot more complicated because the Transform/Hierarchy system gets in the way and needs to be tamed

[sinful cipher]

I'm using Burst 1.4.1

[robust scaffold]

2019.4+. Not 2020.2. Dont know where I got that from

2021.2+ for the standalone version... I think. no clue what BurstAoTCompiler is

[sinful cipher]

Oh Ahead-of-time compiler?

[robust scaffold]

yea

[sinful cipher]

Isnt that what Burst does anyway

or is it JIT

[robust scaffold]

JIT in editor, AoT in build

[sinful cipher]

Ah

[robust scaffold]

I've compiled my project in 2020.3 perfectly fine so no clue what it means

[robust scaffold]

But yea, dont use DOTS right now. It's not friendly to new programmers

[frigid wigeon]

iam not new 😦

[robust scaffold]

Stick with regular Unity. DOTS will not be ready for another 2 or 3 years

[robust scaffold]

Do you know what Burst is?

[frigid wigeon]

i am playing with it right now 😦

[robust scaffold]

Yep, that's Entities. Really dense "documentation" that's great as an overview but honestly doesnt explain any DOTS programming patters at all

[frigid wigeon]

thats true

[robust scaffold]

And here's Burst: https://docs.unity3d.com/Packages/com.unity.burst@1.7/manual/index.html

[frigid wigeon]

thanks 😄

[robust scaffold]

https://docs.unity3d.com/Packages/com.unity.burst@1.7/manual/docs/QuickStart.html

Forget entities, it's basically useless until you know what Burst is inside and out. Try to start in Monobehavior scripts located on your camera and try to do things with IJobs

[frigid wigeon]

this will fix my frame issues 😦 ?

[robust scaffold]

Theoretically: Yes. Practically: No. Not unless you basically rewrite your entire game from the ground up.

Burst isnt just a fancy compiler, it's a "~lifestyle~".

[robust scaffold]

DOTS in a nutshell is: "Is this Burst compile-able? Will Burst be able to read this? Can Burst vectorize this? How can I order my entity components to allow for Burst to optimize this?"

Burst burst burst burst.

One little golden tag, so much pain.

[sinful cipher]

This is how I started.
I only started using the actual entity component system after collecting data to put into jobs took 100 times longer than executing the actual job itself :p

[robust scaffold]

Same here. I started with mass amounts of NativeArrays filled with data before I realized that Entities does that for me.

[sinful cipher]

dear mother of god

[frigid wigeon]

i know

[robust scaffold]

Oh jesus

[frigid wigeon]

so

[sinful cipher]

"Saved by batching: 0"
lel

[frigid wigeon]

i have to optimize the game 😄

[robust scaffold]

DOTS will not help. Custom render pipeline will.

[frigid wigeon]

URP ?

[robust scaffold]

I doubt the bottleneck is information processing. You need to go beyond URP and HDRP and code your own Scriptable Render Pipeline to handle that many entities.

And how many entities is that? 10k? That's nothing.

[frigid wigeon]

😮

[sinful cipher]

It could be. Updating transforms, submitting draw calls, calculating transform matrices....

gets super expensive if you have this amount of stuff on screen

[robust scaffold]

At that point, you need to use a compute shader and do all those calculations GPU side and remove the CPU - GPU transfer bottleneck

[sinful cipher]

For me, total time spent on Graphics.DrawMesh regularly exceeds time spent on updating all my ComponentSystems

@frigid wigeon From my experience, you need to get the "Batches" number down. 41852 is too much for any computer

How many vertices are in that character mesh?

[frigid wigeon]

let me check that out !

[sinful cipher]

What are you going to try? :p

[merry ruin]

Lod system?

[frigid wigeon]

thats crazy

[frigid wigeon]

iam planning to implement it today

[sinful cipher]

What is too bad about SkinnedMeshRenderer is that there is no GPU instancing support for it

[gusty comet]

you need to merge those meshes a bit before considering converting them to entities, DOTS is awesome but 41,000 batches with 38,000 shadow casters and 10,000 animators playing simultaneously is just waaaay too much

[sinful cipher]

I believe rendering 40000 of the same thing should be possible if you could use GPU instancing.

but not with animated character though.

[gusty comet]

i dont think GPUI works with Entities does it?

[sinful cipher]

No clue, but you could use Entities to calculate all the transform matrices, then use Graphics.DrawMeshInstanced(Mesh, Matrix4x4[]) or something like that

[frigid wigeon]

i love you

i was thinking this entire day, i probly can fix it within 3days

[gusty comet]

I would merge the meshes in batches of maybe 100, then convert the resulting meshes into Entities, then shove those merged meshes into Sub-Scenes in maybe batches of 100 themselves

[sinful cipher]

Does batching even work with SkinnedMeshRenderer?

lol guess not

[viral sonnet]

662.4M tris, what is this madness? 😄

[gusty comet]

nope, i know it wouldnt work with Entities anyway, GPUI i wasnt sure about though

[frigid wigeon]

iam trying to kill hte pc

🙂

[sinful cipher]

doing well I'd say

[frigid wigeon]

i literally writing all notes

from ur guys comment

I LOVING WITH IT

❤️ ❤️

I test everything out

[viral sonnet]

my calculator says one mesh has 66k triangles. I think overwatch has like 17k

[frigid wigeon]

🙂

[sinful cipher]

I would use LOD to break a character up into cylinders and spheres for limbs and heads. Then use GPU instancing to just blast thousands of them onto the screen without too much trouble.

[viral sonnet]

those are pretty detailed and like what, 10-15 on screen? what you want to do is really not possible with animated characters

even if you could render them, the animation system breaks down

one part in DOTS that's REALLY experimental is the animation package which could help in this case

but I honestly doubt it for 10k.

[sinful cipher]

I remember once seeing this crazy Unity game or tech demo that had like 10000 animated characters. This was from before DOTS.

No clue how it was done

@frigid wigeon https://medium.com/chenjd-xyz/how-to-render-10-000-animated-characters-with-20-draw-calls-in-unity-e30a3036349a
I found this

This person uses GPU Instancing, BUT...

this is quite clever

he uses an "animation map" to manipulate vertices in the vertex shader to make it appear animated

[frigid wigeon]

This

Is

Cr Az y

[viral sonnet]

https://github.com/joeante/Unity.GPUAnimation

[sinful cipher]

haha this is pretty cool

its like a bunch of animation curves put into a texture

[robust scaffold]

ref NativeReference<float> parameter does not result in vectorized addition while float* [] does. So odd.

[viral sonnet]

6.19ms single threaded for 250k...

[robust scaffold]

That's really good.

With float*[]

[frigid wigeon]

O.o

[robust scaffold]

With ref NativeReference<float>

I cant tell what is better

[viral sonnet]

it's really not. the really bad thing it's even slower parallel

[robust scaffold]

float pointer has less commands but the ref nativereference seems to be unrolled

[robust scaffold]

6ms for 250k entities? Hrm

[viral sonnet]

maybe it is good, in the context of 16.6ms frame budget it's not though 😄

[robust scaffold]

Oh, yea, that bad.

Do you have to use a NMHM? What are you doing with the int's per entity?

Are you summing these together per entity?

[viral sonnet]

the parallel spells can't directly apply the damage amount to the target otherwise I have race conditions between the threads. so the damage amounts are written out and then brought together in the NHM and then they are applied to the entities

[robust scaffold]

Ah ha, allow me to introduce you to the magical world of atomic calculations

Enter stage left: Interlocked. https://docs.microsoft.com/en-us/dotnet/api/system.threading.interlocked?view=net-5.0

[viral sonnet]

those just kill performance further

[robust scaffold]

I had this same issue with summing entity count to dynamic entity targets

[viral sonnet]

I've seen them in action with nativelist.parallelwriter

i've to check if CDFE is atomic

maybe I don't need to do this lol

[robust scaffold]

hear me out, i've profiled this myself. Cut a 4ms singlethreaded job over 2.5 million entities to 0.07ms parallel

Fuck the CTO, he doesnt know what he's talking about. Interlocked is amazing

[viral sonnet]

hm, in my test i'm writing to one single target. that's really worst case

[robust scaffold]

CDFE is not atomic. However, by using pointers in combination with interlocked, you can atomically add values together

Ehhhh, yea

But try it out. If it's equal performance, well you'll cut down on the memory requirements of the nativestream and NMHM

[viral sonnet]

yeah I'll think about it.

i guess it can't get that much worse haha

[robust scaffold]

public unsafe struct Census : ISystemStateComponentData, IDisposable
{
    // ReSharper disable once Unity.RedundantHideInInspectorAttribute
    [HideInInspector] public int* Ptr;
    // ReSharper disable once Unity.RedundantAttributeOnTarget
    [SerializeField] public int Value => Ptr[0];
    public Census(bool i)
    {
        Ptr = (int*) UnsafeUtility.Malloc(1, 4, Allocator.Persistent);
        UnsafeUtility.MemClear(Ptr, 1);
    }
    public void Dispose()
    {
        UnsafeUtility.Free(Ptr, Allocator.Persistent);
        Ptr = null;
    }
}```

That is the target value, the collection of summed ints (or float or whatever floats ya boat).

[BurstCompile]
private struct IteratePopChunks : IJobEntityBatch
{
    [ReadOnly] public ComponentTypeHandle<CitizenshipChunk> Citizenship;

    [NativeDisableParallelForRestriction] public ComponentDataFromEntity<Census> Census;

    public void Execute(ArchetypeChunk batchInChunk, int batchIndex)
    {
        var countryEntity = batchInChunk.GetChunkComponentData(Citizenship).Country;
        unsafe
        {
            // Enabling interlocked pointer from component data from entity cuts time
            // from 4 ms as single-thread to 0.07 ms multi-thread (1 ms total)
            Interlocked.Add(ref Census[countryEntity].Ptr[0], batchInChunk.Count);
        }
    }
}```

That is the process. In this case, I'm summing to the target all relevant chunk counts. In parallel.

If you're using anything other than int variants, you'll need to make your own interlocked extenstion that adds floats together. Plenty on stack overflow

[viral sonnet]

interesting usage of ISystemStateComponentData, I assume this also works for Icomp

[frigid wigeon]

ECS, GPU instancing, LOD, General Optimization code , animatiom map are the solutions to fix this bloody issue 😄

[robust scaffold]

Yea but you need to manually dispose of the pointer. So system state or whatever else is needed

[viral sonnet]

cool thanks, gonna give this a try

[sinful cipher]

Did you ever check performance of:

• Writing to a NativeHashMap<int, int> where key is batchIndex and value is sum.
• Scheduling a second job (just single-threaded) with first job as dependency, in which you sum these values?

I'm kind of curious how that compares to your current implementation.

This is what I would do given the same problem, and is probably what that CTO would want you to do, right?

[robust scaffold]

Yea. Or completely redesign your data structure to remove the need to have intermediate buffers. Somehow

But fuck 'em. Interlocked is my God (read Daddy Gates) given right and I will use it everywhere I can.

[sinful cipher]

lol

Although I have to admit, I cant identify anything that could go super wrong here. Worst case is one or more threads have to wait for eachother to finish their integer addition.

[robust scaffold]

Yea, thats what it does. Just waits in a while loop until a slot opens for it to then add itself onto the value

it's not the most optimal but it completely eliminates any need for intermediate buffers

[sinful cipher]

I wonder what kind of abuse Joachim was thinking of when he wrote that post

[sinful cipher]

Maybe the story would be different if you would iterate over Entities instead of Chunks (to sum some ComponentData value or something)

[robust scaffold]

True. There's only about 7k chunks. Imagine 2.5M entities using interlocked

[sinful cipher]

Hmmm... 7k is still quite a lot though

oh well

[robust scaffold]

7k is nothing, largely

[sinful cipher]

i barely get to 7k entities lol so its a lot to me

[robust scaffold]

bump those numbers up. Stress test ya systems. 700k entities good

[sinful cipher]

Anyway, cool stuff man. Really interesting to see this kind of approach

[robust scaffold]

Honestly, i'm expecting use of around 100k entities at most. But 2.5M and maintaining good CPU times will be key. As only with many many entities will small optimizations show up in the profiler (outside of the usual random changes)

[sinful cipher]

look at my sad worker thread utilization

[robust scaffold]

155ms, what the fuck

holy shit. I'm here counting the ms until my 144fps system will require shifting to FixedUpdate and you're here with 8 FPS

[sinful cipher]

EntityCommandBuffer playback for AddComponent and RemoveComponent takes super long for some reason

[sinful cipher]

its only this bad when I move the "camera" several kilometers in a second

[robust scaffold]

singlethreaded structural change. Of course it'll be very expensive

[sinful cipher]

with normal traversal speed its better, but still not good enough

[sinful cipher]

ooh 😮

[robust scaffold]

And it's on the main thread so it blocks graphics update

There's no way around that unless you're willing to shift all your entities onto a separate world then shift it back, which may be even more expensive

[sinful cipher]

Hmmm...

[robust scaffold]

actually, give me a sec. Let me see how much my 3M entity transfer between worlds cost

[viral sonnet]

implemented Interlocked.Add now and it's working with burst disabled. when I enable it it's throwing errors and that I should disable burst to check ... haha

gonna work on this tomorrow, I think overall it will be an improvement. so thanks again

[sinful cipher]

most of the trouble comes from having to update MeshColliders, MeshRenderers, and normal map textures and such

[sinful cipher]

So I have to return to the main thread every once in a while to do that.

[sinful cipher]

So you're saying Worlds update completely in parallel?

[robust scaffold]

Yes, kinda. You can move a world entirely to a Job thread, independent of the main thread and the graphics update loop. Allowing for a solid FPS while a different world is slogging through a job

[sinful cipher]

HHmmmmmmmmmmm

[robust scaffold]

Assuming that the move chunks job timing scales linearly, the move chunk will cost 0.27ms for 7k entities

Which is nothing

[sinful cipher]

that already beats what I currently have

[robust scaffold]

EconAuthoring Creation job is veeerrrrrryyyyyyyyy expensive. Over 5 seconds in RT. But it's divided over 300 frames allowing for a solid non-blocked 60FPS

[viral sonnet]

burst was acting weird. it's working now. kind of like a free removed race condition. interlocked is my god now too

[robust scaffold]

Performance? What's the timing?

[viral sonnet]

could free up 2 jobs and the write/read to nativestream. pretty huge tbh

[sinful cipher]

#1 cause of slow code and complicated systems for me has been keeping ObjectPools with Meshes and MeshColliders and such. Every TerrainPatchEntity too far to be visible should not take up a large Mesh, so it is released back into a pool. But then reallocating these takes long and is hard to jobify.

I hope DOTS physics and rendering soon becomes good enough for me to use for this...

[viral sonnet]

41ms before and 30ms now.

[robust scaffold]

Interlocked is our God Daddy Gates' given right. Good to see another convert to the Church of Interlocked.

[robust scaffold]

Parallel? That's pretty bad

[viral sonnet]

that's the whole frame timing. i can't measure just the interlocked timing right now

[robust scaffold]

But I guess it is the worse possible case of 250k targeting 1 entity

[robust scaffold]

Ah, Well it shaved off 11ms so good.

[robust scaffold]

No clue about terrain. Unity's supposedly planning something for native DOTS terrain but that was promised 3 years ago

[sinful cipher]

By the way, would it be faster to create a new Entity with an extra component in its archetype instead of adding a component to an existing Entity?

[robust scaffold]

Yes.

[sinful cipher]

My terrain is spherical so I dont expect I can use anything Unity will offer.

[robust scaffold]

thats why you usually always pre-define the components located on every entity in its archetype

[robust scaffold]

True. If regular Terrain doesnt work, DOTS wont either

[viral sonnet]

well not the best absolute timings with profiling now but one worker thread takes 36ms and 2.58ms is the Interlocked.Add

[robust scaffold]

Have you tried to do the job in parallel?

[sinful cipher]

Wtf. I shall try this tomorrow then.

[robust scaffold]

With interlocked add, parallel scheduling is possible

[viral sonnet]

289ms overall and 20ms for interlocked overall

it's already 8 worker threads 🙂

i'd say this was very much worth it because eveyr other solution is pretty bad

[robust scaffold]

2.58ms vs the what, 6ms earlier?

And the worst case scenario. That's over a 2x improvement in performance.

[viral sonnet]

11-12ms before. if not more.

yes, it's absolute worst case. like, it can't get any worse haha

yeah i'm gonna stick to interlocked now for race conditions. all other solutions are not working out and i'm here for performance not for winning some price in style and finesse

[robust scaffold]

Yep. All in a day's work. Anyways I've gotta go to sleep. I'll be around tomorrow or whenever lurking in this channel. Because Burst is Love. Burst is Life.

[viral sonnet]

me too, have a good night

[robust scaffold]

Interlocked can not be vectorized. If you can vectorize, that has a looooot better performance

[robust scaffold]

The main cost of interlocked usage in my case (the census) is the inability to vectorize "reset" the value to 0. Because of the pointer redirection, it can not be properly vectorized.

[sinful cipher]

See you around guys. Thanks for all the great help so far.

[robust scaffold]

im pretty sure I could solve that with intrinsics. And Im looking for it... tomorrow. Sleep now. Yes

[robust scaffold]

https://docs.unity3d.com/Packages/com.unity.burst@1.7/manual/docs/CSharpLanguageSupport_Intrinsics.html#systemthreadinginterlocked

[icy kestrel]

Is there any work around to pass a string into a job, to be used in a switch case statement? I know you can pass it in a NativeArray<char> form, but I'm not sure how to switch case on that, any ideas?

[visual tundra]

How many possible strings and how many switch cases do you have? This might make it easier to figure out how to best do this. And consider what the string is indicative of, and if you can swap it to an enum or int before passing.

[icy kestrel]

Basically, I wanted to do some sort of delegate function passing into my job system, but since found out that's not really possible. So now I want to replace it with a switch case with all the possible static functions referenced inside the switch case. But, the switch case will be under constant additions as I pretty much am always creating new "algorithms" for it to run. Atm, I have each algorithm being a class with a name and run function deriving from some shared interface. I've got all these classes stored inside a dictionary for other external non-job uses. They are paired with an ID, but that ID is different on each run so I couldnt use that to identify them inside of the job. Which is why I went with a name, which ends up being the classes name. An enum does some useful here, do you think that would work?

It's a bit of a shame I couldnt get any delegation working cause the switch case needs to be maintained constantly

[visual tundra]

This sounds like an ideal case (if you'll excuse the pun) for a big enum. That will give you very clear naming, making code easy to read, and switches love nothing more than enums, even if they're huge.

[visual tundra]

enums are superficially a naming of ints, so if you need to be dynamic, you can use those ints in the case, too, by passing in a variable that's got the int value of the case/enum you want, as things change. This can get a bit fiddly, but much less so than messing with strings, dictionaries and classes. Also, I don't know how you're doing it, but those functions need to get into Jobs without reference to the classes/instances... as Jobs don't like references. Have a read of this, for how to think in terms of the world of Jobs vs References: https://www.jacksondunstan.com/articles/5397

[icy kestrel]

Thanks! Atm, I'm calling a static function inside of the job. What I tried to do, was to pass in a class and call it's static method, but the class was a reference type so I couldnt. So is it actually possible to pass in a "static function" to a job?

[icy kestrel]

This is very interesting, so that allows you to reference managed types, but not to read them? So I wouldnt be able to wrap my algorithm class instance and make the call to it's static function would I?

[visual tundra]

I think you'll have to find a way to alleviate class based thinking, altogether. Not merely so that what you do works, but so that it works well. Do you know much about the stack vs the heap, and how that differentiates what's going on? If so, you've gotta (to get the performance boost of Jobs and Burst) think about how to get as much as possible onto the stack. If this doesn't mean anything, then start thinking in terms of arrays of structs, like NativeArrays<structTypeYouMade> so that you can do multiple things at once. Perhaps the best thing I can impart is this: structs can have functions, and they travel with the struct! So if you can pass in a struct, it's got the function you might need within it, ready to go.

[icy kestrel]

Can I pass a struct in and have it guaranteed that the struct does infact have this function?

[coarse turtle]

you can pass in a struct with the method or in your job struct, call a static method with all the arguments you need to do the operation

[icy kestrel]

But if I want to replace this struct with another, how can I call that function instead? Without having to change the code

[coarse turtle]

You might want to consider Burst function pointers

https://docs.unity3d.com/Packages/com.unity.burst@1.6/manual/docs/AdvancedUsages.html#function-pointers

I think you can also have an interface on the struct and hint the compiler that it is completely blittable

struct JobStruct<T> : IJob where T : unmanaged, SomeInterface {
  public T YourStructYouWantToCall;

  public void Execute() {
    YourStructYouWantToCall.SomeMethod();
  }
}

[icy kestrel]

Thanks I'll check this out

[stiff skiff]

I don't think the struct constraint is enough for that as a struct isnt always blittable

https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/proposals/csharp-7.3/blittable

[coarse turtle]

yea, unmanaged works better, couldn't remember if you can have trailing interfaces after unmanaged. Not really sure why I thought you couldn't have interfaces trailing after unmanaged in the first place.

[visual tundra]

Yes, it's always in there, but must be its own function, and must be operating in its own way, without reliance on outer references, or any references. Everything in a struct is copied into a Job, when the struct is passed in. This is known as "copy by value" or some other nonsense. Programming terminology is ridiculous. In other words, don't try to over think this. If you start thinking about how to put different functions inside a struct, you're going to get into a pickle, and need to consider the ideas state, and perhaps making a way to determine what Jobs to issue rather than having the work in the Jobs doing the determinations.

[robust scaffold]

Alright, a few hours later I tracked down my bug. Shared component data properties are not registered with the entity remap functionality. Entity fields / properties located in SharedComponentData are not properly re-index'ed upon transfer to new world. It is thus mandatory to create a temporary component data representing the intended shared component data then manually assign it post-entity transfer to another world. The more you know.

thats a problem because shared component assignment and the resulting mem-copy is extremely expensive. I'm trying to get loading to be non-blocking but if I do the shared component assignment on mainthread, that's a giant lag spike...

[robust scaffold]

Also huh, unity transport, the DOTS ultra lowlevel multiplayer API has been releasing updates regularly and quietly

And apparently it's pre-release flagged. That means full release within the next quarter

That may mean a big announcement on Entities front, because why would they release Transport and not release a "newer" version of Entities?

[viral sonnet]

Hey KornFlaks o/

lol what are these patch notes in transport: Some public APIs that should have always been internal are now internal

[robust scaffold]

'ello. Just looking around the blog and there was a mention in the newly announced Unity Multiplayer Support a small tidbit on the upcoming full release of Transport (along with integration with the new multiplayer services). And when I checked the changelog, it's 1.0 already

[viral sonnet]

yeah, pretty good to see. also hope that means good things for entities and netcode

I'm currently playing around getting anything vectorized but Burst doesn't want to

can you see anything?

[robust scaffold]

Ah, yea. That pointer redirection fucks with autovectorization

I have the same issue, described yesterday about resetting the unmanaged value the pointer was referencing

[viral sonnet]

ah, i feared that. the health struct has more values in it, hm

so in essence we have to use blittable data type arrays?

[robust scaffold]

You have 2 paths forward.

1. Accept the non-vectorization. Isolate the pointer and value combo into its own component and take it out when you need to and eat the performance cost as payment for the magic of Interlocked.
2. Learn assembly. Code out specific assembly instructions to manually pack the ultimate value and then handwrite the vectorization:
   https://blog.unity.com/games/updated-guide-for-using-neon-intrinsics-in-unity-burst

[robust scaffold]

Yea. Direct references to those data, no redirection

I mean indirect references does work. Or else how does NativeList variables get auto-vectorized when iterated through (I think). Unity recognizes that something is a pointer and the ultimate value is a blittable data type but that must be identified in Burst itself.

However that is under question as DynamicBuffers are not autovectorized despite being a pointer to a NativeArray equivalent (which also cant be auto-vectorized apparently or I couldnt get it to work.)

[viral sonnet]

huh, this is really quite limiting :/ had hopes that burst would be smarter.

[robust scaffold]

Dynamic buffers not being vectorized is really concerning. I need to do a lot more tests to figure out why

[viral sonnet]

i know vectorization only from shader programming and this is nearly 10 years ago. but it worked for structs like float3/4 etc... maybe I'm off but this is the same thing

somewhat dissapointing that this is not further along

[robust scaffold]

Vectorization according to the assembly is taking a reference to a value then assuming the next 4 or 8 pointers after it also point to the exact same data type. Packing those pointers together, then doing math on all of them in parallel.

The problem here is that while the pointer itself is packed linearly and able to be vectorized. The int value what the pointer is referencing is not packed linearly

Clearly there is a way to pack it linearly, that's the entire magic of Entities, taking pointers and "dynamically" linearizing component data behind the scenes to allow for burst vectorization.

[viral sonnet]

so that I get this right, the problem is with the Ptr[0]?

[robust scaffold]

Yep

[viral sonnet]

ah ok, makes sense

[robust scaffold]

output[i] += temporaryHealth[i].Ptr would be vectorized, but it's literally adding the pointer's memory index to a value which will produce garbage.

output[i] += temporaryHealth[i].Ptr[0] is not vectorized due to applying the pointer's memory redirection to acquire the actual useful information.

Now that Ptr[0] can also produce a pointer, you can ref the output of that, and thus you can manually pack and vectorize the addition yourself if you use Burst Intrinsics (aka Assembly).

[viral sonnet]

good to know, i'll hold off in this case 🙂

the job doesn't make much sense anyway hehe

a relic from yesterday so I can write the interlocked.add test faster

[robust scaffold]

grab output[i], output[i+1], output[i+2], output[i+3] pointers, pack them, then grab temporaryHealth[i].Ptr[0], 'temporaryHealth[i+1].Ptr[0], 'temporaryHealth[i+2].Ptr[0], 'temporaryHealth[i+3].Ptr[0], pack the pointers, then vaddXX them both.

it'll probably be 1.1x or 1.2x faster since you're doing it manually

The main problem of doing this is that you'll have to swap commands to pack and add for about a dozen or so different processor assembly commands:

So one add function now becomes a 50 line switch statement monstrosity

[viral sonnet]

haha, yeah, no, I get flashbacks to the c++ engine I worked in the past. but thanks for the suggestion

[robust scaffold]

yea. It's an option that if it's the largest contributor to your performance cost, manually hardcoding out vectorization where Burst doesnt recognize it is a very tedious job.

But if it turns your game from a 40FPS to 60FPS (8.3 ms difference per frame), yea it'll be worth it

[viral sonnet]

fully agree

do you know how much speed up Hint.Likely can give? looks interesting

[robust scaffold]

Ehhhhh, never profiled it myself. But it's literally free performance. Might as well do it.

Well, free in that you assume likely / unlikely properly

[viral sonnet]

Hint.Assume reads weird: The assume intrinsic is powerful and dangerous - telling the compiler that a condition is always true

What sense does that make?

[robust scaffold]

When you're debugging and need to see what that branch will say in the inspector. Burst will automatically delete other branches resulting from a Assumed true value

That way you dont get confused by the million other random shit in the inspector

[viral sonnet]

yeah, i just read about safety checks also. okay, not too interesting for me 🙂

[robust scaffold]

Since you cant see what is which branch if the comments dont line up perfectly (I wish we could add compiler comments my god, would make looking through the inspector so much easier).

[viral sonnet]

that would be really useful! most relevant code I find at the bottom lol

[robust scaffold]

Turn on synchronous burst compiling. No-Inlining functions will appear near the top of the inspector.

Asynchronous burst compiling shuffles the burst output and makes reading through it very confusing

[viral sonnet]

ah nice!

great tip! that's so much easier to read now

[robust scaffold]

Huh, the inclusion of BurstCompile compile synchronously throws a lot of errors

wait, function pointer

[robust scaffold]

https://www.jacksondunstan.com/articles/5494

Twisting the burst compiler and C# itself to allow for type inheritance and generic reactions in the form of "spells". A good read

Definitely in no way optimal performance.

[robust scaffold]

Reminder for self tomorrow. Test element at ref function compatible with interlocked and vectorization. Check if stack alloc returns ref-able value.

[viral sonnet]

thanks for the link. sounds really interesting. i'll finish first and ponder what that could gain me

right now the biggest value I'd see is not having so many branches

which is BIG

[robust scaffold]

You still have your testing interlocked job?

How about you try this test for me. Because I can not get it out of my head

And I dont want to walk 30 min back in the rain to code it myself

[viral sonnet]

for the race condition on the health value? or the temporaryHealth job?

[robust scaffold]

Temporary health job

Replace temporary health with this I'm gonna write out

[StructLayout(Explicit)]
Public unsafe struct TempHealth : IComponentData
{
[FieldOffset(0)] Public int Value;

Public unsafe ref int GetReference()
{
fixed(int* ptr = &Value)
{
return ref ptr[0];
}
}
}```

[viral sonnet]

gonna try 🙂 give me a min

[robust scaffold]

Replace the .Ptr[0] in the interlocked.add with .GetReference(). Replace the summation section with .Value.

I wrote that out by memory on my phone. I cant recall what class Explicit so please fill that in with autocomplete

[viral sonnet]

thanks, I wanted to get rid of the manual alloc/dispose

the job is not vectorized

[robust scaffold]

Did it at least work?

[viral sonnet]

i think reading and writing to healths[i].health is the problem

i'll try the interlocked now

[robust scaffold]

+= is vectorized. I've done it many times

Wait. .health

The subsection of the struct, ya, that is not vectorized

Isolate the health parameter and the addition will be vectorized.

[viral sonnet]

success!

[robust scaffold]

What? How?

[viral sonnet]

what you said. isolate the health parameter

[robust scaffold]

Ah. I thought you just recompile it again

You can just reinterpret the health array to int and remove the need for the extra .health

[viral sonnet]

yeah, I saw that you were doing this in your code

would be nice to have a method for vectorized code with more than just 1 value in a struct

I might have some ideas

but first I'm testing the interlocked. you're such a blessing, I actually wanted to rewrite the need for the allocation and then you were just typing it in discord for me 😄

[robust scaffold]

I was hit my inspiration while rain was pouring down my neck. Maybe I would walk outside without my umbrella more often...

[viral sonnet]

Be careful before you catch a cold!

[robust scaffold]

I wanna hear them once ya done testing interlocked

[viral sonnet]

well, not sure if the ideas are any good. i think the the memory layout has to be sequential ints here so there's really no circumventing this then other than to use single values in the struct

[robust scaffold]

I'm personally looking into NativeSlice. If it could "slice" every odd or even index, I could reinterpret a struct containing 2 ints into an int array the native slice every even index to obtain the first property / int using slice with stride to skip every 4 bytes (skipping the odd properties). Then operate on that (assuming it retains pointers to the original array) with vectorized operations and boom, selective vectorization on a struct with multiple propeties.

Or I could just set the for loop index increment to +2 instead of ++. I dont know if that is vectorizable.

[viral sonnet]

hm, interesting, the health value doesn't get updated anymore

[robust scaffold]

Nooooooo

What does the interlocked command look like

[viral sonnet]

Interlocked.Add(ref Health_WriteLookup[destination].GetHealthReference(), amount);

[robust scaffold]

Nooo, fuck

I thought I was onto something. Damn

I'm gonna try it myself but yea, I doubt its gonna work if that doesnt

[robust scaffold]

This is going beyond what I've done before, can you remove the fixed() and just return ref value?

I really doubt that will pass the compiler

[viral sonnet]

doesn't compile. structs can't return this or members by reference

[robust scaffold]

Shame. I'll need to dig into stack alloc to bake in the fixed and I might be able to ref return the [0] index of a stack alloc int[0].

Well, stack alloc needs disposal so that won't remove the Dispose() requirement. Damn

[robust scaffold]

Remove the ref return. Just return the Ptr itself the use .Ptr[0] in the interlocked.

Wait no. It doesnt work like that...

I need to do some research tomorrow to see how I can get a returnable pointer...

Or dig into CDFE to see if I can hook into the entities list before they copy the data for reading

[viral sonnet]

ok, the problem is that CDFE doesn't return the ref. it would work otherwise I think

[robust scaffold]

Yea

It returns a copy. I just remembered. Which is why I had that pointer manually allocated hoops to jump through at creation

[coarse turtle]

You could probably write a wrapper that does this but remove the CopyPtrToStructure

you'll probably need to edit the source directly though

[viral sonnet]

lol just wanted to post that code 😄 exactly. I'll look into exposing this. not having refs on CDFE pains me for a while

[coarse turtle]

since EntityComponentStore is private.

[viral sonnet]

I've been using DreamingImLatios exposed package for a while https://github.com/Dreaming381/Latios-Framework/tree/v0.4.2/EntitiesExposed It can be added there. He already uses the EntityComponentStore

[coarse turtle]

cool didn't know that existed

[viral sonnet]

I had some trouble with his package version so here's my own

[coarse turtle]

i might use the Myri package for audio 👂

[viral sonnet]

@robust scaffold success! I've exposed CDFE and the GetReference is working as expected!

here's the updated version

and don't forget using Unity.Entities.Exposed

off to bed, have a good one o/

[robust scaffold]

slight modification:

/// <summary>
/// Custom returns reference to entity's component as read write access.
/// </summary>
/// <param name="entity">The target entity being accessed.</param>
/// <returns>Direct reference to component on entity for modification.</returns>
public ref T GetReference(Entity entity)
{
#if ENABLE_UNITY_COLLECTIONS_CHECKS
    AtomicSafetyHandle.CheckWriteAndThrow(m_Safety);
#endif
    m_EntityComponentStore->AssertEntityHasComponent(entity, m_TypeIndex);

    CheckComponentIsZeroSized();

    void* ptr = m_EntityComponentStore->GetComponentDataWithTypeRW(entity, m_TypeIndex, m_GlobalSystemVersion, ref m_Cache);
    return ref UnsafeUtility.AsRef<T>(ptr);
}```

Changes GetComponentData to Read/Write access

And merged in the AsRef<>() into the return function so unsafe is not needed in the SystemBase code

And a bit of documentation doesnt hurt.

[viral sonnet]

hey o/ yeah, we will need to cleanup some stuff. what i posted is a proof of concept

[robust scaffold]

Yep, it works. And it's proven to work. Now if unity could actually implement it...

[viral sonnet]

sucks they are so reluctant to add it. having refs/pointers is so powerful. it feels like best of both worlds for c# and c

[robust scaffold]

They pass around NativeArray's GetUnsafePointer() so they have the power and willingness to do it

[viral sonnet]

there are so many instances where you get a struct copy and then have to set it again when you just change one value. there's no sense in it

i thought thta's the whole point of NativeContainers!

otherwise they have all the power and the dev is scrambling around

but yeah, if they don't do it. we have the capability to make it work regardless

when all is cleaned up, i'll post it in the forums. guess some will be interested in it

and maybe they have already targeted it for the next entities release

one year has now passed by since 0.17 was released internally

kind of crazy because I'm really fucking interested what they have done in all this time. I'm certainly not accusing them of slacking around

[robust scaffold]

public static class CustomSystemBaseExtensions
{
    public static ComponentDataFromEntityExposed<T> GetWriteReferenceFromEntity<T>(this ComponentSystemBase systemBase)
        where T : struct, IComponentData
    {
        return systemBase.EntityManager.GetExposedCDFE<T>(false);
    }
}```

this.GetWriteReferenceFromEntity<Census>();

Unfortunately the "this." is required but it's close enough to default system base method calling

var t1 = this.GetWriteReferenceFromEntity<Census>();
ref var t2 = ref t1.GetReference(Entity.Null);

[robust scaffold]

I'm guessing that the next Entities release will be a full blown 1.0.0-pre.1

It's what, 0.21 in the manifest leaks or was it bumped to 0.23?

It's been at least 2 full releases internally since they went offline

[viral sonnet]

yeah, it's still so weird that they stopped releasing. apparently they run on a completely different unity version

[robust scaffold]

yea, 2020.3-DOTS

All the fancy new toys like UI toolkit even still targets 2020LTS even if 2021LTS is out.

[viral sonnet]

i thought using 2020.3 would mean they release this unity version. but they never did. might be a big bug fest incoming

[robust scaffold]

Normally 2020 would be on life support, critical bugs but otherwise pretty much finalized, but the "cutting edge" features are still getting backported

[haughty rampart]

no never. it's far too soon for that. there's still gonna be loads of changes before we're anywhere near 1.0.0

[robust scaffold]

All the burst major features are still min 2020.2 while burst itself is 2019.1

[robust scaffold]

Transport is headed out for shipping to public release soon and the documentation is even more sparse than regular Entities

[viral sonnet]

joachim said next release will be soon some days ago. i hope this means this year 🙂

[haughty rampart]

transport is for dots AND gameobjects though (if i remember correctly) anyway, that is NETWORKING that is not that hard to finalize. because it contains no actual logic

[robust scaffold]

ha, not happening. Probably early 2022, maybe Feb / March.

[haughty rampart]

nah, they said: at the end of this year at the earliest. so it's possible

[viral sonnet]

transport has no dependency to entities, so the release means little for it

[robust scaffold]

I guess, the data in really is generic

[viral sonnet]

still a good thing because overall netcode relies on it and that's already heavily delayed

[robust scaffold]

"com.unity.entities": "0.20.0-preview.26". DOTS still on 0.21 (0.20 internally). From the training packages updated a month ago

[haughty rampart]

can you reference that in a project or where did you get this from?

i'm interested

[robust scaffold]

https://github.com/Unity-Technologies/DOTS-training-samples/branches/active

[north bay]

Hmm can't you use something like this so you don't have to expose a different ComponentDataFromEntity struct?
https://hatebin.com/tmdbhonipp

I think I'm gonna add that to my AsRef extensions lol

[robust scaffold]

Seems a bit more logical than our implementation where we just wholesale copied the EntityManager implementation to get access to the component data storage

Anyways, i got yet more classes, be back in.... 2 or so hours. Maybe I can code during class?

[viral sonnet]

Good thinking to just typecast it. I'm sure we can improve this a lot

[robust scaffold]

Hrm, dont have access EntityComponentStore without assembly modification

[north bay]

Ye you need to put it into the asmref

[robust scaffold]

Yep, definitely a lot simpler to use.

[glacial hazel]

Hii. I'm learning DOTS, and.... I'd appreciate any help xD. I'm very used to code with "event Action<>" because it helps to keep different functionalities decoupled.
Is there any kind of events in dots? Or I'd have to use like component tags (without data), along with entity queries constantly searching for an entity with that component to "simulate" an event? What other option there is?

[haughty rampart]

no. there are no event's in dots. and it's unlikely that something similar will come to dots. if you do truly need events, here's a video where this is implemented. https://www.youtube.com/watch?v=fkJ-7pqnRGo

[robust scaffold]

There's a near daily thread on event implementation in DOTS. Because there is no "official" way.

[glacial hazel]

Awesome, thank you

[glacial hazel]

Wow, that's crazy

[haughty rampart]

keep in mind that events are anti data-oriented-design

[glacial hazel]

Aah... so that means that I'm still thinking "the old way"

[robust scaffold]

Yea, ideally you should somehow design a different way other than events

[haughty rampart]

yes

[viral sonnet]

keep in mind, everything you want to solve with creating or writing more data is also bad 😉

the best case is, do the task where it occurs

there's a lot of lazy patterns going around handling this and none are any good for performance. if you don't care about it, fine, although i wonder why dots then

[haughty rampart]

exactly

[viral sonnet]

some good rule of thumb, best speed ups come from burst code, vectoriziation and good data layout

[haughty rampart]

and most things are doable that way anyways

and the ones that aren't.......may be in the future

[glacial hazel]

No idea what you mean with vectorization

[haughty rampart]

vectorization is when your cpu does e.g. 4 copy operations at once, instead of 1 at a time

[glacial hazel]

Aaah.. ok... where could I read more about that?

I think I need to see concrete examples to understand it better

[haughty rampart]

https://docs.unity3d.com/Packages/com.unity.burst@1.6/manual/docs/OptimizationGuidelines-Aliasing.html

heres the link. it also has lots of useful graphics

[glacial hazel]

Thank you!

I have so many questions lol

For example, no idea how to read this:

.LBB0_4:
vmovups ymm0, ymmword ptr [rcx - 96]
vmovups ymm1, ymmword ptr [rcx - 64]
vmovups ymm2, ymmword ptr [rcx - 32]
vmovups ymm3, ymmword ptr [rcx]
vmovups ymmword ptr [rdx - 96], ymm0
vmovups ymmword ptr [rdx - 64], ymm1
vmovups ymmword ptr [rdx - 32], ymm2
vmovups ymmword ptr [rdx], ymm3
sub rdx, -128
sub rcx, -128
add rsi, -32
jne .LBB0_4
test r10d, r10d
je .LBB0_8

Looks so cryptic

[haughty rampart]

that's assembler language

[glacial hazel]

Should I lean how to read that?

Or I need to be a robot/computer to read it?

[haughty rampart]

assembly is pretty easy to learn. you don't really need to learn it but if you want to know why exactly something doesn't run optimally, it can be useful.

[haughty rampart]

but you don't really need to learn it for this documentation since they describe the important parts below the code snippets

assembler is really interesting. there are the small code pieces, which look (clean) and then there are these huge code pieces which look worse at first glance but perform way better than the small ones (in their use cases). a rule of thumb for assembler i find useful is:
if the words are long: it's optimized, if they are short, it's not XD

[viral sonnet]

does anyone know in which country the dots team is working?

[haughty rampart]

why's that important?

[viral sonnet]

because I might move and I'm going through the options 🙂

[coarse turtle]

I think it's pretty spread out

[haughty rampart]

yeah

[coarse turtle]

You have Burbank, CA and I'm pretty sure there's one in Helsinki

Probably some in france & copenhagen 🤷

[viral sonnet]

hm, alright. thanks 🙂

doesn't seem to really matter then

[viral sonnet]

long good, that's what she said

[robust scaffold]

ymmX is packing of pointers. ymmXXX is the pointer to the set of packed pointers. vXXXXX is the vectorized operation on that packed pointer.

the vXXXXXX is purple and purple is good.

[robust scaffold]

And yea, dont worry about burst optimization right now. First learn to walk before running. Walking in this case is getting a job to be bursted in the first place.

[glacial hazel]

Ok haha, thank you!

[robust scaffold]

Bursting is more than just adding [BurstCompile] to a job, it's an entirely new language "hiding" under C# that Unity is developing called HPC#, High Performance C#.

[robust scaffold]

And first step, ditch the Lambda Entities.ForEach(). Too restrictive, doesnt vectorize. Use IJobEntityBatch struct based jobs.

[glacial hazel]

Whaaat, didn't know that. And where can I read about that or is not out yet?

[robust scaffold]

Burst is fully released, has been for... years?, but it's like a completely different coding language. Like C# suddenly wanted to be C++.

[glacial hazel]

Is IJobForEachWithEntity<> the same than the lambda Entities.ForEach()?

[glacial hazel]

That's why I see pointers and stuff

[robust scaffold]

ForEachWithEntity has been deprecated for years. IJobEntityBatch is ForEach equivalent.

[robust scaffold]

Yep. All that stuff we were talking about over the past few days are burst code

[glacial hazel]

So I'm watching "depreciated tutorials". From where do you recommend me to learn?

[robust scaffold]

Entities.ForEach is actually compiled to IJobEntityChunk. IJobEntityBatch is newer and more optimized

[glacial hazel]

So I don't have to bother you guys all the time haha xD

[robust scaffold]

Yeaaa......... yea. Thats DOTS. There is no tutorials. Just us here on this discord

[glacial hazel]

Ah, feck. You're the tutorial

[robust scaffold]

We learn by hours of trial, error, and figuring out why it didnt work

[glacial hazel]

And, there are no DOTS developers helping you?

[robust scaffold]

Unity DOTS themselves has gone deep undercover and running ultra silent. Nothing constructive from them.

[glacial hazel]

Why you think is that?

[robust scaffold]

there still is a literal gag order on talking about DOTS for all Unity Employees outside the C suite top level. It's in development hell for about a year now

Just ask the Burst employees if something is compatible with DOTS and they will not respond, because they are not allowed to.

[glacial hazel]

Hmm. crazy. And do you have a theory of why they're not allowed to?

[robust scaffold]

Development hell. When progress on code remains stagnant because there's no clear vision forward

DOTS has literal billions of dollars being thrown at it, that's a lot of money for investors that may be going nowhere

So no news coming out to panic investors

I dont remember where I read this but apparently half the core Unity company is working purely on DOTS. Around 40 - 50 people. A normal team like the guys working on the entirety of UI Toolkit (the new UI system) is only 4 people.

[glacial hazel]

Wow. Hopefully there are news in the next Unite now

I'm not too hopeful to learn DOTS after you telling me this though xD

[robust scaffold]

hahaha, my sweet summer child

news? no news.

if they even mention the existance of DOTS

There is supposedly mandatory company wide regular training in DOTS though. Everyone from the people maintaining HDRP to audio have to know DOTS. So it's "alive".

That was from the last Brian's public appearance a few months back

[glacial hazel]

Ok, so it's worth it

[glacial hazel]

Crazy that we had this chat on the Unity's official discord😅

[robust scaffold]

It's worth it ya. How new are you to DOTS? have you made anything with IJobParallel?

[glacial hazel]

I'm super new I'd say, I'm just understanding basic stuff, I've done a lot of stuff with the Job C# system though, but whithout ECS. And I'm watching all the material that I can find. But there are fundamental stuff that I don't understand I think

If you shouldn't use "events" in dots. How would you code a button that when the player presses it opens a door? Or any basic example like that

[safe lintel]

i use entities as events

ie if a button is pushed, ButtonJob spawns an entity with a ButtonPushedComponent that contains data that is then processed/consumed by other systems looking only for the ButtonPushedComponent component

[robust scaffold]

ewww, structural change.

[robust scaffold]

I dont have any events for now but I'm projecting maybe a single singleton containing a buffer with either a function pointer containing the event or an enum mapping to an ultimate event switch statement containing the logic.

Also

God, Burst's dev team is on point in the forums. Literally an hour to reproduce an error then respond that they're fixing it. Imagine if DOTS was remotely as responsive.

[glacial hazel]

So the logic wouldn't by ran by a System?

Like the logic of "opening the door"

[robust scaffold]

Yep. The system is where all the logic is suppose to be. You dont want any logic inside the components

[glacial hazel]

Also, is it not Burst part of DOTS ? Or is not because you also use it with C# job system?

[glacial hazel]

Ah ok, I was thinking more like a Monobehaviour instead of a Component xD

[robust scaffold]

Nope. Burst is Core Unity. Along with Jobs, Mathematics, and Collections (and soon to be Transport).

[glacial hazel]

Aaaaah right!. makes sense

[robust scaffold]

Component is like the values in an array. Systems are like the logic occurring inside and around a for loop accessing that data array

And to finish that analogy, Entities are the index of the component in that array

[glacial hazel]

Yeah, I guess my question is... if you make a job, that's part of a system class. But this job is called in response of an event, instead of running every frame....

Does it matter if that job that you're gonna run, is part of a system class or can be in any class

[robust scaffold]

Yea, honestly it really depends on the frequency of that event

I do have the singleton entity based event flagging currently:

But they're created and operated on once every 3 or 4 seconds, if not more. That's hundreds of frames where they wouldnt exist.

Now if an even may occur every frame or every 10th frame, yea I'll assign a buffer location on a global singleton entity

[glacial hazel]

Not sure what you mean with "singleton entity", for me singletons are a class with a static self-reference that you can access from any code... And I don't understand an entity as a class, but as a collection of components

Also I feel guilty of asking questions because I feel like I'm stealing your time, maybe you guys should make the tutorials xD

More efficient than answering one by one haha