archived-dots

[vivid lotus]

It seems like it isn't worse, at any rate.

Since doesn't every method of entity creation force jobs to complete?

[zenith wyvern]

Oh, when you say spawning from a gameobject prefab...I think you might be a bit mixed up there

Do you mean like - converting a gameobject to an entity at runtime?

Because that's something that should only happen once when your game starts

Preferably via conversion

Then you instantiate from that entity prefab

[vivid lotus]

Oh right, okay, right. The process of spawning from a "GameObject prefab" (which I'm now putting in quotes) involves a conversion to entity anyway first, right?

I know there's optimizations under the hood that make that "GameObject prefab" never actually be a GameObject.

[zenith wyvern]

Yeah, that's expensive, but you would just do that once, then insantiating your entity prefab is no worse than just building the entity from code

Maybe faster

[vivid lotus]

Gotcha, okay. So either way you're generally keeping entities around for the sole purpose of duplicating them.

[zenith wyvern]

Yup

[vivid lotus]

And then yeah, in my case, I'd use a blob or a native data structure to keep track of which entities to duplicate when.

[zenith wyvern]

When you go through the typical conversion process you end up with a single entity with all your components and a Prefab component. Entities with a Prefab component are ignored by queries by default, so that gives you your prefab to instantiate from

[vivid lotus]

Oh, I'm doing it wrong then, because I'm actually spawning the entity and seeing it.

[zenith wyvern]

https://github.com/Unity-Technologies/EntityComponentSystemSamples/tree/master/ECSSamples/Assets/HelloCube/5. SpawnFromEntity

The readme there explains it a bit

Basically you set up your gameobject prefab, reference it in your conversion script with IDeclareReferencedPrefab and when you call GetPrimaryEntity on your gameobject prefab inside your conversion script, that gives you your canonical prefab entity which you can assign to a component or whatever to be instantiated from later

[vivid lotus]

Ah, right, okay, this was my problem.

I want to take a prefab like this, but programmatically create variants as prefab entities.

So from one Unity asset prefab (in the project directory) to multiple actual converted entity Prefabs, each with different values for their RenderMesh.

(One per sprite off of a sprite sheet, in my case.)

I can spawn actual entities like this, but I'm stuck trying to get them to just be prefabs instead of actually existing.

[zenith wyvern]

You can instantiate from the prefab, set the rendermesh on the new entity, then re-add the Prefab component, then you have a new prefab variant

If you add the Prefab component yourself it should be ignored by everything

[vivid lotus]

Oh, seems simple enough.

private static void CreateSpritePrefabEntity(World world, Entity prefab, string name, Mesh mesh, Material material)
{
    EntityManager entityManager = world.EntityManager;
    Entity entity = entityManager.Instantiate(prefab);

    entityManager.SetName(entity, name + "_SpritePrefab");
    entityManager.SetSharedComponentData(entity, new RenderMesh
    {
        mesh = mesh,
        material = material,
    });

    entityManager.AddComponent<Prefab>(entity);
}

[zenith wyvern]

Exactly

[vivid lotus]

Cool, and then that can live in some native data structure, and I can pluck those out when I need to create those sprites. Awesome.

Thank you for explaining this to me, been very helpful.

[ocean tundra]

Anyone messed with SubScenes in the Editor?
I want to have a editor tool that finds a bunch of prefabs, adds them to a subscene and converts them
Then i can use the world as i like and dispose when done

I figure i can use the older Conversion APIs, do they do the same things as subscenes ie the custom conversion systems?

[wary anchor]

Trying to track down the cause of a crash a user reported to me. The log seem to indicate the problem is in the allocation of a NativeArray, which would mean it's one or both of these NativeArrays that have been used once or more than once then disposed of just prior to running a compute shader:

computeShader.Dispatch(kernelHandle, width / 8, height / 8, depth / 8);
nearestColoursComputeBuffer.GetData(nearestColoursOut);
nearestAtomComputeBuffer.GetData(nearestAtomDataOut);

NearestAtomsToGridPointsArray = nearestAtomDataOut;
NearestColoursToGridPointsArray = nearestColoursOut;

NearestAtomArray = new NativeArray<NearestAtomsToGridPoint>(NearestAtomsToGridPointsArray, Allocator.Persistent);
NearestColoursArray = new NativeArray<NearestAtomsToGridPoint>(NearestColoursToGridPointsArray, Allocator.Persistent);

nearestAtomComputeBuffer.Dispose();
nearestColoursComputeBuffer.Dispose();

the log:

I'm unclear why this crash could have happened as I've been very careful to manage the native arrays

[karmic basin]

Yeah that's the way to do it. I suggested inject mode because you said you didn't want to write authoring stuff and in my mind it would keep the object components and make them available in ECS. At least that's what I thought back when I tried with a particle system. Looks like my memory fails me, maybe I had to write a conversion system to make it work 🤔 . Now I need to review the conversion part, maybe I went through it too quickly and mixed things up 😅
Anyway glad you achieved your end result 👍 And yeah that's a great blog

[viral comet]

Hi guys! Am I the only one who missing a "Go from game Object to Entity" feature?

[wary anchor]

it's called "Authoring" if that helps the searches!

[viral comet]

I want to click on my view gameObject in a scene and somehow navigate to corresponding entity in EntityDebugger.

[wary anchor]

Sorry, I generate all my entities from plain old data rather than anything from the Inspector or GO-land, I'm sure others can help more.

[viral comet]

if there was an API for selecting an entity in a debugger... Something like:

#if UNITY_EDITOR
    Unity.Entities.Debugger.SelectEntity(entity);
#endif

It would be super helpful

having such an API, it would be easy to make an MonoBehavior that tracks the click and selects the desired entity

[bright sentinel]

Unfortunately that isn't implemented yet. I hear there are plans for it though.

[pulsar jay]

I am trying to extend Addressables to use it with entities. Anybody has experience with that? I currently hit a roadblock regarding reference counting: https://forum.unity.com/threads/entity-conversion-with-custom-operation.1073135/

[pliant pike]

I don't suppose anyone know why I'm getting this error NullReferenceException: Object reference not set to an instance of an object from this code if (!World.DefaultGameObjectInjectionWorld.IsCreated) return;

the whole point of that code is to make sure there isn't a null reference it's not supposed to need one 😕

[bright sentinel]

World.IsCreated is not the same as World.DefaultGameObjectInjectionWorld. The latter could be a world, but it may not have been created yet, as it could be deferred. You can just check against null directly, and probably also check IsCreated

[pliant pike]

you can't do world.IsCreated

[bright sentinel]

Why not?

[pliant pike]

it doesn't let me, and how would it know which world you could have several

[bright sentinel]

Right, I was thinking on a specific instance

So you would have to do if (World.DefaultGameObjectInjection != null && !World.DefaultGameObjectInjectionWorld.IsCreated) or something like that

[pliant pike]

its weird because that check works in another older scene but just not in this new scene

[bright sentinel]

Probably because of some race condition

Race condition here being loading time

So in your old scene you were just lucky it worked

[pliant pike]

there's no race conditions because its running in the edtior in a gizmos code thing

[bright sentinel]

The world could still not be created though, I'm not sure why that would have anything to do with it

[pliant pike]

that's why I have to check that the world exists so that the code only runs on play because otherwise it runs in editor and I get errors

[bright sentinel]

Then just do what I just told you, I don't see the problem

[pliant pike]

yeah ok np, thanks

nope does not work either

[bright sentinel]

Same error?

[pliant pike]

yep

[bright sentinel]

What does the code look like now?

[pliant pike]

if (!World.DefaultGameObjectInjectionWorld.IsCreated && World.DefaultGameObjectInjectionWorld == null) return;

[bright sentinel]

You have to do the null check first, and you have to turn it into an 'or' instead of 'and'.

The reason is that you will still try to access the world before you check if it's null, so if it's null you'll get the error

And the reason why you want to turn it into an 'or' is because you don't want that code to run if it's null, or if it's not created yet

[pliant pike]

awesome, that works, thanks

[harsh owl]

Is there official support for ECS particles, or I need to go hybrid with it?

[safe lintel]

hybrid unfortunately for now

kind of annoying as vfxgraph doesnt batch calls, so you get a performance hit with lots of similar emitters

[ocean tundra]

I though VFX graph was quite performant 😦

[harsh owl]

well.. I already had a particle manager for GameObject handling of particles... I guess I will try making a ECS bridge for it using that event system thing

[safe lintel]

it does handle like millions of particles from a particular emitter easily, but in my case where i just have a handful of particles for bullet sparks, and then potentially a hundred of those emitters separately, it sorta is a perf hit. at least its on the product board as being addressed(at some point)

[ocean tundra]

Is there a way to have a single emitter and give it your hit locations?

(i havnt used VFX Graph yet)

[safe lintel]

yeah last time i looked into arrays it wasnt implemented, not sure what the status is now

[ocean tundra]

oh well

something to hope will be implemented in the future 😛

Anyone tried storing a burst FunctionPointer in ComponentData OR a BufferElement?

[vagrant lotus]

Hi is anybody good with DOTS audio?

[ocean tundra]

Not sure if many people are using it

I dont see many questions/chat about it here

[odd ridge]

I just realized unity 2020 LTS was out. does that mean there's no chance we get enable/disable component on 2020? I don't want to go through unity upgrade hell again xd

[ocean tundra]

@odd ridge Pretty sure i read somewhere that the next version of Entities (0.18) will depend on 2021 (but cant remember exact version)

[odd ridge]

alright, no enable/disable for me that means xd

[ocean tundra]

what are you doing thats so painful to update?

(so i can advoid it 😛 )

also they said enabled bits wont be complete in the next version anyway (https://forum.unity.com/threads/entities-0-17-changelog.1020202/page-2#post-6774380)

also i cant find the message that said 0.18 will depend on 2021, maybe i made that up sorry :/

But if enabled still wont land in 0.18 then seems higher chance that 0.19 will end up depending on 2021

[odd ridge]

it's just that I have a LOT of tag remove/add every frame and I would get stutters every now and then because of the structure changes

[vivid lotus]

Anyone know where I would go to report some code that doesn't compile in Burst, but probably should? Is there an issue tracker specifically for DOTS/Burst stuff?

[ocean tundra]

@odd ridge I've seen some advice to put a Enabled flag into your components and edit those instead

as that would be very easy to update to the new Enabled/Disabled bits when its finally out

@vivid lotus Use the normal bug reporting, and maybe make a fourm post

@odd ridge but it really depends on what your doing, as there a good cases to use tags

[vivid lotus]

Anyone have an example of a JobComponentSystem that uses an ECB? I can find code examples of one or the other, but not both combined.

Not sure how to go about passing the ECB into the job to be used.

Or are JobComponentSystems out of style now? I see IJobForEach is deprecated, apparently.

[zenith wyvern]

There's an example in the manual

https://docs.unity3d.com/Packages/com.unity.entities@0.17/manual/entity_command_buffer.html

And yeah you should only be using SystemBase now, not ComponentSystem or JobComponentSystem

[stone osprey]

Does it make sense to implement some sort of progress/progressbar as an entity ? It should float somewhere under or beneath a mob/character to show the progress of its current task. Like chopping down trees. So it actually has an location inside the world. And it also has multiple conditions when the progressbar should dissappear or stop the progress. Like when the mob moves away during chopping. So i think its actually worth being an entity. Or ?

[stone osprey]

@craggy vault Store a reference... just have some component that stores like this : PositionReference{ Entity b } on A and loop over them to acess components of B.

[bright sentinel]

@craggy vault Use GetComponent<Translation>(EntityB) to get the position

[hollow sorrel]

yeah I think that makes sense, what are you comparing it to doing instead?
I imagine you could do progressbar as component, and have a system iterate over that for rendering
but i think using separate render entities makes a lot of sense, and means your chunks will be cleaner (can just have 1 chunk full of progressbars regardless of what components the target has)

[stone osprey]

@hollow sorrel Thanks, i already implemented a little prototype and it seems to work fine ^^ Im just always a little... undecided if it comes to decide between using entities or using oop/structs. I just cant find a good guideline for that :/ Actually its possible to implement everything as entities, but not everything should be an entity.

[hollow sorrel]

yeah that's fair, implementing it with just structs outside entities/components would be not bad either, not sure when to do which

guideline I've been trying to follow is if it's game data then it belongs as entity, because ideally I think you should be able to serialize your world and deserialize, and your game should still work as if nothing happened
but when it comes to rendering it becomes more fuzzy, that doesn't need to be part of game data

[stone osprey]

@hollow sorrel That actually seems like a pretty good guideline ^^ But whats game data in your case ? Everything that has a "hard coded" structure like a buff for example : Buff{ name, effect, duration } ? Or everything that has no "logic" ? If so features could vary a lot... for example, if a game has items without any logic on them... they would be game data. But if they have logic ( Some cool stuff happening when used, or something that gets executed every frame ), then they would be an entity ?

[hollow sorrel]

imo anything that is needed to reconstruct your game from a serialized state
so logic, mobs, etc
assets and static data might be an exception, so for buffs you could just have your entities have an id to a buff type stored in a table somewhere, and that table is not part of ecs (could just be an array constructed at startup, or a blobasset/scriptableobject etc)
but the actual remaining/current duration would be stored as part of an entity since that'd be game data

so when it comes to rendering progress bars, if you restore your game state, you could just recalculate the progress and create/display a new progressbar, it doesn't need to be serialized since it can be recreated from existing game state

so in those cases where it's not really part of game state, then you have the choice of making it an entity anyway or not

basically i'd think of the game as a deterministic simulation, even if it's not deterministic, i just mean architecture wise

[frosty siren]

Is there a way in dots to prevent hierarchy conversion? What i want is to be able to have nested gameobjects in my prefab, which are convertable and linked by parent gameobject (entity) but not included to his hierarchy after conversion (no parent, no including to LinkedEnityGroup, no including to Child).

[pliant pike]

do it manually yourself? maybe using something like a GameobjectConversionSystem

[hollow jolt]

how to read entity data from mono behavior ?

found it :

var EM = World.DefaultGameObjectInjectionWorld.EntityManager;
EM.GetComponentData<Translation>( PlayerA.ENT );
// where ENT is set manualy via GameObjectConversionSystem

[safe lintel]

@frosty siren theres https://docs.unity3d.com/Packages/com.unity.entities@0.17/api/Global Namespace.StopConvertToEntity.html?q=Stop

[frosty siren]

As i understand it will completely stop converting

[safe lintel]

think i misunderstood, not sure if theres something only to stop the hierarchy. Could always remove it from the hierarchy with another system?

[frosty siren]

I think about to write late GameObjectConversionSystem which will get all gameobject's tranforms (only which i want to not add transfrom components) and remove LTW and others, and also ask if it is nested HasComponent<Parent> and then exclude it from parent.

[pliant pike]

I don't suppose anyone knows simple way to draw wire cubes and lines in game mode with hybrid renderer

I'm trying to use Gizmos.Draw... etc and it seems it will take about 20 mins to draw a frame

[safe lintel]

you can use ALINE but its a paid asset

its by the guy who made the astarpathfindingproject

works in editor and game, I quite like it 🙂

[pliant pike]

cool thanks, I will definitely buy it, that stuff is really useful

[karmic basin]

Through the EntityManager of your world or schedule it for a CommandBuffer

[safe lintel]

systembase doesnt change how entities are deleted from componentsystems or jobcomponentsystems.

[olive kite]

When reloading the scene it seems to leave all of entity data and rendering. Is there a command to flush it or should I be keeping track and disposing of all the entities manually to recreate or reset the world?

[safe lintel]

dont think theres anything builtin to handle scene changes yet 😦

[olive kite]

cool thanks for the info

[vivid lotus]

If you have a lot of entity pairs (A, B), where an entity type B needs to read information from its linked entity of type A every frame, and there's a lot of both, is there a good way to batch that information transfer? I feel like just looping over every B and having it look up its A one by one would be cache miss city when everything is potentially in different places or different chunks entirely.

[karmic basin]

Are entity pairs components an option for you ? Relationship : IComponentData {Entity entityA; Entity entityB;}

[vivid lotus]

Possibly. To ground it a little, in this example I'm thinking about splitting the gameplay entity from the visual representation entity, and I need to have the visual representation entity receive the world position from the gameplay entity.

But they could all be in arbitrary places in memory relative to one another, and the visual representation entities may be created/destroyed depending on how far the gameplay entity is from the camera.

I'm trying to figure out how best to optimize the position transfer from gameplay entity to representation entity, in big batches, every frame.

[karmic basin]

For camera culling you can look SharedComponentDatas, to group them in chinks

chunks*

[vivid lotus]

I can do that. My situation is complicated because the position of the gameplay entities can't be represented by a normal transform (the world is too big for a traditional float3).

So I can't just, for example, parent the representation entity to the gameplay entity.

I need to do a transformation in the process.

So I have to have a component on the representation entity that points to the gameplay entity, and is responsible for converting the gameplay entity's position into something renderable (the camera never actually moves).

But since there's potentially lots of represented entities, I want to make that transformation as optimal as possible, which means avoiding caches misses as I can.

[karmic basin]

Yeah I see, floating point precision. I will face the same problem soon. I think I'll try to hack my way with subscene "chunks" (not ECS chunks) streaming.

[vivid lotus]

Yeah, my game positions are double-based.

So I need to transform the game positions into camera-relative renderable float positions, if that gameplay entity is close enough to render at all.

[karmic basin]

For the link and transform part... 🤔 maybe I'd try a one-way link yeah. Maybe

[vivid lotus]

That's what I'm thinking yeah.

The render entity just stores the entity it represents. If it exists, it draws, if not, then no work.

[karmic basin]

Sounds like the first idea at least.

[vivid lotus]

But that means each pass each render entity needs to random-access the gameplay entity it represents, which could be anywhere in any chunk basically.

Which is cache miss city.

Just wondering if anyone has wrestled with this before.

What I could do is move gameplay entities that have a representation to a dedicated chunk, and make sure it has the same index in its chunk that the representation entity has in its chunk. Then that's minimum possible cache misses I think.

But that causes fragmentation in other ways, and also means I can't use chunks to optimize those entities any other way, either.

[karmic basin]

Yeah I'd go subscenes chunks octree style as my first try, ignore anything else than current + neighbors
Keep in mind chunks are not infinite anyway, you don't have only one of each archetype. Entities will span multiples chunks anyway

[vivid lotus]

Yeah I'm not well versed in chunk manipulation in DOTS yet.

[karmic basin]

in the entity debugger you can see how much space they take

[vivid lotus]

And yeah I'd prefer to use them for spatial decomposition instead.

The other option is to use some sort of intermediate data structure, where I iterate over every gameplay entity that has a representation (via tag, I guess?), and have it write its position to some structure, and then have the representation entities read it.

But that seems wasteful.

[karmic basin]

another option is, each time you iterate through a gameplay entity, you push it on the front row of a cache collection

you can control size of the cache

[vivid lotus]

At a hardware level?

I don't think so, necessarily.

[karmic basin]

nope your own cache

it's just a raw idea

🤔 uhmmm i wanna try that :p

[vivid lotus]

Mostly I'm trying to make sure that I'm reading data sequentially with lots of locality, so I don't get hardware cache misses.

[karmic basin]

I don't see if there is a better way than spatial partitioning at the moment

[ocean tundra]

Someone is/has concerted most of dots to use doubles instead of floats

Specifically core transform systems and physics

I think rendering was a issue but if your using hdrp there's some sort of camera relative rendering you can use

[vivid lotus]

I'm not too worried about that part.

[hollow sorrel]

@vivid lotus i'd copy the data you need to the render entity (position etc), which is cache miss city but you'd only have to do it once per frame and can then reuse that for whatever other render systems you have
since then you can do whatever rest you need to do on the render entity with proper cache linearity

[vivid lotus]

It's the transfer of gameplay entity position to render entity position every frame that I want to optimize most.

[hollow sorrel]

i think cache misses when dealing with paired data is inevitable, but you can def minimize doing it

[vivid lotus]

Hm, yeah, you're probably right. Even if I did have an intermediate data structure I'd have to contend with concurrency write issues.

[ocean tundra]

I've missed part of the conversation but has a custom renderer been suggested? Using the graphics.draw .. apis?

[vivid lotus]

I was thinking a two phase pass where I have a big list/vector of "mailboxes" basically for this data, and both the gameplay entity and the render entity agree on a slot. Then I pass over all the gameplay entities and have each write to its slot, and then have the render entities read their respective slots. Then the data would all be packed at least, but there's still cache misses to be had there, and it's a lot of extra (hard to parallelize) work.

I'm not too concerned about drawing. I have representation entities that work fine, I just need them to know where to be.

I could skip the representation entities entirely but then I run into issues of having to store reference data (meshes, materials) in the gameplay entities.

And I'm not sure how parallel-friendly the graphics APIs are from the request-making side.

[hollow sorrel]

not sure the mailbox thing is worth the effort, there's no guarantee your render entities in your list ends up the same as your archetype iterations, and if you end up trying to do that then that'd prob cost more perf overall

[vivid lotus]

Yeah.

[hollow sorrel]

What I could do is move gameplay entities that have a representation to a dedicated chunk, and make sure it has the same index in its chunk that the representation entity has in its chunk.
this might technically work but defeats the purpose of having separate render entities imo

[vivid lotus]

Also yeah.

Manually drawing could open some options. If I bypass this process entirely, what I could instead do is iterate over every gameplay entity, decide if it needs to be drawn, and if so, queue up all the draw information for it in a ConcurrentQueue (or some other parallel-producer structure) and then resolve that.

The complication again is that I need to convert the coordinates to be camera-relative, since I can't move the camera out to low-precision land.

It isn't so much about getting DOTS to use doubles, it's about the camera, which is still a Unity GameObject with a float-based transform.

So my plan is to just leave it at (0, 0, 0) and move everything else instead.

[hollow sorrel]

yea you could do that regardless tho, like a culling system that does the coordinate conversion and checks if it's in camera view, and if so marks it for rendering

and if not marked then don't even need to create a render mirror entity for it

[vivid lotus]

Yeah that was what I was originally thinking. Sweep all the gameplay entities, see if they're close enough to make render entities for.

Or if they have render entities and have since moved away, so they need to be destroyed.

[hollow sorrel]

yeah

[vivid lotus]

My issue with manually drawing is that I'd probably lose out on instancing and lots of optimizations there, unless I did that all myself.

[hollow sorrel]

i think the separate render entities thing is independent of however you choose to render
i'm doing the separate render entities + copying data, and then manually drawing, but could just as easily swap out the manual drawing for hybrid if wanted

[vivid lotus]

Interesting. Mind if I ask why that design choice? For me it seems like I'd want either/or. That is, either manually draw and don't bother with distinct render entities, or use render entities as a conceit to take advantage of the hybrid renderer.

[hollow sorrel]

trying to make a deterministic + rollback game, but even without that i'd prob still do it this way
there's a lot of benefits to having render entities, for example your archetype chunks are much cleaner (instead of having 10 AI with slightly different components spread across different chunks, they'd just be split on the gameplay side, they're all the same on the render side so fit in 1 chunk there and have good cache linearity)

and imo it's much easier to think about the game architecture being split this way, because your game still plays exactly the same even with a black screen and 0 rendering, they're basically separate worlds

actually in my case they're literally in separate ecs Worlds

for automated testing, you could add a bunch of prerecorded input data into your simulation and play it at 500x speed and query some test results at the end of it, and not have to worry about rendering since you don't need that during tests

[vivid lotus]

That makes sense. I have a ton of questions about that because that's super interesting but I don't want to get too sidetracked. Maybe another day. That said, for your manual drawing, is there a lot of work on that sort of thing out there for DOTS? I figure this is a common use case. Are you batching and instancing and all of that good stuff?

[hollow sorrel]

nah atm i haven't gone too deep into that side yet, current implementation is very naive and inefficient
one of big reasons i went this route in the beginning is because i didn't like how hybrid rendering has 1 chunk per mesh, i'd prob end up with a lot of mostly empty chunks

overall i'm sure hybrid rendering is much more efficient than mine with batching and all that good stuff, but now i don't have to deal with unity's shit (or less at least) so that's reason enough for me

[vivid lotus]

Hah, fair enough. I've never used Unity's explicit draw tools so I'd have some digging to do to get started, I suppose.

Which render pipeline are you using, or does it even matter?

[hollow sorrel]

atm just the old legacy built-in, which hybrid renderer doesn't support

and yea i don't know too much about the drawing stuff either tbh, i've just started using Graphics.DrawMesh for now and worry about the rest later when it becomes a problem

i know hybrid renderer internally uses BatchRendererGroups which is a relatively new thing but no idea how that works

actually not sure if that's what it was called, but they don't use Graphics.stuff at least

[vivid lotus]

There's CommandBuffer.DrawMesh, I believe.

[hollow sorrel]

yea that's a different thing too i think

not sure if built-in rendering supports those

but yea if you wanted to go the Graphics.DrawMesh route, that's trivial to set up
pass in your mesh, pass in your ltw, pass in your material

pewpew now there's stuff on screen

it's the fancy optimization stuff where it gets hard

[vivid lotus]

Yeah, that's what I'm worried about if I go the homegrown route.

Looking into BatchRendererGroup. This is definitely a rabbit hole.

[hollow sorrel]

yea that's fair, i think if hybrid renderer works for you it's good to go with that
i might switch to it some time too, just for now it seems like a lot of complexity that i don't wanna worry about yet

[vivid lotus]

I don't know, actually after this I'm leaning away from it.

This might be doable manually.

[hollow sorrel]

what kind of game are you making

[vivid lotus]

Think something along the lines of 2D RTS, but at scale.

[hollow sorrel]

oooo

is the actual rendering 2D or 3D

or both 👀

[vivid lotus]

2D, trying to keep it simple for now.

(He says, while diving into DOTS and manual rendering.)

[hollow sorrel]

nice, i'm doing 2D as well

kinda

2D in a 3D world

but similar stuff

[vivid lotus]

Perfect thing for ECS, but certainly comes with complications.

[hollow sorrel]

yea

[vivid lotus]

Since you're doing rollback (I assume for multiplayer), are you using fixed-point?

[hollow sorrel]

what's nice is unity's Sprite assets have vertices data already that you can use to make meshes out of at runtime
i would imagine that's how the default SpriteRenderer also works, not sure tho

[vivid lotus]

I actually don't do that. I just make mesh squares.

I have a thing that takes in a sprite sheet and spits out squares with the UVs set up.

[hollow sorrel]

nah not fixed-point, hoping by the time i start doing actual multiplayer burst will be crossplatform deterministic, no idea if it will but if not i'll prob just inflict pain on myself and convert to fixedpoint when it comes to it

[vivid lotus]

Granted, Unity's approach is probably better since it uses vertex as well as fragment on the GPU, whereas mine is mostly fragment.

[hollow sorrel]

yeah quads works too, not sure which way is better

[vivid lotus]

I want to be able to grab sprite sheets out of Resources.Load for modding, so I can't rely on Unity's sprite tools.

[hollow sorrel]

oo fancyy

[vivid lotus]

So, thinking about it, I've still got all the same problems actually.

I don't think there's a good way for a parallelized system to write out all the batch render information safely in parallel.

Unless there's like a NativeConcurrentQueue or something.

[ocean tundra]

Pretty sure there is a native queue to concurrent

Or it supports concurrent, can't remember which

[vivid lotus]

Oh, guess there is, huh.

[ocean tundra]

You could also look at native streams, they are built for the many producer to many/one consumer

But are kinda painful to use

[vivid lotus]

Well so the issue is that I would have to have each game entity spit out its rendering information, and I would then need to pack it into these arrays (being careful to stay under 1023 per).

[ocean tundra]

Also highly agree with @hollow sorrel saying to split up your data/worlds into simulation and rendering

[vivid lotus]

I agree with that, but I may not need any entities for rendering.

[ocean tundra]

But would focus on poc first then redo it later

[vivid lotus]

I'm curious to see how to make it work first and foremost.

[ocean tundra]

I'm also building a networking stack which is built on the same idea, spit server/clients, and using curves to playback everything. Basically making clients a "dumb" video player

Yup definitely start with getting your core gameplay loop working

[hollow sorrel]

forgot nativestreams exist, that could be a good idea as well (but agreed on the make it work first redo later)

[vivid lotus]

I'm actually less concerned with that. I'm interested in the tech right now. The game is mostly an excuse to learn it for the time being.

[ocean tundra]

Yea building all this "tech" kinda killed my poc. As I had to keep going back and changing things and eventually spent most of my time there instead of either getting the poc happy or finishing the tech stuff

[vivid lotus]

That doesn't bother me. I always come out the other end with more tricks up my sleeve.

But that's all beside the point.

[ocean tundra]

Exactly, I learnt tons and now have really good ideas on how to do rts networking and modding

[vivid lotus]

I guess I need to decide if all the random access of representation-entity reading gameplay-entity is worse, or if doing the batch rendering via a queue or stream or something (and sorting out which goes to which batch) is more expensive.

[ocean tundra]

I honestly found native streams more work then they were worth. But when I start stress testing they might be a better idea

[vivid lotus]

In reality the rendering entity is pretty superfluous, especially if all of its relevant state is changing every frame.

[ocean tundra]

I found the break between ecs data, into native container, processing on those containers and back again to other entities frustrating

[vivid lotus]

In my case I have a finite (but indeterminate) set of possible sprites, each with one unique mesh and one (potentially reused) material, plus some overridden material properties for things like team colors and such.

So each gameplay entity maps to exactly one of those.

The gameplay entity doesn't need to provide, or know, more than a couple of index values.

The rest would just be lookup tables into the assets.

So iterate over all the gameplay entities in parallel, determine if they're worth even trying to draw, if so, spit out into a stream their lookup table data, and then a separate pass reads that stream and packs everything into batches, essentially.

The issue is that that latter part would be very difficult to parallelize.

[hollow sorrel]

ohyea you'd have to map it back so would be a buncha random lookups anyway right

[vivid lotus]

Sorta.

I could prepare one batch group for every sprite, since I know them all ahead of time (they're loaded at startup). And then if a gameplay entity wants to draw, it knows its sprite ID is 342, so that goes into the concurrent stream along with its position (and any additional) information during the preparation pass. Then a process reads that stream and puts that info into the prepared 342 batch.

There's two complications. One is that that stream reading process would be difficult to make parallel since the batch stuff doesn't look like it's set up for that. The second is those batches have a limit of 1023, so I might need to add additional same-batches for a given sprite.

[hollow sorrel]

that sounds like an absolute pain to work with

[vivid lotus]

Could be. I don't know offhand.

It's really just two systems. The ECS system that reads the drawable gameplay entities and ushers their information into the buffer, and then the other system (I guess just a MonoBehaviour?) that reads that buffer and gets everything into batches.

Ah, wait, okay.

This is probably why the HybridRenderer uses a SharedComponent for the MeshRenderer.

To pre-group batches at the chunk level.

[lusty otter]

Burst gives warning about exceptions while building:

An exception was thrown from a function without the correct [Conditional("ENABLE_UNITY_COLLECTIONS_CHECKS")] guard. Exceptions only work in the editor and so should be only thrown in a function protected by this guard
and they originate from come.unity.collections.
Do I need to worry about them?

[vivid lotus]

So, I have it... kinda? working: https://gist.github.com/recatek/fdaa7cecbe6a8c453de90217a56ed52e, if you're interested @hollow sorrel @ocean tundra

On the collection task I can't ScheduleParallel unfortunately, since NativeList doesn't let you Add() in a parallel write context. Makes it sort of a waste to make it a job, but I'm gonna keep playing with that aspect of it.

[vivid lotus]

Uh, does NativeMultiHashMap<TKey, TValue>.Concurrent not exist anymore?

[ocean tundra]

Ok so many comments but base looks good

Native collections, are better to create in on create and keep persistent

With burst is default, don't need it

Never complete a job handle if you can advoid it

[vivid lotus]

Yeah, that's placeholder for now until I can chain a followup job.

Still trying to figure out how to organize it all.

[ocean tundra]

With schedule you don't
Need to pass in handles and store them

By default Dependancy will be tracked for you

[zenith wyvern]

It's called AsParallelWriter now

[ocean tundra]

You can just use World when inside a system

Don't need static access to world.def....

Same with entity manager

It exists as a base property of systembase

[vivid lotus]

Ah, that was code I just ported from a MonoBehaviour, but good to know.

What if you have multiple worlds?

[ocean tundra]

Is Matrix4x4.Translate the new math stuff or old?

Your systems will exist in every world

By default, but you can change that with ICustomBootstrap

[vivid lotus]

It's old, but it needs to be old because that's what BatchRendererGroup wants.

[ocean tundra]

Hmm that sucks, means you'll miss out on some burst magic

[vivid lotus]

With respect to pre-allocating native containers, when would you clear them before use? Couldn't a job still be running during the next frame's OnUpdate()?

Every OnUpdate(), I need to clear an array (or now, a MultiHashMap), and refill and use it.

That's why I was allocating with TempJob.

Though I'm not sure when you dispose when you do that.

[zenith wyvern]

It won't let you clear a container if it's being used in a job before the job gets Complete()ed

Clearing a NativeMultiHashMap is extremely expensive by the way

If it's large

[vivid lotus]

So it's better to Allocator.TempJob?

[ocean tundra]

Oh I head clearing native collections was quick

Probably depends on the collection

[vivid lotus]

NativeMultiHashMap is its own thing, AFAICT.

How do you get a native container to dispose once you're done when you're using Entities.ForEach.ScheduleParallel?

There isn't that thing like in regular jobs where you can mark a field to dispose.

[zenith wyvern]

NativeStream may be better depending on what you're trying to do

[vivid lotus]

I'm trying to organize and bin.

So I need to separate by index.

Essentially I have a whole bunch of entities with a label from 0..n and I want to get them each into batches for rendering.

[zenith wyvern]

Pass in Dependency to Dispose

[vivid lotus]

Like... .ScheduleParallel(myArray.Dispose)?

Or a new job?

Oh wait.

Gotcha, I see now.

Hm, doesn't like this either

            NativeMultiHashMap<int, Matrix4x4> binnedMatrices = new NativeMultiHashMap<int, Matrix4x4>(100, Allocator.TempJob);
            var pleaseJustLetMeWrite = binnedMatrices.AsParallelWriter();

            JobHandle gatherAndBin = 
                Entities
                    .WithName("GatherAndBin")
                    .ForEach((int entityInQueryIndex, in RenderPositionComponent renderPositionComponent) =>
                {
                    Matrix4x4 matrix = Matrix4x4.Translate((Vector2)renderPositionComponent.RenderPosition);
                    pleaseJustLetMeWrite.Add(renderPositionComponent.SpriteIndex, matrix);
                }).ScheduleParallel(default);

            binnedMatrices.Dispose(gatherAndBin);

So the only way to get a handle out of ScheduleParallel is to feed it default for dependencies, but that strikes me as wrong. Is there something else I should be feeding in there?

[zenith wyvern]

You should be passing in Dependency to your job

[ocean tundra]

Try this. Dependancy

[vivid lotus]

Oh. That makes sense.

Perfect, alright, that all works.

I would kill for nested native containers right now.

[ocean tundra]

Haha

Heard that alot

[zenith wyvern]

You can do it with unsafe containers

Apparently it's pretty bad for performance though

[vivid lotus]

Yeah I've been trying to avoid that due to risk as well.

[ocean tundra]

Think I've thought of a faster way to gather the data for your batchs

Use a entity query to component data array

And renturprate (dam phone spelling) into your vector 2

Will still need to matrix it

[vivid lotus]

I'm not familiar with this. Does it just spit out all the components as a big array?

[ocean tundra]

Tho you could change the vector 2 into matrix type

Yup

[vivid lotus]

So my concern with that is most entities will be offscreen, so I'd like to filter them in-place before allocating space for them.

[ocean tundra]

And renturprate (sp??) Allows you to turn one struct to another if they have the same data layout (eg component has a single sub struct)

[vivid lotus]

I did consider putting them all just in a list, and then sorting by type, and then slicing.

What I would need, ideally, is the ability to sort several arrays in parallel treating one as the key and the rest just as data.

[ocean tundra]

The culling should probably be a separate job

[vivid lotus]

In my case the culling is really, really easy.

[ocean tundra]

Where you add/remove a visable tag

[vivid lotus]

Hm, I could do it in three passes maybe.

Gather the components (parallel), sort (not parallel), and then convert (parallel).

There's IJobNativeMultiHashMapMergedSharedKeyIndices maybe?

Or no, that's gone now apparently.

It is really hard to bin heterogeneous types efficiently in Unity jobs, huh.

[zenith wyvern]

NativeStream lets you write arbitrary data/types to any index

[vivid lotus]

Just as bytes, right?

Essentially my issue is, I have 1..n batches, each with a NativeArray<Matrix4x4> and Count that I need to populate. I've created one batch per type of sprite.

And then in the ECS I have just a bag of unsorted, undivided entities, each with an index value of which sprite (and thus, which batch) it should go into for rendering, as well as its position.

And I'm trying to find the most parallel and efficient way to get them to the right batches.

The fact that I can't nest native containers means I can't just do something like NativeArray<NativeQueue<Matrix4x4>>.

I could do a queue, or a stream, or a query, but they would come out in an arbitrary order because I want to do this as ScheduleParallel.

[ocean tundra]

Could using a shared component help, as that would sort your entities into chucks with matching values

So you could use that for your batch id

[vivid lotus]

I'm trying to avoid chunk coercion because I want to use that for spatial decomposition for queries.

[ocean tundra]

Hmmm no reason you can't use both

But also will become less efficient

[vivid lotus]

Can't an entity only be in one chunk at a time?

[ocean tundra]

Yea but your chunks would get smaller

Your chunk "key" would be something like batch id, int 2 positive

[vivid lotus]

Yeah, in that case I'd like to reserve chunk logic for distance queries and the like.

[ocean tundra]

Hmmm my gut feeling is using chunks for spacial stuff might be a mistake

But no real evedince for it

Maybe checkout what physics does

I believe internally it keeps some sort of spatial tree

[vivid lotus]

So far I'm thinking

1. Filter entities and populate an unordered queue from a ScheduleParallel
2. Convert the queue to an array
3. Sort the array
4. Figure out the slices and send them off to batches

[ocean tundra]

It's simple enough to turn off the actual physics work and use it as a pure spatial query system

[vivid lotus]

But 3 is tricky.

[ocean tundra]

Well native arrays have custom sort methods

Think you could just call that from a job

[vivid lotus]

They do, but they don't have multi-collection sorting.

See if I sort, I need a key. That means the key has to be in the struct.

So then I have an array of SomeStruct, not Matrix4x4, which means after they're sorted I can't just direct slice into the batches.

(The batches take a NativeArray<Matrix4x4>)

Ideally I would have two arrays, NativeArray<Matrix4x4> and NativeArray<int> and I would sort them both in lockstep using the values of the latter as a guide.

Then ditch the int array and just slice up the Matrix array so I can very efficiently copy it into the batches.

[ocean tundra]

Have you considered entity slicing?

[vivid lotus]

In what sense?

[ocean tundra]

Using a master entity with most gamepad stuff on it, child with your rendering stuff, your child could use chucks to sort based on batch id. Parent is your spatial stuff

[vivid lotus]

That would just bring it back full circle. I'm trying to avoid having to have a separate child render entity.

That's how this whole thing started in the first place.

[ocean tundra]

Kinda sounds like premature optimization :p without testing it it's hard to say if the random read to copy data will be slow or not

My curve based network relise on the same concept, each frame I check for changes, and if there is a change I copy it

[vivid lotus]

Eh, I'm not afraid of premature optimization. As I said, I'm doing this to explore the tech.

If I was shipping on a deadline it would be one thing, but this isn't paying my bills, it's just to learn.

[ocean tundra]

There's a ton of ways to optimize too

[vivid lotus]

There are, but in this case I want to take advantage of parallelism and locality of data.

[ocean tundra]

Yea I get that

[vivid lotus]

Which are two very reliable ways to optimize.

The issue is mostly in the limitations of the native containers in this case.

[ocean tundra]

But would this complex sorting bit be slower or faster then just copying 1 float2?:p

[vivid lotus]

Depends on when the copy occurs.

The issue is that I need to know the count of instances in the batch ahead of time.

Before I copy the matrices into the array.

Based on how the batch scheduler works.

You need to declare the count and then populate.

[ocean tundra]

If your using chucks and entity querys you could call get entity count

[vivid lotus]

I'm trying to avoid using chunks for this.

There will be other, expensive things I'll need to rely on chunks for. I don't think this is it.

[ocean tundra]

Also another trick I've learnt, is not everything needs to happen instantly on the same frame, if theres a single frame delay from a new unit spawning/deleting is that a issue?

[vivid lotus]

No but I don't see how that applies in this case?

And I absolutely do want each frame to draw the entity in the correct spot without delay.

[ocean tundra]

You could have a 2nd job that updates your counts, it's output data is used in the next frame

[vivid lotus]

I'll likely have several dependent jobs.

[ocean tundra]

Yup

[vivid lotus]

It's just a matter of how I can make the most out of parallelizing those jobs.

[ocean tundra]

Just throwing ideas out there :p

Don't really have answers, just hoping to help spark some good ones

[vivid lotus]

It's tricky. I don't think any of the right ways are possible with how the containers work, without going to unsafe and untyped containers.

Oh yeah, I appreciate that for sure.

Also they deprecated/removed two of the things that could have helped me here, NativeArrayFullSOA, and IJobNativeMultiHashMapMergedSharedKeyIndices

[ocean tundra]

Usually when they remove something in dots there's a better way of doing it

[vivid lotus]

These both just seem to have been very obscure and underused.

[ocean tundra]

So to rephrase the issue
You have entitys with a batch id(sprite Id) and position (a matrix)
You need to break that out into a list of the matrix's per each batch id

?

[vivid lotus]

Pretty much, yeah.

[ocean tundra]

Awesome

[vivid lotus]

The position as stored on the entity isn't actually a matrix, but it can easily be converted to one.

And the challenge is to do this without chunk manipulation, and without additional entities.

[ocean tundra]

Does it have to be a native array?

For the batch thing

[vivid lotus]

The result has to be one native array per sprite ID, and a count.

[ocean tundra]

Yup

And we can create native arrays within a job?

(never actually tried)

[vivid lotus]

Uh, can we? I don't know.

[ocean tundra]

If the answer is yes, then I'm thinking native streams is the best option

Wait no

:(

Native streams would need the data sorted by batch id first

[vivid lotus]

I thought about having a stream where you have a consumer job for each sprite ID that packs its own array, but as the number of sprite IDs grows you're going to have a ton of thread churn.

[ocean tundra]

The consumer doesn't have to be many threads/jobs

Could be a single

But it won't work

You can only begin/end for each index (batch id) 1ce per frame

[vivid lotus]

Well the core problem is you can't have second-order addressing.

Because you can't have nested containers.

[ocean tundra]

2nd order?

[vivid lotus]

NativeArray<int> - first order[]
NativeArray<NativeArray<int>> - second order[][]

If I could do second order indexing in jobs this would be easy.

Just have an array of queues and feed each queue then convert to arrays and load into the batch..

[ocean tundra]

Yea my tricky for that was code gen

My multiplayer kit has a "queue" per player, and the game dev sets max number of players

So I gen out a container with 100s of native queues and things

But your batchs (sprites) are dynamic

Hmm ok what about dynamic buffers?

Attached to a entity

So 1 entity per sprite, with a dynamic buffers of positions

Your entities "add" them selfs into the buffet

And tracks their index within

So they update the same index next frame somehow

[vivid lotus]

So the issue with entities referencing other entities is that it's all random access.

That's tons of cache misses, which is part of what I'm trying to avoid.

[ocean tundra]

I truly believe it's impossible to avoid all cache misses in a game

:p

[vivid lotus]

Well, a big part of ECS is data locality in memory, so I'm trying to play to that strength as best I can.

[ocean tundra]

Your output would be in your required formats (buffers have a as array method)

It's simple to debug/test/understand

[vivid lotus]

I think it would have similar parallelism issues.

[ocean tundra]

Also you only have to update when data changes

Depends

I think cache misses are ok if you focus on keeping them low

Like watching for data changes

And doing things per chunk instead of per entity

[vivid lotus]

I'm assuming that most entities will move most of the time in the sim side.

I'm pretty committed to not relying on chunks for this.

[ocean tundra]

But there's huge amounts of static things in games

No I meant chunk based jobs

Instead of entities.foreach

For each is great to use

But slower then down chunk based jobs

[vivid lotus]

The best is to use ScheduleParallel if I can.

The more I can do in that context, or equivalent job contexts, the better.

[ocean tundra]

Yea you schedule parallel on chunk jobs

[dense crypt]

How can I use a SetChangedVersionFilter in a MonoBehaviour? I have a method I want to run only if a component changes. Creating a query with CreateEntityQuery and applying the filter doesn't do anything. (CalculateChunkCount is never 0) But if I do the same thing in a SystemBase it works as expected.

[stone osprey]

How would you implement server/client communication ? My game server is java based and runs on an ecs and my client is unity based with dots/ecs. Not all server entities are represented in the client. For example only trees, mobs, items on the ground are represented on the client.

Should i serialize those entities on tje server and deserialize them on the client to construct unity ecs-entities ? Should i fire single events like : position_updated from the client to the server ? Whats the common workflow for this ?

So either sending the whole entity from the server to the client to construct the "same" entity... Then send component updates when something changes...

Or instead only send events like, oh a entity of type xy spanwned. Oh the entity moved to that position.

Both waya are actually quite interessting.

[zenith wyvern]

That doesn't sound right, it sounds like you're accessing the component as read/write in the monobehavior

[dense crypt]

That was my first thought as well, but I'm doing nothing with it only CalculateChunkCount, it's also a brand new comp on a brand new entity, so it's completely isolated from other systems. I think it's something related to CreateEntityQuery (which I cache) but I can't wrap my head around it.

[zenith wyvern]

Can you show the monobehavior? Post it to hatebin if it's too large

[dense crypt]

Yeah, here's all there is https://hatebin.com/brqlkopbbp

This logs 1 all the time, but doing the same in SystemBase (Using GetEntityQuery) will return 1 for 1 frame and then always 0, as expected.

[deft stump]

question, not a DOTS question but:
is the Default Unity Rigidbody physics multithreaded?

[zenith wyvern]

Just tested it and you're right. Seems like a bug to me

But maybe I'm missing something

[dense crypt]

I worked around it by having my SystemBase invoke a delegate in a Run job that I can listen to from my MonoBehaviour.

[vivid lotus]

Is there a chart somewhere or some documentation about when systems update relative to MonoBehaviours? Or a way to control the ordering?

Haven’t been able to find anything about it.

Trying to delay a system’s OnCreate until some MonoBehaviours have done Awake and Start.

[shut hare]

What is the difference between ComponentSystem and SystemBase ?

[safe lintel]

ComponentSystem is what came before SystemBase, is mainthread only and shouldnt be used over SystemBase

[shut hare]

it's in its way to deprecation ?

[safe lintel]

yeah

[shut hare]

oh ok

its too bad i liked the PostUpdateCommands field over the SystemBase way

[safe lintel]

JobComponentSystem was ComponentSystem but for multithreading, it too is deprecated in favour of SystemBase

[shut hare]

alright then thank you

[safe lintel]

postupdatecommands is just an entity command buffer, you can do it yourself like

var postUpdateCommands = new EntityCommandBuffer(Allocator.Temp);|
//foreach yadayada
postUpdateCommands.Playback(EntityManager);
postUpdateCommands.Dispose();

[shut hare]

is it better than using existing ones like EndSimulationEntityCommandBufferSystem ?

[safe lintel]

its just different point where you playback an ecb. it is preferrable to try to limit where and when it happens so if you can use EndSimulationEntityCommandBufferSystem then use it, but in some instances you will want results sooner so choose whichever suits the situation.

[shut hare]

i see does creating one causes "sync points" ?

[safe lintel]

https://forum.unity.com/threads/introducing-the-dots-best-practices-guide.1054991/

this can explain it better than i can 🙂

[shut hare]

i'll have look thx @safe lintel !

[stone osprey]

Why exactly does a command buffer improve the performance ? Does it really matter if we create an entity now or the next frame ? ^^

[zenith wyvern]

It groups all your structural changes in once place instead of potentially having them staggered throughout an update cycle which could cause disparate jobs to be completed when they may not have to be

[stone osprey]

@zenith wyvern Ah, so its main advantage is basically that it does not wait for the jobs to finish... and at the start of the next frame, the jobs are finished and we can easily create new entities and components. Lets say we wouldnt use any jobs at all... then it shouldnt have such a big impact on the performance to not use the command buffer, right ?

[zenith wyvern]

I'm not sure what you're getting at there, why would you need a command buffer in the first place if you're not using jobs?

[stone osprey]

Yeah... thats what my question is 😄 If it does make any sense to use it when we arent working with jobs

[zenith wyvern]

AFAIK it's purpose is to queue entity related commands inside a job or even on the main thread to group structural changes in once place. Not sure what you would want to use it for apart from that

By "no jobs" do you mean you're only using main thread jobs?

Or literally no jobs?

[stone osprey]

Alright, thanks. I just heard that it would make things faster. So i thought about using it. But if its only good for multithreading, then im gonna pass here. Only mainthreaded jobs... Just Entities.ForEach(...).withoutBurst(); where ever that runs 😄

[zenith wyvern]

Yeah I'm not sure there any benefit to using command buffers if you're doing everything on the main thread anyways. I can't think of one anyways

[vivid lotus]

Revisiting my earlier question: If I need a system's OnCreate to depend on stuff from a MonoBehavior (like loading assets and making them available via a singleton) is there a way to do that? Or is there a better approach? It's hard to get serialized data into a system, it seems. Or to pass data between systems.

[zenith wyvern]

Pretty sure OnCreate gets called before monobehaviors even exist. I would suggest making your monobehavior into a hybrid component then you can get it in OnUpdate with GetSingletonEntity<MonoBehaviorType> and GetComponentObject

Otherwise you'd need to hook into scene loading in some other way, using the scene manager framework or make a custom static event on your gameobject

Or just access it inside OnStartRunning, I think Monobehaviors exist at that point

[vivid lotus]

Hm, OnStartRunning would work. I couldn't find a lot about how to enable/disable systems. Like if I don't need all my systems running during my main menu scene for example.

Like ideally I'd like to do all of my runtime loading and precomputation during a staging scene before the game scene opens up, and only have systems started during the game scene.

Then they could fetch the MonoBehavior-made lookup tables and such.

[zenith wyvern]

I personally haven't found a reason to manually disable systems outside of testing stuff. I just rely on query matching which will prevent systems from doing work unless they're matching against something

[vivid lotus]

It seems a little extraneous to have to make entities just for things like lookup tables. Surely there's a better way.

[zenith wyvern]

I don't know, GetSingleton/GetSingletonEntity provides a simple built in global point of access to some piece of data

Seems like a good fit for something like that

[vivid lotus]

I think that just moves the problem of when that singleton is created.

And there's a limit to what kinds of data can be stored in a component like that, right? Since it still has to be blitted?

[zenith wyvern]

Not if you make it a class https://docs.unity3d.com/Packages/com.unity.entities@0.17/manual/component_data.html#managed-icomponentdata

[vivid lotus]

So I'm seeing something like this

    public class SingletonTest : SystemBase
    {
        protected override void OnCreate()
        {
            World.EntityManager.CreateEntity(ComponentType.ReadOnly<SomeComponent1>());
            SetSingleton(new SomeComponent1 { value = 1 });
        }
 
        protected override void OnUpdate()
        {
            SomeComponent1 test = GetSingleton<SomeComponent1>();
            Debug.Log(test.value);
        }
    }

That doesn't seem any different from just having data in the system itself.

So the game starts up. As part of the startup process, it reads some config/ini files and such for settings. Ideally the game systems would create after that data is read so they could bind to those settings, but it doesn't seem like there's any way to control that.

[zenith wyvern]

Sounds like you should be setting up the data during conversion

[vivid lotus]

I'm not sure what I'm converting in this case.

This data doesn't live in the Unity editor. It lives in a text file that the standalone game reads at startup.

What I'm trying to do is convert it to shared lookup tables that various systems can easily access.

Oh, also apparently singletons can only be structs, they can't be managed.

public void SetSingleton<T>(T value) where T : struct, IComponentData;

[zenith wyvern]

Use GetSingletonEntity instead

[vivid lotus]

There's gotta be a better way than that. You're paying multiple indirections every update pass (get the entity, get the component from the entity, get the table from the component) then just to get one lookup table.

I'll keep digging for something I can do.

Hooking into the scene manager might be the way to go if it's an option.

You could use a singleton just as a tag I guess, to let a system know that the table is available in static data somewhere.

Have a MonoBehaviour singleton that does the work in its Awake in the scene, and then creates some empty MyDataIsReady singleton, just so the system that RequiresSingletonForUpdate<MyDataIsReady> can start updating and fetch it statically and cache a reference to it, not from the singleton component directly.

Hm, looking at https://docs.unity3d.com/Packages/com.unity.entities@0.1/manual/system_update_order.html

[zenith wyvern]

You can cache it on the first update. I have a hard time believing that could ever become a bottleneck anyways

[vivid lotus]

I could yeah. I'm thinking more that I could just manually create the system after loading the data and giving it the lookup table in the creation process, then adding it to the world myself.

[worthy rampart]

NativeArrays are just pointers right

storing a reference to the same array a bunch of times is relatively cheap

[vivid lotus]

Yep.

Gets more complicated when having to go through managed data though.

All of this also ties in to weird things like what you do if the player wants to go back to the main menu and start another game, etc.

Having systems always exist seems counter to that kind of flow.

Unless you truly only have "pure" systems with no global state, but I think that's an unattainable goal in most ECS approaches.

[bright sentinel]

Also @zenith wyvern It's actually more efficient to create a new ECB and playback it right after an Entities.ForEach than using WithStructuralChanges

[bright sentinel]

What sort of data are you looking for? Also, does it have to be in OnCreate?

[vivid lotus]

Arbitrary game asset data loaded at runtime. Text files containing weapon stats for example, or scripted events, or textures loaded via mods. OnCreate would be nice since that's generally where systems do their setup. I'd like to avoid polling for the existence of the data every OnUpdate.

In MonoBehaviour world it would be pretty easy. You'd do the loading part in a staging scene, and then when that's done you transition to the main menu or gameplay scene.

But I'm scratching my head at how to do this in DOTS world. Or at least, in an elegant way.

[bright sentinel]

Well, OnCreate is called very early in the pipeline - I don't think this would be a good place to depend on any arbitrary data that could come into the game. Unless of course you go with some custom bootstrap, but that will probably be even worse.

You also don't have to poll this every frame - you can always just have a one-off OnUpdate for a system with a RequireForSingleton.

Another thing that might also interest you is just streaming in subscenes, but that's already a specific type of asset, so not sure if it's relevant. But Entities was basically built to have scene streaming built-in.

You might also want to think whether you only want users to be able to load assets when the game is launched. It's a very nice QOL feature to be able to change and load mods without having to close down the game, if possible. This of course wouldn't work with scripts, but definitely with assets.

[bright sentinel]

Is there a reason you can't do the same with ECS? Scenes are still a thing - and even SubScenes might be even better.

[vivid lotus]

Subscenes are a Unity Editor concept, no? For mods, you don't really have that luxury.

Unless I design some way to make subscenes with some sort of mod metadata.

Most of my game assets and configuration numbers are not going to be in the Unity Editor. That's why I need to allocate time at launch or, as you say, in-game during a mod selection stage, to load and preprocess assets.

I think at launch is fine since most games require a reload to change mods anyway.

You also don't have to poll this every frame - you can always just have a one-off OnUpdate for a system with a RequireForSingleton.
The question is when you inject the singleton. Especially if your game has a main menu and such, when do you create the singleton for the system in question, especially if it's shared across multiple systems.

[bright sentinel]

When? You can do it whenever you want

I'm not sure why it matters

[vivid lotus]

Well my options are OnCreate and... that's about it.

As far as I can tell.

[bright sentinel]

Why?

[vivid lotus]

Well, what are the alternatives?

[bright sentinel]

OnUpdate? 🤔

[vivid lotus]

That's what I'm trying to avoid, having to poll in every OnUpdate as per whether the data exists.

[bright sentinel]

That's why you have RequireSingletonForUpdate

[vivid lotus]

But then OnUpdate isn't called.

So how could I set the singleton then?

[bright sentinel]

Well then you create the singleton whenever you want to load the assets

[vivid lotus]

But again, where specifically? In what function?

[bright sentinel]

Well that depends on when you want them to load

[vivid lotus]

At startup, let's say.

Just when the game launches for the first time.

[bright sentinel]

Okay, then you just create the singleton in OnCreate

[vivid lotus]

Can that then be set in other systems that also want that same (managed) singleton?

As in, can the system responsible for loading and processing that data then reach out and provide it to other systems that want access to it?

[bright sentinel]

Well, that system doesn't need to reach out. The other system should also just have the RequireSingletonForUpdate

[vivid lotus]

Are singletons global, per-world, or per-system? The docs made them seem like they were per-system.

[bright sentinel]

Per world - a singleton is actually no different than a normal component on a normal entity. It really only exists because the GetSingleton function is there - you can still create multiple instances of the "singleton" component, in which case GetSingleton would throw an error

There really isn't a concept of singleton in terms of how they're stored

Only in how systems interact with them

So any system can listen to the same singleton

For example, if you just start any entities game right now and look in the entity window, you will see the WorldTime entity with a WorldTime component. That's a singleton, but only because there's a single one of them.

A singleton is only a singleton because we want it to be a singleton - otherwise it's just any other normal component.

If you had a single enemy in your game which was the only one that had some of the components - those components would technically be considered singletons in (Unity) ECS terms

So any system can in fact listen to that singleton

Does that make sense?

[vivid lotus]

It does, but that still seems like a fancy wrapper just for polling.

[bright sentinel]

How would you avoid polling with monobehaviours?

[vivid lotus]

By directly copying the table I want by reference as a field of the MonoBehaviour.

In its Start()

Which I would know is safe to do so, because the source of that data was staged in a prior scene in a DontDestroyOnLoad singleton.

Rather than here where I'm forced to check for the existence of this special entity, retrieve it, and then retrieve its component, every OnUpdate.

Which is minor, sure, in the greater scheme of things, but ugly nonetheless.

[bright sentinel]

If you think that's ugly, then maybe ECS isn't for you, because that's literally what ECS is.

Your systems take all the entities with the specified component and then does some logic on it

The whole idea is to use components to specify what logic should run

[vivid lotus]

Not necessarily. This handling of shared resources is unique to Unity's flavor of ECS, it isn't a fact of life for all ECS systems.

And "pure" systems are almost always a pipe dream. I haven't seen many substantial ECS systems that don't store data.

I think the answer, as I'm digging through docs here, is to just take better control over when and how systems are created.

[bright sentinel]

Well, I just don't see how it's different than looping over any archetype of entities.
That's also literally just polling all the components. The difference is just that you know they're there

You're free to do your own bootstrap, but I don't think you're thinking in (Unity) ECS terms really here.

[vivid lotus]

Right, but many is different from one. This is one shared, read-only resource. There's no need to go through a complicated lookup process every update pass for it -- it isn't going anywhere.

[bright sentinel]

I don't think I've ever said to make a complicated lookup process?

And do it every frame?

[vivid lotus]

That's what GetSingleton does, in my understanding.

It looks up the chunk, finds the component in it, and then you need to follow that through a third indirection to get your actual table.

Whereas, instead, you could defer the creation of the system, construct the system and pass the lookup table as you do so, and never need to do any entity or chunk lookups to get that data, you inherently have a bookmark on it.

[bright sentinel]

Okay, let me ask this: How do you want to construct the lookup table?

Would it be something like looping through a folder and see what files are there?

[vivid lotus]

Within either a system or a MonoBehaviour, in its OnCreate() or Awake() respectively.

[bright sentinel]

I mean, how would you get the data for the table?

[vivid lotus]

Resources.Load

And then parsing it, or if it's a texture or something, processing it into something usable.

And assembling it into an asset and data library for systems and gameplay entities to then use.

[bright sentinel]

Hm okay, well that's definitely going a bit out of Entities land, since it returns any object.
But then you're probably right, OnCreate should be fine. Is your issue then how you would save and use that data?

I think what I would do is load the data in OnCreate, and save it to a managed component.
Any other system can then just have that manged component as part of their query, and you can do whatever with that data as you want

[vivid lotus]

I think I would probably handle the world and manual system creation in a MonoBehaviour, most likely. Then I can easily feed in processed data to the systems as I make them, and exactly when I intend to during the startup process.

That also lets me destroy systems and worlds and recreate them if, say, the player goes back to the main menu and starts a new game.

If I feed in data during system creation I don't need to bother storing it on an entity. I never need the entity part of it to begin with -- I just want it to live somewhere easily accessible in memory as a lookup table.

[bright sentinel]

public class SavedData : IComponentData
{
  Texture2D texture;
}

public class LoadSystem : SystemBase
{


  void OnCreate()
  {
    Texture2D tex = Resources.Load("SomePath");
    var entity = EntityManager.CreateEntity();
    EntityManager.AddComponent(entity, new SavedData{texture = tex});
  }

  // Empty update because it's required
  void OnUpdate() {}
}

// Any system can then just listen for that data
public class ProcessTextureSystem : SystemBase
{
  void OnUpdate()
  {
    Entities.WithoutBurst().ForEach((SavedData savedData) => 
    {
      if (savedData.texture != null)
        // Do whatever processing you want. You can even save it to a different component if you don't want all the data stored in one component.
    }).Run();
  }
}

Just a quick example, the possibilities are endless.

[bright sentinel]

Okay, how would you store the data in classic Unity?

[vivid lotus]

Probably just in a static container.

Or on a singleton MonoBehaviour.

And then after it's created, subsequent MonoBehaviours could cache the pointer to the table itself and avoid later indirection.

[bright sentinel]

Static container isn't really that ECS focused.
And singleton MonoBehaviour is basically the same as a singleton entity no?

[vivid lotus]

It's missing a key step, the Start() pass, which lets me do the indirection once ahead of time and then every subsequent time I can avoid all indirection.

[bright sentinel]

Sorry, what do you mean by indirection?

From quick googling it seems to be the data loading based on the path. But I'm not sure how the above is doing that?

[vivid lotus]

public class DataLibrary : MonoBehaviour
{
    public static Instance { get; private set; }
    
    public Dictionary<int, Mesh> MeshKeys { get; private set; }
    
    protected void Awake()
    {
        Instance = this;
        
        SetupMeshKeys("ResourcePath/LotsOfDataFiles");
    }
}

public class SomeUserOfData : MonoBehaviour
{
    private Dictionary<int, Mesh> keysLookup;
    
    protected void Start()
    {
        keysLookup = DataLibrary.MeshKeys;
    }
    
    protected void Update()
    {
        UseMeshKeysWithoutEverCaringAboutSingletonsEverAgain();
    }
}

[bright sentinel]

Okay, let me ask you now: Why are you even using ECS? It doesn't sound like you want to code ECS here

[vivid lotus]

Well, I do, for everything else.

I just don't want to have my lookup tables be entities.

[bright sentinel]

Why not?

[vivid lotus]

Well, I see no reason why they need to be entities -- they don't do any entity things, they just exist as resource blobs. And looking up entities is more complicated and causes more cache misses than following a single pointer to a data structure.

[bright sentinel]

Well entities are just resource blobs - they're just data

But

You could also create BlobAssets

But they also have to be stored in entities anyways

At least the reference

[vivid lotus]

Right, which is what I'm trying to avoid.

This is essentially just static data. It doesn't really need to live in chunks (let alone have an entire 16kb chunk allocated just for a single pointer to it), and it doesn't need to be accessed through entity indirection.

[bright sentinel]

Right, then I think BlobAssets are what you need

[vivid lotus]

Can they be free-floating?

[bright sentinel]

"A blob asset is an immutable data structure stored in unmanaged memory."

[vivid lotus]

It isn't so much the data structure. How to store the data isn't a problem, it's how to feed it into systems.

Or rather, how to make systems aware of it without having to ask for it.

As I said, I think the "ECS world in a jar" where everything is wrapped in a MonoBehaviour that controls world and system creation/destruction is probably the way to go.

That gives the strongest tools for feeding in data to the right things at the right time during initialization.

[bright sentinel]

I'm still not sure why you want to avoid the system to ask for the data

That's literally what all systems does

Unless you're writing some completely different ECS type than me

[vivid lotus]

There's no reason for it to ask. The data doesn't go anywhere. It's set once at startup and there's no reason not to just have a direct pointer to it as part of the system's creation.

[bright sentinel]

Okay, but then you can just as well store all data like that, right? I feel like you're actively working against ECS here

[vivid lotus]

It isn't working against ECS, it's just storing some data separately.

[bright sentinel]

Well, storing data separately from entities/components is working against it

[vivid lotus]

Only in a situation where you expect everything to be an entity. That's actually kinda unique to Unity ECS, and a little odd.

If you look at other systems they differentiate between entities and resources, and let you access resources without making them ride on entities to do so.

(Rust's bevy is one such example.)

So I'm trying to recreate that here, essentially.

Taking control over the world and system creation process is the likely avenue to do so.

[bright sentinel]

Well, if that's what you want, you're free to recreate it. I've given you a path that uses the tools available, and it seems your worries are only that you don't want to store it as a component on an entity or as a blobasset because you don't like it. If you really want to work against it, be my guest.

[vivid lotus]

No I appreciate you pointing me to the singleton stuff, I think it's more just a lack of ergonomics on Unity's end.

[bright sentinel]

Hm, ergonomics in what way?

Also sorry if that came out a bit harsh, it's been a long day 😅

[vivid lotus]

"Everything is an entity" and "systems are always pure and store no data" are both ECS cliches that don't really work out in practice. I think most more mature ECS systems have recognized that and added more ways to better reflect reality.

If you look at how the HybridRenderer is written, it breaks both of those rules, a lot.

But because of DOTS's expectations, it has to jump through a number of hoops to do so.

Which I think is an ergonomics issue.

If DOTS had a nice way to just float static data for systems to retrieve independently of entities, some pain would be avoided IMO.

[bright sentinel]

Have you looked at BlobAssets at all?

[vivid lotus]

I have, but don't they need to live on entities?

Or rather, in components, on entities.

[bright sentinel]

Hm, actually not 100% sure about that - but I think they do

I haven't worked with them in a while

[vivid lotus]

Hm, I'll take a look. I'm going to go down my ECS-in-a-jar route and see where that takes me for now.

There's secondary benefits there like ensuring easy cleanup if you end one game and start another.

[bright sentinel]

Yeah, that's the worlds in ECS are quite handy, that's true

But bootstrapping is not 😛

[vivid lotus]

I don't think I'd actually use the bootstrapping system. It looks like you can turn it off entirely?

Unless I'm mistaken and you can't just arbitrarily instantiate worlds in MonoBehaviours.

[bright sentinel]

I think you can? You can just create a new world anywhere

And then just add it to the update

[vivid lotus]

Oh, then perfect. Make a MonoBehaviour, have it make a world or two, make the systems for it, and you have total control.

Set all your systems to [DisableAutoCreation]

UNITY_DISABLE_AUTOMATIC_SYSTEM_BOOTSTRAP

Even better, if that still exists.

[bright sentinel]

¯_(ツ)_/¯

But yeah, just looked up and you would still need to store BlobAssetReference on components

But I still don't understand why you don't like using components/entities to store the data

[vivid lotus]

I just see it as unnecessary, I guess. It complicates the lookup process for something that traditionally has always been very simple.

Now, that's true of a lot of ECS things, but here, it's also less efficient.

So there's neither a code cleanliness payoff, nor a performance one.

Where normally I'd be willing to sacrifice the former for the latter.

[bright sentinel]

I mean it's still one line of code. But I do see your point that GetSingleton is a lot slower than storing the reference directly.
But at this point, you should also consider whether it's really worth it. How often would you in reality access that data? GetSingleton isn't too expensive.

[vivid lotus]

Oh it's totally fussing over little things for sure.

That said, I've worked on projects where we needed support from the engine provider because the profiling tools rounded milliseconds to the nearest tenth and we needed to see in hundredths or thousandths because our budget was that razor thin.

So, I'm a little less concerned about premature optimization than some.

Also, the biggest reason of all, it just irks me, and that simply will not do.

Though I appreciate you pointing me in a number of good directions. I'll check out the blob asset thing.

[bright sentinel]

I definitely think it's an interesting problem as well, but I can't imagine that if you're using ECS in the first place, then a single GetSingleton (or even 50 or 100) in a frame will break any budget.

[vivid lotus]

Yeah. It's the raycast thing. A few is fine, but they sneak up on you late in a project.

[bright sentinel]

Probably true. I've looked at the DOTS FPS sample, they seem to have ~500 systems. It's of course not a finished game, but it is a complete vertical at least. And if you have much more than 1000 systems that each need to read the immutable data, then you might have bigger problems 😄

I would really urge you to try and follow the DOTS way at first, and then you can start optimizing with statics for the data if you start running into performance issues.

Something something premature optimization

[north bay]

Is your data coupled to worlds or fully static, unmanaged and/or managed?

[bright sentinel]

Seemed like it's fully managed static

[vivid lotus]

Yeah, fully managed static.

And read-only after initial load (usually).

Or not necessarily managed. Some things are in native containers.

[zenith wyvern]

Good point, wasn't considering that you're losing out on burst if you aren't using an ecb to buffer the structural changes

[bright sentinel]

I think it's actually more to do with the fact that WithStructuralChanges in Entities.ForEach is doing a lot more behind the scenes than you'd think, rather than an ECB

[shut hare]

what replace world.CreateSystem<T>(params object[]) now days ? i've tried world.CreateSystem<T>() but you can't pass params are you just not allowed to do that anymore ?

[bright sentinel]

World.AddSystem(new MySystem(myParams));

[sharp flax]

I need help interpreting this. I'm doing a stress test for my A* pathfinding using jobs & burst (no ecs). This is an IJobParallelFor and innerloopbatchcount is 1. Paths are relatively simple since its just a complete open area on an empty 100x100 grid, so what I would expect is that every thread finished more or less within ~0.1ms from each other, but as you can see that is not happening

I'm using Temp allocator in the job and I have the feeling that is related to that, as I've read that if the memory that it uses is exhausted it fall backs to a much slower allocation, but the thing is that I'm not seeing those pink chunks that I believe are displayed when that happens

I've tried lowering the number of paths from like ~500 to the hundreds and seems to improve a bit but there's always a thread or two taking significant more time

[vagrant surge]

are you sure its not the OS bumping the thread?

[sharp flax]

It is also interesting that on my old pc (4c/8t) it worked fine, i documented it a few months back and had screenshots

[hollow sorrel]

does this also happen if you do a bunch of dummy calcs (e.g. a for loop that does the same thing in same amount on each thread)? to check if it's because of ijobparallelfor

prob not due to temp allocator because that's 16kb per thread, if all those jobs are the same then would expect to see same slowdown in each (and even then, fallback allocator shouldn't take 2ms more)

[sharp flax]

I've got a 10k int3 array so I guess i'm beyond that

[bright sentinel]

Just curious, how many agents do you have in this stresstest?

[sharp flax]

but gonna do that dummy test

with / without the allocs

nearly 500 i think

[bright sentinel]

Couldn't you reduce the int3 array to a int2 array?

Or is this a 3D grid?

[sharp flax]

it's not the grid, it's the way I store pathfinding info (hcost & gcost and parent)

[bright sentinel]

Ah right

[sharp flax]

I tried capping it to ushorts before, but didnt make it better

[bright sentinel]

So you're passing in a nativearray of int3 array with temp allocation?

Or are you allocating that during the job?

[sharp flax]

its allocated during the job

it certainly has to do with the allocs, doing pointless work is totally fine, and removing everything but the allocs still happens

[hollow sorrel]

if you use Allocator.Persistent does it still happen?

don't forget to dispose

[sharp flax]

cant use other allocator inside jobs other than temp no?, if you meant by passing it form the main thread, I already tried making an array with its length * number of agents from the main thread, but since it has to be with cleared mem the bottleneck was so hard that I didn't even look at anything else

the only thing I have some faith in is moving to a bunch of IJobs instead

I've tried with the same number of agents as logical threads and works totally fine, so if temp mem is released after each IJob finishes it should hopefully work fine

[bright sentinel]

So wait, do you mean that you are not disposing the allocations at the end of the job? Or is something else happening?

[sharp flax]

afaik, temp allocations do not need manual disposing, in fact, it does not throw any error like tempjob or persistent does

https://forum.unity.com/threads/allocator-temp-container-need-dispose.852082/

well, schoener reply seems appropiate to my case so i'll just hope that it'll work fine with a bunch of ijobs

[bright sentinel]

So you basically gotta figure out a way to not allocate all that memory huh

[sharp flax]

Fixed it! Now it's 1k paths on a 100x100 grid, but also with obstacles (although the map is simple & open with no dead ends). Lesson learned: be extremely careful with temp allocators in paralleljobs

[heady vortex]

I can run navmesh queries directly on the job system right?

that's what the API says, but I've been told Jobs can't access the main thread

[bright sentinel]

Nice! How did you fix it?

[sharp flax]

Just went from a parallelfor job to an ijob per path request

[vagrant surge]

one job per path request has pretty significant overheads

have you thought of doing 1 job per core, but grabbing the path requests from a parallel queue?

[sharp flax]

Actually I've been doing some more tests and I have realized that now that I'm able to have substantially more paths, if there's very low work having all those path dependencies is pretty terrible, so that's a good idea

[vagrant surge]

im not completely sure how that parallel for works, but a fairly common issue of them is that the work ends up varying too much, and it fucks up the distribution of tasks

generally on parallel fors you want to overlap them with more parallel fors so when threads finish their share they go to something else

[sharp flax]

Never tried to do such small parallel fors with uneven tasks like pathfinding before, I'm doing this just for the sake of learning so I'll see how it turns out 🙂

[lusty otter]

I have an unsafe fixed length array struct implementation that I use with Burst.

To do some very simple memory leak checks, I have something like:

#IF UNITY_EDITOR
    // On allocation
    GlobalStaticClass.counter++;
#ENDIF

and

#IF UNITY_EDITOR
    // On deallocation
    GlobalStaticClass.counter--;
#ENDIF

And check if the counter is 0 at end of game.

Problem is that in Burst inspector, any job that uses it wouldn't compile and I lose the ability to check if I made a mistake in my actual jobs unless I comment out those checks.

Any idea how to get around it?

[north bay]

Are you currently using a shared static?

[lusty otter]

Yeah it's just one static class with a counter

It's only for development anyways to catch the most obvious memory leaks.

[north bay]

Sorry, I meant this https://docs.unity3d.com/Packages/com.unity.burst@1.5/manual/docs/AdvancedUsages.html#shared-static

[lusty otter]

No it's just a plain old static class.

[north bay]

The docs show how you can setup a static field that you can use with burst

[lusty otter]

Thanks

Feels a bit overkill but I guess it's fine since these code will get striped out from production anyways.

[pliant pike]

I love when I update the hybridrender and random code is now broken in a completely unrelated script that has nothing to do with rendering 😕

[safe lintel]

ive found a number of times they introduce leak or error checks where there were none before, often ive been doing things incorrectly but it just happened to work fine until the update

[pliant pike]

yeah that's probably what I've done and things were going so well too

[bright sentinel]

That's probably gonna become better in the near future 🙂

[opaque carbon]

Whats the latest on terrain support in DOTS? I tried attaching a ConvertToEntity to my terrain and while the entity gets created nothing is rendered anymore

[pliant pike]

I don't suppose anyone knows why I'm getting this warning...

IndexOutOfRangeException: Index 0 is out of restricted IJobParallelFor range [4704...161] in ReadWriteBuffer. ReadWriteBuffers are restricted to only read & write the element at the job index. You can use double buffering strategies to avoid race conditions due to reading & writing in parallel to the same elements from a job.

for this job

Entities.WithDisposeOnCompletion(comparisonArray).ForEach((ref CellData cooldata, in DynamicBuffer<FlowfieldVertPointsBuff> flowverts) =>
            {
                
                cooldata.cost = 1;
                for (int j = 0; j < comparisonArray.Length; j++)
                {
                    var tempobsbuff = comparisonArray[j];
                  
                    var tempbool = (flowverts[0].Float3points.x <= tempobsbuff.c0.x && flowverts[1].Float3points.x >= tempobsbuff.c1.x)
                                && (flowverts[0].Float3points.y <= tempobsbuff.c0.y && flowverts[1].Float3points.y >= tempobsbuff.c1.y)
                                && (flowverts[0].Float3points.z <= tempobsbuff.c0.z && flowverts[1].Float3points.z >= tempobsbuff.c1.z);

                    if (tempbool)
                    {
                        cooldata.cost = byte.MaxValue;
                    }
                }
            }).ScheduleParallel();```

is it because of the buffer am I not declaring it readonly properly 😕

[hollow sorrel]

probably because it thinks comparisonArray is readwrite

dunno how to declare that using entities.foreach tho

[pliant pike]

yeah just figured that, I used WithNativeDisableParallelForRestriction(comparisonArray)

and that seems have fixed that problem, now I have tons of others, I wish I'd updated sooner

[whole gyro]

you can use WithReadOnly(comparisonArray) if you aren't writing to it

[bright sentinel]

https://showcase.gdconf.com/session/unity-for-all-2021-tech-and-creator-showcase-presented-by-unity-technologies/880109 Seems there might be some DOTS news during the keynote later today

[karmic basin]

topics including visual scripting, the Data-Oriented Technology Stack (DOTS) workflow, graphics and rendering tools, netcode, quality-of-life updates, and more
❤️ ❤️ ❤️

[karmic basin]

There's also another one later dedicated on visual scripting only, but I'm afraid it means Bolt

[small arch]

will there be any performance gain to generate meshes using DOTS?

as in procedural meshes

[hollow sorrel]

@small arch yeah you can use burst + jobs to generate meshes and it's huge speedup

https://github.com/Unity-Technologies/MeshApiExamples-2020.1 for examples

[small arch]

ty

[half jay]

Can i use Entity field in blob asset?

[small arch]

is it better to use Unity.Mathematics or Mathf performance-wise?

[pliant pike]

Unity.Mathematics

[karmic basin]

Another hidden gem 👀 how did you find it? Did you stumbled upon it by chance ? forum ? ...

OH MY GOD this one too https://github.com/Unity-Technologies/DOTS-training-samples

"2 years ago" 😲

[hollow sorrel]

yeah forums i think

might've been one of the "20xx.x beta" forums when a new release came out, not sure tho
they tend to post new stuff + examples on there

but they could def do a better job making it easier to find

[karmic basin]

Yeah haha

could save a lot of learning time

[gusty comet]

+Stuff deprecates fast these days.

[karmic basin]

THough the ECS samples were great as a start

[karmic basin]

Sure, but seeing how it's supposed to be done is invaluable, then updating is not scary

But I'm happy how they keep their examples up-to-date, very much appreciated

[gusty comet]

Agreed x2

[pulsar jay]

Thats confusing. Is GDC Showcase a different event from GDC as the Website says 19-23 July? And also the Unity for All session is not listed on the unit GDC site: https://unity.com/events/gdc2021 🤔

[karmic basin]

Yeah there's an online free-attending event this week (maybe more)

You just have to register

[pulsar jay]

Yeah I just did. It says I will get a mail from [e-mail] but didnt get anything yet

nevermind I just got it 😅

[karmic basin]

👍

[bright sentinel]

No, blob assets are immutable data

[safe lintel]

will there be a mention of dots at the gdc thing 🥲

[bright sentinel]

Yes, look at the link I posted further up

[safe lintel]

holy moly

time for a morning drinking game, if they mention roadmap or determinism, take a drink

[pliant pike]

will these be available after to view

[twilit coral]

There are supposed to be two GDC events this year, one right now and one in summer for bigger news so they have more time to prepare.

[pulsar jay]

Yeah thanks for clearing things up

[twilit coral]

no probem

[small arch]

hey, the DOTS mesh rendering stops rendering the mesh when half of the mesh is off camera

any way to fix that

[twilit coral]

rebuild bounds maybe? seems like that might be the problem if its right in the middle

Not yet sure caus havent used it yet, will gotta later caus generation is looking kinda steep as it is getting bigger

[small arch]

how would i rebuild bounds

its for both the editor and the camera btw

i moved in the editor a tiny bit down and the whole entity disappeared

[twilit coral]

hmm do you have mesh culling on entity? I am bit lost there sorry if not helping, I stopped working with dots for now till the animations get more userfriendly pass.

[small arch]

this is literally the only code i'm using

[twilit coral]

https://forum.unity.com/threads/createmissingrenderboundsfrommeshrenderer-removed-in-hybrid-renderer-0-3-4.831820/

the post from ivlsiuk is showing how to add your bounds

@small arch

[small arch]

wow that worked

thank you

[twilit coral]

No probem.

[safe lintel]

dots segment on the keynote

so far kind of a non segment to us i think, "its still in development, gonna be the future"

[pliant pike]

there still working on it thats good news 👍

[safe lintel]

they talked about a new netcode sample but i think it was gameobject based? unity.com/boss-room

think we concluded the dots segment

[pliant pike]

so is it normal for the hybrid renderer to not be that great, I get 15fps with 40,000 cubes

is it going to improve or is that just the limits of rendering in general

[bright sentinel]

Yeah that was for GO Unity. Only thing was about the transport layer, but not really too relevant

[bright sentinel]

You should definitely get higher FPS. Are you using V2? Also, URP or HDRP?

[amber flicker]

also assuming you're on desktop with discrete gpu?

[dull copper]

that keynote didn't really bring anything new to the table, mainly the announcement of the new gameobject netcode sample project

[karmic basin]

Yeah they basically confirmed what we already know, "sorry no roadmap, you see we got feature-creep and wanna go all-in dots editor"

[dull copper]

(and new 2D sample project on asset store)

[karmic basin]

And "the forum is the place to go to keep updated about DOTS"

[safe lintel]

yeah, pretty disppointing. i shouldve figured given joachim wasnt on the list of speakers for it

[karmic basin]

Yeah can't wait to see that one

🙂

I think its new netcode-based, so ECS ?

[twilit coral]

I was waiting for Probe Volumes in URP to be showcased but not even HDRP ones got the place, so gotta wait to the summer I guess

[pliant pike]

I don't think I'm using either maybe that is the problem

[karmic basin]

I mixed networking and netcode. You're right @safe lintel, probably GO-based with custom network layer on top of the Transport package

[bright sentinel]

Then that's definitely the problem, you should read this: https://docs.unity3d.com/Packages/com.unity.rendering.hybrid@0.11/manual/requirements-and-compatibility.html and the getting started page as well to set it up

[bright sentinel]

I'd be more inclined to think it's MLAPI tbh

Maybe it's even built on top of Transport???

[karmic basin]

Yeah
I was wondering what is their first-party solution, they don't had one such thing for a long time 🤔 then concluded the same thing, they prob consider the MLAPI ready ? (didn't follow that much to tell)

[bright sentinel]

Yeah it is MLAPI

[karmic basin]

Yeah this one is a fact, that's what they told during the stream

[safe lintel]

didnt realise it was far enough along with integration for them to make a full scale demo like that

[bright sentinel]

Then why'd you say they were making a custom network layer? 😄

[karmic basin]

I meant on top of the Transport package

[pliant pike]

Thanks👍

[karmic basin]

But that would have been dumb to add another solution in the bowl

[bright sentinel]

Yeah, but what's to stop MLAPI being built on top of Transport?

[fluid kiln]

Hey, I'm still stuck on skinned mesh conversion. Anyone has experienced this before and can help me? I lose both the rotation and the scale of the GOs transform during conversion

Happens with both blend files and fbx

[safe lintel]

@fluid kiln need to use a compatible shader, theres vertex and compute shadergraph examples in the animation samples

[fluid kiln]

Well well, thank you

[stone osprey]

Is marking entities ALOT a good idea ? Like... items are entities in my case. They are pretty logic heavy. So i mark them with stuff like "Equiped{ wearer }" or "PickedUp{}" or "Dropped{}"... so there systems that operate on those entities like : Entities.ForEach(Item item, equiped eq, onEquipedSpawnEnemy oese){} is this too much marking ? Or is it flexible and fine to use marking that intensive ?

Some marker only stay one frame... for example theres also a "OnEquip" marker... which i delete after one frame. Only to react with some system to execute onequip logic

[dark cypress]

Might want to make an event system separate from the entities you operate on.
The bigger the entities, the less can fit in a chunk, and more memory needs to be moved around every time you change its archetype.

There have been some discussions on the DOTS forums on the topic.

[stone osprey]

@dark cypress Alright, thanks ! Lets say the performance isnt that important for me... are there any downsides of that intensive marking approach ?

[dark cypress]

No, not really. It's all about micro-optimizations.
I've also heard Unity will optimize adding and removing tag components(data-less) to be very fast, so might want to design keeping that in mind.

[sturdy rune]

It doesn't need to optimize data less tags because they don't need a chunk associated - isn't that the case?

Like they take up next to 0 in the chunk

I was reading this last night in the dots basics guide

[stone osprey]

Ok thanks ^^ I also see no downsides of that design yet ( well performance, but it doesnt matter in my case ). It should be flexible to add new features by adding new components to the items easily... atleast it looks like this, hopefully thats right xD

[dark cypress]

@sturdy rune The entity still needs to be moved to a different chunk(and a chunk created if it's a new Archetype) on a system change.

[sturdy rune]

True

So every tag only associated to one entity would need an entire chunk

[dark cypress]

Yeah, which is why I've also seen a Unity dev recommend to keep entities granular (separate big entities into multiple), so that there are less possible combinations of components.

[ocean tundra]

Weird question, what's the smallest we can make a Icomponentdata?

I'm guessing 8 bytes (the size of byte)?

Sorry I mean what's the smallest we could make a IBufferElement?

OR the largest we could make a IComponentData?

[fluid kiln]

When I connect a second client in NetCode, I get exceptions form the NetCode package querying for singletons of NetworkIdComponent. Has anyone encountered this before?

[karmic basin]

What the hell 😲 yeah you're supposed to have only one per client

[fluid kiln]

One per client? hmmm

I have 2 on my Client0 when Client1 logs in

It's the strangest thing too, I searched my code, nowhere do I create a NetworkIdCompoennt 🤔

[karmic basin]

Yeah that codegen by the netcode package

[fluid kiln]

I'll refresh it

[fluid kiln]

Yeah I mean, never me manually, and I expect that NetCode is supposed to work ^^

[karmic basin]

yeah I see. I'm wondering how it could happen 🤔

[fluid kiln]

Maybe my connection code is not right

I don't see how my code could create more network ids tho :/

        Entities.WithNone<NetworkStreamInGame>().ForEach((Entity ent, ref NetworkIdComponent id) =>
        {
            ecb.AddComponent<NetworkStreamInGame>(ent);

            var req = ecb.CreateEntity();
            ecb.AddComponent<ConnectedRPC>(req);
            ecb.AddComponent(req, new SendRpcCommandRequestComponent { TargetConnection = ent });
        }).Schedule();

        Entities.WithNone<SendRpcCommandRequestComponent>().ForEach((Entity reqEnt, ref ConnectedRPC req, ref ReceiveRpcCommandRequestComponent reqSrc) =>
        {
            int networkId = GetComponent<NetworkIdComponent>(reqSrc.SourceConnection).Value;

            if (!connectedIds.Contains(networkId))
            {
                connectedIds.Add(networkId);

                ecb.AddComponent<NetworkStreamInGame>(reqSrc.SourceConnection);

                var player = ecb.Instantiate(playerPrefab);
                ecb.AddComponent(player, new GhostOwnerComponent() { NetworkId = networkId });
                ecb.AddBuffer<PlayerInput>(player);

                ecb.SetComponent(reqSrc.SourceConnection, new CommandTargetComponent { targetEntity = player });
            }

            ecb.DestroyEntity(reqEnt);
        }).Schedule();

[karmic basin]

Yeah you're not supposed to handle that yourself, just use it to ID the client (who is who)

[fluid kiln]

Exactly

[karmic basin]

refreshing the codegen did not help ?

[fluid kiln]

Maybe the newer version of netcode handles connecting different, I think mine is from like 0.4

No it didn't :/

[karmic basin]

Last time i played with it, I was in 0.6 I guess. WHich one you upgraded to ?

[fluid kiln]

If my code above rings an alarm, let me know I'D appreciate it, the most recent, 0.6

I'll go look at the cube sample again

[karmic basin]

Where are your clients from ? build and/or editor ?

[fluid kiln]

Editor running server/client, and 1 built client

[karmic basin]

Yeah I do the same and never happened to me

[fluid kiln]

If I try with a thin client, I don't get errors but the players are never spawned

Mhh

[karmic basin]

ANd do you try to spawn multiple prefabs per client ? (Looks like no)

[fluid kiln]

No just the player prefab

[karmic basin]

uhm

I'd expect to see 2 on the server world

but not client0

did you launched the build first, the played the editor, stopped it, and played again ? (would mean you don't destroy entites when losing connections)

[fluid kiln]

When I do that, I get the same result as when I run a thin client. No error, but no players

[karmic basin]

ok. If you didnt modify default, only editor can be server anyway

[fluid kiln]

Yeah I checked my build is not server... it's so strange, I built NetCode before the subscene conversion but I never managed to make it work with it

[karmic basin]

when you click on the 2 entities, they are have different values or the same ?

[fluid kiln]

Maybe it has to do with my subscene conversion?

[karmic basin]

I used subscenes, works ok

[fluid kiln]

1 and 2

[karmic basin]

well that all good then

[karmic basin]

only thing weird is why you see both if you filtered the entity debugguer window to ClientWorld0 🤷‍♂️ should see only 1 here

[fluid kiln]

I see some differences in the cube sample, I'll go ahead and make some changes

[karmic basin]

yeah not sure your target component should be empty in last screenshot

yeah do the update, especially the second block of code you shared

In last version they use a GoInGameRequest instead of ConnectedRPC, codegen may expect this name

[fluid kiln]

GoInGameRequest is a used defined component but maybe so 😮

[karmic basin]

you mean user defined ? yeah you're right shouldnt matter

[fluid kiln]

yeah user*

oh well shit

I see people

[karmic basin]

That's strange. I don't see what's wrong. I won't add to the confusion. Maybe yeah it's just you have code not up-to-date

[fluid kiln]

I'm sure it was my second block of code

Seems to work now, I'll try with a build

I should probably build from my build configs rather than the build settings...

But now I'm seeing this

[karmic basin]

Yep. don't use legacy build settings

[fluid kiln]

Which build components are the pipelines

[karmic basin]

I use this

• com.unity.platforms.windows installed

[fluid kiln]

Awesome thank you very much

[karmic basin]

https://docs.unity3d.com/Packages/com.unity.entities@0.17/manual/install_setup.html#standalone-builds

[fluid kiln]

Love that big warning I've never seen but that could've been useful 5 months ago 😂

[karmic basin]

yeah they added it only with 0.17

took me a while to discover too

[fluid kiln]

The warning says that this component is not used by the pipeline. Will my build still be a client?

I'll try it out

[bright sentinel]

It should. But if not, you can add the scripting define yourself as part of the build pipeline, it should just be UNITY_CLIENT

[fluid kiln]

Checks out, now I'm getting Trying to render a batch from the Hybrid Renderer V2 with SRP Batcher OFF. This is not supported. Please turn SRP Batcher ON to use the Hybrid Renderer V2.

But it is?

[bright sentinel]

https://forum.unity.com/threads/how-to-please-turn-srp-batcher-on-to-use-the-hybrid-renderer-v2.1067003/ This reports the same error as a false-positive

[fluid kiln]

DO I have to define ENABLE_HYBRID_RENDERER_V2 in the build settings too?

[bright sentinel]

Uhhh