you can get away with a lot even with millions of entities
#archived-dots
low tangle
is this your first ecs project?
if so I suggest trying and failing but with a simpler game
instead of a 3d grid, try a 2d one
wary anchor
Thanks for the help, @low tangle . I'm still not sure about the hashmap as the terminology isn't clear.
low tangle
public NativeHashMap<uint, Entity>.ParallelWriter outputHashmap;
one of these
in your case it would be a int for the key, but that key would be a hash generated from the x,y,z cords after being divided and floored by the cell size
wary anchor
I'll bang through it brute force and see what sort of numbers I get. I'd like to learn how to do it more intelligently, but honestly some kind of intermediate example project from Unity to bridge the gap would be most welcome
low tangle
did you peek though the different hello cube examples?
wary anchor
yeah I've no problems generating the keys that's all fine
oh yes, been through all that several times. And been reading up as much as possible!
low tangle
a lot of complex ecs stuff is just compounding simple things stringed together, I think you need to learn how the complex bits work in practice, and I really think there should be a tutorial for it
wary anchor
it's like you're swimming in the shallow end, believing them when they say at conference videos "it's not difficult, it's just different" and then the first non-trivial thing you see is the Pacific and you're 250 miles from land.
low tangle
they dont touch on any of the native collection stuff being used with job chains would help a bunch
I can't post snippits from my game, but I'm looking right at a good example :(
maybe I can make some snippits to explain it better
mint iron
I have something similar to you going on and went with a simple NativeArray approach. Its stored on an entity as a DynamicBuffer, and the elements are IBufferElementData so i can be retrieved from anywhere and iterated on in jobs if necessary. Its wrapped in a NativeArray3D, which is really just some sugar for easy 3d indexing on top of the existing array pointer.
wary anchor
oh that would be amazing if you would
low tangle
yes, right totally forgot buffer 1d array approach
basically dynamic buffers are awesome
wary anchor
For the grid ID I was just going to pack the 3 coordinates after flooring into the first 30 bits of an int 32 (3x10, easily big enough)
low tangle
its a list of type attached to any entity, that you can read from inside of jobs
mint iron
I have a system that is able to give any other system a 'layout' (pretty much to just reduce code duplication grabbing it from the dynamic buffer etc). Here's an example of spawning some stuff and assigning them to the grid: https://hatebin.com/rjbzfbaszi
wary anchor
@mint iron thanks for this. So your native array is set up something like:
public struct MyDynamicBuffer : IBufferElementData
{
public NativeArray<int> Value;
}
? Or am I misunderstanding?
mint iron
here's the NativeArray3D https://gist.github.com/jeffvella/fd5b22f5b593f1017ef922eb1b533837
IBufferElementData is this: https://gist.github.com/jeffvella/f20986243979a7fd42d3ccac05e37692
and BoardPosition: https://gist.github.com/jeffvella/16bcd8f36bdd4aa6991c0ccadf954bfe
wary anchor
okay thanks both, I need to read and digest and play a bit to try to get this into my skull.
👍
low tangle
good luck, if I get a free moment I'll try to make a job chain, build hashmap setup for exactly what you are doing
wary anchor
That would brilliant, thanks again for the help!
low tangle
back to overhauling ecs network code 💪🏻
scarlet inlet
A question about the NativeArray, I noticed the indexing operator is not returning by ref. This is annoying, is there an alternative to be able to change just a field of a struct in a native array cell?
mint iron
you could do
var someArray = new NativeArray<Translation>(10, Allocator.Persistent);
ref var item = ref UnsafeUtilityEx.ArrayElementAsRef<Translation>(someArray.GetUnsafePtr(), 5);
item.Value = 0;
perfect @mint iron it's what I was looking for
wait UnsafeUtilityEx?
oh I see it's a package
mint iron
experimental unsafe stuff!
you could also do
ref Translation fifthElement1 = ref ((Translation*)someArray.GetUnsafePtr())[5];
Translation* fifthElement2 = (Translation*)((byte*)someArray.GetUnsafePtr() + UnsafeUtility.SizeOf<Translation>() * 5);
Hi all, sorry it's me again. I'm sorry to be so thick about this, really I am. I know this approach is wrong, but I can't work out why it's wrong nor how to make it not wrong.
https://hatebin.com/uajzdeujrh
@mint iron I got confused reading through your code (my fault not yours!). My understanding from your comments is that the NativeArray3D class contains the BoardLayoutBufferData, each of which in turn contains the BoardPosition data. The NativeArray3D is obviously a custom extended class and tbh I want to keep it as simple as possible right now so I can get the basics down, but in any case I didn't understand where this native array would sit - is it on another component? How do you access it? How do you set it?
mint iron
you could just create one and keep it anywhere, like in a system. I put it inside a DynamicBuffer so that i could view its data in the EntityDebugger, and so that in theory any system could use it, or i could use a job to iterate its values in parallel without extra work.
I'm not going to claim this is pretty, or right :D, but it worked for what i needed. https://gist.github.com/jeffvella/727009911a7cf1cc5064b9754ca4a61f
so systems that need it can go
protected override void OnCreate()
{
_layoutSystem = World.GetOrCreateSystem<BoardLayoutSystem>();
}
_layoutSystem.GetBoardLayout()
Thanks. I am trying really hard to get this, and I promise I'm not actually stupid - I have a doctorate and have supervised doctoral students - but man this ECS is craaaaaaazy hard the way it's currently presented to the public :(
I appreciate the help!!!
mint iron
no worries eh, we're all just picking it up as we go along. blazing trails.
tawdry tree
Everything is hard if you're used to some other very different mindset
wary anchor
I've switched primary uni disciplines and taught at both, I learned an entirely new field in my post-doctoral work, I switched career 4 years ago and have been learning new things every single day. This is different.
tawdry tree
Were those disciplines extremely adjacent with lots of overlap, but with notable, fundamental differences? That's what ECS is to monobehaviour. The issue is the differences in themselves, but how close everything else is.
Or in other words, you have some ingrained 'this is the way to do this thing', but it's not in the new method/framework/principle/discipline/whatever
wary anchor
Well the main issue is actually in the lack of examples of intermediate complexity to take you from the trivial spinning cubes to the Boids.
pliant pike
ECS does need more docs and examples
mint iron
i should mention also that there is an entity with these settings, i know there is only one because the entity gets created from ConvertToEntity workflow. I have one GameObject in the scene that lets me configure settings in the inspector. https://gist.github.com/jeffvella/627f764c8040b471f8d75569f93424ab
pliant pike
but then the lack of is understandable it still being in development
tawdry tree
Well, that is not as much the problem of difference as in making up for it ^.^
wary anchor
I'm open-minded. I'm challenging myself by learning this stuff, and in my experience of learning many different disciplines at different levels, this is by far the most difficulty I've had.
YMMV of course.
tawdry tree
One useful thing with ECS in that regards is that if you adhere to the single responsibility principle (which I do believe is quite central to the "system" part), you can, as here, share the specific code you have issues with, and it should be enough on its own. That is, of course, until you get to intermediate+ levels, and it's more about behavior than, well, lack of such and/or error.
wary anchor
Definitely down with SOLID, and I like the approach - that's why I decided to jump in head first. I guess that's why we're all here.
@mint iron 👍 thank you!
mint iron
here's a simple example of being accessed. https://gist.github.com/jeffvella/5d7deab184f9a1acdb151b25c81f1a3c obviously some other things going on here with events, but its BoardPosition / int3 coordinates being used to switch an Entity between positions.
wary anchor
ahh that's perfect, I see! thank you
pliant pike
well your doing better than me I don't really understand that code at all
wary anchor
I have a glimmer of light through the fog. This is infinitely better than total blindness 😄
wary anchor
@mint iron I feel like I'm getting somewhere with this approach based on your implementation. Just wanted to check before I get too far in... how many elements is reasonable to put on your NativeArray3D (or in general a NativeArray) stored in this way? I think I may be kind of flipping the situation you've got on its head slightly, in that I want access to a fairly large number of IBufferElementData items within each NativeArray.
I'm thinking of around a thousand or so on each one probably
mint iron
@wary anchor i think you could put as much as you want in there, since its just a 1D array / raw allocated space, so hundreds of thousands shouldn't be a problem. Inside a DynamicBuffer should also be okay; they do have a small starting allocation in chunk but if you go beyond that it allocates space on the heap for however much you need. The component space in chunk then just holds a pointer to the real storage location.
winter depot
In the docs for IJobChunk I see they have an example where they set a filter csharp m_Group = GetEntityQuery(typeof(Output), ComponentType.ReadOnly<InputA>(), ComponentType.ReadOnly<InputB>()); m_Group.SetFilterChanged(new ComponentType{ typeof(InputA), typeof(InputB)}); I cannot replicate this, where they are using csharp m_Group.SetFilterChanged(new ComponentType{ typeof(InputA), typeof(InputB)}) I can replicate csharp m_Group.SetFilterChanged(typeof(InputA)); Am I missing something?
soft olive
Does any know how we can pause a simulation using Unity.Physics? Physics simulations seem to run on Time.fixedDeltaTime but that can't be set to zero. TimeScale doesn't seem to affect physics at all. Any suggestions?
vestal hatch
@mint iron I found a typo in your NativeArray3D.
public NativeArray3D(int3 size, Allocator allocator) : this(size.x, size.z, size.z, allocator) { }
there's 2 .z, it should be
public NativeArray3D(int3 size, Allocator allocator) : this(size.x, size.y, size.z, allocator) { }
and
public NativeArray3D(int x, int y, int z, Allocator allocator) : this(x,y,z, new NativeArray<T>(x * y, allocator)) { }
public NativeArray3D(int x, int y, int z, Allocator allocator) : this(x,y,z, new NativeArray<T>(x * y * z, allocator)) { }
Is it generally a good idea to write to a NativeArray or NativeList in jobs, and then do dynamicbuffer.AddRange(array) or is there a way to write in parallel to the buffer itself?
coarse turtle
You might be able to use memset if you want to try to write to a buffer in parallel
I've never tried it myself
mint iron
@winter depot GetEntityQuery is a params argument but for some reason they did it differently with filter changed and so you need the ComponentType[] array.
var m_Group = GetEntityQuery(
ComponentType.ReadOnly<Translation>(),
ComponentType.ReadOnly<Rotation>(),
ComponentType.ReadWrite<LocalToWorld>());
m_Group.SetFilterChanged(new[] {
ComponentType.ReadOnly<Translation>(),
ComponentType.ReadOnly<Rotation>()
});
Yeah, I figured that the documentation was just a typo 😄
Thanks @mint iron
I recently realized that every time I was creating a new entity with a new request to my pathfinding I was changing the chunk, and the entire previous chunk was running again in my IJobChunk. I was thinking that previously unchanged items in the filter wouldn't run again. Now I am using just a bool in a component to filter those already successfully having found a path. Is there a better way to go about doing this? I wanted to avoid the whole add remove components or destroying and adding entities for maximum performance since EntityCommandBuffer is so heavy.
low tangle
bools on components for event entities
avoiding entity invalidation and the mem copy move is important when you hit >10k
winter depot
So I am doing it the best way?
low tangle
well describe the component a bit better
component { bool HasPath }
then a dead simple if check in the jobforeach right?
winter depot
public struct PathRequest : IComponentData
{
public int2 start;
public int2 end;
public bool NavigateToNearestIfBlocked;
public bool NavigateToBestIfIncomplete;
public bool isFufilled;
}```yep thats it, as long as the 'loop' is dead simple to ignore fast enough
winter depot
Its a IJobChunk, I first was trying to use that .setFilter option to not have to filter them at all
low tangle
this ends up being the fastest even though you iterate lots of chunks because of cache
winter depot
I just guess I didnt read documentation close enough because I thought it was saying if new items were added, unchanged old items were ignored
low tangle
hm
not sure what the context is
probally that old chunks are not touched
only the one that has a entity modified in
mint iron
you know changed only works on a chunk level? it can't tell if an individual entity/component value has changed
low tangle
oh make sure you are marking stuff as readonly as much as you can
yep that too, you have to actually check each element in the chunk for the actual change
winter depot
Yeah, everything is readonly
low tangle
it just checks if the version number is changed on the chunk
winter depot
Right, thats what I meant @mint iron
low tangle
which can happen by any jobs touching the chunk that are not read only
winter depot
I read it as it will check for a change of value in the component
Not that it will check for a change in that chunk
low tangle
oh no, basically any filtering and true delta of data type checks you have to do yourself
and it makes sense why when you think about it
winter depot
If you only need to update entities when a component value has changed, you can add that component type to the change filter of the EntityQuery used to select the entities and chunks for the job. For example, if you have a system that reads two components and only needs to update a third when one of the first two has changed, you could use a EntityQuery as follows:
Maybe it actually does, in addition to chunk change
mint iron
yeah that's mildly misleading imo
low tangle
knowing that x single entity changed means something somewhere needs to keep track of that which ruins data locality and starts adding hidden data record keeping
mint iron
they should really rename it and/or make it more of an internal optimization imo
low tangle
you can actually build that system yourself with a new entity type if you wanted, and it still will be reasonable other than memory reference lookups for each one (memory thrashing ahoy)
winter depot
Well if this is wrong as well as the thing I pointed out earlier, than yeah. I can see now why I had more confusion than their should have been
I am still really new to programming, so a lot of this stuff gets lost on me
low tangle
honestly have been enjoying the dod book a lot. really demystified lots of this stuff for me
I forgot who here got it for me in pdf form but thank you a lot. its been a great read the past week in bed
mint iron
low tangle
yup
already managed to save my ass on debugging two really hard to pinpoint networking ecs bugs
almost done with my overhaul because of it
winter depot
I look forward to reading
mint iron
nice!
low tangle
highly recommend it. almost everything unity did has a parallel to this book
just got though the part on finite state machines in a dod approach and its going to be really useful for ai and animation later on
mint iron
i am trying to figure out a bug that just hard crashes Unity, the debug report error logs show a memory access violation but no useful information and i have no idea how to debug that. Somewhere there's something unsafe thats being naughty, just gonna have to go back to the last working version and readd changes until it breaks. 😦
winter depot
Is it worth me going back and changing jobs I created days ago from IJobForEachwithEntity to IJobChunk? Or is the performance difference negligible? Or is it purely situational on chunks that are less likely to change
low tangle
god no
always start with foreach's
if that single job becomes too slow then you go to a chunk approach
always lean towards the simplest implmentation
thats the beauty of ecs, miles wide (in number of systems and types of data) but only a inch deep worth of complexity anywhere in it
honestly I've never once worked on a system that makes problems this simple before now (even when I learned nothing but functional programing for a few months)
winter depot
Heh, I spent a day and a half trying to wrap my head around how to implement a buffer. Then I nearly cried when I realized it was as simple as it is
mint iron
yes!
low tangle
uint id = reader.GetUInt();
//get the player object that represents this player
if (PlayerDataSystem.PlayerLookup.TryGetValue(id, out Entity target))
{
DynamicBuffer<BodyUnpackBuffer> buff = EntityManager.GetBuffer<BodyUnpackBuffer>(target);
if (buff.IsCreated)
{
if (buff.Length < packet.Length)
buff.ResizeUninitialized(packet.Length);
//Raw copy the body pose over to be decoded inside of the unpack jobs
unsafe
{
Buffer.MemoryCopy((byte*)packet.Data + 6, (byte*)buff.GetUnsafePtr() + 2, buff.Length, packet.Length - 6);
}
//Size of pose 0=full
buff[0] = 0;
buff[1]++; //Timebyte to signal that its a new pose
little bit mangled, might put it into paste bin
winter depot
ohh that is really simple
low tangle
but the gist of it is, I wanted to avoid moving this data from a packet around a bunch
so on this entity that is the player on a network, I just offset memcopy the player data into a dynamicbuffer on the player
offsets for labeled structured data on it
after this runs, several burst fast as all hell systems run on that byte buffer decoding the data from that structure to what it really is
it all just works because of dynamic buffers and the fact that they are just memory pointers and you can memory copy into them
reformated it
winter depot
Yeah, I was really dreading everything ECS, because I just learned the last 6 months from nothing to viable OOP... then I started to run into performance issues and found information about it, decided I wanted to at least try something simple and see what its like. Now I just want to optimize, organize and rewrite everything multiple times to ECS for fun. I am still godawful at it, and I am sure I am doing 99/100 things wrong, but its really enjoyable.
low tangle
the best part is that ecs is forgiving for that
it only takes a hour or so to swap out a handful of systems
you get faster at it too
oh, make sure you take notes of your reasoning and everything
also I highly suggest finding a way for you to make diagrams or some way of understanding your data relationships (uml graphs w/e) and stick with it
winter depot
I have a hundred sticky notes in a pile on my desk I write and then never look at again. Thats where I am at
low tangle
I spend about 6 hours working out the kinks on a important critical system, but it only took 40min of writing code when it was all done
planning up front helps a crap ton, I just use tons of text documents embed into the project under source control like the code since they are so important
winter depot
I am pretty bad at the planning phase right now tbh. I just kind of scribble some things and then start typing and hope things work out. Naming conventions (variables/methods/classes) are what I am worst at. I will end up renaming things 20 times before I am done and still not be happy. Know any good resources for how to improve in planning, etc?
low tangle
hm
dont know any offhand, I've just gotten pretty good at it over the years
yeah the only things I can think of is the just general stuff people say all the time
don't want to patronize you by giving you the same ol advice
the only one I will say is be consistent, if you are constantly changing you will have massive friction going back to old code and getting it
winter depot
Yeah, I probably need to go back and watch some general tips/development videos now that I am more familiarized I may catch things I didn't before, in particular those kinds of things. I guess we kind of went off the topic of ECS at this point anyway, my bad!
wary anchor
Given what I'm trying to achieve, I'm beginning to think a SharedComponent might be a better idea than using a Dynamic Buffer.
I'm creating atom entities at the start and they are static. In various jobs, I want to be able to group clusters of them by grid position to avoid stepping through the entire set potentially multiple times per frame. If I add a shared component whose value is their grid ID, I should be able to yoink out native arrays as required with an EntityQuery shouldn't I?
mint iron
this might help you then https://gametorrahod.com/everything-about-isharedcomponentdata/
wary anchor
quickfire linkage, like it 🙂 I'm using shared component data for rendermeshes (obviously), though not yet doing anything downstream with them. Thanks for the info!!
Pretty sure I've read this before, hopefully I'll understand more this time around 🙂
mint iron
tbqh i haven't played with SCD at all, mostly because i read they should be used to purposefully split archetypes into the number of chunks you want to optimize some advanced use cases. And because of that behavior you have to be careful not to cause performance issues. Its on my todo list to figure out at some point.
wary anchor
Notionally, I would love to create a new Component type at runtime for each of my coarse positional groups, add that component on creation of the entities, then run jobs on these specific component types. But that kinda makes no sense in a way. And a single component whose value contains the group ID doesn't make any sense either, as I'd have to go through all entities to group them. The SCD seems like it's designed to do this job. It makes the most sense logically to me - rather than having to create 3 different types of components, a singleton entity, a system to manage the groups... that's getting very bloated very fast.
amber flicker
the spatial data is significant in some way right? Would an atom in cell 1 ever need to know about an atom in cell 2?
amber flicker
the 'best' solution probably depends on understanding your data transformations better - the SCD approach I think would work fine in your case (though I'm not sure every cell being a chunk is necessarily ideal) - but probably so would the boids hashmap approach or even just an IComponent with an int cellIndex - so long as you're aware that a SCD splits it on a chunk level
wary anchor
I'm using it for very coarse spacial filtering, so thinking whole world in max 10x10x10. xzjv kindly linked me scripts to demonstrate a DynamicBuffer example, which I am in the process of trying to implement, but all the while reading and trying to get to grips with all the different possible approaches to the problem. It must be so common, I can't believe UT didn't prepare a simpler example than Boids!
But very grateful for the help on here!
amber flicker
So... my take is a lot of ecs is around making memory layout linear - grids/boids are an example of interdependency which is a pain and so the complexity arises from trying to maintain best linear access while having interdependency. i.e. it's unfortunate but I think as a simple concept, it is a more tricky situation. If you have a bunch of atoms with an Cell:IComponentData you could just do GetComponentFromEntity<Cell>() and then find the cell you want, job done. But we care about speed. It seems to me that your case is more like LOD'ing - culling calculates based on zone/distance to player - it could be interesting to look through some of that code. Does it matter to you if a player is at the very edge of a cell? Do they need to have the adjacent cell also calculated?
wary anchor
If I could distil my query into its simplest form, it'd be:
How do I get the translation and elementData components of the n-nearest atoms to the position of this character, every frame, for custom collision checking and calculation of a new gravity axis?
where character may be player character or NPC or any other dynamic object
You raise a good point about being near the edge of a cell. I was thinking in terms of a radial cell approach - check the number of hits from this cell, expand spherically as required. But I guess that's adding complexity unnecessarily
amber flicker
Ok a bit different to what I understood - this sounds very boids-y to me. Is there anything we can help explain about the boids example?
wary anchor
I've done it in OOD and am wiping that entirely from my mind as I try to learn DOD!
amber flicker
yea it's tricky 🙂
wary anchor
honestly, my feeling about the boids example is I've just been shown how to bob up and down in the shallow end at the baby pool, and then the next thing I'm being flung from the titanic without a life-vest into icy oceanic waters. I'm trying to learn, honestly, I am!!
amber flicker
My guess is you're much closer than you think
wary anchor
I hope so! I will do this, but I really hope to impinge on anyone else's time as little as possible in doing so 🙂
it's definitely a query about how I get the data out. Once I have it, I'm good for the rest, it's just the flow of data, what I can and can't access and the best way to access it so that when I do add lots of NPCs, it doesn't turn into some huge O(n^2) or worse situation
scarlet inlet
What is the suggested way to query an entity through the Entity ID inside a job?
or better saying, I need a job that can query entities by ID
amber flicker
Ok so I'm trying to think of the most simple way to start and what I'd do. I think I would start with knowing I need to find all the atoms within proximity of related players. That means either I have a component on each atom with a buffer of players in proximity (as there could be multiple) or likely better, generate a multi hash map with the player entity as the key. Once the job populates the hashmap, my player can look itself up in the map and find all atoms in proximity. While that 'find in proximity' job runs it could simultaneously e.g. mark an atom as in proximity to a player so that any systems operating on 'atoms' know which to update. Does that help at all?
@scarlet inlet probably need a little more info - do you want to get the component of another entity from within a job?
scarlet inlet
yes through its entityID
amber flicker
you mean its Index or a reference to an Entity?
amber flicker
an Entity is a struct with two ints: Index & Version
wary anchor
@amber flicker thanks, I'm not quite with you yet, but I'm going back over Mike Acton's talk on the boids example while going through the code again. I'll try to tie that in with your comments as well!
More coffee!
amber flicker
@scarlet inlet you pass in e.g. someComponents = GetComponentDataFromEntity<TheComponentYouAreInterestedIn>() to the job then within the job you can do someComponents[anEntity] to access the component related to 'anEntity'
scarlet inlet
interesting
amber flicker
@wary anchor cool - I think there is a step before all the IJobNativeMultiHashMapMergedSharedKeyIndices optimisation - are you familiar with a NativeMultiHashMap structure?
scarlet inlet
I assume there is no allocation in doing so
amber flicker
correct
wary anchor
Hazy, I'd say
amber flicker
the downside is it's random access.. but there's not really a way to avoid it
@wary anchor it's much more simple than the name my sound - v similar to a dictionary. A key and a value. The Multi variant allows you to have more than one value per key.
wary anchor
oh right, okay that is simple
amber flicker
so my suggestion was you essentially make a dictionary where the player entities are the keys and the atoms in proximity are the values
scarlet inlet
ok @amber flicker I think it makes sense
I'll try thanks
why should it not be readonly?
in case I write to the component I guess
amber flicker
yup
scarlet inlet
I guess GetComponentDataFromEntity is a sync point for the jobs as well
mint iron
i dont think so, its just a struct with a typeIndex and version basically
scarlet inlet
maybe, it's not always clear
but according what I see from the official sync points it would make sense it is not
wary anchor
So to check I understand this correctly... the NativeMultiHashMap hashmap in the BoidsSystem is a collection of keys = hashed positions and values = EntityIndex of the boid, is that right?
scarlet inlet
another question, if I need a set of systems to run before the SystemGroup and a set of systems to run after, what would you suggest me to do?
I can use the runbefore, but I can also use other system groups?
like I see there is a LateSimulationSystemGroup
which is a bit scary but well..
amber flicker
@scarlet inlet I define ComponentSystemGroups and their update order. E.g. public class Group1: ComponentSystemGroup { } [UpdateAfter(typeof(Group1))] public class Group2 : ComponentSystemGroup { } then with my systems I say e.g. [UpdateInGroup(typeof(Group1))]
scarlet inlet
and then you register your group in the right order?
amber flicker
@wary anchor from memory, yes
wary anchor
thanks @amber flicker I'm already a step further than I was. I'll get there! Thank you
amber flicker
@scarlet inlet there's nothing more to it than the above - all those systems will run within the SimulationSystemGroup, the ones with the attribute [UpdateInGroup(typeof(Group1))] will run before those [UpdateInGroup(typeof(Group2))].. though NOTE this is just the start of the execution of the job - if you need a system to have finished before the other system, you will need to call Complete on the job
amber flicker
Cool, no worries - good luck
scarlet inlet
@amber flicker ComponentDataFromEntity like the other native containers do not return by ref
I really hate this, but in this case I think there is no workaround
being forced to do a temporary copy is so awkward and slow
amber flicker
? there shouldn't be any copying involved
amber flicker
{
new MyJob(){
someComponents = GetComponentDataFromEntity<MyComponent>()
}.Schedule(query, inputDeps);
}
struct MyJob : IJobChunk
{
[ReadOnly] internal ComponentDataFromEntity<MyComponent> someComponents;
public void Execute(ArchetypeChunk chunk, int chunkIndex, int firstEntityIndex)
{
if (someComponents.Exists(someEntity)) float health = someComponents[someEntity].health;
}
}```hmm sorry I don't agree on this one
did I get it right? you are iterating the whole chunk just to write to one entity?
amber flicker
what? this is just example syntax - maybe take a look through some examples
scarlet inlet
hmm yeah mabe I didn't get it
wary anchor
higher level question, as I'm looking to use the atoms-in-proximity for multiple different purposes (eg calculating up axis for other objects, checking collisions for objects, possibly future systems?), what's the paradigm in terms of separating concerns and staying SOLID.
I can already see 2 separate objectives to be completed by the system, and that feels a bit wrong... though in terms of the data flow it does makes sense.
I presume that the overhead of having multiple systems is greater than for example doing what those systems do within 1 system with multiple stages of jobs?
scarlet inlet
RooBubba I am not a UECS expert, but at an abstract level, splitting responsibilities is always better and if you write the code right you shouldn't have any overhead or minimal
amber flicker
So, my opinion is something (very roughly) like 1 system per set of data you want to transform. Within a system you can chain multiple jobs. If CalcUpAxis & CheckCollisions all iterated over exactly the same data I'd maybe make it within one job for performance. It really depends but I wouldn't overly worry about how expensive running multiple systems is.
wary anchor
current example of how I'm processing input and turning that into movement
amber flicker
(I believe that right now on mobile, many systems can be a perf issue but my understanding is it's being actively addressed over the coming months)
wary anchor
It kinda feels like I've gone overboard with that and maybe they should be combined somewhat
amber flicker
It's always a balance.. I personally have multiple related components within a single file and sometimes multiple systems within a file. I don't like hundreds of files but I'd say what you have above follows quite closely the example Unity have tried to set
wary anchor
Once I've got this first atoms-near-players prototype written, methinks it's refactor O'clock
amber flicker
I'll be surprised if you don't re-write it 10 times 🙂 - I'm on my nth rewrite
scarlet inlet
modularity always wins in my experience
although there is a balance like Timboc says
the trick is to not early abstract
mint iron
maybe this will help u seb, its hacky but u can do ref edits in jobs with it if you want. https://github.com/jeffvella/UnityECSOffsets
scarlet inlet
do what is more comfortable and then abstract/modularize only if needed
amber flicker
Oh wait that's what you wanted to do @scarlet inlet ? my bad I think I misunderstood - @mint iron would you expect much of a speed difference avoiding GetComponentDataFromEntity?
wary anchor
I could make a NativeMultiHashMap for only the atoms - calculated once at start and then readonly. That would contain the hashed positions of the atoms as keys and the indices of the atoms as values. That way I avoid putting all the grid positions into their own chunks, and can play around more with the spacing of the grid/cell
but it feels dirty putting that data into a system... maybe a singleton entity's component?
scarlet inlet
yeah it's OK I am not sure if I explained myself, but surely I don't want to hack 😄
mint iron
@amber flicker GetComponentDataFromEntity is not really a problem, but reading and writing the value can add up if you're working on a lot of data. whereas the ref version can have its members directly accessed / blind written to.
I think perhaps they're deliberately preventing refs being used throughout because it makes it harder to determine if the user is intending to read/write and that impacts their change system.
scarlet inlet
that's an interesting point
mint iron
for example i did an experiment for @scarlet inlet's FasterDictionary in Svelto ECS here https://github.com/jeffvella/NativeFasterDictionary and you can easily see that changing nothing other than using refs makes a big difference if you're working in the order of 100,000's of interations
scarlet inlet
(the check read/write), but doesn't justify the current impleemntation
cool stuff
why do you think native dic is faster?
wait you did your implementation?
I am going to copy that
😄
I actually need it soon
mint iron
its not thoroughly tested 😄
my native version is slower than the managed one, exccept if you put it in burst or use refs
scarlet inlet
i continue the discussion in private as it is off topic
mint iron
kk, good point
amber flicker
Nice, thanks @mint iron
coarse turtle
@low tangle just took a look at your network buffer copy technique, I did the same for my network buffer in my game haha 😅
low tangle
:)
can save a lot of moving around
byte[] FixedBodyPoseBuffer;
var bodybuffer = EntityManager.GetBuffer<BodyPackBuffer>(message.PackBuffer);
unsafe
{
fixed (void* ptr = FixedBodyPoseBuffer)
{
Buffer.MemoryCopy(bodybuffer.GetUnsafePtr(), (byte*)ptr + 6, FixedBodyPoseBuffer.Length - 6, 370);
}
}
this is my send @coarse turtle
wary anchor
Getting stuck at the first part of my testing, trying to set up a NativeMultiHashMap. There are 33005 entities returned by the query with AtomTag and Translation components as reported in the editor.
Not sure what I'm doing wrong, would anyone mind having a look please/
low tangle
interesting choice of doing it once on startup
I dont think I've ran many jobs in oncreate
wary anchor
oh dear, interesting
wary anchor
The thought here is only that I need this calculated once and once only, but to have the data accessible thereafter in an easy to access form, is this not sensible?
low tangle
so far not seeing anything wrong, since you are not running this every frame theres no need for this to be a job component system
wary anchor
it's the first part of the system, I was then going to iterate over these data in update
low tangle
GetEntityQuery has a option for component types directly, its nicer and quicker than writing entityquerydesc's
wary anchor
but can't get the first part to work, debug log gives: AtomGridMap KeyData:
END
low tangle
add a OnDestroy() and dispose the hashmap in it
otherwise your editor will complain about not freed memory
probally already is throwing a warning
you are using the concurrent interface AsParallelWriter then only running one job ScheduleSingle
wary anchor
doh
wary anchor
the gridID should be the key
low tangle
there should be a concurrent index value first before key & value
handshakes = GetEntityQuery(
ComponentType.ReadOnly<ServerLogic>(),
ComponentType.ReadOnly<PlayerID>(),
ComponentType.ReadOnly<PlayerEntityRefrence>(),
ComponentType.ReadOnly<HandShakeMessage>(),
ComponentType.Exclude<ServerSend>()
);
I usually format my query's like this
wary anchor
ahh that's what I was searching for in my other systems, thank you
wary anchor
I used those in my other systems when trying to do this, but this time was mostly following the boids sample so still have some of the older style code!
low tangle
👌🏻 no problem
wary anchor
InvalidOperationException: The NativeArray has been deallocated, it is not allowed to access it when I do protected override void OnDestroy() { AtomGridMap.Dispose(); base.OnDestroy(); }
low tangle
oh, make sure you have a if (AtomGridMap.IsCreated)
wary anchor
hmm, the implication is that is hasn't been created?¬
low tangle
yup, but you still need to dispose of it if this system is ever destroyed
wary anchor
yup, makes sense!
low tangle
its just good practice to always clean up any allocator memory
like writing c code, mallocs & frees go together / go like pairs
wary anchor
totally down with that, just flustered by the new approach!
low tangle
yeah no worries, took me awhile to get used to actually managing memory again
its really nice once you get the hang of it, because you can choose exactly when to get, and free it
wary anchor
so now I have this. Still not getting any keys in the hashmap tho https://hatebin.com/rzknqpoimv
low tangle
one of the things there isn't really a tutorial for is chaining jobs, with a transient native memory being used between them, its a really good trick
let me grab this and test it in my project and figure out whats up
will report back in a sec
wary anchor
👍
low tangle
pretty sure your data is not created before that system is
able to add just fine with this modification
wary anchor
Ahh, right, so I need to create this system after the authoring process, is there a n attribute for that?
low tangle
you can see right at the top I just create the world data right there so I know it exists when the job runs a few lines later
wary anchor
yeah, I see - thanks for this. So I'm creating the entities after reading data in within a monobehaviour at the minute
low tangle
if its a one time thing before the game really starts then you could put it into a init location, but if it can be destroyed or created you would be best to just create the hashmap on demand in this system
ah yeah
awake and start are both waaaaay later than when systems are created
wary anchor
right, so I should put this in a public method and call it once the data processing from disk is finished
low tangle
well it might make more sense to just create a event entity that signals to update the grid after the grid exists
plus you can use that to just update whenever later on
wary anchor
ahh good call, I'll have a crack at that. Thanks for all your help and time, @low tangle !
low tangle
alright no problem
lmk if you get stuck, I'll leave this file open for testing if you need me to look at something
wary anchor
Unforunately you already know me too well, it seems! 😛
low tangle
:)
wary anchor
Sigh, I had to fix a bug I introduced when renaming a temporary script, so have only just been able to start looking at this. I'm not sure of the process you're referring to for the events - there seem to be lots of conflicting approaches on the unity forums about how to structure events. Would you mind giving me the minimal (okay maybe a bit more than minimal!) pointer so I could read up some more and play about trying to achieve it please, @low tangle?
low tangle
sure, component system, or job component system?
job requires that you use a ECB to bring any changes back to the main thread
wary anchor
how would you implement this event? Monobehaviour finishes a job, creates the entities, now I need to signal somehow for this job to run to calculate the grid hash
low tangle
in your monobehaviour, you create one final entity which is the signal
wary anchor
okay
low tangle
that entity would have the signature of { UpdateLookupTable } which is a empty tag IComponentData
and in your system that has the table, has a job or a query that runs on { UpdateLookupTable }, promptly deleting the signal as the first thing they do so to only run that frame once
I can write a example if you would like
wary anchor
yes, I see, so it's basically running nothing on update except for the frame that this entity with the component tag is created
low tangle
actually its update wont even run at all if you do its querys right
wary anchor
I'd like to have a go before getting the right answer, please! 🙂
low tangle
no problem, have you taken a look at the entity debugger?
wary anchor
bingo
low tangle
so a event system will never run, unless there is one in its query
wary anchor
I GOT ONE RIGHT
low tangle
meaning you can stack up thousands of signals, and only respond once a frame if you would like
wary anchor
that is pretty beautiful
low tangle
or respond to every single one
wary anchor
Right, I'll have a crack at it. Can't thank you enough for holding my hand through this!
low tangle
:) my pleasure
wary anchor
I think I have it:
https://hatebin.com/drgthjsmfp
but I've also somehow managed to get Internal: JobTempAlloc has allocations that are more than 4 frames old - this is not allowed and likely a leak (and it's unrelated as I still get it when I comment out my AtomGridUpdateSystem completely). I have no idea what I've done to get that, how do you debug these? I tried the "TLA_DEBUG_STACK_LEAK" preprocessor symbol and enabling full stack traces but can't find where this info is actually presented! I know I haven't yet ordered my systems properly so it could be that...
Of course there's also a warning about the Allocator.Permanent native hash map. Is this the wrong sort of container for these data? I'll need to access it from another system (unless I bloat out this system with all the subsequent player-atom logic)
pliant pike
that could be just a general Unity bug I got that before and just ignored it
See if it happens when you restart Unity
wary anchor
I'll sort out the order of my systems and make sure they're in the right groups. I had put it off for some unknown reason but it needs doing. Will restart tho good shout 😄
amber flicker
Once you get one of those warnings, it will remain until the editor is restarted
low tangle
you should restart, pretty sure you leaked a few containers much earlier
RequireForUpdate(EntityManager.CreateEntityQuery(typeof(AtomGridUpdateEvent)));```only put this in oncreate when you make the query
only make query's inside of oncreate as well
its costly to make them
wary anchor
That makes more sense
low tangle
oof this one too
var atomEntityQuery = GetEntityQuery(ComponentType.ReadOnly<AtomTag>(), ComponentType.ReadOnly<Translation>());```var eventEntity = entityQuery.ToEntityArray(Allocator.TempJob);
//Process 1 per frame right now let's get it working
if (eventEntity.Length > 0)
this isn't quite right
you don't have to gather up the entity array at all, (thats internally a jobchunk that itterates and gathers all entity's into a native array<entity>)
if you are just responding to a event, you just want to only run always when the system runs
no need to check at all how many events
wary anchor
okay, working my way through those. About the hashmap - where's best to store this?
low tangle
persistent is fine, public field if you are going to share it outside of this system
wary anchor
if I were to make it static, for example, would i be strung up from the nearest lamppost?
low tangle
then in other system's that you would want to access it;
in their oncreate grab a reference to that instance of the system using World.GetOrCreateSystem<thatsystem>()
avoid static
hidden coupling and it worms into everything keeping systems locked down from easy replacement or copying into new projects
like this
now you have a very clear, at the top (easy to find) link of this system to that system
this also means, if you create a new world, and add your other system into it, and this new one that needs that hashmap from it, they will auto link to each other
low tangle
and then bam, two different worlds, with both working flawlessly without writing to the same static array
wary anchor
Nice and clean. It's definitely a massive pain to get into, but I am definitely starting to really like this process!
low tangle
make sure you check for and dispose the old version right before you create a new one
just like in ondestroy
wary anchor
ooh yes now it can be updated at any time
low tangle
otherwise the next time you signal to update the hashmap you will leak the old hashmap
coarse turtle
Yea^
low tangle
copy paste of line 43,44 will do the trick
oh and you forgot the require for update up in the on create
wait no
its there
sorry
I always go, query query, then requires
tripped me up for a second
wary anchor
I'll switch to the way you do it, this is my first one it's good to set the consistent way from the start 🙂
low tangle
oh and you can reuse the exact same EntityQuery object from the GetEntityQuery() method
which makes sure the exact same query will light up the event query and the require
wary anchor
RequireForUpdate(atomGridUpdateEventEntityQuery);
low tangle
also you used the bulk DestroyEntity command that takes a query, good job 👌🏻
yup that works
wary anchor
Brilliant! Thank you again 🙂 🙂
tidied it up a bit, next step will be to use these data! I feel much better arms to tackle that now :)
https://hatebin.com/eplfdiymyi
low tangle
its pretty slick isnt it?
wary anchor
arms* yea I meant armed!
low tangle
you now have on demand, super fast to update, hashmap to look up any cell
add a few zeros to your grid and see how big you can go
bet its insane
wary anchor
heh well I currently read in atoms from protein crystal structures - there's a limitation on the number based on some archaic text formatting issues, but before in OOP I could get to about 6k atoms, this 33k atom file loads up and runs like butter
amber flicker
haha yea or just load the file 100 times 🙂
wary anchor
Got that open in my browser, started the 1st chapter already 🙂
I currently read in a text file and parse it with lots of regex (here's an example: https://files.rcsb.org/view/5N93.pdb).
coarse turtle
https://en.wikipedia.org/wiki/Multiple_buffering definitely employ this in a DOD workflow, it's quite nice to ensure read safety 🙂
low tangle
oooh thank you
coarse turtle
np 👍
low tangle
oh simple enough concept, just like a framebuffer double buffer
keep quick access to a older version of the data if it doesn't matter being the latest version
coarse turtle
yep
I was using it in Vulkan and then I figured it would work in Unity's DOTS workflow
low tangle
good idea, I'll add that to my notes to use asap
yeah makes sense
position data, or my body data might be a good place
will keep it front and center and see where I can apply it
coarse turtle
btw does anyone happen to know if it's possible to stick a native container in an IComponentData struct? Would the container have to contain a ptr or something?
low tangle
I've never tried but its something I want to look into
I think that's going to be working around the safety settings possibly
going to try it right now
coarse turtle
Yea...I'll start tinkering around with it, I've been working on a linear quad tree implementation and I'd like to just store it as a component on an entity
wary anchor
Would this be more robust for lookups that occur during an update (thinking along the multiple buffer line):
var temporaryAtomGridHashMap = new NativeMultiHashMap<int3, int>(atomCount, Allocator.Persistent);
var atomGridUpdateJob = new AtomGridUpdateJob
{
HashMap = temporaryAtomGridHashMap.AsParallelWriter(),
GridSize = GridPositionCalculator.GridSize,
};
var handle = atomGridUpdateJob.Schedule(atomEntityQuery);
handle.Complete();
if (AtomGridMap.IsCreated)
AtomGridMap.Dispose();
AtomGridMap = temporaryAtomGridHashMap;
temporaryAtomGridHashMap.Dispose();
return handle;
not really needed, unless you want to make sure there's at least last frames lookup's in there
completing the job isnt very good for performance
wary anchor
fair enough 🙂
low tangle
basically the same as doing all the work on the main thread
coarse turtle
yeah, ideally, you want to schedule early and complete late
completing is really only necessary if you need the data processed immediately and there's no way around it
wary anchor
gotya 🙂
mint iron
@coarse turtle you could, but you'd have to use something without the DisposeSentinel. So a copy of NativeList/NativeArray with safety removed, or use some of the collections that are already blittable like "UnsafeList". If you just have a NativeList in (just tried) it throws this ArgumentException: BlockArchetypeBuilder+TestComponent contains a field of Unity.Collections.LowLevel.Unsafe.DisposeSentinel, which is neither primitive nor blittable
and you'd have to be careful to keep the ptr somewhere safe so you can manually dispose it later OnDestroy or whatever.
coarse turtle
Yeah, makes sense
mint iron
ultimately its just a ptr and a length, so something like this would be simple enough https://github.com/jeffvella/UnityECSOffsets/blob/master/UnityECSOffsets/UnsafeArray.cs
coarse turtle
Ah thanks 👍
wary anchor
To guarantee safety, you must include AtomGridUpdateSystem:AtomGridUpdateJob as a dependency of the newly scheduled job.
So as I understand it, in my new system, I need to ensure it doesn't run concurrently with the atomGridUpdateSystem. I have a private reference to that system, but not sure how to get the relevant job handle to use as the dependency for the new system's job.Schedule
mint iron
are you using a JobComponentSystem or ComponentSystem
mint iron
can you paste your OnUpdate() ? or (HateBin or whatever)
low tangle
oh are you trying to run the job outside of the new system?
wary anchor
accessing the NativeMultiHashMap from the other system now
low tangle
you sure thats pointing to that new job?
wary anchor
no
I thought I'd see that there aren't any errors at this point before going through any of the hash data work, but I do get an error with the bare setup there
InvalidOperationException: The previously scheduled job AtomGridUpdateSystem:AtomGridUpdateJob writes to the NativeArray AtomGridUpdateJob.Data.HashMap. You are trying to schedule a new job CalculateDesiredUpAxisSystem:SetDesiredUpAxisJob, which reads from the same NativeArray (via SetDesiredUpAxisJob.Data.atomGridHashMap). To guarantee safety, you must include AtomGridUpdateSystem:AtomGridUpdateJob as a dependency of the newly scheduled job.
low tangle
oooh I see
the funny thing is the quickest way to fix this would be to just .complete it in the other system
wary anchor
I see how it's done when the jobs are in the same system like in the boids
low tangle
yeah you can chain them just fine
this case is a bit different
since you want this to be a global up to date lookup table you need to update it, then access it multi threaded
you will have to complete() the other job to make sure its done before any other systems can touch it
inside of that other system where you run it
you could expose the handle and consume that (bad practice more coupling)
wary anchor
To be fair, it's unlikely that I will be updating that table after its initial creation
low tangle
yeah I would just have a .complete after the schedule on the update job then
wary anchor
Cool beans
low tangle
it will still be burst and run on multiple cores, but you do loose getting more jobs side by side really getting good performance
tradeoff for making the logic simpler on you, plus its only once in a great while that you update it
wary anchor
it's a necessary setup step, I think it's conceptually fine to block while it works
low tangle
pretty much yeah
wary anchor
ahh that's beautiful, thank you working nicely. Right, I need to go and cook, going to have a stab at using the data later on 🙂
low tangle
👌🏻
mint iron
you could put [UpdateInGroup(typeof(InitializationSystemGroup))] on AtomGridUpdateSystem so that you know it will be done before any of your SimulationGroup systems run (still need to call complete on it though as June suggested)
frigid badge
is there a ready solution for checking if Entity being "seen" by any camera like the one GameObjects have ?
just checking before starting implementing it myself
tawdry tree
So checking if the entity is inside the camera view frustum?
frigid badge
yep
tawdry tree
Haven't heard of anything like that, but if I were to make that, I think I would make a system which essentially takes the camera(s) and make frustum entities from them, which you could then, in turn use wherever you needed. That should scale pretty well, at least on the reasonable assumption that you're not going to have a gazillion cameras each checking if they look at bajillions of entities. But if you somehow get to that point, I suspect you might find easier optimization elsewhere...
frigid badge
yep, thought of making a system that would tag entities that are currently "seen" by cameras, just wasn't sure if there's similar thing already implemented in Unity
coarse turtle
You can project the camera to world space and check to see if an entity is within the bounds of the frustrum
at least that's what I did
stoic mulch
I'm looking to start on the physics of a physics-based game that'll probably take at least a couple years to complete, maybe more. Do you guys think it'd be worthwhile to try using ECS and the new physics engine for it or should I probably stick to the old? The project will be doing simulation of a real-world thing.
frigid badge
Unity team suggests to get familiar with ECS, but use it only to solve particular problems that benefit greatly from ECS, while it's still in preview
ECS works well in conjunction with the conventional Unity, so it won't be super hard to offload some things to ECS
stoic mulch
I'm thinking more about the new physics, like wheels and such.
coarse turtle
Well afaik, Unity has a vehicle example in their physics sample
You can take a look at that for start to see if it's worth your trouble
stoic mulch
Ah yeah just found that
wary anchor
I wouldn't recommend looking at that example as your first foray into ecs though!
winter depot
Something I wish I would have done at first is read the documentation
A few times at that
wary anchor
It's taken me several read throughs, several runs through Mike Acton's gdc talk on the boids and lots of help from the lovely people here, and I feel like I'm beginning - just about - to get a handle on it. If you'll forgive the pun :)
stoic mulch
Have a link for this documentation?
coarse turtle
Hmm when I was first starting out learning DOD in college, one of my professors (who really enjoyed linearization) asked 4 questions to really understand DOD, which honestly I found useful only afterwards
- What is the minimum amount of data you need to operate on?
- Where do you get this data?
- What transformations do you need to do on this data?
- Where do you put this data?
@stoic mulch here you go: https://docs.unity3d.com/Packages/com.unity.physics@0.0/manual/index.html
stoic mulch
This talk btw? @wary anchor https://www.youtube.com/watch?v=p65Yt20pw0g
Thanks @coarse turtle
wary anchor
That's the one. I've watched pretty much all the ecs videos I can find but that one especially with the example code at hand!
stoic mulch
Imma go grab a burrito and dig into that
frigid badge
watched that one too after i found the link in the sources of an example project 🙂
vagrant surge
daily ping http://www.dataorienteddesign.com/dodbook/
winter depot
thanks @vagrant surge
If you aren't busy can you have a loot at https://forum.unity.com/threads/ijobchunk-performance-documentation-misunderstanding.728384/ I really want to learn about using the right tool for the... job
After reading some more threads I am thinking that maybe ECB was not the culprit, but not understanding that the debugger shouldn't be used as a profiler of sorts. Someone mentioned that ECB may show spikes, but that is just waiting for jobs to complete before running
vagrant surge
if you want to do "reactive systems", you need to use System State Components
and use them to store the "old" value
then within your filter changed, you check if value != oldvalue, and calculate the stuff then
and yeah, add/remove components is quite expensive in unity ECS
winter depot
Okay, so I was not so off about that
vagrant surge
it performs a "hard sync", and needs to re-arrange entities from one chunk into a different one
its still way faster than old unity add/remove comps, but be careful
why were you adding/removing components? what use case
winter depot
Okay, so say a unit is standing still. Its still hostile against other units so I was giving it say a Targeter component. This component goes into a job that uses a quad system to just search units close to that unit. If its in range I would apply a HasTarget component with the target. This system that finds targets would have an [Exclude(typeof(HasTarget))] for example. Then it would essentially just jump from one system to the other depending if it has a target or not.
vagrant surge
have you though that the Targeter could be a different entity instead?
winter depot
So when a unit of this type is created I don't add a targeter, but instead create 2 entities?
vagrant surge
one entity doesnt have to be one logical object
yup. And link the "child targeter" to the parent somehow
i would recomend system state components, as they allow you to add/delete things better
so basically, you have your fat Enemy entity, with mesh, animation, controller, etc
but for the targeter part
its just a tiny small entity
that points to its parent
this way you avoid having to copy the entire big character
winter depot
Hmm. I will have to look into both of those options. Is it expensive to have a child ask a parent about information? Such as range and other flags that say what kind of other entites can be targetted?
winter depot
Okay, and if say that unit is upgraded and values change, then you would re-copy to the child?
vagrant surge
yup
this is a pretty good pattern btw. You have your fat "big object" character entity, but then you have "peripheral" small, single-purpose entities
winter depot
I knowingly kind of zoned out and skipped reading about system state components, because it sounded like more than I though I would ever need.
vagrant surge
system state components lets you control the lifetime of a external resource
they dont get deleted when the parent entity gets deleted, and then you can iterate the ones that are "orphaned" to clean up
winter depot
Is it cheaper to create/destroy entities than it is to add/remove components?
winter depot
Oh okay so make the systems poolable
vagrant surge
it is indeed cheaper
in fact, adding a component is roughly the same performance as deleting the entity and creating a new one that has that component
winter depot
That is brutal
vagrant surge
entity addition/deletion performance depends on how many components you have
@winter depot its still STUPID cheap
you can create a million entities in 1 frame no problem
untold night
Object pooling is built-in, in a sense.
That's what a Chunk IS
vagrant surge
indeed
winter depot
I guess I mean poolable in the sense that a unit dies but the targeter remains to be reused by a new entity unit that may be created.. or was I reading into that wrong?
vagrant surge
not exactly
you could do that, but its pointless
if your Targetter is stored in a system state component, when the parent entity gets deleted, the targeter becomes "orphaned"
you can query for orphaned targetters
and delete them
winter depot
Okay, so not queried for reuse, queried for deletion
vagrant surge
or better
remove them from quadtree or whatever
thats what system state components were designed for
@winter depot yeah you just delete and then create again. Its not worth the effort to try to reuse
even if you could do that without much trouble
you can query for "entities need a new targeter" and "orphaned targeter" and reinit them by reuse, but its probably pretty unnecesary
winter depot
Yeah, okay. That's really awesome man I am excited to switch a few of my systems to this. As always, you are a beast. Thank you
safe lintel
im making a simple sprite render system that uses drawmeshinstanced, for things like bullet hole decals, blood splatters etc. generally they'd all be the same except for the materials and mesh(quad) sizes. would I be better off doing this as multiple systems or trying to get them into one system with some sort of management inside of it? its the tackling it as one system that im having difficulty wrapping my head around, not sure if its better to try to do some sort of generic system or just bite the bullet and make lots of separate systems?
stoic mulch
I don't think I can work through that entire presentation, he's talking too fast and going too much into the under-the-hood stuff before just showing us how it works in code with how we'll actually work with it.
wary anchor
It took me at least 2 watches before I sat down today with a notepad, paused it and took notes during the boids walk through while looking at the code to follow what data was where and what was being done with it. I spent about an hour just on that 20 minute section of the talk!
stoic mulch
I'm going to stick to some simpler tutorials and just learn how to code with it first heh
This is a lot easier to digest imo https://www.youtube.com/watch?v=WLfhUKp2gag
A year older but same info
wary anchor
yeah, build up to the boids! I put off going back to it because I'd failed to understand it twice already, but it's worth sticking with it and committing to go back to that sample set!
turbid sundial
Still very new to ECS - and I'm wondering about system design for hex (civ-like) map design. In essence, I want a hex entity per hex. That's cool. But what should handle creating, updating, deleting these hexes, a system - right? So as I move around my world I want my HexSystem to get the position of the camera, determine what hex entities aren't visible anymore, delete those, and create those that should be. Should all of this live in a HexSystem, should HexSystem even know about the camera position?
turbid sundial
I guess I'm asking with the ECS pattern exactly how much responsibility do the Systems actually have
dull copper
low tangle
in a ideal world, exactly one scoped to a single piece of data they need to look at and maybe update @turbid sundial
you probably need to break up what you need to do into multiple steps instead of a single large operation like that.
first start with a system that knows of the camera and simple generates based on the angle the 2d axis aligned area of whats inside of the camera clip bounds (since its a 2d game)
then a different culling system can use that area component to bounds check anything you want (maybe make a Active component with a bool Active in it, or to be more existence programing, simply remove the active component when its not in bounds)
a totally different system runs before the camera-to-area-system which is your camera-movement-system
camera-movement-system
camera-to-area-system
mark-inside-area-system
~more gameplay systems know that you know whats inside the world
Thanks for the thoughts!
rain cedar
Hi! What recent materials would you recommend to take a grasp of real life use of Unity ECS in games? Right now the best source I found (and it was updated to recent changes/deprecations in ECS) is this course (even the preview itself is better than many other deprecated stuff I found elsewhere) https://www.gamevanilla.school/courses/create-your-own-single-player-ccg-with-unity/lectures/6285053
tawdry tree
Honestly, I think that taking a mechanically simple 2D game, something like a sokoban or perhaps a tower defence , and making that from the ground up might be one of the best ways to learn.
How to build a map in mon-world and convert it
How to control a player on a 2D grid (or place a turret for TD)
How that interacts with the next mechanic (box to push, an enemy to aim towards)
Next step (box goal zone, shooting)
etc (victory conditions, hitting enemies, health, resetting levels, more levels/enemies)
Come to think of it, an idle/clicker/incremental game might be good too, the most basic version is a button and a counter, then you can buy an autoclicker, then that gets more expensive as you buy more, alternate resources, recipes, etc. Exceedingly simple (mechanically) to set up the basics, but you can build and build on that.
safe lintel
even just translating a section of a non ecs game to ecs would teach you tons
tawdry tree
True, but there might be a ton more complexity there. Though, if you have a game lying around that has some simple parts...
frosty siren
To let units in my game interact with another ECS objects i use ComponentDataFromEntity/BufferFromEntity. I feel like it's very bad and i'm doing something wrong, but i can't see another way. How u make interaction between objects with ECS?
vagrant surge
@frosty siren nothing wrong about accessing the components of another entity directly
low tangle
if you cant get around remote access its totally fine
you want to avoid it when you can, but its not the end of the world if you can't get around it
winter depot
Is there a magic method of some sort that groups a child to a parent Entity? I have just been using my own Component named Parent..
winter depot
I found something in the documents that was like 20 pages long
vagrant surge
oh... that
thats overengineered to oblivion, but you are only interested on the parenting/child part
safe lintel
that shows all the possible combinations and effects but it makes sense if you read it
winter depot
I think I am actually doing a few things wrong tbh. The only part of my game not ECS now is a single wrapper component with an animator on it for each GameObject. I have one job that syncs all of my gameobjects transforms from my ECS translation, but I have seen people mentioning a EntityToGameObject transform thing anyway that sounds like it may automatically do what I am manually doing
safe lintel
also there is a built in component named Parent(and set the entity value accordingly) and LocalToParent, add those things to a child entity and it will be parented to the parent Entity
safe lintel
there is CopyTransformToGameObject and CopyTransformFromGameObject that either sync a transform's data to ecs components(transform.position & transform.rotation to translation & rotation) or the opposite
winter depot
Is that something that is automatically syncing or is that just something that instead of what I am now doing manually Translation.Value = x.
safe lintel
i cant recall if it does scale tbh
not sure what you are doing manually but you add those components to a hybrid entity and you can sync that data automatically
yeah those things are probably what you want
winter depot
Alright, I will look into hybrid options. I plan on moving my animation to ECS anyway eventually, which will eliminate the need for those gameobjects altogether, but I still want to learn how that works.
Thanks @safe lintel @vagrant surge
safe lintel
also note that ComponentProxies are deprecated (TranslationProxy, CopyTransformToGameObjectProxy etc) and shouldnt be used
frigid badge
so far ECS doesn't index by component data, only by components themselves, right ?
making a system to track Entities' visibility that include a tracker component, and deciding between putting the visibility status as an additional component tag, or as a data inside the tracker component
Can I query only the visible entitles if I go with the field inside the tracker approach ? Or i should use tag components for that
vagrant surge
@frigid badge have you seen the mike acton talk about culling in Megacity?
frigid badge
nope, i guess it covers something similar ?
frigid badge
cool, gotta watch
vagrant surge
what they do, is that they store the chunks that are in-view in an array
frigid badge
although it's not about culling in my case
i just need to control spawn rate based on how many of entities of certain type i have on screen
vagrant surge
you can do what mike acton did, and do it per-chunk
keep an AABB of the entire chunk, and if the AABB is either full-inside or full-outside, then its visible or not. If its midway, you can check aabb of the entities
frigid badge
yeah, that sounds like an interesting video
winter depot
So I must have missed a step here. My game objects now have GameObjectEntity script being placed by AddComponent<GameObjectEntity>();. Then I am adding CopyTransformToGameObject. At that point the system is running in the debugger, so something must be happening, but I don't see transforms changing. From what I read this is using LocalToWorld. Is it odd that my LocalToWorld values 1-3 are wrong and the 4th is the correct one matching my translation?
safe lintel
Sorry, shouldve added that GameObjectEntity is also part of the deprecated proxy workflow. Its recommended to use the Conversion workflow
winter depot
Okay, I will look at https://forum.unity.com/threads/new-subscene-converttoentity-workflows.638785/
safe lintel
the samples repo in the pinned messages has some simple examples of using it
winter depot
Okay thank you
safe lintel
it gets confusing though because some of the physics samples still use GameObjectEntity, imo the hybrid workflow is not really well explained by unity, i guess too much stuff is still in flux for any documentation
winter depot
I tried the workflow examples but it was adding a ton of entities and linking them. For now I am just going with ```csharp
GameObjectEntity.AddToEntity(entityManager, obj, entity);
entityManager.AddComponent<CopyTransformToGameObject>(entity);
Looking at https://docs.unity3d.com/Packages/com.unity.entities@0.0/manual/entity_iteration_foreach.html - It seems to infer that ComponentSystem should be used sparingly - because it is forced to be on the main thread. Every resource I'm reading for ECS bases their systems from this class though, should the default be JobComponentSystem, are there other alternatives (just make whatever systems extend ComponentSystemBase)?
winter depot
JobComponentSystem
"For maximum efficiency, use a JobComponentSystem and an entity command buffer."
turbid sundial
When creating a large number of entities, you can also use a separate World to create the entities and then transfer those entities to the main game world.
What's the advantage of this?
low tangle
Tooling to make it super cheap to bring the final result into the real world
And since it's a isolated word, you can work on it much longer than one frame
That world can stall for as long as it needs without the main thread stalling
turbid sundial
Can't jobs do exactly that, as well?
analog tangle
No... because you need to render a real world.
Think of the "isolate world" as just a buffer that you can load memory into.
At least for your purposes.
Then, once it's in the right format and structure in the isolated world, just memcpy from the isolated world to the real world.
low tangle
the final operation is this
which memcpys all the data, as it also remaps entity id and indexs to match the target chunks
this is what the streaming system in mega city (now the subscene system) uses as the final step
turbid sundial
That's interesting, thanks for the response guys
low tangle
their flow was roughly
main thread:
create a request to load a world inside of a loading world
loading world:
responds to that request, starts the async load from disk inside of a job
takes about 100ms to get the data flowing in from the file (would be a long frame if you waited)
builds and keeps creating entities inside a long chain of jobs
tail end of that job creates the final signal that its done
main thread sees that signal and does the MoveEntitiesFrom on that world copying everything over in 1ms
so the only main thread stall or hitch when loading is that final 1ms
which is easy to budget in when you look at it
turbid sundial
There's still something that's bugging me about how to structure a procedural world (specifically, hexes). I have a pretty straightforward system in old-Unity that I've been trying to convert to ECS to learn. I believe some of my struggles with the original system were due to running everything on the main thread, even though I tried to at least offload the noise mapping to c# threads. The flow was something like
- Check position of camera, if we're a certain distance away, update all my hexes
- For loop through x ** y of hexes, if my "world array" already contains a hex at that key, skip it - otherwise run my procedural code on it on a thread
This was cool - but obviously the main bottleneck here was that when we hit that check there'd still be a small but noticeable hitch while we build a bunch of new hexes (and actually, instantiate and remove gameobjects as well). I felt like this was about as optimised as I could go.
ECS rolls around. Cool. Sounds like I can use this to run all of that generation off-thread. In my mind it works something like this
- HexCellSystem (ComponentSystem) - OnCreate makes a NativeArray of Hexes. OnUpdate runs through the same loop of x * y, and queues up procedural generation for hexes we care about.
However it feels like that has the same problem, since that ComponentSystem still runs in the main thread. Those Hexes also need to be rendered, perhaps that could live in a HexRenderSystem separately - to decide which ones should/not be rendered, but I still feel like I'm missing something here with regards to the ECS pattern / design. The reason I ask about separate worlds is because that at least seems like an obvious way to pass off some of this load to another thread
Rubber duckying out loud kinda helps though 🦆 makes me realise that's at least definitely the wrong approach 😄
low tangle
well not exactly, but you do need to think about it a bit differently
starting with the first, do you need a array of hexes that only that system knows about?
sounds like other systems might want to look at those
like the hex render system
instead of a hidden internal nativearray of them, why not make them individual entities with hex components on them?
turbid sundial
I explained that step poorly, that's actually what I'm doing, I was just using nativearray to create a whole bunch of them at once - but they are created as entities
low tangle
gotcha, just making sure you are on the right basis
turbid sundial
Appreciate it 🙂
low tangle
the next thing I'd say is think about the individual transformations of the data
turbid sundial
hexes themselves are essentially static, with some simple float values for height/moisture/etc. - their data once computed is very lean
low tangle
yep thats good, already have them down to the minimal
but you might even go further even
I'll see if I can ecs this
`- Check position of camera, if we're a certain distance away, update all my hexes
- For loop through x ** y of hexes, if my "world array" already contains a hex at that key, skip it - otherwise run my procedural code on it on a thread`
turbid sundial
I think some confusion lies between deciding on the types of systems that handle these parts, that's really a blocker for me moving away from oop land
and really, what do I actually want to run on the main thread
low tangle
DistanceSystem, takes in the camera position at the start, passes to a jobforeach and updates all Distance components
almost nothing
just want to schedule the jobs and move some arrays around into the jobs
thats the ideal
real world, some will be that clean, lots will be main thread componentsystems (not job) ones that are 'hybrid' and manage some gameobjects as well to make up for what isn't in ecs yet or is not good enough like physics and the hybrid renderer
turbid sundial
that's a good note
low tangle
I just did a quick test yesterday of unity physics (ecs) and the hybrid renderer in my soon to be released game, and the physics work, but the renderer does not at all in a build
likely because I haven't fully switched to hdrp
physics is going to be incompatible with my users modded worlds, so no dice on that really - I just wanted to use it for a interaction system inside of ecs but even that is a bit overkill, so I'm moving back to a hybrid approach like rendering that I already have managing real gameobjects (pooling is really easy with this btw)
turbid sundial
so, if you wouldn't mind, could you elaborate on that a little. DistanceSystem is a ComponentSystem - which does actually run on the main thread? Or is it's role to solely get the cameras position and update any entities which have the Distance component?
turbid sundial
what might a distance component be attached to?
low tangle
and in the jobs public fields is a float3 cameraPos
anything
you no longer care what the distance sync component is, just that it has a position and the camera has one. you logic in that system now gives meaning to that distance component. that it will always be up to date with the distance to the primary camera in the scene
I forgot that the distance component either needs to have its position in it, or one next to it - your choice
so any entity with a {Distance, Position} is what that system loops over
very simple, very easy to understand and write
turbid sundial
Sure, I think that makes sense
cutting up my hex generation seems to be the meat of the problem, though - particularly figuring out what should render, what's too far away to care about and can be removed, and actually running my generation algorithm to get its floats
low tangle
turbid sundial
you're too helpful 🙇
low tangle
the idea with ecs, is to break it down into very simple little combos of things to make a behavior or logic that you care about happen
overall you want procedural generation to happen on several conditions. which means you need to figure out what each one of those steps is. then encode those into a job chain, or a new system that does only that step of the operation
so one of the things you said was a simple step we can now do with this distance component
what's too far away to care about and can be removed,
later on, after you have hexes in the scene, create them with a {Position, Distance} along with whatever else is on them, and have a
ForEach {Hex, Distance}
if (Distance > Threshold)
Destroy(entity)
Ohhh, yes
It didn't really make sense until that part
Coming at it with distance as a component didn't really make sense until then, seemed like a very weird angle
low tangle
when you think about it like this, you actually get chances to actually reuse code when you think about it
turbid sundial
Thank you very much for indulging me
low tangle
now any time you need to do something based on camera distance, just slap a position and distance on it
turbid sundial
Yeah exactly, for the rendering I imagine I'll chunk them together - which can ultimately also have a Distance component and be conditionally rendered
low tangle
yup
simple and fast [BurstCompiled]
its also the faster non squared distance between the two
turbid sundial
so this makes a ton of sense for cleaning up entities, but for creating ones that aren't yet created - it still seems like I just want to do a big dumb for loop and make a bunch of entities
which negates gains made from the Distance approach
because I can't filter through entities which aren't created yet
low tangle
yup
you don't want to use this for that :)
instead you approach this system different
whats the conditions for creating a new hex tile
turbid sundial
just that it's within range of my camera, say 1000 units
turbid sundial
it is, they're roughly 20f in size (vertically a little shorter)
low tangle
alright, so for all grid locations at a 20f cell size, we need to create a new hex tile entity if there isn't already one in that spot / exists
step one is the last thing, first we need to be able to check if there is a cell at that grid location
first job:
ForEach<Hex, Position>
add to hashmap
second job:
schedule job with data:
cell size = 20
for n range in a 2d grid
JobParalellFor
turn n into postion based on cell size
if hashmap does not have position
create new {HexRequest, Position}
in a different system, you process all HexRequests, doing the actual random generation, then create the real hex tile entity, removing the request in the process
turning a 1d n into a 3d position is just a matter of defining x,y,z ranges you want to divide and modulus the n on and then multiplying by the cell size.
if needed add float3 offset to move the whole 3d range
turbid sundial
yep, that math I have down 👌
low tangle
perfect
turbid sundial
little confused by the purpose of job #1 here
low tangle
job one creates the lookup map of all existing hex tiles so we don't create one that already exists
NativeHashMap<key, value>
turbid sundial
ok so - this screams a little redundancy to me - since I wouldn't want to be doing this OnUpdate
turbid sundial
but even that seems like wasted cycles, if the camera is idle for example
low tangle
the bloids demo does this every frame for a million positions/fishes
turbid sundial
😮
low tangle
its very fast, but if you wanted to skip when the camera isn't moving, simply detect that and dont run that system :)
turbid sundial
yeah, I guess I could just check if we're at least 20f from the last position
easy optimization
low tangle
yup
lots of those once you get into this world and mindset
theres also some scaling changes you could do to this algorithm
I'll explain why, but think of the before first not what I'm about to tell you - because memory moment might end up being more expensive and you need to think about this and profile
turbid sundial
I guess I was very premature asking about offloading this work to another world as well 😄 but it's good to be aware of for potential future optimization 👍
low tangle
but here goes, in this, we assume n to be the number of tiles the camera wants to create because they are in range, and m to be the number of currently loaded hexes we have to compare against
n is actually constant and can be much much smaller than m
so most of the time we know that n < m
and since m is being added to the hashmap every frame, our setup actually gets slower the more total loaded entites - m - that get created (its still very fast 1mil+ is nbd).
so in the ideal world, we would want to only hash the smaller consistent set of tiles and reverse the comparison so it has consistence performance. after all the amount of work isn't really changing
doing that isn't always easy though
turbid sundial
no, that makes sense
low tangle
one way you could go about it, is to do a range check on the tiles going into the hash map instead of adding every single one. this allows you to loop very fast, but only consume the memory needed for the smaller set of m that are within the area we are going to check against
thats a simple mod to the very first job, just needing a tiny bit of min/max values to check the position before adding to the hashmap in the first place
job 1 updated:
bring in extra data for min max
ForEach<Hex, Position>
range check to see if its even going to be checked against later
add to hashmap
absolutely
low tangle
you still iterate the entire m, but only hash and keep around the smaller set of it that will be important
its very fast to look at lots of things, just make sure its ReadOnly so other jobs can read from at the same time for good performance
turbid sundial
that's a good note
you've been a huge help, I have one final question - feel free to go to bed if you're in the same timezone and ignore me though!
actually, I think I've figured it out
maybe I'm starting to think in DDS 😁
turbid sundial
thanks so much for taking the time!
low tangle
yeah no problem, happy to help when I can
turbid sundial
this has been a huge needle mover
I felt with the lack of documentation I was moving at 5mph but the last 1.5 hours a lot of parts have clicked
low tangle
both :)
thank you
south birch
Anyone here using ECS without Monobehaviours?
amber flicker
@south birch any specific questions?
Anyone know how the systems are added to the default world? I need to (I guess) manually update all my systems in the correct order at edit-time.
mystic mountain
Anyone have any good reading etc on how to look for how to find what is taking performance on your systems?
This is my profile
amber flicker
@mystic mountain if you expand the main thread area it may give more hints
mystic mountain
I can't do anything about the editor loop taking so much time right?
amber flicker
Profile an attached build instead? 😄 - hard to tell from a cursory glance what's going on - the way your workers have lots of gaps looks like there may be lots of small tasks that have the potential to be amalgamated.. but it could just be the constraints of what you're trying to do
mystic mountain
Looked over 95% over the time spent was from systems from dots, network, physics etc. The other few which each took ~0.5ms something were from using commandbuffer it seems.
amber flicker
yea, looks like there aren't any big obvious targets for optimisation from the profiler - if you need gains I guess you'll have to first check everything's using burst that can, second remove any .Complete calls you can then third go deeper and work out conceptually if there's anything you don't need, can be done spread over multiple frames etc etc
amber flicker
I'm guessing there's not much chance of getting burst to compile for in-editor at edit-time?
pliant pike
If I use GetSingleton() do I have to use SetSingleton to write to it?, it seems I cant use it in a job
frosty siren
Is there a way to store a ref to an element from ComponentDataFromEntity? I mean i don't want to call [] operator twice if i need read CDFE first and then write to it
untold night
There's no ref support on NativeArrays and CDFEs
there were plans, but apparently there were performance issues doing it that way.
You can get usually get a Pointer if you know what you're doing and feel comfortable turning off the safety system
turbid sundial
in an IJobForEach<> I'd like to add a component to the current entity, is that possible?
winter depot
You will need an entitycommandbuffer, which means you will not be able to run burst on that job
Best to break the job into 2 if that is the case
turbid sundial
Interesting, so in my current IForEach selection query, I have entities which have been processed assigned with a Processed tag. I was planning on IForEach excluding these, working on those which haven't been processed - and then adding that component to the entity. I assume it's instead faster to have that as a boolean property and just loop through every entity?
winter depot
A few ways to do that sort of thing, but yes having the second job just iterate and skip based on that tag having a completion bool would work. I am not sure its optimal though?
Maybe create a process entity that instead has a buffer of ones needing processing, or create a bunch of single entities linked to the ones needing processing and just run the ones with that tag.
This way you are not changing the entire chunk and though small, wasting iterations just checking that single bool
Does that make sense?
turbid sundial
The process entity I think does, the second idea seems to fall into the same trap though - those entities would need removing at some point
winter depot
Yes, but removing an entity is much cheaper than adding and removing components
turbid sundial
That I was not aware of, it would still require a command buffer and sync point correct?
winter depot
So you still split that into 2 jobs, and a sync point but it’s much smaller as it only runs on ones that finished processing and need to be removed. This takes almost zero time. The other primary job that links sets the data for the big entity
safe lintel
they do plan to make all ecb's burstable in the future so may want to bear that in mind
turbid sundial
That's interesting 😮
winter depot
That is exciting
frigid badge
hey guys
I have an issue regarding adding a tag component to an entity - LocalToWorld gets reset back to its original matrix whenever i tag an entity with a component. What steps must be taken to preserve its value when adding new components onto the same entity ?
the reset occurs even when another entity gets tagged - whenever AddComponent<TagComponent>(index, entity) called, all entities' LocalToWorld get reverted back to their initial values
frigid badge
so, the only way to fix that was making updates to Entities' Translation component, instead of LocalToWorld component
otherwise, Entities were repeatedly copy Translation into LocalToWorld each time their Archetype was changing
I was under the impression that the right way to move Entities around was to write into LocalToWorld, because that's what they're doing in the examples, granted, in those examples entities never change their archetype
so is it something I'm doing wrong with the Archetype change and writing to LocalToWorld, or was my assumption to move Entities via LocalToWorld wrong, and i should use Translation Rotation Scale components ?
low tangle
basically the LocalToWorld is the result of any and all outside data feeding into it
frigid badge
yep, I understand that, it's just that it also can be written to by any system and all is fine and well until the Archetype change, at that moment, all accumulated writes into it are discarded and it's recalculated based on TRS components
I'm trying to figure out if this behavior caused by my mistakes in handling it, or it is expected
low tangle
I think theres actually a component for uploading direct changes to the localtoworld
or maybe it was the absence of any components other than the localtoworld
wispy walrus
Hey everyone,
Are there any examples (maybe in unity demo projects?) of big complex systems that one can look into as a textbook example of ECS / Job System.
Individually all the things make sense, I listened to the talks, read whatever I could and experimented a bit myself, but I feel like I am still failing to grasp what are the best practices and how would one implement complex interconnected systems. Of course that comes with experience but I just want to have a glimpse of it if there is something like that.
mint iron
I think the Unity Physics examples are some of the better ones, because they have a character controller that can run around a level and shoot, and a car that can drive around. We are still lacking intermediate and complete game examples, i think partly because once you get outside of 'cool i can spawn a 100k things and move them' it becomes apparent how many gaps there are in rounding off a complete solution with ECS.
https://github.com/Unity-Technologies/EntityComponentSystemSamples/tree/master/UnityPhysicsSamples
frigid badge
or maybe it was the absence of any components other than the localtoworld
funny thing is i also had a thought "what if i just remove TRS components during instantiation", but didn't follow on it
I'll check it out rn
yup, worked like a charm
so, if entity has TRS, you have to write to TRS to preserve the transform, or get rid of TRS and use LocalToWorld
or never change the archetype
wispy walrus
@mint iron thank you, that's really helpful
dull copper
@mint iron the car example isn't good ECS example tho
it's like super quickly hacked together thing that does almost everything in monobehaviours
actual vehicle code is in MB
it just uses physics raycasts from it
and impulses
and obviously physics engine itself will deal with the collisions
wispy walrus
what would you recommend then?
dull copper
well, these official samples are still the only ones that are somewhat up-to-date
so, that's what we go with
wispy walrus
And which ones would you consider good pure ecs example
frigid badge
Boids example is pretty extensive, up to date and showcases good real world problem where ECS is a good tool for the task
pliant pike
I dont think there is many good practice full games/projects, as its all still in development, everyones still figuring how best to use it(including Unity)
pretty much however you use it, its going to be more efficient than monobehaviour at least
uncut nexus
How is the action of grabing being made in 2D
frosty siren
I have a very strange error for me. The native container has been declared as [WriteOnly] in the job, but you are reading from it.. This container was returned by DynamicBuffer<T>.AsNativeArray(), which
was returned by BufferFromEntity element access. We can't get BufferFromEntity with WO, only with RO and RW through attribute. Also i have no [WriteOnly] with BufferFromEntity declaration in job struct.
But the more strange thing is that this error doesn't appear every frame, but only rare frames.
When not using AsNativeArray() and iterate through DynamicBuffer there is no problem. Does it mean that AsNativeArray is WO?
mint iron
Sounds like a bug or bad exception description; i'm not sure why it would be treating it as WriteOnly without you explicitly asking for that. I had a quick look at AsNativeArray() and didn't see anything strange there. Perhaps it only occurs sometimes when the buffer is invalidated by running out of in-chunk space and source being reallocated but the new array struct doesn't get the memo. Try settings a large [InternalBufferCapacity()] and see if it still happens.
frosty siren
I find an error in my code: i use AsNativeArray() at start and then iterate through it and try to modify buffer. Will try your advise
So that was a core of error. But why so strange message
coarse turtle
what kind of job interface are you implementing? nvm I misread
mint iron
i usually post about those sort of error messages or submit a bug because its going to keep coming up for other developers and confusing people, best they improve it.
turbid sundial
So there's a RenderMesh component type, with a mesh/material property - but I can't make a job with a mesh/material because:
mesh is not a value type. Job structs may not contain any reference types
So how do I set a mesh / material for an entity? 😮
turbid sundial
Or I suppose I could just use the ComponentSystem
queen prism
Does anyone have any good tutorials about getting started with ECS and getting familiar with the different thought process for learning it? I've tried so many times to jump into ECS but I always get confused and end up straying away from it. I've looked at the pin links and their GitHub but it doesn't necessarily teach me much.
untold night
@queen prism
If you can get ahold of the Data-Oriented Design Book by R. Fabian, I recommend that:
https://smile.amazon.com/Data-oriented-design-engineering-resources-schedules/dp/1916478700/
It's very helpful in breaking it down in a way that the limited amount of Unity Resources can't yet.
queen prism
I'll definitely try to buy it and check it out 🙂 thank you!
low tangle
it was a very good read, I think I'll try to get a physical copy at some point as well
making me really itch to make my own game engine fully data driven style
but who has time for that :^)
tardy locust
What is the ECS way of adding a mesh renderer to an entity?
Because I can't do it as an IComponentData
I wanted to make an Actor Archetype which is literally just something that can exist in the world with a rotation, position, scale and then some sort of renderer so that I could get familiar with Unity's way of doing ECS
So I have this class that I could have some premade archetypes in that are used a lot
public static class ArcheTypes
{
public static ComponentType[] Actor = new ComponentType[]
{
typeof(Translation),
typeof(RotationComp),
typeof(ScaleComp)
};
}```
Then I can do something like this:
```csharp
EntityArchetype actorType = entityManager.CreateArchetype(ArcheTypes.Actor);
entityManager.CreateEntity(actorType);```
But of course that only gives me a Translation, a quaternion and a float3 for Scale.If I wanted to visualise the creation in the world, how would I add a MeshRenderer?
turbid sundial
There's RenderMesh in Unity.Rendering
tardy locust
RenderMesh
Okay so that's the name then?
manic nebula
Quick question, is the ECS worth learning at this point to build a relatively lightweight game if i already know the classic scripting system and if so, is it actually production ready or still missing features?
tardy locust
It's still preview
But you could likely build something with it now if you wanted to
But you'd need to want to seek out the information on places like here, because you won't really find it elsewhere atm
The question is whether your game warrants it as ECS can give you a lot of benefits but might not actually be necessary
Whether you should learn it or not is up to you I guess. Look at it as another tool in the toolbox. If you understand it, you can benefit greatly from it in projects that requires it.
manic nebula
ok, thanks
yeah until documentation gets good with it its gonna be a bit of a task to learn when ive got a bit of a time limit to work with
tardy locust
Thanks for the help @turbid sundial
pliant pike
you need the Hybrid Renderer package to render mesh's in ECS @tardy locust
tardy locust
Yeah I found it. Thanks though @pliant pike
pliant pike
you also need the component localtoworld as well
pliant pike
no
tardy locust
The one in Unity.Transforms?
oh.
namespace Unity.Transforms
{
[WriteGroup(typeof(LocalToWorld))]
[WriteGroup(typeof(LocalToParent))]
public struct Translation : IComponentData
{
public float3 Value;
}
}```I thought this took care of things
pliant pike
you need RenderMesh and LocalToWorld as bare minimum I think
tardy locust
What does the second one do?
public static ComponentType[] Actor = new ComponentType[]
{
typeof(Translation),
typeof(RotationComp),
typeof(ScaleComp),
typeof(RenderMesh)
};```I have this
pliant pike
entymanger.SetSharedComponentData(CentralBank, new RenderMesh { mesh = CentralBankMesh, material = CentralBankMat });```thats how I set the rendermesh
tardy locust
I have not yet tried SetSharedComponentData
I'm still trying to understand how Unity wants this to work
pliant pike
yeah it took me a while to figure out the rendermesh stuff its kind of recent
tardy locust
What does SetSharedComponentData really do? I am guessing it makes entity share data
tardy locust
Why would you do that with a RenderMesh? Is that like mesh instancing or something?
oh
So you give one mesh
But can personalise each mesh if you wanted
Through Materials I guess?
And/Or animations if you wanted to do that?
pliant pike
yeah you have to set the mesh and material using variables and then apply them in the inspector
tardy locust
In the inspector?
pliant pike
you could do also do it by loading in a resource as well I guess
tardy locust
Yeah I'm thinking that is kind of the intention?
But then if a designer wanted to make a level with static elements, would they still just use prefabs?
pliant pike
you cant personalise the mesh for each entity all entity's have the same mesh and mat for that sharedcomponent
tardy locust
Hm ._.
To give a real example
Lets say I am making an RTS and I want to place a building
The building I place is for a specific race
So it looks in a specific way, have specific hp and might even have a team colour because someone else might also be that same race
In an example like that, I might have to change the team colour of the building I'm placing for player 1 and 2
So they can be distinguished
I am guessing the intention for something like that is to use shared data for the mesh
But change the material?
pliant pike
you would use a different archetype or different entity's
tardy locust
hm ._.
So are they just letting people repeat that stuff because the entity system is faster and thus makes up for it?
pliant pike
what do you mean "people repeat what stuff"?
tardy locust
From what you just said it sounds like in my two player scenario with the same building but different team colours
I'd have to make the building twice as two different archetypes
pliant pike
it would have to be the same in the old monobehaviour just different objects
tardy locust
Might just give up today try again tomorrow =_=
pliant pike
you'll get it eventually, I struggled and am still struggling with it
tardy locust
The thing is that I've read quite a lot about ECS before Unity decided to implement their version and it doesn't feel all that intuitive on some of these points
pliant pike
how so?
tardy locust
Well just this whole idea of some of these things are predefined in the Unity namespace and some of them are not
Like I couldn't just add the quaternion from the mathematics package
I had to make a separate component struct with the quaternion in it alone and then I could add it to an entity
I can't add the MeshRenderer I now have to go hunt down the RenderMesh
And not only can I not just add the RenderMesh, I apparently also have to add some other component called LocalToWorld?
How was I supposed to infer that?
pliant pike
its still kind of early in development, lots of things are still being implemented and figured out
tardy locust
To me it feels like it's one of those things that are out too early.
pliant pike
it will be a long while before its all finished
its not really out per say, its in beta
tardy locust
A lot of the videos you look for nowadays don't actually matter any more because there was a period of time where the ECS packages changed like every month or something
pliant pike
free for people to test and help Unity develop it, its not really ready for active professional game dev yet
tardy locust
Even as a testable I feel it's a little too early but I guess that's just me
pliant pike
there's lots of stuff you have to implement yourself
you dont have to use it on its own you can just use their convertoentity's method
and jobs you can use completely without using entity's
analog horizon
Er I'm having some issues with the editor crashing when I try to create this architype
EntityManager.CreateEntity(bulletArchetype);
protected override void OnCreate()
{
base.OnCreate();
bulletArchetype = EntityManager.CreateArchetype(
typeof(Bullet),
typeof(Translation),
typeof(Rotation),
typeof(LocalToWorld),
typeof(RenderMesh)
);
}
Would anyone happen to know what would be causing this?
tardy locust
What is in your Bullet component?
turbid sundial
What are the errors
analog horizon
No errors it just immediately closes Unity
The bullet
However having only "Bullet" in the archetype does not crash things
Oh it seems to be caused by atleast the Rotation component
tardy locust
Well there you go then I guess
turbid sundial
Still trying to figure out a good way to use RenderMesh with jobs. Is it just impossible to pass a mesh + material to a job? because Job structs may not contain any reference types.?
Or should I be looking at instead using a hybrid component, via IConvertGameObjectToEntity - seems like the only option available to me 🤔
low tangle
if I remember right from one of the hello cube examples they use a entity prefab to pull that off, since the ecb just records the spawn prefab target, and the main thread is the one actually doing the creation and assignment of the mesh
turbid sundial
Yeah I think that's the route I'm going to have to take
golden elk
what is the best way to use a 2d array (result of terrain GetHeights) in a job? do I have to just flatten the array and pass it as a nativearray<float>?
turbid sundial
I think that's the most performant, since then you can run it in an IJobParallelFor
golden elk
I wonder if it'd be faster to just utilize the heightmap texture and a compute shader to generate a mesh from the heightmap texture vs passing getheights to a C# job to do it
low tangle
anyone know if these NativeList methods even work? I can't seem to figure out how they are intended
public NativeArray<T> AsDeferredJobArray();
public NativeArray<T> AsParallelReader();
public NativeArray<T> AsParallelWriter();
they all return a array instead of a parallel writer like hashmaps
NativeMultiHashMap:
public ParallelWriter AsParallelWriter();
I'm assuming they are just unfinished?
low tangle
okay so usage is only for after you've basically filled the list once, then you can write to that list/internal array in parallel. you can't use it to append to the list in the first place
vestal hatch
I've been using NativeHashMap<type, byte>.AsParallelWriter and then .GetKeyArray because it seems silly that NativeList doesn't have an actual parallel writer with append and NativeQueue requires dequeuing on the main thread.
Is there any other way to get list like functionality in parallel?
maybe IJobParallelForFilter is what i'm looking for?
untold night
You can Deque NativeQueue in an IJob off the main thread, but only after all queued elements have been written
Joachim mentioned something called NativeStream that was derived from the Unity.Physics work. It supports Parallel read and parallel write, maybe that would work better for your use case?
There's no real docs other than some tests I think, and I haven't used it myself
IJobParallelForFilter can also work with NativeLists
I think
vestal hatch
i'll give that a try
untold night
or rather, it can be used with a deferred list for either append (add) or filter (remove)
vestal hatch
I've never messed with the deferred stuff. Does that basically allow you to chain jobs without .complete in between them?
golden elk
anyone use something like clipperlib with jobs before? is there a job friendly version of that already somewhere? seems like it'd be a good candidate for jobification
I am basically generating an area of the terrain as its own mesh, and clipping it to some polygon. currently doing it brute force on main thread and trying to jobify it
low tangle
I looked at the interface for the stream and it doesn't seem very intuitive
I'm trying to avoid the main thread at all
I managed to ditch the list since I knew the max possible elements and can just skip extra
mystic mountain
Anyone ever had this error when watching EntityDebugger on specific entity (all inspector stuff wont show up)
nvm, seems to be known issue that is fixed in next version of properties
wary anchor
hi all, basic query again.
I've got a NativeMultiHashMap where the keys are positionalID (int3 for now) and the values are the index of a query - the hashmap is created with IJobForEachWithEntity.
In a separate system, I'm wondering how best to get other components from those entities indexed in the NativeMultiHashMap (which is a non-changing or very infrequently changing reference lookup).
Wondering if I've gone about this the right way for my data but also unsure about all the methods actually available!
if I use myNativeMultiHashMap.GetValuesForKey(), I'll have the subset I'm interested in, but I'm not sure how to use this result to get the components I'm actually after.
I suppose I could just brute force it every time and do away with the lookup table, that way I can set NativeArrays with the positional/scale data I need and I know that my indices will match up, but it seems very wasteful to be rehashing everything potentially many times per frame when I know those data won't change
wary anchor
Alternatively, generate several multihashmaps at the same time, one for positions, one for scales...?
mystic mountain
How does JobHandles really work? If I have a "public ArchetypeChunkComponentType<Translation> translationType;" in my IJobChunk will the next job not start if it is also using the same?
wary anchor
How are people generally doing this? I need to calculate vectors between player or NPC's Translation.Value and nearby surrounding objects' Translation.Values.
I can't work out how to use the nativeMultiHashMap to get the translations of the components, the entities for which I have references. Maybe it's just the wrong approach. Some examples of this basic stuff would really be amazing 😦
frosty siren
What am i doing wrong? GameInfo.entityManager.SetComponentData(newHex, new Rotation { Value = quaternion.RotateX(90) });
It's seems like 45 degree rotation
amber flicker
probably need to pass in radians @frosty siren
@mystic mountain I believe that's correct unless both jobs have [readonly]
@wary anchor I was going to type something out but you could look at the second to last post here: https://forum.unity.com/threads/jobparallelfor-and-nativemultihashmap.560896/ - you need to allocate the hashmap then when you create a job, pass in the .AsParallelWriter() (which replaces the depreciated .ToConrurrent()) version to write to it in parallel then e.g. pass it into a new job without .AsParallelWriter and you should be able to read from it in parallel. Might be easier to help with specific code.
frosty siren
Yeap, that works quaternion.RotateX(math.radians(90))
fringe sinew
Hmm, I think I might be missing something.
I have a task that operates with Unity's Mesh API, thus it cannot be done fully in jobs. It's also on 2019.2, where we don't yet have Mesh APIs that work with native arrays.
I approached the problem with this solution: operate on entities as much as possible in JobComponents and write data to meshes in the main thread system (with foreach)