its also a lot harder to write slow code in burst/jobs than on C++
#archived-dots
vagrant surge
the limited feature set and heavy restrictions burst has means you essentially are obligated to do data-oriented design or stuff similar to it. Its stuff that works amazing on modern cpus. On C++ you can ruin performance quite quick by abusing OOP and using stuff like linked lists
dull copper
tbh, I'd be really curious to see how AVX would perform but something tells me Unity has planned a lot of these optimizations to work specifically up to 4-wide SIMD
I also saw some comments about some early tests where they said AVX just made some chip thermal throttle so much it didn't really give any better results (should probably find this post as that doesn't make much sense to me and surely that can't be the case on most high end PC's)
I'd personally want AVX to be able to do double precision math 4-wide but it seems that Unity's stance is that doubles will not get any special Burst treatment :/
they kinda put doubles on the new math lib as people asked for them but they are just sitting there, unaccelerated by the fancy stuff
vagrant surge
@dull copper thats avx 512
the one that throttles
the thing is that for now, burst mostly deals with a vec4 using a vec4 instruction
so thats why its only SSE
to use 8-wide vectorization, you need to SoA EVERYTHING
tho avx itself is kinda like just sse duplicated
dull copper
@vagrant surge I dunno about the 512, there's like two cpu's that support that? ๐
it also makes no sense to support any other AVX than the first one in a game engine as it's only one that's somewhat wide spread on gamers CPUs
and yeah, I assume the AVX gains would be slimmer in regular use cases
why I want it myself is mainly because I'd love to use double math on some places
steam hw survey doesn't even list AVX 2 or 512 but regular AVX is at 88.6% atm
analog tangle
Eh... AVX2 is anything Zen from AMD or Haswell+ from Intel.
Also Excavator from AMD but that's not too common
dull copper
it's still way lower percentage than AVX ๐
untold night
I posted this on the forums, but if you're working with blob data, don't try passing a BlobArray<T> to a function at all.
You will have a bad time.
Right now there's no other way than to use it directly as part of the referenced blob structure, or by using GetUnsafePtr() and passing that along if you want to process the array in a subroutine
minor sapphire
Anyone think the next physics drop is going to be accompanies by havok pricing announcement? It still days havok coming in June in the havok site...
dull copper
I dunno about that but the next one is going to have some api changes
I still wonder if they postpone havok further
because since these both use the same api
minor sapphire
Hmm
dull copper
having havok out now would make it a lot harder to fix the other physics package api too
IMO they should first stabilize Unity Physics so they know everything they need for it
before putting more to the mix
minor sapphire
We're in an updates drought I want my updates :D
dull copper
I just want more functional Unity Physics package ๐
Havok deal doesn't seem that great to me
there's some cost to it, and by the sounds of it, it's going to be a sub or rev share, and I don't want either
minor sapphire
Don't know what the deal is yet
dull copper
and then, it's not even "full" Havok
but like black box Havok wrapped in Unity Physics API
so it's same crap we get now with physx
and can't control about anything extra
they said that many havok specific techs are not going to be included
it's just bare bones havok physics
but for stuff like that, physx is fine too
and besides, Havok is not crossplatform deterministic either
which Unity Physics will be
of course it's too early to tell what the whole deal will be like, but it doesn't look good so far
minor sapphire
My priority is less CPU time. I'm making the kind of game where if I have spare CPU, I can put in more entities for a larger / more active universe
If I see the caching in havok lead to lower CPU time, then I want it lol
I hope the pricing is not shitty
mint iron
is there a way to exit a job once you've done what you need? in my case i'm testing elements for collision for a selection mechanism but currently only one selection is allowed, so there is no reason to keep searching through entities in IJobForEachWithEntity once a match is found. I suppose i could set a bool and just do nothing for the rest of the iterations but that's not exactly elegant.
minor sapphire
Not sure, can't think of anything better really.
gentle osprey
Possibly stupid question, but how are you supposed to add child entities in code within ECS? I tried the code below, but get told that the children native array has already been deallocated.
var owner = EntityManager.CreateEntity();
EntityManager.AddComponentData(owner, new StreamedInVoxel() { Layer = layer, Index = idx });
var children = EntityManager.AddBuffer<Unity.Transforms.Child>(owner);
// ...
var e = EntityManager.CreateEntity();
children.Add(new Unity.Transforms.Child { Value = e });
Edit: Think I figured it out, was not aware that the entity was invalidated after you added or changed component (if I understand it correctly)
Wait, if that is the case, how the heck are you supposed to be able to keep parent child relationships? Add a component to the child then the child value breaks?
amber flicker
@gentle osprey It's tricky to tell exactly what's happening in your code but in case it helps, be aware that every time you add a component to an entity, that causes the archetype to change which causes the structure of your entities (their indices etc) to all change. That and the fact that that all has to happen on the main thread. If you have Entity newEntity = EntityManager.CreateEntity() the reference to newEntity will remain correct while adding/removing of components. In a more broad sense, parent/child relationships are a bit of a pain - fundamentally you have to have a component on one entity that references the parent/child entity.
gentle osprey
Right, I see, I was starting to get that idea after reading around some more, thank you for clarifying, it was a bit confusing :)
Though, I a question I am still stuck with is: If adding or removing components changes the indices (and then effectively the entityID (?)), are there any ways to have a "reference" to one particular entity? Because as I noted above, if entityID's change when you do structural changes to an entity, then child/parent relationships based on referencing id's will break as soon as you do a structural change to a child or a parent
amber flicker
If you use an Entity (as opposed to an int), the reference will hold just fine ๐
But not while within a job and triggering a change
gentle osprey
Right, I think I just managed to come up with a test case that made me understood how it actually works. Essentially, the Entity always refers to the same element, but it is problematic to access the components belonging to that entity right after I have made a structure change?
amber flicker
yes but when going wide right? doing so on the main thread you should be fine I think
gentle osprey
I also got it when I was doing regular componentsystem. Hold on and Il see if I can write up a complete minimal example, because now I need to really understand this :P
amber flicker
ahh ok yea... I think I can guess - this is because you are grabbing a NativeArray<Entitiy> or similar, then adding a component to e.g. the first entity, that causes a restructure which makes your for loop no longer make sense going over the previously referenced array
maybe - this is what I've mistakenly done in the past at any rate
gentle osprey
var streamedIn = Entities.WithAll<StreamedInVoxel>().ToEntityQuery().ToEntityArray(Allocator.TempJob);
for (int i = 0; i < streamedIn.Length; i++)
{
var e = streamedIn[i];
var children = EntityManager.GetBuffer<Children>(e);
for (int j = 0; j < children.Length; j++)
{
if (EntityManager.HasComponent<Unity.Transforms.Frozen>(children[j].Value))
{
EntityManager.RemoveComponent<Unity.Transforms.Frozen>(children[j].Value);
}
else
{
EntityManager.AddComponentData(children[j].Value, new Unity.Transforms.Frozen());
}
var trans = EntityManager.GetComponentObject<Transform>(children[j].Value);
trans.position += Vector3.forward * Time.deltaTime;
}
}
That breaks, or at least similar logic, however, if I just change EntityManager.Remove/Add with PostUpdateBuffer.Remove/Add everything works.
Yeah, I think it's like you say. However, I didn't do it exactly this way, but in another way where it wasn't super obvious what was going wrong, so it totally looked like it was the Entity that was "invalidated" as an accessor, rather than what was really happening :P. And if Entity's could get invalidated, then there would be no reason to store them, as they would be invalidated upon as soon as they experienced a structural change. Luckily it isn't like that, so my worries was a result of a missunderstanding :P
Thanks for helping me clear that up :)
Still don't understand why Unity removes my Child Buffer though :/, guess I'll just create my own.
amber flicker
fyi there are bulk add/remove methods etc that you want to be using as much as possible
they work on entity queries & NativeArray<Entitiy>
so I might make an entity query that selected all entities with StreamedInVoxel & Frozen for example and then do query.RemoveComponent<Frozen>()
the awesome thing about that is it just changes the archetype so is super cheap compared to individually changing entities
gentle osprey
Oh yeah, My plan is to move over to using as much of that as possible (as well as the job system), just need to figure out the algorithm first :)
sonic oar
Been out of the loop for a while and now ECS seems quite different. Can anyone recommend a good place to learn the current version?
dull copper
you can look at the links on the pinned message here
official manual and dots sample repo especially
native shuttle
Is there anyway to draw editor stuff for entities?
like an editor component system or something like that
something to iterate through entities and draw handles and stuff
tawdry tree
Can't you use the draw in editor stuff in a system?
And conversely, I'm pretty sure you can grab the entity manager in a normal monobehaviour or editor script
gusty saffron
Is it possible to store a dictionary in a component? I know it's possible to store arrays in dynamic buffers.
merry oasis
don't think so
i also wouldn't recommend putting one in a SharedComponentData either
trail burrow
@gusty saffron yes, you can make your own dict using native memory, then you can store it on a component
mystic mountain
So I'm trying to convert a prefab to an entity(prefab?), I see that there is a GameObjectConversionSystem that can be used in the IConvertGAmeObjectToEntity interface, but I want to do this on a scriptable object that holds the prefab. Anyone know how to do this in any way?
dull copper
Burst 1.1.0-preview.4 on staging now:
## [1.1.0-preview.4] - 2019-07-05
- Burst will now report a compilation error when writing to a `[ReadOnly]` container/variable
- Fix regression with nested generics resolution for interface calls
- Fix issue for UWP with burst generating non appcert compliant binaries
- Fix issue when reading/writing vector types to a field of an explicit layout
- Fix build issue on iOS, use only hash names for platforms with clang toolchain to mitigate issues with long names in LLVM IR
- Allow calls to intrinsic functions (e.g `System.Math.Log`) inside static constructors
- Improve performance when detecting if a method needs to be recompiled at JIT time
- Fix an issue with explicit struct layout and vector types```I get this when I try to spawn entities with Physics body of Kinematic(and dynamic) from server to client, but static type works. In my ghost settings (the replication settings), I send translation and rotation. Anyone who have a guess what could be wrong? x)
Using ECS physics and Network with NetCode.
sonic oyster
No idea if this goes to ECS (Unity.Mathematics lib), but i have trouble understanding noise.pnoise
i have two arguments, both float2 or float3 for 3D noise
float2 p, float2 rep
what does those mean?
i'm trying to feed X and Y coordinate to P and leave rep as 1f,1f or 10f,10f
and reverse, coordinates to rep and static values to P
tawdry tree
Have you simply tested what happens when you vary one but not the other?
neon shore
from what I get rep is the periodic variable for repetition / tiling
low tangle
^
if you open the source file it has a full description that isn't picked up from intelisense
blazing mural
I hope this is okay to ask here, but what is the Data-Oriented way to implement "helper" classes or similar? For instance, I have a class full of static methods that allow me to easily determine the direction (N, NE, etc) from angle, 2 points, etc.
This is useful because I reuse this code all the time...
Is this "data oriented" or is there a better way to have similar functionality?
untold night
pure (no side effects) static helper functions are perfectly fine
that's how the new math lib works.
blazing mural
thanks. I was looking at the math library not long ago. appreciate it @untold night
low tangle
basically if c# allowed for freestanding functions, all of those 'helper' methods would just be in a math namespace
which would be some nice piece of mind
but effectively the forced static class ends up being like a namespace as long as you are diligent with your programing
pure functions are very data oriented - they should be a thought of a data transformation encoded into a function
cunning perch
What are some examples where data oriented design shouldn't be used? I don't want to force everything to be if it's better off not
tawdry tree
I can't really answer that question, but a related one:
Where should you not use Unity's DOT stack?
The answer to that is: When the cost and risk outweighs the benefits. (true for most paradigms and patterns, really). The way I see it, in its current form, there are some things you are simply better off doing in MonoBehaviour, be that because it's too cumbersome(time consuming and/or leads to bad code) to do in ECS, the perf impact difference is practically zero, or it's (nigh) impossible with our current tools.
dull copper
to put that differently, from my POV only reason to jump to the DOTS today in actual production is if you need the perf improvements on the things at hands - which probably makes it irrelevant to most of the existing games as your gameplay scripts rarely affect the overall perf in any meaningful way :p but if your game has brute force computing, this starts to matter more
do note that I'm not talking about people doing stupid things on their code now, making it perf heavy for no good reason. if you do that on monobehaviours, you can be sure to shoot yourself into feet on DOTS too
tawdry tree
Simulation type games can benefit, the kind which boasts "X units in the map at the same time!" or factory/city sims and such, and there are some other games which can benefit from using at least part of the stack (notably burst jobs), if nothing else. Basically, if your game would get great benefits from multithreading, DOTS makes that a lot easier than MonoBehaviours.
dull copper
sure, that would go into brute force stuff as you compute a lot of things at once
minor sapphire
Dammnn I hate when burst is not identical to non-burst. Can't debug log to find burst yay
cunning perch
Makes sense, thanks
low tangle
there's one extra thing alongside needing the perf of DOTS that I was glad I bought in for, simplicity. after porting the majority of my networking over to a dots friendly setup, it went from monolithic server/client with various submodule stacks and interfaces, to a wide collection of single responsibility systems that could manage each stage by themselves.
on the forms someone recommended reading this DOD book (which I think is nicknamed the dod book)
http://www.dataorienteddesign.com/dodbook/dodmain.html
@cunning perch
tawdry tree
Oh yeah, data oriented code with proper single-responsibility components is exceedingly simple to work with, as long as you know (or can look up) what each system does (more traditional OOP might be even worse if you can't do that, too)
And ECS makes that so easy. Will have to remember that when questions of what and how come up.
low tangle
exactly
I really want to do some posts on ecs-ifying how I thought about and broke down my systems over the last year of development
had a lot of strife and success I'd love to help people skip
hollow sorrel
thanks for the book link
i still get confused when thinking about data oriented architecture
for example let's say you have a builder/management type game like rollercoaster tycoon
and each building costs upkeep so i suppose you'd have an Upkeep component with a cost field
and at the end of the month you wanna have a report that shows what categories took up the most upkeep
you could put a category enum in the Upkeep component?
but then if you also wanna show what specific buildings in those categories took the most upkeep then what
low tangle
you would create another component with the total counter and add it to
then to display month to day cost you would just sample from that counter
resetting the counter every month
hollow sorrel
hmmm
so you'd have like Building1Counter Building2Counter Building3Counter components?
low tangle
[] is a single entity
[BuildingType, MonthCounter, CurrentUpkeep]
which then you would have a UpkeepCaclulationSystem
which would look at all MonthCounters and CurrentUpkeep
update the current upkeep on all buildings (based on any factors you choose: on/off, performance)
and then every tick up the month counter
as well
which can even be a separate system; MonthUpkeepCountingSystem
which just runs after/before and takes current calculated upkeep and adds to the counter
and running off a event entity [EndOfMonth] would trigger another system UpdateLastMonthUpkeepSystem which would take in each MonthCounter and set it to zero, and store the historical data elsewhere into a spot so it can still be looked at in graphs or UI table data
[EndOfMonth] 's would be spawned from another system that updates the current time of the game world simulation (allowing for pausing, 1x 8x fast simulation of time), every month advancement would just spawn it as a event entity to be consumed and destroyed in one frame.
hollow sorrel
ahhhh i see
hadn't considered using events for time as well
that helps a lot
thanks!
low tangle
no problem
tawdry tree
I would very much like to hear your experience with ECS-ifying existing code and what challenges you've met and how you solved them, June, so please do share.
hollow sorrel
yea am interested too
low tangle
Alright, I will find time soon and somewhere to put them (forms probably) I've been taking lots of notes to digest later on that will help
My game is a vr sandbox with as much logic as I can put into ecs
I did a few game jam practices to figure out how to do things early on (made terraria clone) which I highly suggest
Really hope ecs animation comes out soon
tawdry tree
Animation is a bitch without good tooling, yeah. Had a course called (creatively) 'Unity development' while I did my bachelor, which had the task of remaking Super Mario's 1-1 level, complete with enemies and animation.
Setting up basic platforming took no time, respawning and coins and enemies killing you on touch easy-peazy, but making it so you would kill them if you hit them from above and be hurt from the side was a pain, and animation, oh god, that took so much time, despite having the state machine thingy (to be fair, this was our learning process, so not unsurprising).
Altogether I used maybe a week worth of full-time work on the 'finished' version over the semester, and now that I have more experience I could probably do it in 2 or 3 days with more polish.
I really should do some game jams, but it's really hard for me to come up with ideas for themes and I don't really have a community around me I could lean on for a team, so it'd be solo and take like a day just to come up with an idea. ๐คท Still get practice by (re)making mechanics, such as RTS controls/movement, a simple AI combat system, procedural terrain generation, etc, though. Works for me.
low tangle
Exactly, I don't take them to completion. Just until I feel they provided a good learning experience or in happy with where I got the prototype to
Oh and take notes
I even include little bits of what my rationale at the time was. I find it very helpful to compare the initial state of the problem and how it was actually solved
I very much approach problems differently now that I've been doing ecs for a year
hollow sorrel
@low tangle out of curiosity, how do you structure your folders
do you seperate components and systems?
low tangle
yes
I do not put my components into a new sub namespace though
they stay within the same as systems for ergonomics
low tangle
correct
hollow sorrel
is that working out better
low tangle
much better
hollow sorrel
howso
low tangle
hm let me think
increased control over the network thread
I had to write it myself so I was able to remove a bit of the layering there
I also used a ring buffer for events and object polling for those
I'm working on a native memory version but haven't had the time nor need for more network performance yet
I'd like to make it fully able to queue and or send from jobs
but I hit my performance target before I needed that with a single network thread
low tangle
it sounds more complex than it is
I really do apricate how simple ecs systems make this
hollow sorrel
i been looking at litenetlib/enet but seems like being able to keep everything C# side seems easier
especially if you were to go multiplat
hollow sorrel
yea but you're still calling into C++ right
low tangle
c and yeah it does
but that c library is very thin and easy to port to other platforms
hollow sorrel
ah that's good
low tangle
one of things I really enjoyed about setting it up with enet, is I left no networking logic or high level stuff to the first system, the connection management one
it simply gets the events from the network and converts them to entities
with only taking care of mangled packets as its only logic
because of how well ecs scales with millions of entities I can count on a writing a secondary system to analyze the amount of packets coming from someone and then disconnect them from the server
its very nice being able to just add a system to observe data flow
hollow sorrel
ah that's pretty neat
how do you decide what components get sent over the network tho
do you write a new system every time
low tangle
tagging, then adding support to the send and receive systems
I'm trying to avoid code generation (protobuf) and avoid reflection magic
because I need a solid deterministic networking
I already got bit in the ass for a few weeks debugging my first network implmentation
lots of users
now I can just save the current entity world to disk (still working on this) when a problem happens and observe the world, because it is the network state and game state
hollow sorrel
ooo
that sounds like a pretty big benefit
whatcha mean by tagging
like you just add a NetworkComponent tag to your entity?
low tangle
let me find my tags
[ServerSend] [ClientSend] [ClientLogic] [ServerLogic]
the sends would've been merged, but server needed a extra few prams and one extra type doesn't make a big difference when the archtype is already going to be split
I also convert ENet Peer's to a component: [NetworkID] for fast lookups to entitys they belong to (mostly root player entity)
hollow sorrel
oh man
i feel like i just wanna pick your brain for hours
i'll try not to
also
low tangle
if ECB's were not such a pain in the performance I'd have kept my first version of the network sim group
hollow sorrel
i sometimes feel like putting everything in different systems is bad for performance
i mean when iterating the same types of components just to do a different op
low tangle
which even converted [NetworkID]'s to [NetworkEntity] and added a direct entity ref right into a component
if you feel there is too much overlap, try to break the systems even more
dont feel bad about adding more 'loop's over the data
hollow sorrel
mmm
just feels like merging the systems would be better because then i'd be doing one loop instead of multiple loops over the same data
but that goes against separation of concerns
low tangle
exactly
hollow sorrel
such inner turmoil
low tangle
one of the most important high performance systems in my game is only 80 lines in total
a single foreach job running over a million entities
low tangle
it is
I'm working on scaling it to 1k players in a world
I'm gonna head back to work, I'll be around.
silent epoch
hi all
i wanna learn sumthin like u guys
where should i startd ?
thanks
low tangle
check the pins first for resources
theres a pretty good beginning tutorial series on the forms
frosty siren
get some strange error in console
Unexpected error while processing function `PathFind.Find(int fromIndex, int toIndex, Unity.Collections.NativeArray`1<HexEnergyCost> hexesCosts, ref NavPath[] result)`. Reason: Object reference not set to an instance of an object
at PathFind.Find(int fromIndex, int toIndex, Unity.Collections.NativeArray`1<HexEnergyCost> hexesCosts, ref NavPath[] result)
But execution of my code not calls this method, cause it's in false IF. I tried to use Run() method to execute job on main thread and debug it. So VS shows that this method is not called.
Have u any ideas why
frosty siren
Yea, I understand ๐ I mean maybe somebody knows what kind of magic is in that case
mystic mountain
Can you use a fixed array size in IComponentData in some way?
tawdry tree
How fixed? Compile time or you know when you add the component?
mint iron
seems like you should be able to,
public fixed int MyFixedArray[5];
Yeah, I see. I need unsafe context for that, saw someone said that it was removed for the DynamicBuffer.
tawdry tree
If there's few enough, you could work around it by making individual fields instead of one array field, but obviously that has a few.... issues.
mystic mountain
Hmm, yeah. Those are my input actions. But I guess it's simpler to do that for now anyway ._.
trail burrow
@mystic mountain if you are fine with unsafe you can do:
public fixed byte ArrayData[ARRAY_LENGTH * SIZE_OF_ELEMENT];
MyData* array = (MyData*)&component.ArrayData[0];
Unsafe context isn't ... unsafe
tawdry tree
Unsafe context is potentially unsafe. As long as you're careful it should generally be fine, though. Just... make sure you know exactly why you're making it unsafe.
trail burrow
Eh, I mean
Yes if you don't know how pointers work,or are confused by non classes then yeah stay away
But there's nothing inherently unsafe about unsafe context, if you don't know what you are doing sure
You can blow your own foot away absolutely
But if the argument is that because you can fuck up then its unsafe, well then loops are also unsafe :p
Its probably the most poorly picked keyword on all of c#
Should've been unmanaged
Like they are using for the new type constrain in c# 7.3
</endrant>
coarse turtle
Well I would definitely agree that "unsafe" should be named unmanaged - would make so much more sense haha
native shuttle
you also have IntPtr
safe lintel
is there a way to search an entity with the entity debugger? besides just scrolling the list?
native shuttle
which are kinda a managed pointer
tawdry tree
@safe lintel If you know what components it has, you can find a system which targets it, but other than that you'd just need to click around 'til you find its ID. (AFAIK)
safe lintel
yeah was hoping there was some search by index option i was missing ๐ฆ
native shuttle
you could add a tag to make it easier
safe lintel
super easy to make a ton of entities, but hard figuring out whats what. when is that new entities release i feel like we are overdue for some project breaking changes ๐
tawdry tree
Hmm, you could make a hybrid system which finds an entity by ID and adds a tag for you, then just look for that tag for a drastically smaller search volume
As in, you just input the id in the editor(monobehaviour/wrapper) and BAM
native shuttle
dont forget to add the sounds effect
safe lintel
hmm those could be a winning ideas
tawdry tree
Ahh, how convenient proper tooling will be
The #1 thing on my ECS todo is a system which uses scriptableObjects to do basically prefab/archetype work (for spawning).
safe lintel
aw crap hybrid components dont work with inheritance, at least not with the originating class, only the derrived one
safe lintel
ahh the ultimate in dirty feeling cheap hacks, a monobehaviour, with a cached reference to a secondary monobehaviour, for the system to call to get around the inheritance thing
frosty siren
can we somehow customize Entity inspector that shows components data?
cunning perch
Just reading about the index parameter in the manual. https://docs.unity3d.com/Packages/com.unity.entities@0.0/manual/entity_iteration_job.html#sidetoggle
Why would one want to use multiple jobs in a system?
`Iterating with IJobChunk requires more code setup than does IJobForEach, but is also more explicit and represents the most direct access to the data, as it is actually stored.
Another benefit of using iterating by chunks is that you can check whether an optional component is present in each chunk (with Archetype.Has) and process all the entities in the chunk accordingly. `
But why? Wouldn't it be better to just to require or exclude a component for each case on different systems?
merry oasis
mmm
generally if you have optional components, you're doing shared logic, but then also some part differently according to the optional components
if you block out each individual case, well that seems like a great way to bloat your code by having to copy all of the shared logic that you're doing
i think the transform system should have something that uses that
archetype chunk iteration with some optional components i mean
oh man there was also something written up about this that went in depth but idk where it was
cunning perch
But You can have a base class to share logic right ?
merry oasis
how would you use that in a job though?
cunning perch
๐ ah right
merry oasis
ok here it is, this is old af and is using outdated API but at least reading should be good https://github.com/Unity-Technologies/EntityComponentSystemSamples/blob/659c819ec45c6efde5c18889b3cc6022235393da/Documentation/content/chunk_iteration.md
golden heron
Has anyone tried using ecs for the behind the scenes simulation of a server?
Plus, @merry oasis , @cunning perch You can write methods to do your logic and call them in jobs, just so long as they operate with the right kinds of data, then you dont need to bloat the systems themselves.
Thats all unity.mathematics is really, a set of thread safe methods for maths. Altho of course they did some tricks to maximise efficiency, but nothing stops you from making your own.
tawdry tree
@low tangle is doing some kinda multiplayer server/client stuff, but I don't know if it's what you're referring to, @golden heron
golden heron
Im intending to run a server with an authoritative simulation controlling the players and ai
Its an mmo setup im aiming for
I think i have met June, she was nice :)
Atm i have a simulation, and i have a client-authoritative multiplayer setup which has a dumb server which just sends transform data back and forth, not secure at all really. Altho... Its amazing how smooth it runs considering that it has absolutely no attempts to smooth or interpolate the movements, it is currently relying purely on the arrival of udp packets to define the movement timings, which is very much variable, and yet still provides a movement which is surprisingly responsive.
I want to make the simulation take over in the middle for the server hehe
I have however written the entire udp layer myself, it uses a sort of half and half setup, between the inbuild c# client classes and access of the sockets themselves directly.
tawdry tree
https://discordapp.com/channels/489222168727519232/497874303463850004/597198948662509598 You can read the convo from this message and down quite a bit, I don't know any more than what she said there.
golden heron
Ah ok thanks hehe
It looks like she was using a library - mine is pure - i needed someone who has the experience of basically writing a library hehe but its ok, thanks still :) Altho of course, still interested to hear from june :)
tawdry tree
I reckon a lot of the methods and possibly patterns should be reusable, but yeah. If you're making something form the ground up it's not unreasonable that you might be on your own.
tawdry tree
The source code? You can find it in your packages folder when you download the package, dunno if the Git repo is publicly available
cunning perch
ah good point
mint iron
Hey umm, i'm seeing a 2-4 second delay the first time i run systems that are triggered by mouse down input, but in only when Burst is enabled. Its really weird, is this an anomaly in the Editor or will i really have to pre-prime all my jobs to avoid this kind of hang at time. ๐ฟ im assuming its the reflection etc
Edit: Figured it out, when Synchronous Compilation is on, its struggling, causing the delay.
cunning perch
I want to experiment using burst and ECS, and I want to benchmark my findings. But I'm not sure what constitutes as a "good" test. Obviously the best is if I tweek minor things and document how a particular change affects performance.
But I don't know what environment and tools to use. Should I use unity unit tests, or unity profile or external tool, or is debug logger enough?
It's creating 1000 or more instances enough, should I skip my benchmarks for the first loop? So many things to think about
I guess just measuring fps on a build is the most accurate, but don't want to build every time...
hollow sorrel
yea building in release mode is most accurate because editor adds a lot of overhead
modest beacon
is ecs now stable?
hollow sorrel
@cunning perch for small stuff you can use the Stopwatch class to time stuff
@modest beacon nah api still changes every month or so
modest beacon
wew.
modest beacon
thanks.
wicked patrol
hey guys, is there some well rounded resource to make the step from regular programming in unity to using ECS?
fallen sleet
@cunning perch For profiling individual jobs, the Unity profiler is pretty useful at least as a guideline. For a proper benchmark you'd likely want to take the min/avg/max over many runs (ideally 30 or more).
mint iron
@wicked patrol not really that im ware of.
There are some great links in the pinned posts for this discord channel, if you're starting out make sure you take the time to get a feel for the concepts of ECS so you have some perspective.
There was a GDC2019 talk that i think had some good explanations for how Components/Systems work together https://www.youtube.com/embed/0_Byw9UMn9g?autoplay=1&disablekb=1&iv_load_policy=3&modestbranding=1&showinfo=0&html5=1&rel=0
Unfortunately I think a lot of the talks you'll find can focus a bit too much on justifying it and explaining why it can be more performant and as a result there's a lot of stuff about cache lines and data storage techniques, multi-threading etc but depending on your level of knowledge and learning style that information might be largely irrelevant.
I recently found a good code example of some basic game functionality from someone on github:
https://github.com/skhamis/Unity-ECS-RTS
Also, unity's tutorial page on ECS:
https://learn.unity.com/tutorial/entity-component-system#5c7f8528edbc2a002053b678
And the documentation:
https://docs.unity3d.com/Packages/com.unity.entities@0.0/manual/index.html
The DOTS Unity forum is also good to lurk around and see what kind of issues people are having and the solutions
https://forum.unity.com/forums/data-oriented-technology-stack.147/
amber flicker
@fallen sleet & @cunning perch that's what the unity performance reporting is designed to do - it can be quite frustrating but it does take measurements over time and present a report - great for automated testing but mostly I just debug a build attached to profiler - tend to be fairly accurate timings in my experience
wicked patrol
@mint iron hey, thank you for all the resources ๐
cunning perch
yeah thanks all
wicked patrol
also, to the more experienced people with this system. when do you make use of ecs an jobs? only when you encounter performance issues or are you designing your code by default along the ecs?
*and
tawdry tree
Right now you can't really go pure ECS, and the tooling for ECS is a bit lacking (ie nonexistent), but long term you should have the option between pure ECS, hybrid, and MonoBehaviour. MonoBehaviour might be phased out at some point, but that would be years into the future.
Right now there are two approaches to (possibly) using ECS.
The first is most useful when you have an existing project, and involves finding points that could benefit from the performance or simplicity of jobs/ECS and 'jobifying' and then potentially moving the code to ECS. This is most relevant to any code that affects a lot of 'stuff' at the same time, think simulation things or RTS unit stuff.
The other is to start a project with ECS in mind, and make as much as possible in ECS. As mentioned, you can't really do pure ECS, but in this variant you would do whatever you could in ECS and fall back to MonoBehaviour where needed (or where you would simply save yourself a lot of work/headache).
Well, that's how I see it as well, and there's grey areas of course. Jobs, by the way, can be used completely separate from ECS and is not in preview state.
My main recommendation for ECS right now is to play around with it and try to wrap your head around the differences in paradigm and how you need to think about things differently, not necessarily the specific APIs and such. That, and if your game is simulation heavy you might consider using jobs and/or ECS for heavy parts, if only to get that sweet sweet multithreading.
wicked patrol
great thanks. I already expected something like that, now I have confirmation :)
The job system in itself is great! takes away soo much headache when trying to execute code on multiple threads
minor sapphire
Damn is this still not fixed, did I miss the memo? lol. Well I suppose they haven't had an updated for ages now, maybe in next update.
https://forum.unity.com/threads/ijobforeach_b-dynamic-buffer-is-write-only.681667/
coarse turtle
Yeah, I guess the alternative for now is BufferFromEntity<T> and grabbing the buffer with an IJobForEachWithEntity
hollow sorrel
trying to port my 2d animation system to ecs, before i'd just do
animator.SetAnimation("animationName");
but then i'd have to store a name string for each animated entity, might not be the ecs way?
could do a lookup table but then i'd have to do something weird like
animator.SetAnimation(lookup.GetAnimationId("animationName"));
trying to gather ideas on how best to approach this
could be something i'm missing
coarse turtle
if the animation names are consistent and shared amonst all entities
you can just store it into a component which holds a BlobAssetReference
and use that as your 'lookup table' to grab ID
if its different per entity, I think putting each animation name into its own BlobAssetReference should suffice
AnimationNameBlob is an IComponentData struct with a BlobAssetReference<T>
entity 1: AnimationNameBlob: [ has all of your animations names specific to entity 1]
entity 2: AnimationNameBlob [ has all of your animation names specific to entity 2]
// In a ComponentSystem
Entities.ForEach((Animator anim, ref AnimationNameBlob animBlob) => {
// Do some stuff with the anim and animBlob
});
oo i haven't used blobassets before
animation names themselves can be allocated at start so i guess they don't change but each animated entity can have a different state (name)
like walking, idle etc
coarse turtle
Ah then maybe option 2 might work for you with the pseudocode above and for Blob stuff, you can look at the BloblifcationTests.cs in the Entities.Test dir
hollow sorrel
so if i understand right, blobassets are basically like a lookup table? an animator would basically just hold a pointer to a blobasset, which can be referenced by different entities at the same time?
coarse turtle
Well blob assets can hold immutable data and reference data, so you can store objects, arrays, dictionaries, etc
merry oasis
oh so thats what they are for
coarse turtle
let me pull up an example I've used it for
public struct CardBlobData {
public BlobArray<int2> CategoryDmgMap;
public BlobArray<float3> CastTimes;
}
public struct CardLibrary : IComponentData {
public BlobAssetReference<CardBlobData> Value;
}
so the top struct are just arrays of int2 and float3 which work as my "blob data", the data I populate there is actually authored in a scriptable object, and in a Conversion script, i copy the data from the scriptable object to a newly constructed card blob data
And the CardLibraryis the component data which my systems will usually process on
so you can create your own BlobData which holds a Dictionary if you need to and then create a BlobAssetReference of that BlobData
hollow sorrel
ooo converting scriptableobject to blobasset, i like that
i guess the only thing i'm not sure about is how it's different than keeping a dictionary of CardBlobData somewhere else and put the key in CardLibrary
coarse turtle
Well, i think the lovely thing about BlobAssets is that you can have multiple BlobAssets
of the same archetype and live in the same chunk
Compared to using an ISharedComponentData which puts multiple entities in their own chunk
hollow sorrel
yea but i mean you can still have multiple if you have a dictionary in a static lookup class somewhere
hollow sorrel
just trying to figure out the pros n cons
if blobassets have better memory management that sounds like a good reason
btw i noticed you have cast time in a float3, is there a difference with float3 to 3 floats
coarse turtle
so the float3 is SIMD friendly so it can work with burst compiler well according to unity
and the reason I have it as a float3 was that it was a vector3 in the scriptable object
hollow sorrel
oic
coarse turtle
so it was just an implicit cast without me having to worry too much about the details ยฏ_(ใ)_/ยฏ
hollow sorrel
makes sense
hollow sorrel
so native containers like NativeList are structs
and this is fine because they just hold a pointer to the unmanaged version right
but then why does physicsWorld.RayCast want you to use a ref nativeList instead of passing by value
are you supposed to pass by ref?
mint iron
browsing through the code there it looks like they pass it around a bunch of times before it actually does any raycasting. In that case the small cost of copying two ints for allocator and pointer (or even more if you include the AtomicSafetyHandle and DisposeSentinel) could add up.
tawdry tree
Well, if you don't refit you just copied the entire damn thing... except not really, because of the pointer. Makes sense for it to be passed by reference instead of value then, doesn't it?
hollow sorrel
hmmm yea could be due to passing it a few times
and yea hodhandr but that sounds more like a reason to pass by value
if it's just an int to copy then doesn't make much sense to pass by ref
candid willow
Is ECS HDRP ready yet
safe lintel
no
safe lintel
well you can use hdrp with ecs but pure fully featured hdrp in dots doesnt exist yet, only the hybrid renderer
low tangle
it is independent of the rendering pipeline you choose
hybrid render is a renderer on top of the scriptable pipeline
at the moment most people are not directly rendering from ECS (some are though)
and instead doing most of the game in ECS, but traditional renderers and meshes as gameobjects
candid willow
is hybrid renderer a partial hdrp
low tangle
the whole pipeline is being reworked to be fully ECS friendly but its taking time to do it right
can be though of like that yes
I don't know the exact location of where it resides, but it works with stock unity renderer on my game, and worked with my hdrp test project
candid willow
I see. I've been MIA for past few months so just catching back up with ECS updates. Of course, my old project files won't render at the moment ๐ฉ
safe lintel
hybrid renderer doesnt work with lightmapping or lightprobes
low tangle
its very isolated and preview at the moment
candid willow
oh ๐
Well, I use OctaneRender for the standard renderer, lets give that a go with ECS
I think it should work since I will just use standard shaders and gameobjects
safe lintel
was mentioned light conversion is coming in the next release but detailed info about hdrp/lwrp in relation to dots has been super vague
candid willow
Yeap...did a search all around the unity roadmaps and docs for ecs hdrp and got nada
before coming here to you fine folks
safe lintel
i guess its in flux
low tangle
I saw a post on the forms not too long ago that basically they will create a HDRP-ECS renderer
which will replace HDRP when the engine goes full dots
candid willow
Anyone know of other dots renderers
safe lintel
the chunk thing with sharedcomponents seems like it would be limiting to me? not sure how they plan to work around or within that but my brain overheats just reading the transform system combinations
i would love to be a fly on the wall to just listen to them spitballing on how to approach tackling such things even if i probably wouldnt understand most of it
candid willow
Probably a lot of F bombs
fringe sinew
I don't think I understand something regarding the job sytstem and ECS
Let's say hypothetically that I have 2 separate systems that perform work on unrelated set of components (neither of the systems operate on a component that the other system uses).
How do I ensure that both of those systems operate parallel to each other instead of one of them waiting for the other to complete before starting?
fringe sinew
I heard about ECS handling all of the race conditions and exception by the means of warning you, but not that
safe lintel
probably easiest to just combine the work from the two systems into one system if you need that kind of concurrency
tawdry tree
Unless you specifically make a job dependent on another one Unity should just run it whenever it thinks is good. Now, if you check the profiler you'll see that this usually (almost always?) means that jobs will run sequentially anyway, but the important bit is efficient use of threads
safe lintel
otherwise i think you could hamstring yourself by restricting when those systems can run as core counts, workloads can all vary
tawdry tree
But yes, if you need the code to run parallel you gotta do that yourself.
I think a scenario where a job (job C) needs two other jobs (A and B) to be done before it does stuff is more likely than a scenario where you need two jobs to be parallel.
Like, why would it matter if they're parallel or not? You just care that they're done when you need them to...
fringe sinew
Because I have quite a heavy job for a procedural effect that does not read or write to the components that the gameplay uses, with gameplay systems themselves not reading or writing to the effect components. It is possible for the effect to be scheduled at the start of the frame and leave it be and save 1 ms or so.
If I have unused cores and I can schedule that heavy job to start working while gameplay jobs don't even touch it but don't actually do it - it's a derp move and a total waste.
safe lintel
i dont understand, scheduling the job to start - which you mention doesnt touch gameplay elements, while there are unused cores is a waste?
you mean gameplay isnt requesting the work from the job specifically?
fringe sinew
No, I didn't mean it that way. I mean that if I schedule something that is not touching the gameplay but the gameplay systems wait until that effect system is complete (unless there are no more threads) - that is a waste
safe lintel
so this effect doesnt interact with gameplay systems but gameplay systems expect/require the effect in order to complete?
anyway if there isnt an entity archetype for the effectsystem to work on, it wont need to do any work and gameplay wont be stood up by this one system if thats what you mean?
fringe sinew
Yes. They are completely separate and do not interact with each other. What do I need to do to ensure that they run in parallel?
safe lintel
parallelism is a byproduct from jobifying code, but for specific guarantees of parallelism between systems I think is left to the user to try to figure out
quote from joachim
"Jobifying code and parallelism are two very different things. Writing parallel algorithm's especially when there is interaction between elements is a very large and difficult topic. There is no magic to make that inherently simpler. Except... For just not doing parallelism..."
tawdry tree
To be clear here, do you want it to run separately, ie. frame-independently, as a sort of on-demand, don't interfere with other code thing, or does it need to be run within a frame or even every frame?
fringe sinew
It needs to be ran each frame alongside the gameplay loop.
tawdry tree
Then I don't see a problem? It sounds an awful lot like you're attempting premature optimization, ie. optimizing before you know you have a problem.
Have you looked at the profiler during runtime and spotted an issue?
fringe sinew
It's all only a hypothetical question. It's not about premature optimization, but about writing something that scales from the start
It's the same thing as teaching yourself to constantly write const in shaders since it forces the compiler (and you) to make some optimizations.
It's exactly the same here: how do I make sure that unity's ECS and Job Systems will execute code in parallel as I want?
tawdry tree
Not 100% sure on this, but in general I'm preeetty sure it (by default) runs jobs one by one(but each job in ECS can easily be worked on several threads). If you want anything else, you need to engineer that yourself. that aside, it sounds like the only real change you'd need to optimize it would be to make Unity start it at the beginning of the frame, which i'm pretty sure you can convince it to do.
fringe sinew
Alright, now the opposite question: how do I make a system depend on another system to complete before scheduling a job?
I mean, there's this thing, but what if it holds some dependencies that I don't want to be completed?
Like that effect that should run in parallel?
tawdry tree
So inputDeps are made for you automagically, but you can influence it.
AFAIK, by default it just contains a dependency for the default update.
I know that you can tell Unity to use another update group, but not how.
Pretty sure you can also tell unity that you want a specific system as a dependency, but I also haven't used that function.
@safe lintel any pointers here? How do you influence what unity gives you as inputdeps?
fringe sinew
Alright, ran some tests.
-inputDependencies doesn't have dependencies for jobs that require them to be completed before scheduling (so if I have 2 systems that do not depend on the same components, then none of them will be included in the dependencies to each other)
-If I have 2 systems that do not depend on the same components, then they will execute in parallel, but the system that was scheduled later will have to wait until there'll be a free thread.
mint iron
In Unity Physics they schedule lots of jobs from various systems, and then have a final system that and blocks/waits for all the previous jobs to finish. If you're scheduling the jobs early enough, they should have time to run and mostly finish before the waiting system is reached.
So if you wanted to do that, you keep a field in your system called 'FinalHandle' or whatever, and let other systems register themselves as a dependency via a method AddSystemToWaitFor() or whatever. That method does a JobHandle.CombineDependencies() and when your system Update runs you just FinalHandle.Complete() and it waits until all the jobs have finished.
In my current project i have a bunch of systems that are scheduled early in the tick and almost all in parallel jobs. These do all the processing work, in theory take the most time, and their output is queued events. Then i have an EventSystem which creates all the queued EventEntities in batch. Then another group that comes after, which does structural changes (pretty much anything with EntityManager). They're all grouped together at the end because they would force all jobs to complete.
The point of all that is to not interrupt jobs by having systems that wait for them to complete, you give em the space to run concurrently and be scheduled efficiently.
safe lintel
i personally haven't really forced any dependencies other than the UpdateAfter attribute for 2 systems, I just let unity figure out the scheduling unless I specifically run into a problem
low tangle
the automatic job dependency detection will work and automatically run the jobs at the same time assuming you have two job sets
joba [compA, compB]
jobb [compC, compD]
they will be able to run at the same time, because the signature of the job is used to dependency check that they operate on two different independent sets
the default is that it works magically like that
but two things break it easy you need to be aware of
*if the components are not all read only, say compA and compC, and said entities reside in the same chunk it will be unable to run both at the same time, but instead sequentially
*if there is a sync point at the end or start of either system, or in between either system in the update order they will have to finish the job that was scheduled before that point not allowing them to run at the same time
its usually possible to rework your logic to get better job utilization, make sure you use your tags to mark readonly, readwrite, writeonly so you can make a clear picture of what needs to be read at the same time
mint iron
what's the approach for sound currently? i see they have something in the works for it - ECS Track: Graph Driven Audio in an ECS World - Unite LA https://www.youtube.com/watch?v=kDE-KxQBiYQ but that was back in November.
cunning perch
Yeah, they're still working at it.
https://forum.unity.com/threads/dots-audio-discussion.651982/
hollow scroll
Have anyone find a way of doing Pure ECS with Character Animations? I was just testing Playables and IAnimationJob (https://github.com/Unity-Technologies/animation-jobs-samples), but that requires a SkinnedMeshRenderer, which is not supported if you go full Pure ECS
lime shoal
When DOTS will be fully released? Any information? I mean possibility of making any game on DOTS only.
safe lintel
@hollow scroll there is joachim's gpu implementation https://github.com/joeante/Unity.GPUAnimation
the full replacement is in development (and on the staging registry), they havent given really any info on it besides mentioning it may make it to preview late this year
coarse turtle
@lime shoal dots still has a pretty long roadmap and they're going to try and support the conversion workflow more from what I recall than a pure dots approach
hollow scroll
@safe lintel thx, I'll take a look at that
analog tangle
Yeah... last I heard they want to push stuff out as fast as they can, so they're leaning on existing MonoBehaviour functionality rather than taking the time to carve out an ECS workflow.
Makes sense... that's a lot of lead time... and the ECS design pattern will involve a lot of meandering so there'd be a lot of "chasing a moving target" if they spent too much time on the workflow anyway.
I think it's a super-super-long play anyway.
What propped up Unity is all the marketplace content. Focusing on converting core engine bits to C# and making them modular, with a dependency-walking package manager...
Not too far a stretch to assume their goal is to have an ecosystem of paid and free engine parts, which they toss in enough first-party modules to assemble a full game, but intending for the end-user to mix and match as best suits their needs.
upper tiger
Hi all,
If I have a job handle and that job has some command buffer calls, does calling Complete() on the job execute those commands on the main thread before any code after the Complete() line is executed?
low tangle
@upper tiger it does not, you have to manually play back the command changes in that case
upper tiger
@low tangle how do i manually do that?
low tangle
in that system, instead of taking a command buffer from say, EndFrameCommandSystem instead create the ECB yourself, feed it to the job, then after calling job.complete, run EnitityManager.Playback(ECB) iirc
I'll get the exact syntax in a snippet if you need it
hollow sorrel
@minor sapphire yea works with hybrid renderer
minor sapphire
ty
upper tiger
@low tangle ah yes that makes sense! thanks for that
low tangle
@upper tiger double checked the syntax just now, had it backwards! its ECB.Playback(EntityManager)
upper tiger
thanks ๐
low tangle
oh and dont forget to dispose the ECB after you are done
upper tiger
I'm a little bit wary that a job in my system relies on a previous job main thread instantiation
hopefully ill think of a way to parallel it properly
low tangle
ah, yeah I ran into that with my hybrid player object spawning system
I just had to rework it into a entityquery loop writing to a ecb, playback the ecb, run new entityquery loop hitting the types from a, then disposing the ecb, and letting the auto created ecb do the final changes from loop b
it dealt with gameobjects so it was all inside a stock non job system
upper tiger
ah ok
how are you finding hybrid to work with?
im going for a pure ecs approach and letting my project develop as they add more
low tangle
ecs logic and structing works very very well for everything but animation stuff
I'm really really ready for the ecs animation package
its very nice to just be able to entity logic for everything, then store a little lookup dictionary to find a gameobject and do some hybrid work on
upper tiger
agreed
I can understand why you would want hybrid as there are a number of systems that arent in pure ecs yet, but the speed gains im getting currently are insane, im hooked
got 10000+ entities spawning and destroying at 60 fps
low tangle
yeah its pretty fantastic for performance
my core game features, all the networking is pure ecs
and the performance is ๐ฉ
low tangle
also because its just public and so easy to get around, I'm able to just keep chopping off abstractions I've used in the past, so that say applying some rotations across the network is a single step
no lookups or complex entity ticking / deserialization
upper tiger
it kind of forces you to structure your code in a way that is efficient
upper tiger
agreeed
my code from previous years was a spaghetti mess of cross class static variable manipulation and checks
given i was still learning and not that great at OOC
but what killed me was the heap allocation and garbage collection just nuking frames
hollow sorrel
speaking of hybrid, whenever i need a gameobject i usually have it track its own entity and set its own transform.pos to its entity's Translation in lateupdate
but i could also imagine doing it the other way around and adding the monobehaviour on the entity in ecs and setting the transform from an ecs system
is there a good reason to do one over the other
low tangle
well not exactly
but theres a even better way using TransformAccessArray
you can set the gameobjects position and rotation from a job
never have to apply or lookup (which causes thrashing) data from a monobehaviour
hollow sorrel
ooo
i forgot that's possible now
so you usually have systems track the gameobjects then?
instead of gameobject track entity
low tangle
correct, and the gameobjects are marked with DontDestroyOnLoad as well
and the gameobjects when created are tracked in that system by adding a SystemStateComponent with a int id pulled from the gameobjects hashcode
so that way when something deletes that entity, that system can delete the gameobject
this also allows for you to treat that entity as the gameobject one in the same
low tangle
any other hybrid systems can grab the gameobject linked to that entity though the public dictionary lookup id->gameobject in it
hollow sorrel
i read about systemstatecomponents but couldn't think of a reason to use them
but that makes sense
low tangle
so you can say, put all your animator properties into a component, then grab and apply values to the animator from another system
but you never have to think of the animator, its just a component on a entity instead
I've been trying to keep everything in the 'ecs world' as much and long as I can
before going back to the slow messy hybrid world
hollow sorrel
yeah that sounds good
i've been using gameobjects for spriterenderer and have an ecs animator change the sprite
for now seems easier than making an ecs spriterenderer + tooling
low tangle
pretty much yep
god I cant wait till they finish rendering and animation into ecs
I want to do everything in ecs
hollow sorrel
hahah
upper tiger
Any idea if they have put UI stuff into ecs yet or soon>?
all my button on click listeners are static function calls
hollow sorrel
afaik they haven't mentioned anything and i don't expect them to any time soon because the new big thing for UI is UIElements
which isn't ecs
low tangle
UIElements seem super neat
hollow sorrel
and prob won't be cuz the editor uses it too
low tangle
it does have a slight data driven angle to it though
from one of the videos on it iirc
so I think it will end up getting a good integration eventually
hollow sorrel
yea probably
upper tiger
Honestly, as long as ecs can free a lot of heavy lifting to parallellism then im okay with a few things remaining in the main thread, as long as they make it nice to use both
hollow sorrel
there was a dude on the forums who made his own UI system in 2 weeks and it performed a lot better than ugui (tho then again what doesn't)
but there's no pure ecs mesh/font/etc yet so it was still using hybrid stuff
also there's still no materialpropertyblocks in hybrid feelsbadman
low tangle
yeah :(
hollow sorrel
low tangle
I had to come up with a new UI for my game because PowerUI was murdering gc every frame
so I went though a few
and man
they all suck so much
wait not powerUI
ZFBrowser
which is just CEF
so now I'm back on ugui because it at least works much faster for building in VR
hollow sorrel
yea
all the ui frameworks i've seen don't have great performance and ugui at least has nice workflow (imo)
man i thought of myself as a pretty decent unity and C# programmer but ecs feels like learning how to program all over again
every day is learning something new
low tangle
its a great feeling innit?
upper tiger
i love it
its like carving the fat off everything
it stops you from taking the first dumb idea you had and implementing it, cause it just doenst work
so you make a system that pertains to one piece of functionality in your game and all of a sudden you have good separation
hollow sorrel
yeah but it's strange because normally when you don't know the best way to do something in unity you google it and there's 100 dudes who already solved it and you can figure out the pros and cons of different solutions
with unity ecs it's like you gotta scour forums and blogs and hope you find one of the 10 people that have an overlap of both good understanding of ecs and also taking the time to explain to others
it's kinda like learning how to dev for VR
i know ecs and data oriented design is nothing new but still kinda feels like uncharted waters since there's not that many resources about it
civic field
Anyone using Rider with ECS? I cannot seem to be able to even open a simple project. Rider just hangs immedietly ๐คท Not even going to Unresponsive state just... stops doing anything (0% cpu usage)
vagrant surge
http://dataorienteddesign.com/dodbook/ obligatory plug, again
more for C++, but it explains how to architect stuff in the data-oriented model
upper tiger
@low tangle btw I tried using the EntityCommandBuffer and cant see any PlayBack method on that variable
low tangle
you sure?
low tangle
oh
you have to store the source non concurrent version
class DummyJobSystem : JobComponentSystem
{
struct DummyComponent : IComponentData { }
struct DummyJob : IJobForEachWithEntity<DummyComponent>
{
internal EntityCommandBuffer.Concurrent ECB;
public void Execute(Entity ent, int index, ref DummyComponent c0)
{
var a = ECB.CreateEntity(index);
ECB.AddComponent(index, a, new DummyComponent());
}
}
protected override JobHandle OnUpdate(JobHandle inputDeps)
{
var ECB = new EntityCommandBuffer(Allocator.TempJob);
var handle = new DummyJob
{
ECB = ECB.ToConcurrent()
}.Schedule(this, inputDeps);
handle.Complete();
ECB.Playback(EntityManager);
ECB.Dispose();
return default;
}
}
@upper tiger
upper tiger
if i have more jobs, can I just reuse the ECB before disposing?
upper tiger
@low tangle worked a treat
low tangle
correct
upper tiger
@low tangle
coarse turtle
Nice editor
amber flicker
Intuitively Iโd expect a bulk addcomponent operation to do something clever where it doesnโt do a full restructure of all the chunks until the end. Looking through source though, it just does a for loop of each one by one so Iโm clearly mistaken. If I want to e.g. change the archetype of half the entities of a certain archetype (which I commonly want to do), surely thereโs a way to do something smarter in bulk than add/remove components individually?
safe lintel
@civic field im using rider 2018.x to 2019.1 with unity ecs, no problems here. jetbrains support team is very responsive, maybe they can help if you submit something?
frosty siren
Got a strange bug:
When job down below compiled with Burst i see some strange things. Only the first schedul/run will work. When i attaching VS to Unity Editor i see like code execution goes to run(finalistsCount) and then goes nothing. If disable burst compilation on this job everything goes nice. Is there some problems of using CDFE/BFE inside bursted jobs or something else?
[BurstCompile]
struct PathFollowingJob : IJobParallelFor {
[DeallocateOnJobCompletion] [ReadOnly] public NativeArray<Entity> finalists;
[NativeDisableParallelForRestriction] public ComponentDataFromEntity<NavProgress> navProgressCDFE;
[ReadOnly] public BufferFromEntity<NavPath> navPathBFE;
[NativeDisableParallelForRestriction] public ComponentDataFromEntity<HexDestination> hexDestinationCDFE;
[NativeDisableParallelForRestriction] public ComponentDataFromEntity<MoveDestination> moveDestinationCDFE;
[NativeDisableParallelForRestriction] public ComponentDataFromEntity<MoveProgress> moveProgressCDFE;
[NativeDisableParallelForRestriction] public ComponentDataFromEntity<NeedStatus> needStatusCDFE;
public void Execute(int index)
{
var finalist = finalists[index];
var navPath = navPathBFE[finalist];
var navProgress = navProgressCDFE[finalist].index;
if (navProgress < navPath.Length)
{
var nextHexIndex = navPath[navProgress].hexIndex;
hexDestinationCDFE[finalist] = new HexDestination { hexIndex = nextHexIndex };
moveDestinationCDFE[finalist] = new MoveDestination { destination = HexInfo.XZPositionFromIndex(nextHexIndex) };
navProgressCDFE[finalist] = new NavProgress { index = navProgress + 1 };
}
else {
navProgressCDFE[finalist] = new NavProgress { index = 0 };
moveProgressCDFE[finalist] = new MoveProgress { progress = -1 };
needStatusCDFE[finalist] = new NeedStatus { need = Needs.idle };
}
}
}
also the results in a game looks like it actually works but wrong, and in a string
moveDestinationCDFE[finalist] = new MoveDestination { destination = HexInfo.XZPositionFromIndex(nextHexIndex) };
```nextHexIndex equal to zero, cause in game i get float2 (0,0)@upper tiger very nicely done!
upper tiger
video is pretty choppy, I had to keep removing frames until the gif was under 8mb
but works so smoothly and fast
low tangle
try recording with sharex video mode, or obs for smooth encoding even while taxed (assuming you use nvec)
I have to use obs if I want my videos to not be choppy in vr
mint iron
is there a way to set a LWRP shader property from ECS / ConvertToEntity ?
hollow sorrel
you can copy the material (in code), assign the shader properties and set it
but the usual way would be materialpropertyblocks and cant use that in hybrid renderer yet unfortunately
tho some dude on forums made a workaround
not home atm but search unity ecs materialpropertyblocks
mint iron
thanks, gives me a place to start digging
fathom trout
@frosty siren one thing to note- you can't debug burst jobs, which is why only the first schedule gets run when breakpointing it, probably due to how it compiles in editor as it runs (you need to disable burst compilation to breakpoint)
frosty siren
Ok now it's clearer, thank you for answer.
But i still can't understand how burst broke code execution, cause all goes nice after just disable burst compilation. I thought new update of burst package will fix it, but it's still broken
pliant pike
I have kind of a basic question but I couldn't find many examples. I'm wondering how I control the flow of the program with JobcomponentSystems, like how do I start and stop them from external scripts, how do I make them activate after another event or job, or activate after an input?
tawdry tree
Systems do their work as long as entities they target exist. You can intentionally stop and start systems, too.
Example OnUpdate from a system I have and use to spawn stuff
protected override JobHandle OnUpdate(JobHandle inputDeps) {
if (!Input.GetKeyDown(KeyCode.KeypadPlus)) return inputDeps;
var handle = SpawnThing(inputDeps);
_commandBufferSystem.AddJobHandleForProducer(handle);
return handle;
}
ah yeah thanks, I think the onupdate part in jobcomponents is the part that confuses me
tawdry tree
SpawnThing is a helper method to create a job to spawn a thing, by the way, it literally just creates and schedules the job. Said job is what actually spawns the thing, I guess. (using entitycommandbuffers)
pliant pike
yeah I was wondering how to create the job without scheduling it immediately
tawdry tree
Well,
var unscheduledJob = new JobOfSomeType{
//Setup variables
};
var scheduledJobHandle = unscheduledJob.Schedule(/*Relevant job arguments here*/);
It's not scheduled until you actually, y'know, .Schedule it. I highly suggest reading the examples and (crucially) make a very simple project that can spawn and despawn entities, and a system which moves them around or some other visible change.
pliant pike
yeah that's what I'm currently doing, I'm trying to use pure ecs though
I've seen the samples with rotating cubes but I feel like they are to simple
tawdry tree
You can't really do pure ECS if you want stuff rendered (technically you can, but it would be pretty damn complex, so not for ECS beginners)
pliant pike
I am rendering stuff simple cubes at least
It took me a while to figure out the rendermesh stuff
soft olive
Hey guys, I need some help dealing with Collision Filters.
First, the documentation seems to talk about the "Belongs To" and the "Collides With" fields, but through code (in the version I'm in) it's the "Category Bits" and the "Mask Bits".
Are those just renames of the same thing, or is there some mapping happening underneath that I'm not aware of?
And second, is it possible to have collisions enabled for two types of Physics Bodies and then filter out (i.e. ignore) those collisions in a cast/query?
soft olive
Nvm, got my answer. Category Bits and Mask Bits are just a rename, and yes we can filter out independent of collision setting on the Physics Body.
coarse turtle
๐
soft olive
The issue I was running into had to do with the fact that classic-unity colliders seem to be converted by the GameObjectConversionUtility.
Any idea how to control that?
coarse turtle
Right now the most sure fire soln is to create your own ConversionSystem to selectively ignore some Unity components
But doesn't Unity.Physics have their own colliders? I've only messed around with it just a bit
hollow sorrel
yea they do
but i guess they also convert classic ones to the new ones
i didn't know that
that's neat
soft olive
It's neat, but a little annoying because you can't control the collision filter on the classic-to-ecs collider.
soft olive
Because we're using Raycasts to be able to click on the object, yes.
Unless there's a better way to handle that?
hollow sorrel
you can raycast with the new physics
are you using both physics systems at the same time 
soft olive
Luckily, no actual classic-unity rigidbodies.
but yeah, we are kind of straddling classic with ecs a bit.
hollow sorrel
depending on how deep into physx you are, i'd just go all in and convert your current raycasts to ecs.physics raycasts (you can use them from monobehaviours too)
soft olive
Good point. Might we worth looking into.
It's hard handling ECS in a team when some of the people who write code aren't comfortable with ECS.
hollow sorrel
yeah for sure
the physics samples has some good examples for raycasting https://github.com/Unity-Technologies/EntityComponentSystemSamples/tree/master/UnityPhysicsExamples
i think mousepick uses it from monobehaviour among others
looks pretty much same as old raycasting just a bit different syntax
working in a team that doesn't have much exp with ecs is a valid reason not to use it, but i think using two physics systems at the same time would be more confusing in the long run
coarse turtle
^ that
I'm currently in a pickle where I'm using Unity's old physics system for 2d and I'm bundling it with my own custom physics code
stuck saffron
It definitely is. I made the conscious decision to remove the more performant physics stuff I wrote using Unity.Physics so we could stick with old physics to strictly avoid that confusion. Made sure to write the API in a way though that I can simply swap out between them in the future. Which I recommend if you think you will use it when it is stable
elder garnet
anyone here who knows a thing or two about textmesh pro?
untold night
@elder garnet - you should probably ask in #๐ฒโui-ux about TMP
native shuttle
Sorry if its an irrelevant question, but I havent used ECS in a while and didnt found update notes. Are we already allowed to create entitys in editor and set component data via EntityCommandBuffer?
mint iron
@soft olive here's how im doing raycasting if it helps. https://gist.github.com/jeffvella/fa5b7e4eaf3fa4b42e0b77ad26ed3535
@native shuttle you can only convert existing game objects to entities using the ConvertToEntity workflow, after the entity is created when the game starts you can't modify it in the editor, or change anything directly in EntityDebugger yet.
minor sapphire
can anyone tell me an intuitive use case for using BurstCompiler.CompileFunctionPointer<T>?
upper tiger
Hi all,
I have a entity which I change the material of like so:
var r = EntityManager.GetSharedComponentData<RenderMesh>(selected[0]);
r.material = outlineMat;
EntityManager.SetSharedComponentData<RenderMesh>(selected[0], r);
However when I change the material, the whole object disappears. The material has GPU instancing on. Whats the go?
tawdry tree
Could it be that the new material is broken/not renders?
upper tiger
the new material is created simply by right click -> new material
dont see why it would be broken
tawdry tree
Have you tested that it works on other objects?
upper tiger
this is the simplest possible object
wouldnt make sense to try on anything more complex
tawdry tree
What I meant was: did you make sure the second material(the one you swap to) renders, if you use that to begin with?
upper tiger
yeah i can put the material on an object and it works fine
tawdry tree
The whole hybrid rendering thing is a pain IMHO, and I don't understand it particularly deeply. Making sure both materials work was the only potential issue I could think of.
Actually... Have you tried making a completely new RenderMesh and just copy over the important fields?
tawdry tree
Might be that something gets lost because of value/reference weirdness
upper tiger
oh fuck me thats gonna be it
tawdry tree
Heck, even if that does work, make a forum post. Should be useful feedback to the Unity folks, and other people might have the same issue and stumble upon it
upper tiger
hmm still doesnt seem to work
amber flicker
just two quick things to check - if you 'change' the material to the original material does it still disappear? also, this isn't in a for loop is it (I notice selection[0])? just in case you're changing the archetype within the loop
upper tiger
the [0] is because im getting objects by tag in an entityarray and I know the object is the 0 one
var r = EntityManager.GetSharedComponentData<RenderMesh>(selected[0]);
RenderMesh newR = new RenderMesh();
newR = r;
//newR.material = outlineMat;
EntityManager.SetSharedComponentData<RenderMesh>(selected[0], newR);
This doesnt make the object disappear at least
basically just setting it to what it was before
although not sure if that is doing anything special
but if I uncomment that line out, it disappears
amber flicker
are you 100% that your material you were using has a valid shader & setup?
upper tiger
outlineMat = (Material)Resources.Load("_testMat", typeof(Material));
unless thats failing and not throwing any errors
amber flicker
I think this is the region in which your problem lies - does _testMat use the standard shader?
amber flicker
a wild guess is maybe using resources.load means the material is stripped or somehow unknown by the hybrid render system? Can you serialize the reference to the material for a test at least?
upper tiger
hmm
mint iron
This is the way its working for me. I'm using ConvertToEntity and LWRP. Maybe there is some special sauce that gets picked up from the clone that doesn't get setup up properly when using resources load. Its ripped form a larger snippet but you get the idea.
public void Test(Entity entity)
{
var selectedMesh = EntityManager.GetSharedComponentData<RenderMesh>(entity);
// Makes a copy using the one provided as a template
var newMaterial = new Material(selectedMesh.material);
newMaterial.SetFloat("_someProperty", 1);
selectedMesh.material = newMaterial;
EntityManager.SetSharedComponentData(entity, selectedMesh);
}
mellow remnant
I have a question about the ECS
i just watched the talk (https://www.youtube.com/watch?v=0_Byw9UMn9g) regarding it, i think i got everything boiled down, but i still dont understand how inheritance is solved with this system or replaced
or how to think about inheritance when working with ECS
amber flicker
I find it helpful to think in terms of composition - for the common behaviour, rather than put it in a base class, separate it out as a system
mellow remnant
yeah im just watching the system part
and realised i was again thinking about OOP
where data handles itself
here it's the system that handles the data
amber flicker
In a lot of ways I think DOD is actually more simple than OOP - after writing a couple of things you'll get it. The process is constantly something like 1) what is the data? 2) what operations do I want to do on that data?
If there is something in common (e.g. animal, ape etc get hungry), think about what data is in common and how you want to manipulate it. i.e. if it's Hunger well, that's just a piece of data that you can call Hunger and then write a system that changes it based on only the data related to what affects it
mellow remnant
tbh
i think the best anaology is...
a bag
vs a keychain
the Bag being OOP, a bag holds all the data but you have to carry it everywhere to reach into it and do something with it
while the keychain just keeps track of wherever that data is and you can do whatever with it while only using one part of it, and not carrying everything else around
(bad latter anaology, but there's no magical keychain yet which keeps track of every key that is attached to it without it being physically attached to it)
another analogy:
Pipeline vs Commander instruction list
ECS does a pipeline of operations with a wide range of data on specific parts of that data, passing onto each part of that data to the next set in the pipeline, literally "piping" it through
while OOP's design is basically an instruction list of a commander which tells all other Objects to do "something", which's something might involve telling other pieces of objects to do something
Every "person" (object) has to personally show up to get their order, forming a line where they probably have to loop through multiple times (again, maybe a bad analogy)
(sorry for just ranting in here, i had a coffee rush and an epiphany about this)
vagrant surge
@mellow remnant with data oriented logic, specially with stuff like unity ECS, you jhust dont need inheritance
at all
you can emulate the inheritance through composition
tawdry tree
The usual tip here applies:
Create a project where you can spawn and despawn entities, and make them move or some other simple, visible change. Doing so let's you get a 'feel' for ECS you just can't get from simply reading and watching talks. It's what I tell everyone new to and interested in ECS.
vagrant surge
i recomend more to check out Entitas
as it has proper editor support and its much easier to use
frosty siren
I spent a lot of time doing things without creating/destroying entities at runtime, but now i need to deal with it and the first step ended with obvious error: The NativeArray has been deallocated, it is not allowed to access it. I assume that happens because memory layout changes. So i have a couple of questions about it:
- What is the best practice now to avoid this problem? Using EntityCommandBuffer?
- Why creating new entity of one archetype broke accessing to another entity dynamic buffer?
tawdry tree
Which piece of code causes that error?
I have seen it before in a system when the person forgot to initialize it to begin with (oops!).
amber flicker
@frosty siren 1) If at all possible to destroy entities based on an archetype / query, try to use the relevant bulk commands as they're much quicker, 2) To add/remove components or create/destroy entities you need to be on the main thread. These operations all change chunks around. To get the benefits of jobs you can either a) fill a command buffer in parallel (then after the job completes, it will then block the main thread while it executes), or b) populate a NativeArray (if you know the number of elements your job will find) or c) populate e.g. NativeQueue (which has the downside that you have to dequeue on the main thread one by one)
frosty siren
No i can't do it based on archetype or query cause i need to destroy concrete entities and i don't know what archetype they matches.
I do it on the main thread.
I'm in case when one entity store a dynamic buffer which contains another entities. I try to iterate through this dynamic buffer and call Instantiate(Entity), but after first entity creation editor shows an error at the next try to access dynamic buffer.
amber flicker
hmm I haven't done much with dynamic buffers but if it's main thread and you're not modifying the buffer at the same time, that sounds like something I'd expect to work? I wonder does it work if you copy the buffer into a temp nativearray and use PostUpdateCommands?
frosty siren
PostUpdateCommands that's what i wanted to find. Is this an ECB that playBack automatically after Update of a system?
Stupid question cause it's clear by its name
untold night
as a note for PostUpdateCommands: It's also only valid during OnUpdate(), and it only exists in ComponentSystem, not JobComponentSystem
frosty siren
ok, what if i have 2 systems in which i use PostUpdateCommands. How do it more efficient? Is it ok that there will be 2 EBC that will PlayBack() somewhere in the frame? Or all PostUpdateCommands plays at the same time at the end of a simulation?
I see that in ComponentSystem.cs m_DeferredEntities (which returned by PostUpdateCommnad) call PlayBack inside AfterOnUpdate() method.
untold night
If you have 2 systems and use PostUpdateCommands, the structural changes will be applied before any other systems can run. It effectively turns the ComponentSystem into a main thread sync point.
You can use a EntityCommandBufferSystem, such as EndSimulationEntityCommandBufferSystem (or others depending on what group your system(s) execute in). To defer the changes to that particular sync point instead in your own system.
This is possible to do with a ComponentSystem, as well as in parallel with a JobComponentSystem
shut cosmos
is it fine to use components with a lot of variables (around 70) for ecs+jobs?
hollow sorrel
are those 70 related and are you making use of all of them in the same system?
even if the answer is yes it's prob still not ideal
i can't even think of a case where i'd have a class in OOP that has 70 variables and generally ecs is more split up
split that shit up fam
shut cosmos
I actually wanted to split them up, but there's a limit on how many components you can process at once with a system
I'm trying to implement features from stepmania into a rhythm game
in stepmania, you can have a lot of different effects on the notes, and they need to be calculated together
that's why I kinda need them to be easily accessible but I figured having a struct with that many variables wouldn't be optimal
frosty siren
@shut cosmos u can iterate manually through your components. I think IJobChunk is a best way to avoid limitations.
shut cosmos
what if I need to calculate everything together? each effect would affect the notes transform and color in different ways, so I can't have then on separate batches
safe lintel
go for it(with chunk iteration), but it sounds kinda like a nightmare ๐
mint iron
@frosty siren i've also had this frustration with DynamicBuffer because it gets invalidated when you do pretty much anything. I've had to on numerous occasions ToNativeArray() and then loop through that.
frosty siren
yeap, i use AsNativeArray() on my two DB and the first one ChildMemory works well, but the second one Visor still throw an error
mint iron
perhaps its because ToNativeArray() allocates a new and copies but AsNativeArray() will just use the same (invalidated) safety.
analog horizon
Hullo! So I've been dinking around with ECS and have run into trouble with my "GunSystem"
using Unity.Entities;
using UnityEngine;
using UnityEngine.Jobs;
#pragma warning disable 0649
public class GunSystem : ComponentSystem
{
struct Group
{
public Gun gun;
public Transform transform;
}
protected override void OnUpdate()
{
float time = Time.time;
foreach (Group e in GetEntities<Group>())
{
if (time - e.gun.lastFireTime > e.gun.cooldown)
{
e.gun.lastFireTime = time;
e.gun.createBullet();
e.gun.particleSystem.Play(true);
}
}
}
}
The code works fine with one hybrid entity, but once I duplicated the gun gameobject I get a bunch of errors in regards to
Unity.Collections.LowLevel.Unsafe.AtomicSafetyHandle.CheckWriteAndThrowNoEarlyOut```
Being called on the `foreach (Group e in GetEntities<Group>())` line. Does anyone know what I'm doing wrong here?its
Entities.With(myEntityQuery).ForEach((Entity entity, ref MyComponentData myComponentData) =>
{
//stuff
});
the old inject method is obsolete
the samples repo(see the pinned message) has examples of this too
analog horizon
Oh good to know! I'm using the package version 0.0.12 v.24 where would I find an updated version that contains Entities.With?
tawdry tree
Same place you got the original package - Go to the package manager and select the Entities package, should be an update button down right.
Any up to date getting started tutorial should go through that in a bit more depth.
analog horizon
Ah I see my mistake ๐คฆ I was attempting to find later package versions in 2018.3.6 thanks!
elder wharf
So, I'm sharing a small prototype of Entities.ForEach() without using delegates, I don't know if it would help anyone, but here is the code:
https://gist.github.com/guerro323/d03c875f454531b53f7aa08796483a63
small example for iterating on a data component and class:
// declare your variable as you'll reference them, they need to be in the same scope... (safety reason)
Translation translation = default;
Transform transform = default;
// 'i' is the current iteration.
// 'entity' is the... entity.
// you also have 'chunk' and 'entityIndex' for more usages... (for queries using 'Any' components?)
// you also need to end the function name with the type of components you want... (D = IComponentData, C = Class, B = Buffer, ...)
foreach (var (i, entity) in this.ToEnumerator_DC(m_Query, ref translation, ref transform))
{
Debug.Log($"i={i}, entity={entity}, transform gameObject={transform.gameObject.name}");
translation.Value = new float3(i);
}
It may help some people who hate using delegates (because of allocations?)
coarse turtle
https://forum.unity.com/threads/rules-and-whys-behind-blobs-and-colliders.690451/#post-4620274 just came across this for anyone interested in using blobs and wondering about how they work
soft olive
Have you guys noticed any weird crashing when you have many jobs that use Debug.Log (those jobs aren't burst compiled) ? Also, does anyone know how to use DebugStream?
frosty siren
@soft olive, I have not encountered such a problem. You can use debug from VS simply with calling run instead schedule on your jobs. And without [BurstCompile]
soft olive
@frosty siren oh, that helps. Thank you!
I am still curious about DebugStream though. I came across a forum post saying it was used in the physics debugger, but can't find the actual way it was used.
young nymph
hi all!! I wish improve my trails on my game using jobs + ecs.. does anyone has some resource to start learn? thanks
stuck saffron
Pins
frosty siren
@young nymph maybe this will help u https://learn.unity.com/tutorial/entity-component-system?language=en
young nymph
thanks @frosty siren I'll look
frosty siren
I need to iterate through NativeMultiHashMap with shared keys. I mean that values with the same key should be computed on the same thread. There is IJobNativeMultiHashMapMergedSharedKeyIndices that have ExecuteFirst and ExecuteNext methods. But i need to do some computations after first and all next values. In manual iteration i do it like
if (TryGetFirstValue(...)) {
do {...} while(TryGetNexValue(...))
//do some extra work here
}
How can i do it with HashMap jobs?
frosty siren
Does every HashMap jobs goes parallel per bucket?
fathom trout
i usually use an IJobParallelFor when iterating NativeMultiHashMaps
(& iterate them exactly the same as your code snippet)
frosty siren
yeah, but i need to have key array and GetKeyArray() returns not merged keys. I try to avoid extra work of sorting array
mint iron
Is there a way to do ConvertToEntity without it setting up the transform / localToWorld etc? i thought i remember seeing some attribute for that but can't seem to find it.
upper tiger
How do I get the children entities of a parent entity?
dull copper
New physics package https://bintray.com/unity/unity/com.unity.physics/0.2.0-preview
golden heron
Quick double checking, when dealing with parent and child relationships in ecs, you dont need to change the input for the child, you just continue using translation to move and rotation for rotation, and it just refers to the local space to the parent instead?
amber flicker
translation & rotation are always local, yes
@upper tiger likely you already know but the conversion system will populate a 'Child' array on the parent so accessing that would be the right a way. I think you want something like var children = EntityManager.GetBuffer<Child>(parentEntity);
the source of ParentSystem.cs may be useful to look at
golden heron
Good suggestion @amber flicker thanks
upper tiger
thanks @amber flicker that seems to work
random question
any idea how I can get shared component data from entity in a job?
something like ComponentDataFromEntity
amber flicker
as it's on the chunk level, you can use e.g. chunk.GetSharedComponentData
upper tiger
in a job?
i need get shared component dat afrom entity
i have a entity and I want to get its current rendermesh
amber flicker
all entities within a chunk will have the same rendermesh - if there is one entity per mesh that means each chunk will only have one entity in it
I'm not aware of any other syntax to get the shared component data from within a job on a per-entity level - I think deliberately as conceptually it doesn't make too much sense
upper tiger
i guess what im trying to achieve is this:
Take all entities with a component tag, get their current rendermesh and change the material to a random color.
I'm not even sure if its possible without a job. I guess I would have to create a new material for each entity.
lets say I have 100 objects from a prefab. They will all have the same renderMesh to start with but after I change their material im guessing each one will occupy its own archetype and chunk?
amber flicker
correct - and beware that you don't want to be changing archetypes within a job so you either just want to do it on the main thread in a normal system or else use a command buffer (I'd just go with a normal ComponentSystem)
that's why there's no api to e.g. set shared component data on a per entity basis within a job
amber flicker
yea, know the feeling ๐ - good luck!
upper tiger
another question. Is there a way to query whether an entity has a component type already
amber flicker
sure - you would usually define it as a requirement for the system via a query or an attribute to the system but you can also do it within a system using entityManager.hascomponent() or a job through chunk.hascomponent etc
upper tiger
ahh cool hascomponent
amber flicker
yes - also there are [RequireComponent()] and [ExcludeComponent()] attributes which can be useful for ComponentSystems
upper tiger
well its a job system where I raycast from my mouse outwards and if i hit an entity, i put a tag on it
problem is, if i click it twice, it tries to put the tag on it again and causes an error since it already has it
amber flicker
yea, so, I think it sounds like you wouldn't want the job system to be checking items already with that tag? So I'd put it as a component to exclude in the query
upper tiger
actually the job is an IJob so i can just return the selected entity natively and then do the check and dispose after
amber flicker
but the point is, if something already has the component that a system would add, you wouldn't want to waste resources and increase dependencies by checking those entities that already have it right?
upper tiger
true
the thing is the job isnt getting any sort of specific types
it just does a raycast which could potentially hit any entity in the game
im not sure how to exclude
amber flicker
cool, so yea actually although I said ComponentSystems I think it should just work if you add [ExcludeComponent()] above your IJob struct
upper tiger
struct Pick : IJob
{
[ReadOnly] public EntityCommandBuffer commandBuffer;
[ReadOnly] public CollisionWorld CollisionWorld;
[ReadOnly] public int NumStaticBodies;
public RaycastInput RayInput;
public void Execute()
{
float fraction = 1.0f;
RigidBody? hitBody = null;
if (CollisionWorld.CastRay(RayInput, out RaycastHit hit))
{
if (hit.RigidBodyIndex < NumStaticBodies)
{
hitBody = CollisionWorld.Bodies[hit.RigidBodyIndex];
//Debug.Log(hitBody);
commandBuffer.AddComponent(hitBody.Value.Entity, new Selected());
}
}
}
}
struct Pick : IJob
{}```so that will prevent the collisionworld from hitting bodies that contain that component type?
amber flicker
errrr actually good question
upper tiger
it doesnt btw
I might be able to pass the picked entity to another job that has an exclude on it
amber flicker
yes sorry for being unhelpful - I guess ideally you'd want the collision world to ignore those entities - I guess there's some kind of collision mask or something? Sorry I haven't done anything with physics yet
upper tiger
you are definitely helpful. It helps to think about these things critically and definitely jogs my memory as well
the more people that understand DOTS< the better
amber flicker
yea also I will just say that if this is the start of your own mouse-picking, a) I think it's likely someone/Unity? has already started some of this and b) I guess ultimately you'll end up removing Select at the end of every frame anyway otherwise you'll have no way to tell when something has stopped being touched? might be unimportant for your use-case of course
plus thinking of drag etc
upper tiger
the code is taken from unity physics mouse pick behaviour
the selected tag may exist for any amount of time
think about selecting a unit in a game, you can choose to have that unit selected for a while
tbh ill probably change it later, but i need to go through the process
amber flicker
cool, makes sense
golden heron
entities = AIoosQuery.ToEntityArray(Allocator.Persistent);
AIs = AIoosQuery.ToComponentDataArray<AISchedulerData>(Allocator.Persistent);
Why do these lines keep reporting the "A Native Collection has not been disposed, resulting in a memory leak"
???
I am disposing - in this case they are being set to by...
amber flicker
do you dispose them in ondestroymanager or similar? you know that once they appear, they're permanent until you restart the editor?
golden heron
only that... duh...
protected override void OnStopRunning()
{
base.OnStopRunning();
entities.Dispose();
AIs.Dispose();
OOSAIs.Dispose();
OOSentities.Dispose();
}
are there any other errors in your system? sometimes if another error is present, it doesnt run the dispose method
golden heron
no other errors.
Its a simple iteration of the arrays
and the activity is just incrementing a float for a timer
OnStopRunning is sufficient? It gets called in the editor too?
minor sapphire
hmm I would just be questioning if you really are disposing
golden heron
Clearly i am, it says...
minor sapphire
oh it didn't scroll
lemme read new comments lol
are the instantiation lines happening only once?
golden heron
The lines calling the number of entities and components are done every frame to update the information in case a ship dies or something
entities = AIoosQuery.ToEntityArray(Allocator.Persistent);
AIs = AIoosQuery.ToComponentDataArray<AISchedulerData>(Allocator.Persistent);
this is in onupdate
minor sapphire
that's why then
upper tiger
what is the necessity of persistent?
golden heron
not majorly important.
minor sapphire
you are creating collections each frame, but only disposing the last one
so you are not really disposing ๐
upper tiger
need to dispose in update
golden heron
Ideally, i would have liked to be able to make the nativearray and then dispose it in the same frame but the fucked up allocation setup moans whatever i do
I started with that being a defined variable in onupdate, with tempjob allocation, and being disposed at the end of the fraem
ie, last lines in onupdate dispose them
minor sapphire
there are multiple approaches you can take
amber flicker
TempJob is right.. I'm not 100% but I think the ..To..Array calls actually auto-dispose? @minor sapphire do you know?
golden heron
But im not running any jobs, so it seems to make the tempjob allocation fail, and if i try to use temp, it gives compiler errors
minor sapphire
I don't think they autodispose but I'm not a guru
golden heron
Either way, all i want is an array of the objects that acts like it should
minor sapphire
tempjob can be used even if you're not passing it to a job
golden heron
this is a ridiculously unclear setup
minor sapphire
๐
golden heron
Then why the fuck does it give errors? I do this...
upper tiger
You do have to dispose ToEntityArray
golden heron
OnUpdate
// allocate arrays
// iterate arrays
// dispose arrays
and still, i get not disposed errors
upper tiger
that seems like a normal approach ^
golden heron
Surely trying to dispose an array that does not require it should give a different error????
thats fucking stupid
golden heron
Please take note arkor, i am not calling you stupid
A Native Collection has not been disposed, resulting in a memory leak
minor sapphire
it should give you a line number if you turn on the leak detection
golden heron
It did... thats how i know the lines above...
I have been looking through all this there is no obvious thing being overlooked
minor sapphire
sometimes even when you think you are disposing so evidently, you're missing something... you really are just likely not disposing something...
full code?
upper tiger
^^
golden heron
gimme a minute ill put it back how it was originally
i fucking hate unity sometimes. I just put it back how it was originally... no error.
upper tiger
rip
golden heron
Actually... its not exactly the same.
i originally had the tempjob, now i have persistent, but that is the first thing i tried.
So yeah... sorted, but damn... that was stupidly unclear.
upper tiger
tempjob should be fine
i am wary as to why you need persistent being allocated in an OnUpdate
amber flicker
Joachim: We have landed necessary changes for entitycommandbuffers to be burstable in 19.3. ๐ - much excite
minor sapphire
hollyy ssheeeeeett!!!
upper tiger
ooo
upper tiger
do you guys know if theres a DOTS roadmap somewhere i can look at?
i want to know when they are going to tackle certain systems like animation, navmesh, UI etc.
golden heron
arkor, i did say, i didnt need persistent, i just changed it cos of the errors.
minor sapphire
dunno alkor I'd like to know too heh
golden heron
Im using tempjob again... i seriously think that this was a unity error bringing up errors that dont exist. I think possibly an interrupted execution of the system due to hitting stop. The system is now in exactly the same state it started in/
upper tiger
actually
in my experience I did come across allocation errors that would persist until i restarted unity
minor sapphire
โ me too
upper tiger
if you allocate a lot of things poorly, sometimes they dont all get GC'ed
basically the code you had was a programmed memory leak
persistent allocations every frame only being removed when you stop the game
golden heron
thats the errors i mean
the latter code i can understand there being errors in, what was annoying me was that one moment it was fine, the next throwing up errors all over the place and refusing to respond... beats head against the wall
Still, my original code was fine... i reverted back and no errors.
Isnt there an overrideable method in monobehaviours for "on hit stop" in the editor
?
minor sapphire
@golden heron it could have been a persistent error from a previous memory leak that needed to be cleared with an editor restart
golden heron
Yeah i tried that, i rebooted, but i had already been convinced that i was doing something wrong, i just wasted an hour and a half on this when there was nothing wrong with my code.
And.... i think i just figured out a better way to do this anyways.
minor sapphire
has anyone seen this before?
Unexpected exception System.IO.FileNotFoundException: Could not load file or assembly 'Smash, Version=0.3.0.0, Culture=neutral, PublicKeyToken=null' or one of its dependencies.
File name: 'Smash, Version=0.3.0.0, Culture=neutral, PublicKeyToken=null'
at Burst.Compiler.IL.Jit.JitCompiler.CompileMethod (Mono.Cecil.MethodReference methodReference, Burst.Compiler.IL.Jit.JitOptions jitOptions) [0x000bb] in <bbb9a8f1ad8e40f882ae7118e307be8e>:0
at Burst.Compiler.IL.Jit.JitCompilerService.Compile (Burst.Compiler.IL.Jit.JitCompilerService+CompileJob job) [0x002c6] in <bbb9a8f1ad8e40f882ae7118e307be8e>:0
While compiling job: System.Void Unity.Jobs.IJobExtensions/JobStruct`1<Unity.Entities.GatherChunksAndOffsetsJob>::Execute(T&,System.IntPtr,System.IntPtr,Unity.Jobs.LowLevel.Unsafe.JobRanges&,System.Int32)
I've upgraded to burst 1.1.1 from 1.0.4, had to replace the burst asemdef reference, but there are two others... what else broke??
safe lintel
there were notesnon the forums about burst upgrading, certain versions maynrequire some manual edits
'While upgrading from a previous 1.0.x version, you might encounter an error saying that the assembly Unity.Burst cannot be found and the GUID for the assembly doesnโt match. We actually unintentionally broke this GUID while moving from 1.0.x to 1.1.x, our apologies for this breaking change. In order to fix this, you need to manually edit your asmdef in case it is using the GUID. The most common case is while you reference Unity.Burst, you need to use GUID 2665a8d13d1b3f18800f46e256720795'
dull copper
Dots sample repo has now updated physics samples
minor sapphire
yeah I did the burst fix, this was something else, I restarted a couple of times and it seemed to resolve
safe lintel
ho hum, no notes of changes to raycasting from inside a collider?
soft olive
About what I was asking earlier about Debug.Log causing crashes on jobs, some poking around in the crash logs seems to tell me that sometimes Debug.Log causes an access violation. The very specific scenario that causes this is when I'm catching an exception and then using Debug.LogError from within a job. I also have a weird suspicion that volume (you see like 999+ errors within a second of this happening) has something to do with it.
I wonder if this is a known issue.
safe lintel
missed the conversation but you cant use debugl.log with burst if that hasnt already been ruled out
untold night
You have to use [BurstDiscard] on any Burst Job function that calls Debug.Log
something like this:
[BurstCompile]
struct MyJob : IJob
{
// data ...
public void Execute()
{
//do stuff...
Log(a,b,c);
}
[BurstDiscard]
private void Log(int a, int b, int c)
{
Debug.Log($"{a.ToString()}...{b.ToString()}...{c.ToString()}");
}
}
Does anyone know if I can have multiple materials on a RenderMesh? Similar to how in the editor you can add more than one material
upper tiger
interesting, so if you have a gameobject with two materials on it and you use convertToEntity script, it creates 3 separate entities
muted flare
can anyone write here json line with the newest entities package form packages/manifest?
"com.unity.entities"
: ver ??
cuz something gone wrong and I cant download it
by package manager
muted flare
oke, i figure it out now preview packages are hidden ๐
dry snow
Monthly check-in on the status of ECS: is it production ready?
vagrant surge
@dry snow Jobs and burst are, ECS no
tawdry tree
As before, however, it is at the level where you could consider if it is viable to use in your project, especially if you expect it will take a few years before you're feature-stable and start increasing your testers.
Mostly I suggest just keeping yourself updated for now, and playing with it to understand, to get the patterns and DOD and ECS.
safe lintel
my non professional opinion is that the entity component systems, jobs & burst are production ready. the ecosystem that dots comprises or will comprise of(physics, everything else) is not production ready
tawdry tree
You mean, if you use DOTS but make all systems yourself, it's production ready, but if you need/want builtin physics/animation/audio/etc, then it's too early?
trail burrow
basically
the core loop is there
but everything you actually need to build a complex game is not
low tangle
expect to use a lot of hybrid systems. but even still its a very nice change of pace how you build and manage a game anyways.
analog horizon
On a scale of 1 to 10 how normal is this amount of GC (in preview .24 it was way less 1k per frame)
safe lintel
looks like more than normal
hollow sorrel
is that with leak detection enabled
safe gate
Hey all! Long time since I've been here. How is ECS Physics looking? Does it have some sort of collision/trigger callbacks or do I have to do those manually?
pliant pike
I don't suppose anyone knows how I use a DynamicBuffer in an Ijobforeach do they?
this works in an ComponentSystem, Entities.ForEach((DynamicBuffer<EmployeeEntitysArray> currentbusiness) =>
surely there must be an equivalent way for Ijobforeach I just cant figure out the code formatting
safe lintel
there is an IJobForEach variant with buffers
IJobForEach_B<> & many other variants
last i used it they were read only though but maybe they fixed that by now
pliant pike
I think that works, thanks a lot
analog horizon
Gaah! It was with leak detection! Thanks for the help lol
safe gate
How would I use a EntityCommandBuffer on a ICollisionEventsJob?
Because I need the job index
[BurstCompile]
private struct CollisionEventJob : ICollisionEventsJob
{
public EntityCommandBuffer.Concurrent EntityCommandBuffer;
[ReadOnly] public ComponentDataFromEntity<PlayerTag> PlayerTagGroup;
public void Execute(CollisionEvent collisionEvent)
{
Entity entityA = collisionEvent.Entities.EntityA;
Entity entityB = collisionEvent.Entities.EntityB;
bool isBodyAPlayer = PlayerTagGroup.Exists(entityA);
bool isBodyBPlayer = PlayerTagGroup.Exists(entityB);
if (isBodyAPlayer)
{
EntityCommandBuffer.DestroyEntity(jobIndex????, entityA);
}
else if (isBodyBPlayer)
{
EntityCommandBuffer.DestroyEntity(jobIndex????, entityB);
}
}
}
This is just a test
safe lintel
i dont think you can use .Concurrent
safe gate
But concurrent is not used on jobs?
safe lintel
just means the job/command buffer has only one worker
so just remove concurrent and the jobindex parameter from destroyentity
to elaborate i dont think you can use concurrent commandbuffers with collisionevent jiobs
safe gate
ArgumentException: The previously scheduled job CollisionTest:CollisionEventJob writes to the NativeArray CollisionEventJob.UserJobData.EntityCommandBuffer. You must call JobHandle.Complete() on the job CollisionTest:CollisionEventJob, before you can write to the NativeArray safely.
EntityCommandBuffer was recorded in CollisionTest and played back in Unity.Entities.BeginInitializationEntityCommandBufferSystem.
call .Complete on its jobhandle ๐
safe lintel
what do you mean
safe gate
Because this job is used in a JobComponentSystem
And I run it on the override JobHandle OnUpdate method
Well, I just returned new JobHandle() and it worked!
Thanks a lot!
One question, if I want to keep track on which state the player is in, like moving or attacking, which approach is better? Having a component with a enum representing the state or a tag component for each state?
The tag component makes that systems have to iterate over less entities as they will have the tag or not, but I will have to make more usage of adding and removing componets.
Using the enum makes that the component is always there, but when systems want to iterate over entities with the StateComponent, they will have to iterate over all of them and check the enum value.
Am I right?
safe lintel
i think it depends how fast you plan to be switching states
so personally id use a bool or enum, and not be adding/removing a component for something like playermovement
safe gate
But if both players and enemies have this state logic?
Enemies would change less frequently, but the player will change as fast as someone can press a key
safe lintel
im personally doing attack/move as bool states rather than component, but for being Dead means adding a new component
safe gate
What do you mean by bool states?
safe lintel
well technically its just data within a component that the system determines if a character should attack or move, but I dont do any component changes for it
anyway im far from an expert on this at all so dont read too much into my take on architectural planning for your project!
not even sure id call myself intermediate lol
safe gate
I can't even call myself beginner lmao
How do I create a IJobForEachWithEntity where the entity does not have certain component?
safe lintel
[ExcludeComponent]
experiments with making ecs physics drive a skinnedmeshrenderer ragdoll https://gfycat.com/littlethickbrownbear lol so bad
tawdry tree
For a start, that's not half bad.... but it's 'just' a start.
safe lintel
yeah it is a start, funnily enough it was way more convincing without doing the legs and hips, as the joint limits from hybrid kinda holds the rest in place.
naive yacht
https://gyazo.com/ad252891d6c275509ab1ee4a37338fd1
guys when ever I go in the ball goes through walls
how do I fix this
agile dome
@naive yacht This is not the right channel for this at all. <#497874196274348032-489222168727519232-physics> <#497874004401586176-489222168727519232-general-code> or <#497872424281440267-489222168727519232-general-unity> would fit better
indigo orchid
Is there a recommended way to deal with optional components in job systems? Two example options:
Example 1: Use two different jobs where one takes the optional component and the other excludes it
public class AccelerateSystem : JobComponentSystem
{
struct AccelerateWithMaxSpeed : IJobForEach<MoveState, Acceleration, Velocity, MaxSpeed>
{
public void Execute(ref MoveState c0, ref Acceleration c1, ref Velocity c2, ref MaxSpeed c3) {}
}
[ExcludeComponent(typeof(MaxSpeed))]
struct AccelerateWithoutMaxSpeed : IJobForEach<MoveState, Acceleration, Velocity>
{
public void Execute(ref MoveState c0, ref Acceleration c1, ref Velocity c2) {}
}
}
Example 2: Would use the same job structure as AccelerateWithoutMaxSpeed but would use IJobForEachWithEntity and an ECB to check if the entity has the optional component
Alternatively, is there a better way?
safe lintel
something like: maxSpeedData = GetComponentDataFromEntity<MaxSpeed>();
if(maxSpeedData.Exists(entity){ //move with max speed params}
@indigo orchid
indigo orchid
Okay yeah, so that was what I meant wrt to Example 2 although I wasn't aware of the GetComponentDataFromEntity way,
Thanks!
mint iron
@safe gate which approach is better? Having a component with a enum representing the state or a tag component for each state?
Both options have a cost, its just about how you want to balance it. If you go with modifying data in a component that is always there, then if you have 100k entities you'll have to loop through all of them every time and check the component data. Which would be wasteful if say, there's only 5 entities you're interested in.
The Add/Remove component approach (in Unity ECS) is designed to solve that by physically moving entities together that have the same stuff, essentially pre-filtering. There's a cost to the add/remove you pay up-front, in exchange for the ability to easily loop through the entities you're after.
tawdry tree
To elaborate on the latter, adding and removing components costs a bit (as xzjv said), meaning the cost scales primarily, but quite drastically by frequency of change (and of course amount of changes, ie entities), where changing an enum takes perf hits primarily from the amount of entities you have to enumerate.
In other words, if you expect few entities, changing an enum might well be both fastest and easiest. It's harder to give a generic situation where the other option would be more ideal, simply because it's more complex (How many entities do you have? How often do each of them change state on average? Do you get times where a lot change at once?). Also consider easy of reading and maintenance/bug hunting.
As with all things perf, you can't really know until you test (profile) it.
low tangle
On the forms someone made a thread profiling the difference. Rule of thumb was 50k entities changing on desktop was peak. 10k was reasonable and 1k ish on phones. That is changing archtype every frame
Profile if you will have more than that changing every frame potentially. And also do chunk filtering when you can
For small simple things encoding logic as adds and removes is very powerful in how simpler it is to understand, create on the first place, and debug later on
fallen sleet
I think there's also some kind of fast path if you happen to modify an entire chunk in the same manner. For example, remove a tag component from every entity in a chunk.
low tangle
Correct you can get batching where it just changes the archtype int for the chunk and does nothing else
fallen sleet
Because then the entire chunk can just change archetypes, but the actual data does not need to be moved.
low tangle
But you should not count on things like that
fallen sleet
I'm not sure in which other cases that fast path fires, but AFAIK at least adding and removing tags is one such case.
low tangle
Entity query bulk adds I think do as well
hollow sorrel
https://gametorrahod.com/tag-component/ is the benchmark june means i think