#archived-dots
1 messages · Page 107 of 1
I only started using DOTS about a month ago but I already realized the benefits of using it so if I can I use it
excellent
So i am trying to raycast in a job but i am gettting errors(it was fine on ComponentSystem but not fine on JobComponentSystem)
my job:
public struct RayCastJob : IJob
{
[ReadOnly]
public PhysicsWorld physicsWorld;
[ReadOnly]
public UnityEngine.Ray ray;
public ComponentDataFromEntity<PlayerMovementData> movementDatas;
[ReadOnly]
public Entity playerEntity;
public void Execute()
{
uint layer = 1 << 8;
var input = new RaycastInput
{
Start = ray.origin,
End = ray.direction*1000 + ray.origin,
Filter = new CollisionFilter
{
BelongsTo = ~0u,
CollidesWith = layer,
}
};
physicsWorld.CollisionWorld.CastRay(input, out Unity.Physics.RaycastHit raycastHit);
var playerMovementData = movementDatas[playerEntity];
playerMovementData.targetPos = raycastHit.Position;
movementDatas[playerEntity] = playerMovementData;
}
}
OnUpdate :
protected override JobHandle OnUpdate(JobHandle inputDependincies)
{
if (clicked)
{
clicked = false;
var physicsWorldLocal = physicsWorld.PhysicsWorld;
var playerMovementDatas = GetComponentDataFromEntity<PlayerMovementData>();
var playerEntity = GetSingletonEntity<PlayerMovementData>();
var ray = Camera.main.ScreenPointToRay(pointerPos);
inputDependincies = JobHandle.CombineDependencies(inputDependincies, physicsWorld.FinalJobHandle);
var job = new RayCastJob
{
physicsWorld = physicsWorldLocal,
movementDatas = playerMovementDatas,
playerEntity = playerEntity,
ray = ray
}.Schedule(inputDependincies);
return job;
}
return inputDependincies;
}
i added [UpdateAfter(typeof(BuildPhysicsWorld))] attrbitute to system, its working now
Tip: put csharp or cs after the triple backticks (these: ```), and a linebreak immediately after that (no whitespace at the end of the line) for C# syntax highlighting.
This will make your code easier to parse.
public static void Main(string[] args){
//Example comment
ExampleMethod();
}
thanks 🙂
it still blows my mind how much you can drill down and speed up things that were already fast
moved 0.120ms on the main thread to 0.004 and shaved the final job (my code instead of the transport layer) down to this
just blows my mind
same june
i dont recognize that profile pic 👀
decided to update my pfp
ah cool
var transDatas = GetComponentDataFromEntity<Translation>();
var playerEntity = GetSingletonEntity<PlayerTag>();
var posArray = new NativeArray<float3>(1, Allocator.TempJob);
new FollowPlayerJob
{
transDatas = transDatas,
playerEntity = playerEntity,
pos = posArray,
}.Schedule(this, inputDeps).Complete();
UnityEngine.Camera.main.transform.position = posArray[0];
posArray.Dispose();
return inputDeps;```
So this is my camera movement job system, do you guys think that this is the proper way of doing this ?
nope but it will work
😄
i need to learn some hybrid approaches then, but most of the info is like 2018-2019
ah its a converted gameobject then?
yeah
yeah you will need to make a new authoring component for it
there isn't one for what you need to add to the entity
ConvertAndInject on the ConvertToEntity as well
What should i do when i am authoring tho ?
ConvertAndInject doesnt work since it gives me a warnning that converttoentity script wont work because its in subscene
yeah you need to pull the player entity out of the subscene
cameras and players are usually not part of the static physics and mesh world
they are the dynamic interactive part of the world and need to be handled like the special case they are (until unity converts everything to ecs in 2032)
alright done but they are not syncing, transforms i mean
aw man i hate this 😄
tried to add CopyTransformToGameObject but it doesnt work
nvm it does work, i was using [GenerateAuthoring] instead of my own
ironic I'm trying to do the same thing as well move a camera with a player using jobs, and figure out how to use Copytransformtogameobject
How do you add the Main Camera game object to the CopytransformtoGameobject component?
main thread,
EntityManager.AddComponentObject(your entity, maincamera's transform);
EntityManager.AddComponentData(your entity, CopyTransformToGameobject);
CopyTransformFromGameobject is a option as well
cool thanks, I wasnt if there was some other way
So i was wondering (again)
if i did entity query in a monobehaviour, will dependincies automatically handled ? and/or race conditions prevented ?
monobehaviours already update outside of the loop but you can't actually access any entites data without using a entitymanager method or a entityquery.ToComponentData() method
which all do a sync internally
in my monobehavour testing ongui's I typically do this:
oh i understand, is it because monobehaviours run on main thread right ? but what if some random system's job is running at the same time ?
oooh, it does syncornize always ?
the internal sync on those will force them to stop
it is smart where the types you are selecting from are the only jobs forced to stop
jobs that access those types, will be forced to sync if they are running at the time
ah great
@opaque ledge did you figure out how to get CopyTransformtoGameObject working?
Yeah i did
cool, you couldn't tell me how could you?
why wouldnt i 👀 , i would love to 😄
Fortunately its.. easy
dstManager.AddComponent(entity, typeof(CopyTransformToGameObject));
When i am authoring
and thats it really
I've got the component on the entity I just need to figure out how to add the object to the entitys component
oh ? like a monobeaviour ? well i was just going to experiment on that actually
EntityManager.AddComponentData(cam, new CameraTransferData {value = Camera.main});```
I think I need to do something like that except to update the objects camera position with the entitys camera
are your camera positions updating?
Well, if you just add CopyTransformToGameObject to camera's entity, and when that entity's translation/rotation change that entity's GameObject transform will change as wll
yeah
so you've just added the component to the entity and the gameobject is updating?
yep
I'm using a [generateauthoringcomponent] to add it maybe that is what I'm doing wrong?
ah yeah, you have to add that type somehow, is your camera entity instatiated on code or from editor/subscene ?
I'm just using converttoentity with inject thing
I can move the entity camera version just the gameobject is not moving
Are you using a tag for your camera ?
then you can make a custom authoring for that and simply add the type
you just need to create a new monobehaviour to add onto that gameobject thats being converted and injected
dstManager.AddComponentData(entity, new CameraTag());
dstManager.AddComponent(entity, typeof(CopyTransformToGameObject));```
this is my custom tag authoring
no I'm not using a tag didn't think I needed to
the convert and inject already attaches the Transform for you
so you only need the CopyTransformToGameObject
I dont understand
[GenerateAuthoringComponent]
public class CameraTransferData : IComponentData
{
public CopyTransformToGameObject value;
}
that is what I'm using on the camera object to add it to the entity
No you dont need to do it like that
sec
[DisallowMultipleComponent]
[RequiresEntityConversion]
public class CameraTagAuthoring : MonoBehaviour, IConvertGameObjectToEntity
{
public void Convert(Entity entity, EntityManager dstManager, GameObjectConversionSystem conversionSystem)
{
dstManager.AddComponentData(entity, new CameraTag());
dstManager.AddComponent(entity, typeof(CopyTransformToGameObject));
}
}
So this is my camera tag authoring, authoring means that unity will run this code to convert your gameobject to an entity
So you can make an empty authoring for example and simply drag it to your camera gameobject
[DisallowMultipleComponent]
[RequiresEntityConversion]
public class EmptyAuthoring : MonoBehaviour, IConvertGameObjectToEntity
{
public void Convert(Entity entity, EntityManager dstManager, GameObjectConversionSystem conversionSystem)
{
dstManager.AddComponent(entity,
typeof(CopyTransformToGameObject));
}
}
For example
you can literally just copy this and make a script, and drag it to your camera gameobject
@low tangle I was wondering (again) i added a monobehaviour to my entity in authoring, i set the entity field in my mono to entity that is being converted, so... is there way to access this mono from jobs ? or does monobehaviour always needs to check the data that every frame ?
enjoy Calabi 😄
you can just get them like any other component
no ref but just place them first inside the lambda
well, when i want to get data from entity inside a job, i use GetComponentDataFromEntity<> and Buffer version, i didnt see any object version
ooooooh
wow i never knew that
there isn't a way to get the monobehaviours in jobs
so always from ForEach ?
and for direct getting you can use EntityManager.GetComponentObject()
i thought ForEach and Jobs were the same
i think you or someone else said that ForEach constructs those job structs, so it didnt really matter
as long as you do .Schedule at the end
yeah it makes a job for you as long as you are using .Schedule()
if you use .Run() its going to run on the main thread
@opaque ledge You should really really watch that pong video I linked to you yesterday. It explains all this very clearly
and even still if you dont use WithoutBurst() you cant use managed stuff (monobehaviours) anyways
i actually watched it before you mentioned that link but i wasnt really paying attention to it much, i should check it again
so the only time you can use monobehaviours is if you .WithoutBurst() and .Run() on a ForEach in a JobComponentSystem
but dont worry about that really, if you need to use monobehaviours they have to be used on the main thread anyways
So, in order to reach monobehaviours that i added to my entities, i have to run ForEach on my JobComponentSystems with .Run()
yeah
which includes all classes
Or get it directly from the EntityManager with GetComponentObject
If you already have the entity
thats still main thread only access
one think about GetComponentObject I added an object supposedly to an entity how do I find it?
even with main thread mb stuff you could be writing in serial processes / loops / foreach's
we are just literally talking about that Calabi 😄
I know I tried this
are you using this 'hybrid' approach for your UIs June ?
EntityManager.AddComponentObject(cam, Camera.main.transform);```
but then I dont see how to find it with GetComponentObject
for UI and quite a bit of stuff
but I've reworked lots of stuff where its all done in entities, then a few linked entities is where it 'writes out'
like Sark said, you can use EntityManager.GetComponentObject<T>(entity) Calabi
I'm trying to minimize the number of hybrid things I have to the bare minimum
yep
oh I see yeah thanks
i was trying to use these monobehaviours to 'interact', like.. to show UI and such
instead of interaction with the mb's or invoking events on them
modifiy the mb's public data to be exactly what you want
yeah, thats what i was going to do
hopefully
when my entity dies, will that mb also dies ? will it's memory cleaned etc.
actually nvm, i probably could have a bool field in mb and if thats true i simply destroy mb
Yeah it won't be deleted
But you can call GameObject.Destroy from anywhere if you want to removed it
So i was wondering again
in burst documentation it says using wide primitives such as float4, double4 etc, should be preferable because of performance reasons, so i have my player movement data and i have float3 for target position(where to go) and float for player speed, should i make them into float4 ?
This session addresses how we are expanding the scope of the Burst Compiler to enable even the most demanding, hand-coded engine and gameplay problems to be expressed in HPC# via direct CPU intrinsics. Andreas shares the reasoning and use cases; as well as discussing implemen...
yeah i am just watching that actually xD
😅
i gave up yesterday because it was too techincal, but imma watch it
How to read NativeArray<float> as NativeArray<float4> ?🤔
sounds like more trouble than its worth
😄
anyone benchmarked a dots+burst raycasting job vs traditional method? Say 10,000 casts, are we talking a (numOfCores * 0.5)% difference or something even more significant?
if you compare jobified DOTS physics raycast, at least compare it against RaycastCommand on that implements jobified raycasts on physx
k, do u know if anyone has done this already?
I'd expect the results depends also on the way you use it
I dont really like comparing things randomly without a real use case
I am just thinking its fairly common to queue up a bunch of raycasts for say projectiles (say big space battle) per game step then resolve them for the next frame, this can be fairly expensive.
There is a way to do it as batch in their manual but i never tried on performance
Sigh, I see PhysX added origin shifting, but doesnt appear either havok or unity physics has similar functionality. I am curious what type of performance could be achieved in a large world with unity physics + dots and a origin shifting scheme.
There was a post not too long ago from a Unity dev who said they are working on origin shifting
Or "floating origin" he called it which I assume is the same thing
tbh the batch raycastcommand for physx might rival the dots raycast but if you are anchored to physx then you wont be able to scale as well as dots if you wanted say 10,000 spaceships to be raycasting using gameobjects. like 0lento said a use case is what matters, both are really performant though.
queries are supposedly a thing where unity physics should be really performant
at least according to their own docs
its funny, the perf of raycastcommand was really good when I was using it, if it hadnt been for the clunky way of using it(ie it is its own job rather than being able to use them in jobs like dots version) i might not have gone the full ecs route
technically, the Unity Physics docs do imply that you should actually do similar approach with it's raycasts if you want optimal performance (add all raycasts operations to a native container and perform them on a separate job
oh wait, they just said that if you don't use it on burst jobs already, you should
so nevermind
ya I was wondering if burstCompile is where u would see the big speed up, due to SIMD and the noaliasing stuff
It seems like I found a bug setting InverseInertia all to zero crashes unity
most of the physics stuff will already be bursted so your code will be the only slow part, until you burst compile it as well
physx got jobified a bit in the latest versions and its pretty dang fast
you really shouldnt think of ecs - gameobjects as either or
Hey has anyone seen this exception
Unity.Properties.CodeGeneratedPropertyBagRegistry.System_Collections_Generic_List_1PropertyBag has invalid vtable method slot 4 with method
happens only in Mono build 😦
so doing some digging, propertybag's are generated for ISharedComponentData's and object IComponentData's
What are the rules around them? no generices? no other managed references??
yep, afaik no generics and no managed references in component data
You can have managed references with unsafe code, I'm not familiar with the process though
And you can have generics, you need to explicitly declare them, I forget the syntax. I'll see if I can find it
There we go, it's in the changelog
[assembly: RegisterGenericComponentType(typeof(TypeManagerTests.GenericComponent<int>))]
hmmm... there must be some more complex rules somewhere
as i changed/removed most of my object IComponentData's
i still have some generic types but i have them all registered like that
and im now getting
TypeLoadException: Type Unity.Properties.CodeGeneratedPropertyBagRegistry.System_ObjectPropertyBag has invalid vtable method slot 4 with method Unity.Properties.PropertyBag`1<object>:Accept<TVisitor> (object&,TVisitor&,Unity.Properties.ChangeTracker&)
Not sure, sorry, those are some pretty damn arcane errors
yup 😛
Are you using the latest version?
i think so
0.5.1?
All I can suggest is to post about it on the forum. Make sure you make it clear in the thread title that you're dealing with an indecipherable error, hopefully that will make someone from unity pop their head up since they seem to care about making the errors more user-friendly
yea thats a good idea
im loving most of this DOTS stuff
but when getting errors like this it drive me mad
😛
Yeah it's insanely frustrating when you get an error that doesn't explain anything
maybe upgrading to 2019.3.0f6 will help
ECS works for 2D projects as well, right?
yep
a terraria clone was my first project to test ecs when it first came out over a year ago
Thanks, good to know.
Hmm, my text is shown properly on subscene but not when its turned into an entity 🤔
Is there an alternative to OverlapBox with PhysicsShapes?
@plush portal have you read this? https://docs.unity3d.com/Packages/com.unity.physics@0.2/manual/collision_queries.html#query-types
I didn't, thanks!
RTM
What is DOTS? How is it different than ECS?
I don't suppose anyone knows how or if you can draw stuff in the editor view, something like this using dots
@pseudo plover DOTS = Data-Oriented Technology Stack , it's just Unity's umbrella term to communicate everything that goes under this tech, including their ECS, burst compiler, job system, all compatible packages etc
I do notice that Unity deliberately tries to stop using term ECS on their promo pages, which can cause some confusion: https://unity.com/dots
I see that page also got some weird sentences like, "The DOTS stack consists of a growing number of packages." where they are essentially saying: "the data-oriented tech stack stack consists of.."
(yeah, I'm nitpicking :p)
@pliant pike Unfortunately I don't think there's anything official, I think you would have to manually create a world and do updates in the editor. Something like https://forum.unity.com/threads/editor-scripting-with-ecs.527687/#post-4845995
With the conversion system worfklow it seems like they want us to avoid things like this and just use the normal EditorWindow/Inspector workflow to represent our entities at edit time
thanks, figure's so how would I display things like lines bezier curves just in the editor view?
Should be plenty of material online about creating fancy editors. Once you have a normal editor working you then just have to set up a IConvertGameObjectToEntity script to convert your edit data to entities
yeah I'm looking at one of the physics.examples and they seem to use a monobehaviour with OnDrawGizmos to draw some things maybe that will do
Yeah they have a whole fancy system for that, it seems like they've taken steps to make it less accessible for us to use, but it should be possible
then again it may be to complicated for me
I would honestly recommend just learning the normal workflow for creating fancy editors. It's a completely separate problem from ECS so there's plenty of material online about it and that style of editor isn't going anywhere now that they've made the conversion system the "normal" way to do things
I'll give it a try yeah thanks
its not nitpicking, thats a totally valid complaint
it should be "The DOT stack consists of a growing number of packages."
or they should just not use acryonyms
I don't know about that, most people accept ATM machine as completely sensible english
accept, but many more just say "I need to go to the ATM"
It's just about what sounds more natural, and dots stack sounds natural
if they want natural they should just say "The Data-Oriented Tech Stack consists of"
also, its not casual english, its documentation and press releases, grammar matters in these contexts
Also in Technical Documentation the grammar is exceedingly important. The text should be technically accurate at all times to minimize confusion for those being prudent in their reading
heh my username overlaps with this; the le baron or the the baron 😄
I thought you were thebaron all along,
I must have tuned out le
Is there any particular trick to using concurrent EntityCommandBuffer.Concurrent with IJobForEachWithEntity? I manage to use it just fine in my other jobs (IJobChunk), but now I suddenly run head into The previously scheduled job BleedOutSystem:BleedOutSystemJob writes to the NativeArray BleedOutSystemJob.Data.CommandBuffer. You must call JobHandle.Complete() on the job BleedOutSystem:BleedOutSystemJob, before you can write to the NativeArray safely.
public class BleedOutSystem : JobComponentSystem
{
[BurstCompile][ExcludeComponent(typeof(DeathTag))][RequireComponentTag(typeof(BeeTag))]
struct BleedOutSystemJob : IJobForEachWithEntity<SecondsToDeath>
{
public float DeltaTime;
public EntityCommandBuffer.Concurrent CommandBuffer;
public void Execute(Entity entity, int index, ref SecondsToDeath secondsToDeath)
{
secondsToDeath.Value -= DeltaTime;
if (secondsToDeath.Value <= 0.0f)
{
CommandBuffer.AddComponent<DeathTag>(index, entity);
}
}
}
private EntityCommandBufferSystem mCommandBufferSystem;
protected override void OnCreate()
{
mCommandBufferSystem = World.GetOrCreateSystem<EntityCommandBufferSystem>();
}
protected override JobHandle OnUpdate(JobHandle inputDependencies)
{
var job = new BleedOutSystemJob();
job.DeltaTime = Time.DeltaTime;
job.CommandBuffer = mCommandBufferSystem.CreateCommandBuffer().ToConcurrent();
return job.Schedule(this, inputDependencies);
}
}
i think you need to add the job to system's producer or something, so you need to do something like
var jobHandle = job.schedule(this, inputDependincies);
mCommandBufferSystem.AddHandlerForProducer(jobHandle);
return jobHandle;
Ah, now it stopped complaining. Thank you, appreciate it :)
👍 , also i think you should get EndSimulationEntityCommandBufferSystem's command buffer.
I am not sure if it actually makes any difference but EntityCommandBufferSystem itself is abstract class, also in their manual they use the class i mentioned
but glad its working now, happy hunting 😄
I'll look into it, thanks for the tip :)
* BeginInitializationEntityCommandBufferSystem
- -
- -
- -
* EndInitializationEntityCommandBufferSystem
* SimulationSystemGroup
* BeginSimulationEntityCommandBufferSystem
- -
- -
*TransformSystemGroup
* EndSimulationEntityCommandBufferSystem
* PresentationSystemGroup
* BeginPresentationEntityCommandBufferSystem
- -
- -
- -``` They are default sync points. For example if you spawn something with Transation and LocalToWorld in simulation system group using EndSimulationECBS, Translation won't be applied to LTW untill next frame and this something will render in 0,0,0🤔 So choose wisely😄
@warped trail this may be why my instantiations keep flashing for one frame on 0,0,0 thanks
you can just set Translation and LocalToWorld at the instantiation to avoid this
ooooh 👀 thats literally happened to me
i do set the translation but didnt bother with localtoworld
same here thanks man
I have a job system that creates nativecollection. What is the recommended for passing this data for other systems to use.
NativeList<float> list = GetComponentSystemFromWorld.Distances?
Entities.WithReadOnly(list)
Use a dynamic buffer
You can attach it to an entity in the creating system and query for it in the consuming systems
Otherwise you're headed for dependency management hell
you can also use tag components
NativeList<float> was just an example, I am using https://github.com/marijnz/NativeQuadtree as a broadphase to figure out close elements.
@twin raven I am not sure what you are trying to do, but you can have multiple jobs in a single system
But I also find the quadtree useful in other systems
Then yeah what Sark said, you write to Dynamic Buffer from 1 system, and read those and process from another
@twin raven If you're absolutely sure your data won't fit into a ComponentData/DynamicBuffer then you'll have to basically do what you showed and keep track of any consuming systems. Basically think about how the EntityCommandBuffer pattern works with AddJobHandleForProducer. That's what you need to do.
also, i think you need to make sure write system updates first and read system updates later
not entirely sure
@twin raven If you're confident you could also take a look at fholms collections. He made a bunch of fancy collections that can be used in components, so you can basically use them like a dynamicbuffer. You could do whatever he's doing and re-create a quad-tree in the same way
No worries, but you should thank @trail burrow , he wrote it, hahah
So
I have this piece of code for updating UI, as June advised
protected override JobHandle OnUpdate(JobHandle inputDependencies)
{
var cameraEntity = GetSingletonEntity<CameraTag>();
var rotations = GetComponentDataFromEntity<Rotation>(true);
var positions = GetComponentDataFromEntity<Translation>(true);
Entities.ForEach(
(Entity entity, TestMono testMono) =>
{
var entityPosition = positions[entity].Value;
var cameraRotation = rotations[cameraEntity].Value;
var cameraRot = new Quaternion(cameraRotation.value.x, cameraRotation.value.y, cameraRotation.value.z, cameraRotation.value.w);
var canvas = testMono.worldUI;
canvas.transform.position = new Vector3(entityPosition.x, entityPosition.y - 2, entityPosition.z);
canvas.transform.LookAt(canvas.transform.position + cameraRot*Vector3.forward, cameraRot * Vector3.up);
canvas.transform.Translate(Vector3.back, Space.Self);
})
.WithoutBurst()
.WithReadOnly(typeof(ComponentDataFromEntity<Rotation>))
.WithReadOnly(typeof(ComponentDataFromEntity<Translation>))
.Run();
return inputDependencies;
}
The thing is.. one of my systems changes Rotation, so this system gives me error, one thing i could do is adding UpdateBefore, but that means text position will be moved before entity position changes so it gives 'glitches' so to speak(since it will update its position 1 frame later).
I am guessing its because this job system isnt depending on inputDependincies ? but since Run runs on main thread it shouldnt either way right ?
Ok i added "AlwaysSyncSystem" it works properly now
I should rewatch that ping pong video before Sark beats me up
DynamicBuffer methods create sync points, yes?
Do they just play back at the end buffer command system of whatever group they're processed in?
A lot of DynamicBuffer methods call for reallocations, it seems.
reallocations of plain old data don't cause sync points.
It's structural changes.
You can change the buffers contents in a job as long as it's not adding or removing the buffer itself, writing to the buffer doesn't change the entity's structure, only the buffer pointer in the DynamicBuffer itself.
Of course, you can't write/add/remove elements in the buffer from multiple jobs, but that's the case with most data access in a job anyway.
Am I doing something wrong if I am using the same sub-scene for two different scenes?
i dont think so, but pretty sure you cant use the same subscene twice in a scene
It's fine, I am not using it twice in the same scene.
Hi all! I'm getting started with DOTS and was curious what are the latest versions of hybrid/entities/physics/netcode that work well together? I've spun up multiple projects recently and always run into issues. Even using the versions from the DOTSSample are causing me problems. I'm running on 2019.3.0f6 and usually just setup the simple netcode cube demo. At the moment i'm just running hybrid/entities/physics with a physics block above a table. When it hits the table unity crashes with BlobArray.cs: IndexOutOfRangeException: -1 is out of range Length 24. But that seems to be a common error across my setups. Any advice would be appreciated! 🙂
I am using ECS in a more hybrid way, and I decided to refactor my code by following some sort of principle/pattern I thought of.
Basically, the concept is that my entity world only handles game logic. And the managers, UI, etc. are all in the gameobject world. The manager sends events to the entity world (by creating/setting entities with "event" components), like movement/input events and other stuff that changes game logic.
And so the "principle" I am trying to follow is that the entity world should never be aware of the game object world, and only knows about those events it receives.
My game/code flow feels cleaner like this, but I don't want to hit a wall and have to refactor again later.
So am I going in the wrong direction with this?
Should I go back to having "manager" components that handles inputs and acts on game logic in my entity world?
Will I regret structuring my game like this later?
Aint really gonna know till later truthfully - you can plan ahead, but needs can change - I typically didn't find it too much of a hassle restructuring since the data stayed the same - just the system order/logic ended up changing (I dont exactly know the full context of the manager stuff so I can't really comment much about that)
@surreal grail Last week you were asking more or less if you could do polymorphism in burst? Turns out you kinda can:
public interface IVisibilityMap
{
bool IsOpaque(int2 p);
bool IsInBounds(int2 p);
}
public static NativeHashSet<int2> VisiblePointsInRange<T>(int2 origin, int range, T visibilityMap) where T : IVisibilityMap
{
NativeHashSet<int2> points = new NativeHashSet<int2>((range * 2) * (range * 2), Allocator.TempJob);
new GenericFOVJob<T>
{
origin = origin,
range = range,
visiblePoints = points,
map = visibilityMap
}.Run();
return points;
}
You just need to ensure the compiler generates the concrete struct via generics
Works beautifully
I was kinda amazed, I thought for sure Burst would thwart me somehow
@plush portal you wont run into issues that way, but you need to think about your code base and keep everything separated proper. its far to easy to screw up your separation and have something to fix later on
at first I actually did what your doing now, but I eventually figured out clean ECS is way more sane to manage on large projects instead of lots of monobehaviours
so I slowly transitioned from gameobject / MBs doing some logic, to porting it all over to MBs processed by systems
then transitioning those MBs to full ecs systems, with proper componentdatas
then after that I've moved to a setup where a few select systems handle output one way to gameobjects
and a few systems work on a process MBs way (mostly built in stuff like Animators)
@coarse turtle
Okay.
So basically, it's like a sandbox, and the player can interact with objects in the sandbox(entity world), selecting them, moving them, etc.
And for a comparison here's an example on how I would do manager stuff:
Before, I would handle selecting objects with a component that stored the selection area's coordinates(basically a selection manager) and had a system that did the casting when it recieved inputs. A scriptable object contained the component, so I could retrieve it in my UI.
Now, I have the selection manager in the gameobject world, and it recieves inputs sends an "event" to the entity world, which is retrieved by that systems that does the casting and destroys the event.
That way I can easily disable and enable features by removing monobehaviors, instead of having to disable systems, while useless "manager" entities hanging around in the entity world.
I can make new gameobject scenes that interacts with the same entity world differently, so I can have a scenes where the sandbox features are different, or even make a playable world scene and throw in a physical player.
ah you have it backwards in the ecs world
you shouldn't be disabling systems ever
just removing data that causes things to happen
Yeah that's what I am doing right now.
Since the monobehaviour doesn't throw events(entities) when desactivated, the system doesn't do anything because it doesn't find the data.
Raw input is polled and set into values on a CD, every frame, always if the input poll component exists
the input component also has the KeyCode's it should use (so another system could update or rebind them if needed)
another system, handles all RayCastInteractiors
every frame these are processed and the pimary selection is added if one exists
another system then processes InputPoll to see if Activation is being pressed this frame, which then emits Activation (point that interaction entity) entites
Sounds like you're doing a simple messaging system from MonoBehaviours -> Component Systems, that should be fine as long as you know how those events are triggered, should be easy to debug too
then instead of processing all Activations, its done in reverse, and all Interactable things are looped though of x interaction type. At the start of the system all activations are hashed to their targets. This allows for the exact kind of interaction to check the hashmap if that entitiy is in the map, its being activated this frame
this inversion of processing allows for grouped types of interaction entities to be processed of the same kind
since Activation events are separated from source, you can simply change what emits them, instead of a RayCastInteractor it can be a PointInteractor
this also alows for activations to even come directly from the server, or scripting languages
at no point does any of that pull in data it doesnt need. and only the very ending might even touch a mb/gameobject. like for InteractToggleGameobject
also if you read the DOD book
the input processing first step is a condition table style
and activations are emissions
and all of it is done as existance based processing
since if the entity or data doesn't exist, then nothing happens
want to disable interaction while in a menu?
add a condition to the input polling stage
Thanks
I can help explain better if you have questions
events from monobehaviours is a good approach though
I use that for my http / web stuff
they produce entity events
same here regarding addressable assets - until an alternative comes out - it works for me for now 🙂
Yeah that's a lot of stuff all at once, but I get the general idea.
I would have some questions, but I have to go, so I might ask them another time, if you don't mind 🙂
yep I'm around if you need me
I dont like just directly giving away my source code to my game but I'll see what I'm comfortable with
Nah, I won't try inquire your source code from you, there's just some stuff you said I don't fully understand 🙂 Anyways, goodbye.
@zenith wyvern ^^^^ that's very nice! thank you! 🙂
Not sure if this is the correct place:
Trying to replace the Colliders on my gameobjects with the new Unity Physics Shapes, however the object are made with probuilder. This means I cannot link the mesh in the Physics Shape component when changing the shape type to mesh (unless I export them). I tried to make an editor script that runs through my selected objects to replace the original Collider with the Physics shape and link the mesh on the object in there. Is there a way to achieve this? It seems the Physics Shape component is made up from different ComponentData and only visible in the editor for easy use.
@vale nymph Have you seen this video? https://www.youtube.com/watch?time_continue=1274&v=tI9QfqQ9ATA&feature=emb_title
This session gives an overview of the physics systems and workflows powering our Data-Oriented Technology Stack (DOTS). Get insight into design considerations underlying Unity Physics, how its use cases differ from those of Havok Physics, and how both offerings express concep...
I don't know if it applies to your case but he is able to use their physics authoring script to convert complex physics objects
To the entity versions
Oh I haven't seen this one yet. Thanks @zenith wyvern I will check it out!
does exists an method to using Physics.Raycast and Physics.OverlapSphere as Jobs?
Check https://docs.unity3d.com/ScriptReference/RaycastCommand.html if you are not using ECS Physics
also https://docs.unity3d.com/ScriptReference/SpherecastCommand.html (and bunch others but no direct overlap ones)
Trying to ask again x) Anyone know the correct way to "dynamically" add subscenes? E.g. I have different maps which are equal to a few subscenes, but I don't want all subscenes to exist in the mainscene from start, but want to load them if requested. Currently I do this by instantiating a prefab that is the subscene GameObject, but this doesn't seem to work in built player.
I remembered joachim said something about it way back when but I remember a scene load component being mentioned
skimming the rest of the thread for you
Thanks. I'll check out megacity and try using normal scenes as well then (?)
@low tangle May I ask do you use it for singleplayer or netCode? I'm a bit curious how you would solve the mixed part, handle the "normal scene" as a resource that can be split between the two?
Hah, this problem I need to solve soon as well (manual loading/unloading of subscenes). I'll know to ask you about it Jaws when the time comes 😉
custom networking multi player
its very very easy to write a entity to represent the current scenes in whatever transition they might be
then you can make systems to create load/unload requests
@mystic mountain
@low tangle Yeah, that part is ok. But since I use the NetCode where both of the worlds exist alongside the same scene, there is some problematics. You cant create two subscenes of the same, as well as can't load an additive scene twice which has a subscene in it (same thing). So you'll have to make some logic around to load the scene when either get request from client or server, and keep the scene there as long as both want it as well.
oh yeah thats going to be really tricky then
weird restriction on them
I went custom netcode a long time ago but I noticed the new netcode wasn't going to be a fit for my game
what was your reasoning for going with it. if you dont mind me asking
Mostly that I wanted to use data oriented programming, compatible with all other unity stuff, not write netcode myself, and release date far ahead in time : )
ah so not write netcode then
I get that
I've always enjoyed netcode programming myself
It's not that I don't enjoy it, just that if I would do that part as well I would not really have time to make the game to use it with x)
yeah but now you get to deal with working around the new netcode lol
I get it though, time management is number one
its why were all smart and not writing our own engines
I was 50/50 on using that or rolling with the new steamworks/valve based client/server. On DOTS Netcode for now.
Valve does some interesting things to wall against ddos
at the side cost of being locked down to valve
Hmm, so i am trying to create UI gameobject everytime i am creating a certain entity, but.. when i am instantiating an entity, monobehaviour wont be copied so they are using the same mono, so... in order to create a monobehaviour and add it to an entity i must use the EntityManager on OnUpdate, and right now my entity is created on my job.. so i will simply store the data on that entity to native array, and i must call Job.Complete() and instantiate my entity after that using EntityManager and create a new GameObject, add my mono to it and add that mono to my entity
Which i feel like this would create a code abomination
is there a better way to handle this ?
Perhaps i should make a Dynamic Buffer, and when i create that new entity i would add it that to dynamic buffer, and some other system reads that and creates a new monobehaviour and add that monobehaviour to that entity and remove it from dynamic buffer ?
Suggestions are welcomed 👀 😄
dont create the entity in a job
i will store the component datas to native array then and use Entity manager to do it ?
I mean cant you just create the entity on main thread
instead of in a job
that way you can create the gameobjects and join them all as one?
well, not locked to Valve, just the added ddos protection would be locked to valve + their special convenient steam id handshake
i create the entity after i process some stuff tho
Whats this 👀
I.. i dont know how to use system state components for my example
im too noob 😭
I've used them briefly its just basically a component that persists and tells you when structural changes have occurred
like you can search for that component and if it exists and the other components don't then that tells you the entity has been destroyed
hmm, i understand now i think, i will tinker, thanks
iam looking for a good learning ressource for the DOTS physics package. can anyone throw me a good link? 👾
uh perfect, exactly what i needed 😄
@pliant pike @warped trail thanks 😄 i was actually able to do the thing i tried to
me is happy now
@digital scarab I know you can manually load subscenes, but you still need to keep track on what exist in managed code in scenes and whatnot. Right now I've set it up to have one level in one normal scene which consist of subscenes that don't auto load, and I keep references of the request to load this scene through SO. My thoughts now are that this will lead to all subscenes will end up in both binary when building later, so I could divide the client,server,mixed into normal scenes as well - but it will require some more logic built around it, and this might just be a non problem in the end.
Anyone else having issues losing one frame of position information when spawning using a buffer? I am having a problem where if I use EndSimBuffer it spawns entities for one frame on 0,0,0. If I use beginSimBuffer I just lose one frame of information, so my bullets will appear to spawn a bit further away from the spawn point. I have set LocalToWorld values on spawn as well but it still flashes at 0,0,0.
Are you spawning using Create Entity or Instantiate ?
Sounds like you need to use BeginSimBuffer and tell your generating system to [UpdateBefore] the processing system
The manual shows the update order of systems so you need to order your own systems around that https://docs.unity3d.com/Packages/com.unity.entities@0.5/manual/system_update_order.html
@opaque ledge I am using Instantiate. I will try these @zenith wyvern as well thanks
I had the same problem with instatiating from a prefab entity, i made it's position to some arbitary number, like 0, 1000, 0 in my case, so when it spawns it spawns there for 1 frame then it comes back to where i want it to be. i am using EndSimBuffer.
This is probably not the proper way to solve this issue, but it works for now 😄
Definitely not. If you read the default system update order it should be obvious why an entity would appear at origin for a single frame if you create it with EndSimBuffer
just set Translation and LocalToWorld when you instantiating👍
I think generally you wouldn't really know the proper values to set until you were inside the processing system, which is probably separate from the generating system in a lot of cases
So you might need to fiddle with the [UpdateBefore] and [UdateAfter]. Just try to avoid just setting random update orders and hoping it works - read the default update order and set your order based on that.
@warped trail I tried this by setting the LocalToWorld using a float4x4.TRS and it still flashed on 0,0,0. Do you have a different approach for setting this?
yeah tried that too haha. I think the initial settings on the Translation component is set to 0,0,0 that's what may be overriding the LocalToWorld?
Sorry for the obvious question but are you sure you're instnatiating before TransformSystem updates?
I also tried removing translation and re-adding it but it seemed wrong
TransformSystem updates between begin sim and endsim, so you should be instantiating at least before BeginSim
If you're sure you're instantiating in the correct place and it's still not working you may have to resort to the Nightmare Page https://docs.unity3d.com/Packages/com.unity.entities@0.5/manual/transform_system.html
damn yeah thanks ill be seeing this in my dreams tn
instantiate before TransformSystemGroup, or apply your transforms yourself🤔
instantiate, set Translation, set LocalToWorld👍 😅
Yeah it's been a long time since I've used the transform system but when I was remember having a similar problem and setting both translation and local to world fixed it, aside from ensuring my update order was correct
Yeah I must be missing something here is what I have for my spawner:
public class SpawnerSystem : JobComponentSystem
{
private EndSimulationEntityCommandBufferSystem simBuffer;
protected override void OnCreate()
{
base.OnCreate();
simBuffer = World
.DefaultGameObjectInjectionWorld
.GetOrCreateSystem<EndSimulationEntityCommandBufferSystem>();
}
protected override JobHandle OnUpdate(JobHandle inputDependencies)
{
var ecb = simBuffer.CreateCommandBuffer().ToConcurrent();
var job = Entities.WithAll<Spawner>().ForEach((DynamicBuffer<BufferElementSpawn> buffer, int entityInQueryIndex) =>
{
for (int i = 0; i < buffer.Length; i++)
{
var spawn = buffer[i].entitySpawn;
var pos = buffer[i].spawnPosition;
var rot = buffer[i].spawnRotation;
ecb.Instantiate(entityInQueryIndex, spawn, pos, rot);
}
buffer.Clear();
}).Schedule(inputDependencies);
job.Complete();
simBuffer.AddJobHandleForProducer(job);
return inputDependencies;
}
}```
static method
{
Entity ent = _man.Instantiate(_jobIndex, _entity);
_man.SetComponent(_jobIndex, ent, new Translation { Value = _pos });
_man.SetComponent(_jobIndex, ent, new Rotation { Value = _rot });
_man.SetComponent(_jobIndex, ent, new LocalToWorld { Value = float4x4.TRS(_pos, _rot, new float3(1,1,1)) });
return ent;
}```
If you change EndSimulationEntityCommandBufferSystem to BeginSimulationEntityCommandBufferSystem does it work?
yes kind of.
It's possible the transformsystem needs to run on it to set something up for the renderer
it doesnt flash anymore at 0,0,0 but I still lose one frame
Oh, so like, if it's a moving bullet it appears one frame ahead of where you would expect to see it the first time?
Not nitpicky at all, it's obviously incorrect behaviour
It sounds like your movement system is updating after the transform system
😬 ....ah mah gawd
Hahah, was that it?
just sec
Or I guess that would be your movement system updating before the transform system, so the momement system moves your object after it spawns but before it's had a chance to render for the first frame
So I guess...your movement system should update BEFORE your generating system, and your generating system should update before the transform system?
That made my head hurt more than it should have
sorry about that! haha
No worries, my fault for being so tired
Thanks for your help though finally got this load order stuff figured out
I got some weird errors when I tried to build my project (it failed to build), anyone seen this before, or knows what it is? what player assemblies is it missing, is it a unity bug? (unity version 2020.1.0a9)
some kind of a burst error
oh actually when I click on it it says: Unable to retrieve windows 10 sdk base
I maaay download the win10 sdk from their website, see if that fixes it, idk if it should come with visual studio by default but I'll try
probably need to update your visual studio install with the correct sdk, burst documentation has info on what you need https://docs.unity3d.com/Packages/com.unity.burst@1.2/manual/index.html#standalone-player-support
make sure you restart the editor after burst fails to start like that. it seems to error out the process and never run burst again or something @surreal grail
well it must run burst since in playmode it is working, I guess its having issues with the windows specific part of burst
works now, thanks guys!
what was the problem ?
well it was either windows build support not installed in the version of unity I'm using (I thought it is for building on universal windows platforms) or windows sdk not being installed on the computer
if it was the windows build support then its really weird how it's only needed for Burst
@digital scarab You have client and server in the same project, the worlds are side by side, but use the same scene. You can't create two instances of the same subscene. If I start my game with both client and server, the server first wants to load the world X, then I authenticate player and connect to the server who says it should load world X. Now the client disconnects and I should unload world X for client, but not server.
Can, but it throws error or warning.
Not that I know of, it just says you can't have two of the same same subscene.
@digital scarab
Right now I've tried with either loading a scene twice (which contains a subscene) or load two different scenes which contain the same subscene.
Alright, so it should work even thought the warnings then. I'll still have to tinker a bit how I will connect the normal scene to the specific world. Thanks for the help! @digital scarab
I can share issues all day x) haha
@digital scarab hey, anything new on the wonky jobs scheduling?
also any info on being able to limit the worker threads to the FIRST available worker threads? 😄
trying to debug unity job system with 23 worker threads is not fun
and when I limit it to few workers, they get split to random workers
so I might get worker 22 active
which I have to scroll down to see and it then hides the all the regular info
on DOTS dev I do miss my old 4 core 4 hw thread cpu
Hey folks... I might have missed something but I get (again) some compile errors on the latest packages on 2020.1.0a21. Seems the Build package and the Platforms package conflict for some reason. Already tried deleting the whole Library folder, but to no avail. It seems downgrading all Platforms packages to 0.20 seems to fix it, but I wonder why this conflict exists in the first place. Also I'm not sure what the Platforms packages are even about, tbh, they seem to be not a dependency for anything really.
@sharp nacelle https://forum.unity.com/threads/compilation-error-in-unity-platforms-preview-2-0-2-0.813063/
Thank you! didn't find the post with the search, because I was looking for unity.builds and not platforms 🙂 well, it's meh, but I can revert for now to the older packages and wait or entities 0.6.0. thanks again
@low tangle So here's a question. When you talked about Activations, you mean event entities/components that determine if a feature is activated and interacts with entities right? I am not sure I fully understand what you said about inverting the processing, but did you mean that instead of giving entities info about the Activation, you give the Activation info about all the entities it interacts with? And if that's the case, an Activation that only acts on enemies the frame it is emitted, doing that would be useless in that specific case, right?
Or am I misunderstanding?
Because the way I am doing it is that my systems are looking for "events" that gets destroyed by the system on the next frame.
Except for stuff that happen every frame, then the event emmiters(in my case, monobehaviours) sends info to event entities that stays alive and get destroyed by the emmiters themselves when those get disabled. But even then, the events themselves are simply data that never get written by systems, only read and destroyed.
Considering I am not misunderstanding, why would it be better for the events to act as activators and contain the entities they act upon?
And 2 other questions, but those I am asking to anybody that can help:
- Is there a safe way/a preferred way to activate/deactivate physics? I want to be able to freeze/unfreeze the world whenever I want, like on the click of a button.
- Is there a safe and efficient way to create a new entity world and transfer entities between it and the default world? How will systems react to having two entity worlds in the same scene?
you can change PhysicsStep.SimulationType to SimulationType.NoPhysics
thanks
Anyone has a rotate around function who can share ? because i just crashed my editor while i was creating my own lol
nvm i got it
Is there a way to get all entites that has a certain tag without making a job struct ? Right now i am doing like this:
[BurstCompile]
struct GetMarketEntitiesJob : IJobForEachWithEntity_EB<MarketDataBuffer>
{
public NativeArray<Entity> entities;
public void Execute(Entity entity, int index, [ReadOnly]DynamicBuffer<MarketDataBuffer> b0)
{
entities[index] = entity;
}
}
As you can see i am not interested in MarketDataBuffer, doing like this is fine, but i feel like there is a waste of performance, is there a way to do it or should i just go with it ?
maybe an EntityQuery?
Entities foreach can as well
the lambda that is
But really it comes down to what level of verbosity you want
What's the best way to handle setting up new entities? As an example, I have a Block entity (think, city block) that I need to subdivide into Lots, and I'd like to create a system to handle the subdivision process. This only needs to be done once per Block. Right now my thinking is that I use WithChangeFilter and add an IsSetup bool, only running the code for values set to false. Is there a better way to go about doing this?
@opaque ledge cs NativeArray<Entity> entities = m_Group.ToEntityArray(Allocator.TempJob, out JobHandle gatherArrayJH);
So you want to do something once an entity is created ? i asked that yesterday as well, you should use SystemComponentData
m_Group is entity query ?
yes
alright so, create entity query at OnCreate, and put this to OnUpdate then, thanks
@opaque ledge Ah okay, so you mean that the IsSetup bool should be a ComponentSystemData?
yeah
I mean you should have a seperate system that catches these newly created entities and do something and put the SystemData tag on them.
i have a 'working' example if you like, i can DM you or post here
do you recommend something similar for doing instant "events" when adding a component to an Entity?
trying to figure out how to do something instantly once I add a component instead of waiting a frame for the logic to see the component
Yep, that is indeed possible
@opaque ledge I found the docs for that which has an example so I'll start with that for now. Thank you though!
however.. SystemData is special, when entity is destroyed, the entity isnt actually destroyed until all of the SystemData on it is removed. So this behaviour can let you do something like..
Do something when entity is created, add the system data
Do something when entity is destroyed, remove the system data
I am using it to instantiate some monobehaviour and add it to that entity.
You can also do it to do something when a component is added or removed, but that means more tags to create
and good luck Edvard o/
it was kinda hard for me to grasp at first 😄
I don't suppose anyone knows the simplest way to create/render a bunch of points/lines in ECS?
Anything I have needed to render enmasse be like points I have used the VFX system with entity backing;however that has some limitations
I mainly just need it for debug purposes, so I can see what I'm doing
I could just create small spheres I guess
spheres would potentially be ideal if you just need something quick 🙂
@pliant pike You can call Debug.DrawLine in a Entities.WithoutBurst.Foreach(...).Run()
yeah I did try that thanks
Yooo anyone have any satisfying examples of extreme performance gains with dots?
Along the lines of this:
https://www.reddit.com/r/Unity3D/comments/e5rjpv/converted_my_character_controller_to_dots_and_the/
I just love to see the crazy improvements
Hahah, that's a great video 😄
I dont really have any modern videos or screenshots
but when I was first working on a way to drive 1000 animated skinned meshes I had to really structure the job right per instance
at like 52 bones per each one of the thousand with one entity each and a jobchunk that ran over each grouping I just barely got it running
at first with a simple foreach it was somewhere in the ballpark of 40fps on a quad core
there's another good recent example here https://www.youtube.com/watch?v=ubUPVu_DeVk&t=852s
✅ Get the Project files and Utilities at https://unitycodemonkey.com/video.php?v=ubUPVu_DeVk
Let's take our Unity DOTS Pathfinding and apply it to Unity ECS so we have Entities roaming around calculating their path.
📍 Support on Patreon https://www.patreon.com/unitycode...
eventually after lots of profiling I got it to about 400fps
then I had the next problem which was networking
visible sets means you need to check per each thing your observing
if you have a 1000 players its 999 other observations in memory somewhere
rounding up thats a million iterations with random access
no mater how you slice that its a big n
but yeah again dots made it easy to come up with a flexible setup that can push that many at about 200fps
thats kind of ridiculous
other than sheer gpu crying from all the computing all the bones and then rendering all those unique meshes, I can actually run 1000 players. if I can ever get that many of my users on
full skinned animated models
it kind of shows how badly we've been using these hardware all along
yup
stream processing and focusing on memory access is a godsend
I really wish hardware vendors would give low level access to the cache
so that way smart people like the dots team could get even more out of l2/l3 cache
well 10 times better performance is good enough for a start
I've gotten more than 10x
some things being ported from monobehaviours got that three times as I improved it
yeah I think thats like the average you can get without really trying though
keep in mind this doesnt always happen
mb 1x
ecs main thread 10x
ecs jobed, burst 100x
ecs jobed wayyy better arch 1000x
but it has a few times now gone like this ^
one of the hardest to understand is how much performance you can net by not doing any work at all
which you get by structuring the data so you only process stuff that you want said logic to happen
you can really kill most dirt bits and IsDone checking and what not
but not always, as you still need to touch other code which might depend on that kind of polling
yeah and that leaves much more space to do other stuff
@pliant pike There this blog post where they claim a "2250x" speedup from switching to burst for their enemy movement system https://blogs.unity3d.com/2019/11/27/creating-a-third-person-zombie-shooter-with-dots/
hmm - interesting - I think I might've missed that blog post
i guess it depends on how bad your code was before also 😄
Well it is oop 😏
@zenith wyvern That post is just so bullshit lol
The numbers did seem a bit fishy to me
Is it possible to make something that is 2250x slower than what DOTS would be? yes.
Like @mint iron they must have been doing something pretty insane to see that kind of boost
But you gotta be a proper fucking idiot to achieve something that performs so badly lol
Lol
But it's not like it's an atypical case. Horrible OOP code is unfortunately an industry standard. ECS forces you to write better code, so.
To be fair, DOTS have 'performance by default', whereas a naive solution in OOP can be really bad for some tasks
Yes, threading + simd makes things faster, nobody that has any idea what they are doing is surprised.
All the marketing wank really gets on your nerves eh?
They are just doing themselves a disservice pushing this "oh god we had 1000x perf improvement" and then it's some... fairly shitty looking game that's being demo:ed?
well I guess DOTS is a double bonus its teaching them how to code properly as well
Hmmmm - I am curious as to what it looked like before - would be nice to see a before and after case study 🤔
Yeah, but they don't even know why
@trail burrow No need to be rude, there's a good chance those developers hang out in here.
from my testings it seems dots improvement correlates directly with the cores of the system while the oop code is relative consistent in its performance (the bad and the good)
Eh
Eh... well, all I'm saying is that them pushing this 'several thousand times' improvement, then demoing something that looks 'subpar', is just doing unity a disservice?
- Most things will not see that much improvement unless it was written very very badly to begin with
- The game being demoed looks at best like a 3 month indie project
It makes Unity look bad tbh. Because anybody that knows anything knows the 2250x number is bullshit.
People never really seem to notice how much cache optimising helps
If that is the 'best' they could find so they have to push it on their blog... I mean... it doesn't look good
That's a lot more reasonable than saying their game "looks shitty".
I think that game was just free labor wasn't it?
sure @zenith wyvern ¯_(ツ)_/¯
Or at least the dots port of it
The devs of that game are from my city i just realized
One of these days ||never|| I'll get around to writing up that damn post and profiling of dots
And obviously the ECS+DOD approach is good for massive amounts of identical things
Well, a game is made up of a shit ton of little things
Separate them well and you get good systems
Good systems then can get job'd usually
If their jobs you can likely as is flick on clang, I mean burst
Lots of easy paths to performance and simple programming
Wait shit
Meant llvm
Close enough
@low tangle what is cache optimizing ?
There exists small temp memory that lies near the CPU - typically called L1 $, L2 $, L3 $ (depends on the processor). CPU can prefetch data and store them in the caches for quick easy access compared to fetching from system RAM
optimizing for the caches - means you want to take advantage of that aspect of CPUs 🙂
It's basically the primary goal of ecs in how unity set it up
It all comes down to those unity memory managed byte chunks
We interpret then as a structure of array in a foreach loop over said data
The foreach is the kernel
If you have a small kernel running over a linear array of bytes, the CPU basically can't fuck up
I still don't understand how the cache works personally. Is it only relevant for "for loops"?
So it will hopefully be in cache and won't miss
Honestly, depending on what kind of system you are working on a x2000 increase in performance isn't out of the question. Was working on a project that had a pretty straight-forward dynamic navmesh solution from a previous project that was designed for around 100 agents at the same time. Now the new project needed 50k+ agents and of course it didn't scale well.
Code got moved to a more data-oriented structure, got the proper SIMD-treatment, cache-line optimizing and some threading on top. This was on a custom C++ engine quite a few years back. I think in the end they got the navmesh generation to be around 50x faster, but the new pathfinder handled the 50k agents nearly twice as fast as the old approach handled the 100 agents. So depending on how you calculate it, you could say they got an improvement of around 1000 times. And I think the engineers working on it considered some more approaches to make it even faster.
Unfortunately I didn't work on that system myself at all, was only a hired freelancer for one specific other feature, but it got me interested in optimization techniques and data-oriented design overall 🙂
yeah - effectively by storing memory linearly and in chunks - you can store those slices of memory into the cache (if the slice is too big to fit into the L1 $, then it can likely be stored in L2 $ / L3 $ as those are bigger in capacity but has slower access compared to the L1 $)
The cache is constantly in use always in a cpu
@zenith wyvern Every instruction will pre-load memory it needs. If the next instruction can run on the same cache then nothing needs to be pre-loaded and that's where the speedup comes from. It works for anything, but you will see the biggest gain in loops.
CPUs load strips of memory from ram instead of the but you asked for
The CPU will also escalate from l1 to l4 (ram) looking for the data at each stage
In multi core, l3 requires a some sort of sync that I don't really get. But the layman explanation is you want to get your shit into l2
yeah
Which are always way smaller than the fat l3 processor companies like to advertise
Thankfully like mentioned before, that's exactly what chunks (soa in a friendly form) and ecs helps you make
You can still make systems that jump around and do random access which is a cache miss
But sometimes it isn't
dod also talks about the problem of loading instructions based on not-yet-loaded data, which really fucks up a cpus ability to work out of order
So any time I access a new piece of data it's potentially bumping something out of the cache? Is that right?
including instructions
The os could decide to freeze that thread to jump somewhere and do some driver code
Totally flushing your data
We have to help suggest for it to be in cache and your code to be fast to process as much as it can
This is why sometimes it's just faster to always do the work instead of checking if you should
So basically we should try not to think literally about what's going on in the cache, but try to structure how we process our data so that, if we're lucky, it will fit?
Well you have a framework helping you make smart choices
And giving you the option in the first place
Most of the time you will be lucky though. You have design your code to be able to cope with the few random cache misses and stuff like SIMD also helps a lot in it. Basically the more you can do in a few simple instructions, the higher the chance not to run in a cache miss.
because rn unity won't use more than 10% of my cpu and 16% of gpu whilst running a stress test
@tight badge spam more meshes to render? 👀
Focus on entity processing that does one thing in that system. For getting better cache usage you want to have all the data there on that entity to do work, and the entity should be as simple of a archetype as it can be
@tight badge battery safe mode on?
I did I'm trying ECS for the first time and I spawned 100000 entities
if you can make branch prediction more... predictable, you're also saving on cpu work that you just throw away
I'm running on a desktop I overclocked
Does that mean if I have data on two separate entities ( so we know it's in two different chunks that might not be close to eachother ) then I'm automatically getting cache misses if I access data on either entity?
The classic ecs example of a move forward component tied to a position
not if you're processing all entities in both chunks
Depends on how you access the data and in which order
and how big your components are
Yes but there's a chance the other chunk might get accessed in order so after a lump loads in you might run though a few 'random' accesses without a miss
stream processing is the default way to mitigate cache miss
But if this is the case you might be able to just keep that data in the entity
I'm following this tutorial rn and everything's the same https://www.youtube.com/watch?v=ILfUuBLfzGI
Learn how to get started using Unity ECS.
✅ Get the Project files and Utilities at https://unitycodemonkey.com/video.php?v=ILfUuBLfzGI
Unity DOTS Explained
https://www.youtube.com/watch?v=Z9-WkwdDoNY
If you have any questions post them in the comments and I'll do my best ...
By stream processing you mean loading the separate data into a single array for processing, is that right?
Yes but the chunk full of entities is just a array of data that is 16k long
If your job only needs simple arrays then you should totally keep it that way
ideally, you're applying the same transform to every element in the stream
Yeah
Right, so that's only something I consider for very performance intensive code basically
This is all for getting even more performance
On a general sense - yes
I've shaved jobs that were taking like .02ms with burst down to 0.001
It's pointless but I get curious
I did about 12 systems like this one day to shave .5ms
Basically the more you can do in a few simple instructions, the higher the chance not to run in a cache miss.
@sharp nacelle
It sounds like at my level of knowledge this is pretty much what I should keep in mind for maximizing performance
I appreciate all the insight everyone, thanks so much. I have no formal training or education as a programmer (obviously) so this is really valuable
Awhile back with a earlier version of burst I noticed it didn't do local hoisting out of loops
So I was able to get better stack usage by moving the local variable out of the loop
Was able to measure it at 10k+ iterations
For{
Int I = 0
Do something
}
Vrs
Int I = 0
For {
Do something
}
That's the trick, making sure the compiler can tell what's dependent on yet-to-be-loaded data, and what's consistent across the transform
cause if you have to wait to load an element before you know how to process it...
So how do you optimize for a cache exactly ?
There is a ton of performance you can squeeze
Single responsibility systems
Transform data
Emit data
Remove data
That's how the dod book describes it
Replace data with entity and you have your primary tools in ecs
All the talk of databases in that book keeps confusing me. I guess the chunks are the database equivalent?
You focus on keeping your system doing the single action it should do to achieve the logic you want and you will have about as good as cache usage as you will get
The chunk lookup system is the database
And entity archetype
So basically ignore the concept of the database and only think about how I access the entities via queries
Yes
We have +- type and for joins you have to do it manually by creating the new array of data, then looping to query by your keys
So.. i should make systems that their query is lower than cache limit ?
Basicly just try not to do joins
I think wikipedia describes it generally well in a few sentences - if you want to optimize for caches -
Naive implementations of ECS where you iterate through all entities regardless of whether you need it may not be inherently good
But if you can do a 'smart' implementation where the entities are what you exactly need (e.g. the chunks) - you can then start optimizing for caches
Or do the simple pattern of hash all potential targets then query that in a simple loop over your primary
Exactly what we have with unity's version @coarse turtle
yep
@opaque ledge no
@opaque ledge At our level of knowledge the best thing we can do is minimize what each system does as much possible. Also has the benefit of making the code more readable
Yes
each system or each job ?
It's worth doing for multiple reasons not just performant
But you have got to remember
You can still do it 'wrong'
Both I suppose. If you have a system that's running a ton of different jobs, that sounds like your system is doing too much
There is nothing wrong with that
You might have a few job chain to prep some nice linear data for your final job, or maybe a lookup table
It's systems that have too much in them
You really can build an entire complicated bit of logic with discrete simple systems
I got a recent example
Give me a bit to type it out
okay so, in order to understand more, i have this system called CooperationCreateTradeShipSystem which does the following:
FirstJob = Get Owned Markets with IJobForEachWithEntity_EB
SecondJob = Find Comparisions between these markets(which is written to a Buffer from another job) with IJobForEachWithEntity_EB
ThirdJob = Process these comparisions find the best route and create the ship with IJob
So this is my most complex system so far, should try to improve it somehow ?
it takes 0.6 ms on entity debugger
Nah that's fine
Imagine your game has this entity for the player that is created at some point
[] For a component
It might be something like [create player] [resource Id] and while it does its thing it gets a few new types added as it goes like, [downloading] and [is spawned] [has body]
So you can query at any point and have that system do whenever type of thing to the player it needs at that stage
Have we a way to get union EntityArchetype/EntityQuery/IComponentData collection based on 2 entities?
But your only doing one kind of thing to this entity, which is mutation
Not yet @frosty siren
Each system in this first example is also loading in the entire 'fat' entity for each stage when it really doesn't need it
So a better approach would be to not have one player entity, instead there isn't one, there is just a collection that produce the player you expect
lots of smaller systems are slower than monolythic ones
but the exact inflection point is not so easy to find
Yeah not too small
the best you can really do is good use of chunk-level logic
But it's not slower
like, a Transform system will access a bunch of stuff at once, so better look at what the chunk has, and do the proper thing, instead of doing 99 different queries
a lot of the examples imply you should be doing addition of 2 variables in a job
Did you get that from profiling?
and that really bothers me
@low tangle i have my own ecs implementation in cpp. same style as unity
in fact i think faster for some things
Oh right I used it
You posted it awhile back in here when I asked
ah, yeah
I'm in situation when i need to build entity at runtime based on some groups of IComponentData. So i need to store collection of IComponentDatas somewhere or i need to have prefab entities with it.
We can't today assign IComponentData with values in editor through one collection. Only just have collection of types. And can't get types that entity have. And also can't get union of archetype. Looks like impossible task.
At least I think that was you
i made a newer version not long ago, much thinner and faster
Sweet
but its only like the "absolute basics"
I used this c one for Bit in that style
anyway, ive done a bunch of random experiments with it, on full stuff with graphics and simulation and stuff
Flecs is really well made
it has some truly galaxy brain level stuff that is way ahead of what unity ecs does
but i dont really like how you have to register systems, i prefer the way mine works, where its like a very fancy data structure
Yeah
I like unity's bit of magic
Anyways
In pratice in my game increasing the system count has helped drop the number of wasted cycles spent on fat systems
There is a sweet spot of many-jobs vs few-jobs. This depends on what you are doing. For tryhard level stuff, using things like manual chunk iteration is the way to fastest speed. There is a bit of overhead every time you run a system
I'm pretty much only paying for what's being done that frame
but as a general rule of thumb, smaller systems = better
Yeah
specially if they generally work on different sets of components
if you have 5 systems that run stuff on components A,B,C, then its pointless to have 5 systems. Merge them into 1
but if system 1 is A,B, system 2 is B-D, and system 3 is A-C, then of course separate them
it depends on how many of the components overlap
Even if the systems were doing completely different things you would merge them so long as they were processing the same component types?
Think thin archetypes so you get Max count per chunk
@zenith wyvern keep in mind you can just put your logic as a function
5 actions on the same archetype
and then your system is just something like ```
for-each(A, B , C){
stuff1(A,B)
stuff2(B,C)
stuff3(A,B,C)
}
o wow hadn't heard of flecs till you both mentioned it - thanks for that lol 🙂
Np
flecs is a bit on the over-engineered part, but its extremelly solid
Yeah
the guy also added a cpp api
I suppose so, I realize I may be getting a little too caught up in SRP lately
which is easier to use
It was fun to bolt together a little game with it
I really want to attempt to make one in dlang at some point
also seems to be super solid
yes - that's the one i used legion 🙂
Yeah
That's the hip thing to do with ecs
Did you update the repo for yours? @vagrant surge
yeah you can see the newer one
i should wrap to native array if i want to send Component Data or Buffer to a job right ?
the code of the new one is much more simple
Chunk management and archetype scanning is what I need to figure out so I can start learning how to make mine
Awesome
Thank you
it has the downside that adding/removing components while iterating is not supported, and its feature set is fairly basic (no chunk-level comps or similar stuff)
That's fine
the coolest part is the fact that this actually works:
Man you guys are doing all these games in different languages and frameworks, what a wild bunch
I prefer delayed structure changes like unity's anyways
haha - i've been wanting to make my own personal toy engine
world.registry_decs.for_each([&](EntityID entity, PositionComponent& campos, CameraComponent& cam) {
XMVECTOR eyePosition = XMVectorSet(campos.Position.x, campos.Position.y, campos.Position.z, 1);;
XMVECTOR focusPoint = cam.focusPoint;
XMVECTOR upDirection = XMVectorSet(0, 1, 0, 0);
Globals->g_ViewMatrix = XMMatrixLookAtLH(eyePosition, focusPoint, upDirection);
Globals->g_d3dDeviceContext->UpdateSubresource(Globals->g_d3dConstantBuffers[CB_Frame], 0, nullptr, &Globals->g_ViewMatrix, 0, 0);
});
fully automatic for-each
has the option to also run parallelized
@low tangle i dont like it. Adding/removing components while iterating is a very common usage, specially in gameplay code
That's fantastic
of course, im talking about add/remove on the current entity you are iterating
It just bothers me because performance
i think unity cant do it
@low tangle its faster than delaying the change for later
- no memory to hold the change command
Yeah
Sorry for cutting in the conversation, for 2D projects I'm seeing some rather than using RenderMesh, instead they write their own rendering system with Graphics.DrawMeshInstancedIndirect or similar methods.
Is there a reason for doing so?
in older version of decs, adding/removing while iterating did work, but it was a bit buggy. It was significantly faster
Assuming there isn't something else contenting with that chunk/entity
@lusty otter RenderMesh is horribly slow in almost all use cases right now.
@lusty otter I do the same - for me it's due to using my own srp
There are improvements coming though.
Ahh..
Can't do work on a job if your wanting to work on the latest version of that chunks data. Which means you can't modify the contents
Imo if you're starting now there's a good case for using RenderMesh. Unity has been saying that massive performance improvements should be incoming within a month or two
So only one access at that time if you do structure changes
@low tangle btw, Flecs CAN add/remove while iterating, while parallel
oo good to know @zenith wyvern - now i wonder if they have a 2d pipeline that 👀
I'd be wary about those @zenith wyvern
absolute madlad
Only do it if you're a hobbyist though
As usual don't depend on Unity's release timing if you're a professional
he does it by creating a clone of the entity in a new chunk, and then at the end of the update, those newly created staging chunks get added to the main sim
Yeah
done in parallel by having multiple of those staging areas, one per thread
@low tangle Jaochim sounded pretty confident, he did literally say "massive perf improvements". As usual we can always be skeptical of their hype but it sounds promising at least
No I mean the timeline part
Oh, hahah. Fair
still better than godot where they dont even batch 2d sprites
I'm sure it will be fast
I guess I'll do the same and write my own rendering as well.
Boggles the mind
posted about new Unity code formatting tool here: https://discordapp.com/channels/489222168727519232/497872424281440267/674679661145358362
wonder if dots needs additional considerations for that
m_Variable names for all
ah interesting - some more static analysis tools
Maybe
@dull copper neat, lets see what it does to my project
In the future this package will also configure compilers to emit warnings about code that violates our code conventions
well, you could always just...not to install that package
unless they put it as dependency
i wonder if we can turn that off 👀
or if you just do like #pragma warning disable some_code_#
For more infor refer to https://github.com/Unity-Technologies/com.unity.coding
heh 404 why am I surprised about this
I'm planning a user scriptable (and highly customizable with user textures and what not) 2D game targeting mobile platforms. Due to that nature ECS is the only solution that can be performant enough to run on low end devices.
I have never touched Unity before (though with an okay amount of experience in programming) and I won't be able to get started until a few more weeks due to other projects, so for now I'm just reading documentations and watching tutorials to get prepared.
For now though I really do feel like jumping straight into ECS, so much of the stuffs in tutorials and what not are completely irrelevant. I would appreciate some general pointers and advice, thanks!
Hmm - I'd play around with it ecs and a bit of Unity to get the feel of things
maybe tinkering with the current Tiny could be a good way to get started
after all it has a working renderer
and a very good demo on the tiny racer thing
thats probably the best current ecs example unity has
yea
I probably wouldn't use tiny if I was a professional personally
It's way too rough right now and they still haven't released a proper timeline
for basic stuff - I'd say the entity component system samples on Github might be a good start too
thats too basic
doesnt show how to actually do a project
the tiny racer demo does show it
check out the pinned messages on the channel here for a list of starters
Okay
that tiny racing will be super annoying to start with because it doesn't even render properly in the editor due to subpar hybrid support
didnt they kinda fix that in the latest update?
or is still broken af in the editor
ah, I haven't tried on latest
complete and utter embarrasment btw
but yeah @lusty otter if you happen to have a decent way for 2d stuff i'd be happy to look into it 🙂 I'm currently still using the sprite renderer + animator cause the workflow is too nice to give up
I do know there's some URP fixes on upcoming SRPs tho
like seriously tho
for hybrid that is
releasing a demo that is hopelessly broken in-editor...
Gotcha
I guess they did have it working on their end, just not on the version that was published
the URP hybrid fixes that are coming up were done during the hackweek already
so a long time ago
I'm not going to be too hard on Tiny, they responded well to user feedback. No one wanted their javascript only version, remaking it as a proper C# version that utilizes the normal editor seems like a good idea given their pivot away from working entities in the editor
all in all, most of the dots samples we get are quite hacked to work
That's a bit worrying to hear.
Honestly when I saw 2019.3 release note and DOTS was in there I thought it's matured to a good level already.
are IJobForEach parallel ?
the stuff that is usable is to use DOTS tol speed up some stuff
Yeeaah...unfortunately they are being kinda misleading with their marketing imo
i'd much rather they focus on URP than anything else; because can't use HDRP on mobile.
but editor support and rendering/nonlogic support are atrociously bad
if you want to tinker with ECS models, i still keep recomending Entitas
at least its not a disaster in-editor
even if its much slower
Oh can we use URP with ECS too?
You can but people have had a lot of issues with the hybrid renderer in particular
@lusty otter expect Entities pacakge to be released this year... anything other.. not, this includes hybrid, audio, rendering, anything DOTS related
If you use a custom rendering solution it's fine
you can.... but last time i tried it had many issues and gaping holes, no property block support, troublesome upgrades, lighting issues when building to device.
😰
but it's not going to happen for all systems (just my opinion based on how things have evolved)
man, thats such a long time line
I wonder how the GameCraft fellas feel about that
that game is being developed on the bleeding edge
i think Tiny is not using engine at all in builds, thats why it is broken in editor🤔
hmm aren't they using Svelto.ECS? I haven't been following their updates lately
of course things can evolve a lot faster now that Unity has put more resources on whole dots thing
Well I guess for my simple game the current DOTS stuffs are enough but man
They really got me with that marketing LOL
so who knows, maybe they do get it done by 2022
There are some people who are utilizing dots to make really great looking full fledged games. Look at Ezienhorn's game on the forums for example. Definitely the minority though.
You can do a lot with what is already there, you're just still limited by other systems
people using current DOTS in production are more like exceptions right now
there's like 5 commercial products using it
also many of them don't use most of the DOTS packages, just entities
or entities + hybrid but even that's iffy
@coarse turtle they do use svelto ecs
but i think they integrated it with some of DOTS stuff, like jobs or burst
Gamecraft is using ECS, Jobs and the new Havoc Physics package
ah
nah its not really using the ecs itself, they have their own. Unless you know more 🤔
i just follow the guy on twitter
@stable fog I thought they used Unity Physics
They did
did they swap it to havok?
same - I remember seeing a few tweets abt URP support being lackluster w/ PP - also I'm pretty sure the guy is also on this discord? 🤔
well, I mean if they move from DOTS physics package to another, it's not going to break many things
ultimately it's a dropdown option
I think they said they had to tweak some values
yeah I was just typing that they probably need to tweak all hacked stuff too
as there's always small finetuning on things to not glitch
@stable fog pretty sure
tho they do kind of interface beetween the 2 ecs
he doesnt need unity ecs
his own is pretty fast
However I can confirm that, once compiled, Unity Jobs runs at the same speed of Svelto.Tasks with very similar results.
you know whats funny?
watching this robocraft guy, which is a small team, have way more productivity and a much more stable ECS stuff compared to unity themselves
and stuff like Entitas having decent editor support, and ECS being unable to
i am getting this when i am instantiating from command buffer
is this normal ? or am making some mistake somewhere
@vagrant surge agreed, I moved to Entitas because of rich features and Editor support