livid comet
This had more interest than I expected so instead of flooding the DOTS general discussion just going to put any updates here.
Am automatic 1:1 ordered mapping between chunks where from the users perspective you have single mega chunks but under the hood the data is split into logical groups where it guarantees the index in 1 chunk matches its corresponding index in another chunk.
Setup a single entity
Automatically split into multiple entities with data moved to correct chunks
Put VirtualChunk attribute on your data that you want to be grouped in a specific chunk
[VirtualChunk(1)]
public struct AIData : IBufferElementData
{
public int Value;
public int Value2;
}```
Then add VirtualChunkAuthoring to your authoring object and that's it. It will auto split into chunks during baking.VirtualComponentTypeHandle
VirtualBufferTypeHandle
VirtualComponentLookup
(VirtualBufferLookup etc coming when I can bebothered)
Work the same as their entity counter parts.
[BurstCompile]
public unsafe struct TestJob : IJobChunk
{
public VirtualComponentTypeHandle<TestPosition> TestPositionHandle;
[ReadOnly]
public VirtualComponentTypeHandle<TestReference> TestReferenceHandle;
[ReadOnly]
public VirtualComponentLookup<LocalTransform> TestDatas;
public void Execute(in ArchetypeChunk chunk, int unfilteredChunkIndex, bool useEnabledMask, in v128 chunkEnabledMask)
{
// These components might exist on different entities/chunks
TestPosition* testPosition = chunk.GetComponentDataPtrRW(ref this.TestPositionHandle);
TestReference* testReferences = chunk.GetComponentDataPtrRO(ref this.TestReferenceHandle);
for (var i = 0; i < chunk.Count; i++)
{
Entity entity = testReferences[i].Reference;
float3 position = this.TestDatas.GetRefRO(entity).ValueRO.Position;
testPosition[i].Value = position;
}
}
}```Should you use this?
No. I can't guarantee I'll support this long term yet. It's also thoroughly untested. If you want to look at the source it's available here : https://gitlab.com/tertle/com.bovinelabs.core/-/tree/virtual/BovineLabs.Core.Extensions/Chunks
No guarantees it's in a state that won't instantly crash your Unity though.
Why did you implement it then?
I've been thinking about this concept for a long time. I was going to bed the other night and it just clicked, I wrote down a bunch of notes, woke up the next day and implemented the core concept in 2 hours. (This is how most of my interesting system development happens. Months of planning, minutes to implement.)
Should Unity implement something similar?
Not anytime soon. I'm not advocating for anything. It's simply an idea I've had in my head for ages for pushing performance to the next level for how I design dots architecture.
It'd be a big task to ensure safety for their standards and I haven't even confirmed if there are enough performance benefits to offset overhead.
Maybe somethig like this would be cool to research post 1.0 or further in the future?
How did you do this without editing the entities package?
A very good understanding how things work and a bunch of replicated code.
For example I now have TypeManagerEx, Virtual variations of Handles/Lookups etcs. I am limiting methods within these to make it easier to maintain.
There are definitely performance and safety benefits to this if I edited the package.
Gotchas
So many. As I am not modifying entities package
Honestly I suspect no one but myself unless you spend a bunch of time working out why this is good will really reap the benefits.
Performance benefits (or downsides)?
Tests coming soon~
reserved
reserved 2
So the big update since I last showed this off is it now automatically splits a single baked entity into multiple, with this I have completed the basic loop I was envisioning (currently it doesn't clone data, should be pretty easy just was lazy yesterday).
livid comet
I'm writing a decent sized simulation to try actually see if there are any benefits to this (as well as finally test my AI system after 2 years) and I'm finding there is this mental advantage to these virtual chunks
I'm worrying a lot less about putting data on my entities
I've struggled with this for a while and somewhat why I haven't touched my AI much.
Things like storing, CompletedOrders in a buffer. Did I really need this? If I keep adding buffers to my entities which are going to impact chunk size.
Well now I don't care! I have a dedicated chunk just for AI components and I can load myself up with all the data I want for my simulation.
graceful carbon
if I understand correctly, this is quite similar to shared components in the sense that the data is not stored in the chunk where the entity is?
I think with the hard-coded limitations on chunk size I can easily see this being useful; after all, a lot of the benefits of chunk organization are reduced if you have only 10 entities per chunk.
I'll keep this in mind!
livid comet
I've changed from using integer chunk identifiers to strings and you can configure what chunk they end up via settings
this plays better with multiple libraries and makes final chunk balancing much easier
also didn't mention this before but you can use the attribute on an assembly level
[assembly: VirtualChunk("memory")]
Also fixed up issues with netcode ghosts and them using begin simulation command buffer
livid comet
Finally got component and buffer data copying working in baking. Took a bunch of extensions, thankfully I already had already written an UntypedDynamicBuffer
limber pagoda
You have many interesting ideas, but their information is being everywhere. Could you write down some readmes, even a concise one, for each feature you are developing or want to discuss, and put them in their repo? 😂
livid comet
I'm not really developing anything here you should be using, let's call it an academic experiment
halcyon tide
would this help improve performance and chunk utilisation when you have really large entities?