Avoid using generic jobs if possible | Official Unity Discord | Page 1

rugged plover Mar 3, 2023, 3:51 PM

#

I have generic job which takes arbitrary component with known data layout, reinterpret it to T array and copy to output array (which is ComputeBuffer by the way). What I want is to try to avoid using generic job here, because of need to register them for every component case.

Can job below avoid using generic in any way? All it does actually just copy one array to another with same size and same data layout.

[BurstCompile]
    // takes all chunks by query, and then just copy component data to compute buffer starting from 1st entity in query index
    internal struct SyncPropertyByQueryJob<TProperty> : IJobChunk
            where TProperty : unmanaged
    {
        [ReadOnly][DeallocateOnJobCompletion] public NativeArray<int> chunkBaseEntityIndices;
        // this should be filled every frame with GetDynamicComponentTypeHandle
        [ReadOnly]public DynamicComponentTypeHandle componentTypeHandle;
        public int typeSize;
        [WriteOnly][NativeDisableParallelForRestriction] public NativeArray<TProperty> outputArray;

        public void Execute(in ArchetypeChunk chunk, int unfilteredChunkIndex, bool useEnabledMask, in v128 chunkEnabledMask)
        {
            var data = chunk.GetDynamicComponentDataArrayReinterpret<TProperty>(ref componentTypeHandle, typeSize);
            NativeArray<TProperty>.Copy(data, 0, outputArray, startCopyToIndex, data.Length);
        }
    }

severe fable Mar 3, 2023, 4:51 PM

#

you could just pass a void* from NativeArray.GetUnsafePtr() into each job and UnsafeUtility.MemCpy() it over

rugged plover Mar 3, 2023, 6:08 PM

#

@severe fable thank you for answer, but I'm very unfamiliar with all this pointer stuff. Can you please show quick example of such a job?

severe fable Mar 3, 2023, 6:36 PM

#

actually looking at it more, i don't see how you'd know which chunks to copy from if you don't use a generic. i didn't read close enough to notice that at first, and just read your text

rugged plover Mar 3, 2023, 7:02 PM

#

@severe fable but if chunk could return ptr to use it with MemCpy, then how code should look like?

severe fable Mar 3, 2023, 7:04 PM

#

well but it can't, because you don't know where in the chunk the data array for your particular component is, since chunks can have more than one component type in them.

#

anyway, if you could find correct pointers and sizes somehow(tm), you could use this https://docs.unity3d.com/ScriptReference/Unity.Collections.LowLevel.Unsafe.UnsafeUtility.MemCpy.html with said pointers and sizes, but i don't know how you could

tardy sun Mar 3, 2023, 7:11 PM

#

You can totally avoid generics here, just switch it to a byte array and just pass in the size of your data

#

This is basically what I do for my save system

#

The only thing in your code that is the generic is the output array

rugged plover Mar 3, 2023, 7:15 PM

#

tardy sun The only thing in your code that is the generic is the output array

as @severe fable mentioned it is also chunk.GetDynamicComponentDataArrayReinterpret<T>

tardy sun Mar 3, 2023, 7:16 PM

#

Use byte there as well

rugged plover Mar 3, 2023, 7:16 PM

#

as T?

tardy sun Mar 3, 2023, 7:16 PM

#

Yeah sec let me find a pc

#

I use dynamic type handles a lot to replace generic jobs

#

But I forgot pattern

rugged plover Mar 3, 2023, 7:17 PM

#

ok, I see now. But how to convert any array to byte array?

severe fable Mar 3, 2023, 7:17 PM

#

and here we show that i haven't used dynamic component type handles before...

rugged plover Mar 3, 2023, 7:17 PM

#

Reinterpret?

tardy sun Mar 3, 2023, 7:17 PM

#

var components = chunk.GetDynamicComponentDataArrayReinterpret<byte>(ref this.ComponentType, this.ElementSize);
Check.Assume(components.Length > 0);
this.Serializer.AddBufferNoResize(components);```

#

Yeah from my save system

#

Serializer is just a native list

#

https://github.com/tertle/com.bovinelabs.saving.free/blob/6d4a9d303b5acc85033b2cf9047efaf6ac3f5fde/BovineLabs.Saving/Save/ComponentDataSave.cs#L202

#

that's from here if you want an example

#

you already pass in type size so you don't even need to change anything you're passing

#

[BurstCompile]
// takes all chunks by query, and then just copy component data to compute buffer starting from 1st entity in query index
internal struct SyncPropertyByQueryJob : IJobChunk
{
    [ReadOnly][DeallocateOnJobCompletion] public NativeArray<int> chunkBaseEntityIndices;
    // this should be filled every frame with GetDynamicComponentTypeHandle
    [ReadOnly]public DynamicComponentTypeHandle componentTypeHandle;
    public int typeSize;
    [WriteOnly][NativeDisableParallelForRestriction] public NativeArray<byte> outputArray;

    public void Execute(in ArchetypeChunk chunk, int unfilteredChunkIndex, bool useEnabledMask, in v128 chunkEnabledMask)
    {
        var data = chunk.GetDynamicComponentDataArrayReinterpret<byte>(ref componentTypeHandle, typeSize);
        NativeArray<byte>.Copy(data, 0, outputArray, startCopyToIndex, data.Length);
    }
}```

when scheduling job
outputArray = previousOutputARray.Reinterpret<byte>(sizeof(TProperty))

rugged plover Mar 3, 2023, 7:24 PM

#

Ok, so I can try to use ComputeBuffer.BeginWrite<byte>() with any actual data inside and chunk.GetDynamicComponentDataArrayReinterpret<byte>(ref this.ComponentType, this.ElementSize); also with any actual data inside and then just copy without any generic

#

One thing still confusing me. Why we're passing stride of our actual data layout?

tardy sun Mar 3, 2023, 7:25 PM

#

the typeSize?

rugged plover Mar 3, 2023, 7:25 PM

#

yes

tardy sun Mar 3, 2023, 7:27 PM

#

🤷‍♂️ DynamicComponentTypeHandle doesn't have the info

#

they could look it up and store it inside when created though

rugged plover Mar 3, 2023, 7:28 PM

#

I'm more about: we reinterpreting as byte but passing another type's size

tardy sun Mar 3, 2023, 7:28 PM

#

rugged plover I'm more about: we reinterpreting as `byte` but passing another type's size

it still has to declare the array length

rugged plover Mar 3, 2023, 7:29 PM

#

oh, ok

tardy sun Mar 3, 2023, 7:29 PM

#

which is going to be sizeOfElement * length / sizeof<T>

#

T is byte so 1

#

but you know

#

looking at source, it just reads this from the chunk anyway 😄

#

CheckComponentSizeMatches(typeHandle, typeSize, expectedTypeSize);
it's only used to confirm it matches

#

var typeSize = archetype->SizeOfs[typeIndexInArchetype];
it doesn't actually use it for the array length, just reads it directly

#

oh well

rugged plover Mar 3, 2023, 8:09 PM

#

WOW! It works!!!! Just a little mind work with indecies and it WORKS!

#

@tardy sun @severe fable thank you for help, it will improve my system a lot!

peak lava Mar 3, 2023, 9:54 PM

#

Could someone explain IJobChunk for dummies?

tardy sun Mar 3, 2023, 10:10 PM

#

Ijc is basically the way you iterate entities in jobs

#

Things like IJobEntity code gen into ijc to provide a more convenient api

#

I'm not a fan of most tutorials because they skip over this, personally when teaching new devs we start with ijc to understand how entities work then move onto the code gen convenience

#

Anyway the manual will do a better job explaining how to iterate than I will on a phone

#Avoid using generic jobs if possible