#archived-dots

1 messages · Page 38 of 1

misty wedge
#

Which system creates and schedules the job that populates the drawer with commands

rotund token
#

whatever system wants to

#

it can be a specific debug system inside one of my debug assemblies

#

an example

    [UpdateInGroup(typeof(DebugSystemGroup))]
    [RequireMatchingQueriesForUpdate]
    [BurstCompile]
    public partial struct NavMeshPathDrawSystem : ISystem
    {
        /// <inheritdoc/>
        public void OnCreate(ref SystemState state) { }

        /// <inheritdoc/>
        public void OnDestroy(ref SystemState state) { }

        /// <inheritdoc/>
        [BurstCompile]
        public void OnUpdate(ref SystemState state)
        {
            var drawer = SystemAPI.GetSingleton<DrawSystem.Singleton>().CreateDrawer<NavMeshPathDrawSystem>();
            state.Dependency = new DrawNavMeshPathJob { Drawer = drawer }.ScheduleParallel(state.Dependency);
        }

        [BurstCompile]
        private partial struct DrawNavMeshPathJob : IJobEntity
        {
            public Drawer Drawer;

            public void Execute(in DynamicBuffer<Path> paths, in FindPath findPath)
            {
                if (paths.Length == 0)
                {
                    return;
                }

                var ps = paths.AsNativeArray();
                for (var index = 0; index < ps.Length - 1; index++)
                {
                    var p = ps[index];
                    var p1 = ps[index + 1];

                    Drawer.Line(p.Position, p1.Position, Color.green);
                }

                Drawer.Line(ps[^1].Position, findPath.Target, Color.red);
            }
        }
    }```
misty wedge
#

Ah alright, so you just have a dedicated system for it

rotund token
#

again it's not required

#

it's just my preference

misty wedge
#

Yeah I know, I was just asking for your approach 🙂

rotund token
#

my drawers also can have optional categories assigned

#

and can be filtered by type

#

so you can turn them on/off at runtime

misty wedge
#

I just switch between server side and client side gizmos 😅

rotund token
# 
            {
#if UNITY_EDITOR || DEVELOPMENT_BUILD
                var systemTypeIndex = TypeManager.GetSystemTypeIndex<T>();

                var enabled = IsEnabled.Data &&
                              (this.SystemFilterSet.Contains(systemTypeIndex) || (category != default && this.CategoryFilterSet.Contains(category)));

                this.KnownSystemSet.Add(systemTypeIndex);

                if (category != default)
                {
                    this.KnownCategorySet.Add(category);
                }

                NativeEventStream.Writer writer;

                if (enabled)
                {
                    var drawer = new NativeEventStream(this.allocator);
                    this.drawers->Add(drawer);
                    writer = drawer.AsWriter();
                }
                else
                {
                    writer = default;
                }

                return new Drawer(enabled, writer);
#else
                return default;
#endif
            }```
rotund token
#

previously filters were shared

#

so if you turned on physics drawer it'd draw both client/server

#

(well in this particular case no as i wrote a specific workaround for it, but in general this is how it'd work)

misty wedge
#

nice

rotund token
# 
        {
            [ConfigVar(CV.DrawEnabled, true, CV.DrawEnabledDesc)]
            internal static readonly SharedStatic<bool> IsEnabled = SharedStatic<bool>.GetOrCreate<CV.DrawEnabledTagType>();

            private readonly UnsafeList<NativeEventStream>* drawers;
            internal NativeHashSet<int> SystemFilterSet;
            internal NativeHashSet<FixedString32Bytes> CategoryFilterSet;
            internal NativeHashSet<int> KnownSystemSet;
            internal NativeHashSet<FixedString32Bytes> KnownCategorySet;

            private Allocator allocator;```
#

but yeah i just use the command buffer approach of using a UnsafeList<NativeEventStream>* drawers;

misty wedge
#

What's this? [ConfigVar(CV.DrawEnabled, true, CV.DrawEnabledDesc)]

rotund token
#

oh i have this system that lets me write to a static variable

#

and it can be populated from environmental variables

#

it's basically a preference + environmental variable system combined into 1

#

in editor it reads editorprefs to remember a users state and in builds it reads from environmental variables (and optionally saves state) to config instances of the app

#

(it's highly inspired/copied from the dots shooter sample, i just made it burstable)

misty wedge
#

this is my hardcoded lazy version of that 😅

misty wedge
rotund token
#

nah the really old multiplayer shooter sample

#

i dont think that's been updated right?

misty wedge
#

I have no clue, I've not looked at the dots samples much

rotund token
#

this thing

#

yeah 3 years since update

robust scaffold
rotund token
robust scaffold
#

That's the newest DOTS netcode samples.

rotund token
#

though i have jazzed it up a lot - mine uses shared statics not static fields

#

and i generate the whole UI etc from it, save state etc

robust scaffold
#

All this work for debugging features. I just debug log and hope.

misty wedge
rotund token
#

psh burstable break points where it's at now

#

but yeah visual debugging (drawing) is mandatory imo

misty wedge
rotund token
#

it catches so many bugs

rotund token
#

it's been working fantastic for me

#

i think i've only had it fail once and a project sync fixed it

misty wedge
#

1.7.4 so I guess that explains that

rotund token
#

yeah it only works in 1.8+

#

being able to break point in burst* is game changing though

misty wedge
#

Did they fix all of the weird burst caching issues?

rotund token
#

as far as i'm aware yes

robust scaffold
misty wedge
#

Guess I'll upgrade then

robust scaffold
#

Breakpoints seem nice but... i've never used them.

rotund token
#

o_O

#

what

robust scaffold
#

just code gooder

rustic rain
rotund token
robust scaffold
#

shader code cant use breakpoints and the other code can largely be debugged using log.

rotund token
#

it just turns burst off for that specific job you have broken into

#

debug log is so inefficient though

#

you have to recompile

rustic rain
#

Oh man

rotund token
#

you can't just catch an exception randomly

rustic rain
#

Log is all we have in modding 😅

#

Ppl made hotswap libs to ease pain

rotund token
#

what stops you hooking up a native debugger when modding?

rustic rain
#

Also no PDB files, so you have to look up error line through IL

rotund token
#

seeing code is less important than seeing the data

rustic rain
#

Luckily Brainzzz

#

The guy who made Harmony

#

Made an app that makes it easy

#

Just to debug RimWorld kek

rotund token
#

wait these are .net games aren't they

#

unity mono*

rustic rain
#

It's unity game

rotund token
#

you can literally just read source code with a debugger without pdbs

rustic rain
#

That's what we do, yes

rotund token
#

i often just reverse games and read source

#

it's extremely interesting to see different peoples approaches

#

(but i also like finding uses of my libraries =D)

rustic rain
#

But game only generated limited stacktraces

#

Without a line of code

#

Only IL instruction ID in specific method

rotund token
#

an interesting approach could be selecting a system from the system window

#

the thing that's being inspected is InspectorContent

#

which is a dynamic drawer for 6 different types

rustic rain
#

Hm. Does Aline work outside of runtime?

rotund token
#

but what you're interested in is if Content is SystemContentProvider
so you could have a gizmo drawer when a system is selected in inspector

rotund token
#

no idea

rustic rain
#

Yeah

#

Instead of gizmos

rotund token
#

no reason it couldn't though i don't know how it's setup

#

my drawer works by simply adding it to editor world

misty wedge
#

You need to specifically tell it to render in game view

rustic rain
#

Would be interesting if Unity bought it to swap it's own implementation 😅

#

Just like rival

rotund token
#

you mean buy it and not update it anymore? 😄

#

sigh back to documentation

unknown.png
misty wedge
#

yay saving

#

looking forward to that 👍

rustic rain
#

I don't really like the idea of saving through attribute or interface

rotund token
#

you don't have to use attributes

#

you can just tell the saveprocessor what types you want to save

rustic rain
#

Yea

#

This is what I used too

rotund token
#

you might not have access to the component

rustic rain
#

I also have another idea in mind

rotund token
#

so can't add the attribute

rustic rain
#

Just like bakers

rotund token
#

this is actually kind of how my system works

#

it just generates the 'baker' for you

rustic rain
#

To define saving you declare type Saver<t>

robust scaffold
#

Serialize and deserialize?

rustic rain
#

And it will feed component data one by one to it

rotund token
#

literally what i do

#

i just automate the process for you

robust scaffold
#

Does 7zip have a c# API?

#

Is there a stream compression algo?

rotund token
#

new ComponentDataSave(this.builder, stableTypeHash);
literally create 1 'saver' per type you have specified. You can do this manually instead.

rotund token
#

unity has a built in burst friendly wrapper

#

damn bot

#

if you go back a directory and into l4z-1.9.1 includes the per platform libraries

robust scaffold
#

Huh, neat.

rustic rain
#

Save file compression?

rotund token
#

but yeah unity has this in com.unity.entities@1.0.0-exp.8\Unity.Core\Compression

#

but they strip the stuff in builds

rustic rain
#

There are tons of 7z libs for . net though

rotund token
#

l4z is much faster than lzma in compression though

robust scaffold
#

I would imagine for save files, size is more important than creation and decompression speed.

rotund token
#

hmm i disagree

#

compress speed was already a noticeable issue for l4z

#

so i split my compression per component instead of combined

#

and the difference between 10mb save file and 9mb save file

#

is not a big deal

#

lz4 time: 0.0008859634399414062
lzma time: 0.23617887496948242

#

its magnitudes different compression time

#

lz4 compression ratio: 0.20672
lzma compression ratio: 0.11150

#

sure it compressed quite a bit better

rustic rain
#

Yeah, Id say saving time is more cruical

rotund token
#

compression time is really all i care about for real time saving

robust scaffold
#

Yea, that's pretty big difference.

rustic rain
#

Nothing more annoying than waiting 2 secs to save

rotund token
#

decompression isn't important as it's usually hidden behind a load screen

robust scaffold
#

Maybe an option? Manual saves uses the better compression but quicksaves uses lz4?

rotund token
#

zip processing speed: 69.78 MByte/s
lz4 processing speed: 718.92 MByte/s
lzma processing speed: 2.70 MByte/s

zip processing speed: 121.29 MByte/s
lz4 processing speed: 9259.26 MByte/s
lzma processing speed: 7.97 MByte/s

#

top is real world data

#

bottom is synthetic data

#

imagine you have 50MB of entity data

#

5 seconds later you'll have saved

#

(ok to be fair, lzma is only 1 of 7z algorithms. 7z is just a container format, supports multiple compression methods. the others might be quite a bit faster)

rotund token
#

something i might consider in the future

robust scaffold
robust scaffold
#

ToEntityArray does not filter by enabled component existence. Annoying...

rotund token
#

hmm thought it should

#

it seems to go through the filter path

robust scaffold
#

Seems incomplete though so probably in the future?

rotund token
#

ah

# 
            {
                GatherEntities((Entity*)entities.GetUnsafePtr(), in cache, in matchingArchetypes);
            }
            else
            {
                var filter = entityQuery.__impl->_Filter;
                GatherEntitiesWithBatching((Entity*)entities.GetUnsafePtr(), ref filter, in cache, ref matchingArchetypes);
            }```
#

it should be going through the GatherEntitiesWithBatching path

#

from public static NativeArray<Entity> CreateEntityArray

misty wedge
#

Do I need to call CompleteDependency if I'm writing to a singleton using GetSingletonRW?

rotund token
#

if it errors yes 😄

#

(i don't think so though)

robust scaffold
#

Ah, it does.

unknown.png
#

I wish singleton handled enabled components as well...

viral sonnet
rotund token
#

Spacebase Startopia uses my event system, though i knew this already

viral sonnet
#

a startopia remake? noice

#

never heard of it 😄

rotund token
#

didn't seem to do as well as i expected for them 😦 (the fact you haven't heard of it probably didn't help!)

#

looked cool

#

though i've never played startopia so i don't know how accurate the comments about it being different are

viral sonnet
#

i've played the original in 2001 or smth. tried to get into the gog version but these type of games have died, rightfully so in my opinion. gamers are more into persistent big sims now rather than small mission based ones. the price point also doesn't help i guess.

misty wedge
#

Why is the trailer in german even on english steam? thonk

#

I think my ALINE is broken :[

#

These are all the gizmos I'm drawing

#

The weirdest thing is it's lagging my input insanely. I'll press a button to move and nothing will happen, then it will keep holding it, as if it was delayed by like 5 seconds

#

This job also takes an insane amount of time for some reason

rotund token
#

how many verts are you drawing?

#

that'd be about how long it takes for me to draw 1mill+

#

though im not sure what stage that is for aline

#

my actual mesh building is done multi threaded

#

are you drawing this all via the same thread?

viral sonnet
#

when we need access to an isystem, are we supposed to cache the handle or can we cache the isystem directly which we get from GetUnsafeSystemRef, even though it's a ref?

robust scaffold
#

@rustic rain @misty wedge IJobChunk does filter out disabled components entities if they are in the query as a .WithAll<EnabledComp>(). Took me about 2 or 3 hours of debugging why my prediction switcher wasnt working with multiple thin clients connected

rotund token
#

to answer your question, i would probably just get it again when needed

#

not cache anything

#

(though realistically, my first answer avoid it =D)

viral sonnet
#

yeah i know 🙂 i'm using a singleton to acces

rotund token
#

i converted a bunch of code to use dynamic type handles instead of generics last week

#

was an interesting experience trying to do work on long arrays of bytes

#

makes you think about what's going on under hood

viral sonnet
#

i love byte pointers 😄

rotund token
#

gets a bit hard

#

when you don't know the type

#

or fields

#

etc

#

basically implementing interfaces on a byte level

viral sonnet
#

yep, it's the type of code you look at in a few months and not understand anything 😄

viral sonnet
rotund token
#

yeah

viral sonnet
#

i have everything converted now to ISystem. the only thing that are still SystemBase are ones that use GOs and the generic stat system

rotund token
#

i converted my state system to a helper struct

viral sonnet
#

nice was about to ask 😄

rotund token
#

compare it ot the previous version

#

where i had to inherit from systems etc

viral sonnet
#

less files? hehe

rotund token
#

to the systems entity

#

the fixed 1 byte version was easy to change from generic

#

to just do a memcmp

#

the unknown length (though i've limited it to 256 bits) was a bit more doing my head in

robust scaffold
#

Huh, NAs can now return Spans. A safer pointer with (nearly) no overhead is nice.

unknown.png
viral sonnet
#

how's your mainthread timing of ISystem? mine are mostly around 0.01ms

#

still wonder how those with 1k systems are managing. that will be 10ms straight systems

#

it was said SystemAPI.GetSingletonRW calls .Complete. doesn't work for me. I need to call .Complete beforehand otherwise i get errors

#

i should finish up my asset. stuff i don't like has been ironed out, 1.0 is fully working, no known bugs, 50k casters every frame run at 3ms sim time. time to work on some docs then 😄

rotund token
robust scaffold
rotund token
#

they're not allowed for entry point arguments

#

Added support for System.Span<T> and System.ReadOnlySpan<T> within Bursted code. These types are not allowed as entry-point arguments.

robust scaffold
#

So a unbursted job can pass spans as fields back and forth? How does that work?

viral sonnet
#

i don't see much value for span in burst and unmanaged space. i never once had pointer code that was overflowing

robust scaffold
#

Sure, once everything is nailed down and properly tested, the span can be replaced with a raw pointer. It's like a very specific type of Assert.IsTrue()

viral sonnet
#

ah i see, guess spans are useful instead of writing watch evaluators

robust scaffold
#

The downside is that it's not easily reinterpreted. You need to fix the span and converting it back to a pointer and then reinterpret. Which is very annoying.

hard wagon
#

I can't seem to find JobComponentSystem class?

rotund token
#

it doesn't exist

#

it's been deprecated for like 2 years

hard wagon
#

Has it been substituted to something else?

#

I guess "ComponentSystem" was the replacement?

rustic rain
proud jackal
#

Or ISystem, you can pick either or 🤷‍♀️

solemn hollow
#

Is there now a way to find an Entity in the inspector that has a set of components like it was possible with EntityDebuggers Filter option?
Lets say i want to get a list of all Entities that have an AbilityTag AND Prefab.

unknown.png
#

Doing it like this in the Hierarchy just gives you AbilityTag OR Prefab

#

Thats the number one feature i miss from entity debugger.

dense storm
#

How to get the amount of entity with certain tag?

#

Like this? int count = SystemAPI.Query<SomeTag>().Count();

#

InvalidOperationException: No suitable code replacement generated, this is either due to generators failing, or lack of support in your current context

#

int count = SystemAPI.QueryBuilder().WithAll<SomeTag>().Build().CalculateEntityCount();

solemn hollow
dense storm
#

Yep, got it, its getting the right amount ty

misty wedge
rotund token
#

that it does an Any query instead of an All

#

i'm | | close to writing my own hierarchy window

#

i really expected 1.0 to update behaviour...

solemn hollow
rotund token
#

oh yeah

#

you can't name something more than 64 characters

#

or is it 62

#

whatever fixedstring64 is

solemn hollow
#

i really hope there will be an option to get more characters

rotund token
#

i guess even with 1mill entities

solemn hollow
#

sure in most cases its not needed but i really like naming extensivly

rotund token
#

doubling it would only be 64mb

#

so while i personally dislike long names it does seem silly not to support something a bit longer

solemn hollow
#

i mean even if they would just trigger a warning once id be fine with it. the constant spam is too much

rotund token
#

just truncate it

#

at worst

spice vale
#

I need a bit more info. What is your type UIBase? Do you still have a gameobject with UIDocument in the scene?

rustic rain
#

I do have 1 GO with UIDocument

#

I create root VE in it

#

and then just attach screen VE's to it

spice vale
#

So whole hud is in same tree? Sorry if this is obvious, first time using UI toolkit.

rustic rain
#

yeah

spice vale
#

okok, yeah I don't see a way around UIDocument.

rustic rain
#

ngl, I just had an idea of...

#

entity UI

#

kek

#

literally draw entities same as you used to draw GO UI

#

I'd guess that would be the case for some world space UI elements

spice vale
#

not sure what you mean 😅

spice vale
#

when using addressables with systems, our only option is using magic strings, right? Or is there a more robust way?

brittle anvil
#

What I want is a workflow where Entities replaces GameObjects completely in the editor
I get why they're not going for that right now because people would have to re-learn the engine, but I do think having a workflow that completely ignores the existence of GameObjects is the way forward

#

Conversion/Baking workflows just add a layer of complication

solemn hollow
#

@brittle anvil i think baking is just a necassary evil. the authoring side just needs a different datalayout to give an optimal interface for gamedesigners while runtime data needs to be optimized for performance. I saw how a AA Studio implemented their own baking/conversion pipeline (before entities existed) exactly for that reason.

rustic rain
spice vale
#

what do you mean?

rustic rain
#

in monob

#

so you just store everything on prefab

spice vale
#

and you instantiate the prefab from a system or what?

rustic rain
#

no even need to instantiate

#

as long as you got your reference

spice vale
#

ah true

viral sonnet
#

you have 1 addressable as sort of dictionary/list that references other addressables?

rustic rain
#

nah, I use magic strings

#

just shared the other way it can be done

viral sonnet
#

and that works for dots or just MBs?

spice vale
#

no, I think he means that you use baker to create a singleton entity with reference to a prefab that holds addressables

rustic rain
#

that's the other way

#

allthough

#

I'm not sure

#

whether it's really required

#

if you have subscene, why not just have game object in the first place

spice vale
#

which go?

rustic rain
#

the one you attach your references to

#

you can just keep it inside of scene and find it in there

#

don't see a point to drop it into conversion

#

but you do you

spice vale
#

How would you find it in a system if it's not baked into entity?

rustic rain
#

Object.FindOfType<T>

spice vale
#

That works, I'm too unfamiliar with vanilla unity lol.

rustic rain
#

me too

#

straight dive into ECS 😅

spice vale
#

same here basically

brittle anvil
#

This would also do away with the need to still use MonoBehaviours

viral sonnet
#

Newcomers always voice the same thing until the learn about the workflow. Even though you work with GOs and MBs in the editor it's just something that already has been established and no GOs or MBs will exist at runtime. It's editor only data.

brittle anvil
#

Just seems a bit counter-intuitive unless you are already coming from the old GameObject way of doing things

#

The way Entities currently works means you have to really learn the "old" way first, then graduate to Entities in order to understand it

rustic rain
#

you don't

#

you just need to learn your tools

#

and game object is one of them

brittle anvil
#

I know why it works this way, it's because they don't want to alienate the experienced developers

rustic rain
#

well... not really

brittle anvil
#

But if you come from a custom engine or something that already uses a Pure ECS, GameObjects and MonoBehaviours can feel weird fitting into that concept

rustic rain
#

yeah

#

at first they wanted conversion

#

which was fully OOP

#

now they switch to ECSy approach - Baking

#

and it does feel odd

#

and GameObjects are only used because they are already ready part of engine and easy to work with

brittle anvil
#

I do hope they add a separate workflow in the future, it needs to be possible to ignore the GameObject/Monobehaviour workflow in its entirety if you want to

#

The building blocks for it are there already

rustic rain
#

I don't really see how a better way, because in the end you still want to have initial authoring data, which makes sense, compared to runtime data which doesn't always do

#

so instead of game objects you'll have... Editor entities or smth

#

same thing under the hood

brittle anvil
#

I suppose this is where it works differently to the engine I wrote, where the ECS worked in the editor too

rustic rain
#

ECS does work in the editor too in Unity

brittle anvil
#

It does, yes

#

But only when playing

rustic rain
#

no

#

you have Editor World outside Play mode

#

as soon as you close subscene it's converted

#

and now you have entities

brittle anvil
#

Right, so they're most the way there then with that

#

They're not not letting you actually create entities and add components to them as entities in the editor, they convert it after you've used the "familiar" workflow instead

rustic rain
#

yeah

#

you have to use bakers for that

brittle anvil
#

Whereas my engine worked where you would create an entity and attach the components to it, this worked live in the editor

#

They weren't a separate thing, they were literally an entity existing in an ECS World created for the editor runtime

rustic rain
#

I get it

#

well, it's still doable in Unity

brittle anvil
#

I hope Unity does a similar thing in the future

rustic rain
#

just requires extra steps

#

which yeah, is a bit downside

#

for debugging

#

I wish for a feature to remove comps in Editor windows

#

or add them

brittle anvil
#

The only benefit I see to just sticking with the current workflow is compatibility with GameObjects for those that want that

rustic rain
#

well, it'll be a while until Unity will have animation rolled out

#

UI

#

and smth else I had in mind..

#

animation uses hybrid for now

#

and GOs in general a good thing

#

just objects in a scene

#

doesn't have to be related to actual gameplay at all

brittle anvil
#

What I did was made Scenes be a basically a collection of Entities that had a Transform component at the very least

#

But most the time you'd have a MeshRendererComponent too

#

Then in the engine itself, a system existed that would iterate over all MeshRenderer components in the loaded scene and render them

rustic rain
#

maybe it does work like that, kek

#

it's written in c++, so we can't really know

#

but since there's dynamic batcher

#

I'd guess it does work like that at least in a way

#

anyways, by scnee I meant actual Unity Scene, not SubScene (which is a collection of entities)

brittle anvil
#

Yep, the ones that have GameObjects in, that I want obsolete in the future

robust scaffold
viral sonnet
brittle anvil
#

It isn't exactly the same thing, since data never needed to be converted in my engine

#

The data was stored and retrieved from the components

#

It's the MonoBehaviour step that's the real difference here

viral sonnet
#

The decoupling is a conscious decision to split designer and runtime data.

#

It has nothing to do with MBs in the conventional sense

robust scaffold
#

loading subscenes when switching scenes keeps freezing unity editor, so annoying

balmy thistle
robust scaffold
balmy thistle
robust scaffold
robust scaffold
# balmy thistle Is it a complicated repro?

I've just decided that every time I change a subscene, any of the prefabs inside of it, or add a system, I open all subscenes, clear cache, and restart unity. Then once reloaded, i go through my two scenes, close all subscenes, run each scene individually to ensure no baking errors, then go to my launcher scene and run the game again. If I had more than 2 scenes, i might be more concerned. But as it is right now, it's only 10 seconds occasionally so long as I dont touch the subscene or any prefabs or create a new system in the code. Modifying code itself seems to not break things most of the time.

#

If I leave Unity running (not play mode, just leaving the editor alive) for a long enough time, a baker concurrency error gets thrown breaking all subscene caches requiring this whole process to repeat.

rotund token
#

Oh I get the concurrency issue very frequently

#

But for me it just enter play mode a second time and it's fine

#

Very annoying though

#

Cache breaks frequently from nothing though

robust scaffold
#

Yea, no clue what's causing it because I'm doing the same thing and sometimes it just stops working

rotund token
#

i keep finding cool features i forgot i implemented in my save library while documenting it -_-

robust scaffold
#

So many errors, tilemap errors, editor hub errors, and dots errors. It's like unity programming is reloading the editor over and over again...

#

Ah, it happened again. Here's the concurrency error message: "InvalidOperationException: Operations that change non-concurrent collections must have exclusive access. A concurrent update was performed on this collection and corrupted its state. The collection's state is no longer correct." Originating from "Library/PackageCache/com.unity.serialization@2.0.0-exp.11/Runtime/Unity.Serialization/Utility/DefaultTypeConstruction.cs:79"

viral sonnet
#

are you corrupting memory by any chance?

#

with pointers

#

my main project is stable

#

i've taken a look at rival platformer sample. the main entity is 2.1k bytes 😅

#

most data should be in a blob 🤷

rotund token
#

i get the same concurrent error

#

im pretty sure unity is aware of it (and maybe even fixed it already)

robust scaffold
#

And plus, if it was me, it'll be happening every time I press play.

rotund token
#

@robust scaffold

#

when i reported it

#

and response

ecuzzillo — 07/10/2022
fixed in an upcoming editor release iirc

#

mine seems to be a different stack but same error

#

so hopefully same problem

gusty comet
#

Is there a way to prevent a disabled GameObject in a Subscene from being baked?

#

Right I think it converts the GameObject anyway, but adds a Disabled component

robust scaffold
#

Oh yea, and leaving a c:<ComponentType> search in the entities hierarchy leads to a Allocator.Temp mem-leak warning.

#

Dont know if that does anything but I always restart Unity when I forget to clear the component type search when stopping play mode.

robust scaffold
gusty comet
#

Gotcha. Guess I'll have to write a Destroy authoring that collects GameObjects marked with DestroyThis, injects those entities into a Buffer, and a DestroySystem executes once to destroy every entity in that Destroy's dynamic buffer

#

Is that a good solution?

robust scaffold
gusty comet
#

I use GameObjects as a folder sometimes

#

You know, to group GameObjects that I actually want to convert inside the Editor

robust scaffold
#

Use a subscene. You can use subscenes inside subscenes.

gusty comet
#

Ah, and the subscene doesn't get converted?

robust scaffold
#

Well, technically yes? There's a subscene entity but that's for monitoring if the subscene has loaded in completely.

gusty comet
#

Sure. There's one problem though and that's that the GameObjects between subscenes can't reference each other, whether it's dragging one GameObject into a different GameObject's field, or using GetComponentsInParent/Children to get parent/child GammeObjects in different subscenes

#

I wrote some scripts that greatly simplifies wiring together entities by treating the GameObject hierarchy as an organizational tool

robust scaffold
#

Ideally, you wont have those references to begin with. And instead find relevant entities with queries. Using tag components and so on.

gusty comet
#

What are the alternatives?

#

Like, I want a StatAdder entity to reference Stat

robust scaffold
#

Queries or have it exist on the same entity

gusty comet
#

Forgive me for not knowing much about subscenes, but are you saying that a Baking System that queries at baking time can only find Entities within the same subscene?

#

So I could take advantage of that by attaching a tag component, and the subscene only knows the tag components within itself?

robust scaffold
#

No no, at runtime.

gusty comet
#

I see. But the point that queries only return entities under the Subscene GameObject still stands?

robust scaffold
#

Simply dont have references at bake. Construct your entity relationship web dynamically at runtime by querying the existence of components and data within them.

gusty comet
#

I'm starting to understand. What are the disadvantages of wiring together GameObjects at editor time instead of Entities at runtime?

robust scaffold
#

Otherwise uh, stick with GOs then and yea, a tag stating DestroyThis and query in a baking system to destroy those entities.

robust scaffold
#

More modular programming, easier-ish to debug and expand on.

#

ECS in general is big on modular coding. Everything is isolated from each other using general type handles and operating on a set of data from them.

gusty comet
#

Gotcha. I've actually replaced all direct field references with a way to inject references around the place within a GameObject "folder". Guess I'll write a system to do that but in ECS?

robust scaffold
gusty comet
#

One Entity performing actions on another entity through ECB or Component/BufferLookup

#

Those actions occur really rarely though. IT's stuff like a DecisionMakerStop entity running some stopping logic on a DecisionMaker entity, for example

#

There's actually a ton of Entity references in my design. Not sure if that's good or bad

#

Fortunately most interactions are event-driven, with next to no constant-frame operations

robust scaffold
#

Entity references are unavoidable ultimately but hrm, event driven actions.

#

Have you seen the forums with their discussion on event driven approaches?

gusty comet
#

Yeah, there's tertle's event system and the whole IStateComponentData thing

robust scaffold
#

I hate rehashing it, especially since it's pretty well written.

#

^ The guy who wrote that was hired by unity to work on dots as well which is sweet.

#

I dont know, what I'm getting from that post is that monobehavior isnt actually that slow.

#

DOTS isnt end all be all programming style. If you have a lot of interconnected relationships, a very performance focused OOS might work better. Like a Rust or C++ implementation.

gusty comet
#

Thanks for sharing that link. It made me rethink DOTS design

#

So it looks like the most performant event-driven solution would be using an EventStream scheme, like maybe tertle's Event System, and for uncommon events, using Component/BufferLookup to execute the event

robust scaffold
#

Where are these events being triggered? And how?

viral sonnet
gusty comet
#

I'm just worried because each one of these events, like TraverseRuggedTerrain and AddStats, already read and write to a bunch of components through IJobEntity. Turning them into events like that would mean a ton of more Component/BuffeRLookups

viral sonnet
#

mono will fall apart on anything else though

gusty comet
#

It does the following in order, and then repeats:

#
  1. Evaluate Conditions of all Plans
  2. Choose the first Plan in a list that fulfills all of its Conditions
  3. Executes the Plan's Task, and any Events (abstraction for a single operation)
  4. Once the Task generates a Complete event, that Complete event restarts the DecisionMaker
#

It's my simple psuedo-GOAP system

#

It works for my game, but it's design is clearly flawed according to that guy's performance benchmarks

#

Right now events are done through attaching a ConditionEvalute, TaskProgress, or EventExecute component, and then once they are done doing stuff, they remove that component

robust scaffold
robust scaffold
viral sonnet
#

yeah add/remove ends in terrible performance. write to a nativecontainer instead. no need to go through entities

gusty comet
#

Thanks guys. I don't want a massive rewrite, but the simplest solution would be to just use Enable/Disable component

#

Are ECBs slow even when just enabling/disabling components?

robust scaffold
#

Final system at the end using that query disables all the condition enable-able components.

robust scaffold
#

See, no entity relationships, no hierarchy needed.

gusty comet
#

True. I don't like introducing a write dependency by using ComponentLookup.Enable though

robust scaffold
#

No component lookups required.

viral sonnet
#

dependencies are normal. what you want to avoid are sync points with jobHandle.Complete

robust scaffold
#

it's a system query operating laterally across plan entities.

gusty comet
gusty comet
robust scaffold
gusty comet
#

Also, some Plans require more than one condition, and some plans generate more than one event. I guess the plan is to condense all possible Condition/Task/Events into one generalized component and system for each of those three?

robust scaffold
gusty comet
#

For example, there are Events that do things like StatAdd, SearchFor, Pathfind,

#

I see now!

#

Wait, so a plan can have everything...

#

A buffer of Condition data

#

A single Task data

#

A buffer of Event data

#

and enabling ConditionEvalute, TaskProgress, or EventExecute will do things to those events?

robust scaffold
#

Not a buffer, a condition component and the fields within that condition IComponentData struct will be the data related to it's conditionals.

gusty comet
#

Right, but a plan may require mulitple conditions to be fulfilled before the decisionmaker allows it to be executed

robust scaffold
#

Yea, so a system will query for all of those

#

That _query checks for the enabled state of 4 components and the disabled state of two others

gusty comet
#

That queries for every single qualifying entity at runtime, unless you're saying that it's scoped to the subscene?

robust scaffold
#

It's for every single entity. Subscenes are only an organization tool in the editor

#

upon running / play, all entities get plopped into a giant mixing pool known as a world.

gusty comet
#

yep, sounds familiar

robust scaffold
#

Imagine those 6 components as your conditions. This system, or event, requires that 4 components / conditions be fulfilled and 2 others that shouldnt in order to execute.

#

You can mix and match whatever conditions you require for each type of event you want triggered.

gusty comet
#

Gotcha. Some Plans don't require the completion of all possible Condition types though. Some Plans might be wired to a StatAbove Condition, while another might only require SearchFor + FoundAtLeastOne, for example

robust scaffold
#

And that query will only return the entities / plans that have furfilled those conditions.

gusty comet
#

Gotcha

#

Oooooohhh

#

So If a Plan requires StatAdd, then part of the query is UsesStatAdd and SuccessfulStatAdd?

#

that will check for any plans that uses StatAdd AND succeeded in it

robust scaffold
#

Yea. And that system will be the event that does the work you want happened once that condition has triggered.

#

Now depending on what you want done, that might require entity lookups and there's no way around that sadly.

gusty comet
#

That's perfectly fine, your amazing plan reduces the only entity relationships to just DecisionMaker -> Plan (since plan already contains buffers)

#

Thanks a ton!

#

That was really eye-opening

robust scaffold
#

Ideally, what you want done will also exist on the same plan entity, like a transform position or health property, and you wont need any lookups.

gusty comet
#

Just one more caveat though

#

How can I account for every single combination of operations that act on Stat, operations that act on Trait, and so on? So there are (n * z)! permutations

#

n is the number of "data types"

#

z is the average number of operation types per data type

#

EntityQueries don't have nested WithAll/WithAny IIRC

robust scaffold
#

Well, there can be that many combinations but you dont need to code an event for every single combination of the conditions. Unless you want to.

gusty comet
#

Right, it's just that for a plan to have all its condition succeed, it needs a query for stuff like
WithAll
WithAll<UsesStatCompare, SuccessfulStatCompare>()
WithAll<UsesSearchFor, SuccessfulSearchFor()

#

And for every single possible condition combination, a query will be needed

#

are there any query building constructs that can do this?

robust scaffold
#

It just increments.

#

Alternatively, with a enabled component, you can just have a WithAll<StatCompare>() and it will only return entities with an enabled StatCompare component.

gusty comet
#

Could this represent "If it has UsesStatCompare, then it needs to also have SuccessfulStatCompare"

robust scaffold
#

Yea, that is the enabled component. The existence of a component StatCompare will signify that it uses StatCompare and enabled state is that it's successful.

gusty comet
#

Could each operation type enable an XFailed if it failed, and for a Plan's Conditions to succeed, it must WithNone<Every single possible failed condition component>?

robust scaffold
#

Yea. I dont see why not.

gusty comet
#

It's kinda doing a God Object anti-pattern, but if you're good with it then I am

robust scaffold
#

I dont know what sort of pattern this is but it's pretty straight forward and completely alien from traditional OOS programming.

#

I was blessed by the fact that I didnt know how to code when I started out so picking up on these patterns were very easy as I didnt know any other way to do it.

gusty comet
#

Fair, my current codebase is pretty "tainted" by OOP relationships

robust scaffold
#

Minimize and ideally eliminate those relationships for best performance. That doesnt mean to stack everything onto a single entity, rather work with systems and the query filtering process to get what you want done.

gusty comet
#

Yep, sounds good. You used a lot of SystemAPI.Query. Is that a main-thread only construcct?

robust scaffold
#

Query as in EntityQuery, not SystemAPI. I use threaded jobs.

#

I dont use SystemAPI.Query actually. Havent found a good reason to use it.

#

@rotund token I just realized, how does IJC iteration on chunks with enabled components work? From my testing, IJC operating on .WithAll<EnableableComponent>() will only iterate over entities with the EnableableComponent actually being enabled. But the chunk itself is noncontiguous in terms of components that are enabled. Are the final values of each enabled component mem-copied back to their respective positions within the chunk or is the chunk itself rearranged to provide a region in which the components are enabled for a single mem-copy operation?

rotund token
#

doesn't it pass you the component by ref?

#

the chunk is not arrange for you

#

there are gaps where enable components are disabled

robust scaffold
#

No, it passes back an archetype chunk that contains only the enabled components within the query. there are no gaps within the NAs obtained from the chunk.

rotund token
#

that's not true as far as i'm aware

#

that's not how the enable iterator works

robust scaffold
#

That's what I'm seeing in my code. Unless I'm hallucinating. I am not using the enabled iterator unless I have .WithAny<>() queries.

#

GhostOwnerIsLocal enabled component is in the above query with .WithNone<>() query. And the IJC scheduled with it will not contain any component data relating to entities (well one in this case) that has a disabled GhostOwnerIsLocal without checking the enabled mask.

robust scaffold
#

Okay, there's either a mem-copy going on following jobs with enabled components found within the query or some black magic that allows for vectorization across gaps.

rotund token
#

no idea

#

IJobEntity uses EnabledBitUtility

robust scaffold
#

I just traversed all the way up to entity storage in the source and nowhere in there does disabled components get filtered out, I think.

#

I think it's something to do with the matching of the archetypes... probably not since that indicates chunk fragmenting

unknown.png
rotund token
#

are you testing in actual split chunks

#

where some things are disabled

#

and some aren't

#

or are they all disabled/enabled

robust scaffold
#

This is GhostOwnerIsLocal, there are one enabled and thinClientCount disabled. So it's internally discontinuous.

#

Wait, hrm

#

The enabled version is structurally different.

#

Thats why it's not matching the archetype.

#

Alright, mystery solved. Im just dumb.

#

Netcode has a lot of things that break chunks with interpolated and predicted states using ghost components so a lot of what I'm imagining as identical archetypes entities are in fact not.

#

You know what would be nice, if the hierarchy or inspector had a little number on the side indicating archetype. And entities with the same archetype would have the same number.

#

Or maybe some grouping in the hierarchy list indicating as such.

gusty comet
#

Last question, I promise. @robust scaffold by condensing reasonably related components into fewer Entities, this would have the consequence of drastically increasing both the number of unique archetypes, and the size of each Entity in an archetype chunk. This results in much more cache misses everytime a chunk is loaded since there are many chunks to choose from, and each one containers fewer entities. Is this an acceptable tradeoff for drastically reduced lookups?

robust scaffold
gusty comet
#

I've also read online that cache misses happen anyways due to the OS constantly context switching between threads

#

So your design still holds

robust scaffold
#

If you operate within component data existing on a single entity, there will be no cache misses. No as in none.

#

If there is no entity relationships, there will be no cache misses.

#

Component data, as in IComponentData. Not buffer data. Those may still result in cache misses.

#

And linear component data access. Same index operations.

gusty comet
#

Right, but remember that the IJobEntity queries will only act on Plans at certain phases like Evaluate, KickoffEvent, etc, and those are already few per frame. Then you have all the different combinations of Conditions, Tasks, and Events that go with each Plan

#

So it's like one or two entities per chunk

#

But you're saying that large and sparse entities are an acceptable tradeoff?

robust scaffold
#

Yes. Well, each event. You will have an entity containing universal data and multiple 0 sized component datas indicating conditions for each event on that entity.

gusty comet
#

Okay. What I'm getting from you is that chunk iteration is already super fast, and that a chunk not having that many entities is not too much of a concern, again due to cache misses happening anyway?

robust scaffold
#

For example, if you want to have an event that sets the state of a person to "unconscious" if the health drops below 10%.

#

The person will be the entity and it will have a IComponentData called Health with a float field called Value.

#

On that person entity, it'll also have a 0 sized struct ChecksHealth implementing IComponentData, IEnableableComponent

#

There will first be a system that queries all entities .WithAll<Health, CheckHealth>().WithOptions(IgnoresEnabledComponentState)

#

Inside that system, there's a job that has Execute(EnabledRefRW<CheckHealth> check, in Health health) { if (health.Value < 0.1) check.ValueRW = true }

#

Or something like that, I dont use IJobEntity so I dont know the exacts of that.

#

And then in a second system called KnockUnconscious, there's a query with .WithAll<CheckHealth>()

#

And a job in that system iterates over entities with enabled CheckHealth tag and if so, set a separate component on that player entity to unconscious enum.

#

And a separate system can also reuse the same CheckHealth tag component, not just KnockUnconscious system. It can be another one that also checks Mana or something.

#

The combination of queries makes the event run.

#

You can also just run a conditional check off the health directly within the KnockUnconscious job but that's a really simplistic use of it.

#

There wont be any cache misses in this process. The issue is the sparse nature of these enabled components so you might be jumping / skipping a lot of disabled entities.

gusty comet
#

I see. I really appreciate you sharing your knowledge - thanks! I'll refactor my DecisionMaker code and rethink some other things.

light mason
#

Does entities 1.0 not support quest 2?

light mason
#

it was working on 0.17

gentle gyro
light mason
#

used to render my entities but no longer does 😦

rotund token
dense storm
#

Why Enable/Disable component data requires Lookup?

rotund token
#

What do you mean by requires?

dense storm
# rotund token What do you mean by requires?

Well, instead of this:

            public ComponentLookup<TeleportData> TeleportLookup;

            void Execute(Entity e, ref TransformAspect transform)
            {
                transform.Position = TeleportLookup[e].Position;
                transform.Rotation = TeleportLookup[e].Rotation;

                TeleportLookup.SetComponentEnabled(e, false);
            }

Why not this:

            void Execute(ref TransformAspect transform, ref TeleportData teleport)
            {
                transform.Position = teleport.Position;
                transform.Rotation = teleport.Rotation;

                teleport.SetComponentEnabled(false);
            }

Way simpler.

rotund token
#

How do you imagine this working

#

Teleport is your struct

#

It doesn't actually hold its enabled state

dense storm
#

Hmm true 🤔

solemn hollow
#

i get different ingame behaviour depending on wether the subscene is closed or opened...
anyone else seeing this?

rotund token
#

things break when they're open

#

physics falls through ground if you select itt

#

disabled components dont bake

#

etc

solemn hollow
#

oh no. makes the debugging experience even worse 😦
weird thing is i had something work when the scene was open and not work while closed xD

rotund token
#

interesting i never have open subscenes

#

what debugging features am i missing out on?

solemn hollow
#

mainly that the entities are named in the hierarchy at runtime if they come from an open subscene

queen dome
#

Hello everyone, How do you add entity dynamically rather than with Bakers at Start?

frosty siren
queen dome
#

The problem for me was to access entity manager from behaviour

#

thank you @frosty siren

raw mica
#

Ah it happened again Here s the

robust scaffold
gusty comet
#

Is it a good idea to implement spatial partitioning by having a ISharedComponentData storing just an int2 representing the cell of a grid with an optimal cell length? This may be a solution to frequent Component/BufferLookup, because in my game, nearly every operation operates on just the Soldier itself, or surrounding Soldiers, and both categories are likely to be within one cell, or the surrounding ones. Meaning that in the average case scenario, all the soldiers are divided between 4 or 9 chunks. This does however require that:

  1. Constantly setting shared component data if any soldier moves between cells, or constantly doing it anyways, if sharedcomponentdata causes chunk moving only if the value itself changes
  2. That a Job could query for a sharedcomponentdata by just supplying it a value like that int2.

I’m already doing spatial hashing where each data includes the entity, but that entity will have a list of “parts” like Brain, Reactor, Stats, Traits, etc that are better off stored in a DynamicBuffer in that entity. Hence, the prerequisite BufferLookup to fetch the part before each operation. For example, a MoraleBuff for all nearby Allies will have to fetch the index of the Trait within the Part returned by the spatial hashing. However, with the SCD, those Allies are highly likely to reside within the same chunk, which mean all those lookups on the list of nearby allies are highly likely to not suffer from cache misses. The implementation will be simple, but is this a generally accepted pattern? I do recall people talking about using SCD for spatial partitioning, like octrees

#

Actually, please ignore point #2, since that’s already done by my spatial partitioning. None of the search logic will change, but some code to sparingly update the SCD need to be inserted into the Spatial Partitioning logic

viral sonnet
#

shared comps create their own chunk and archetype. it's grouping so no this is not a good solution to fragment this much

gusty comet
#

That’s fair. I think I’ll stop asking all these optimization questions and just code the game, profile later

#

But I do want to ask if DOTS is meant to make it so that we generally don’t have to concern with these factors, and instead just follow general guidelines to get good performance

#

Right now I feel like DOTS programmers have to constantly be aware of low level details, which defeats the purpose of an “easier” tool to write performant code

#

I mean, it’s true that you always have to be aware of performance details, but for DOTS there’s so many conflicting factors

#

Like Entity size. Small entities mean fast iteration, but potentially more lookups, while large entities mean slow iteration, but potentially fewer lookups

#

Should we just not worry about entity size then, since it’s too difficult to split hairs over this unless we spend time writing a profile test?

#

And instead focus on which solution is more scalable, extensible, and maintainable?

#

This is my 3rd year of on and off writing DOTS code, and I feel like that experience just burdens me with anxiety over all these performance trade offs

#

At this point, stressing over preventing cache misses seems pointless and instead it’s far easier to just follow Burst and Job guidelines

rustic rain
rustic rain
#

so it'll be easy to find bottlenecks

proper charm
#

Hey anyone have any idea on this error?

unknown.png
#

I have it installed

unknown.png
robust scaffold
proper charm
robust scaffold
proper charm
robust scaffold
#

cool

proper charm
#

What the hell is this. THis is not done I wasted 30 min on this freaking thing

proper charm
robust scaffold
#

Oh my god, it does exist.

unknown.png
rustic rain
#

kek

#

it's just that it was internal

robust scaffold
rustic rain
#

and enjoy unsafe Entities

proper charm
#

Hey can I call other methods inside of void execute in IJob

robust scaffold
robust scaffold
#

You cant schedule jobs inside jobs but otherwise it's just like a thread

#

if you want performance, use burst compile and it's a lot more restrictive on what you can and can not do

proper charm
#

yes burst returned that it can't compile when a method is called.

rustic rain
#

depends on what you call

#

take a look at burst manual

#

to see what is allowed

proper charm
robust scaffold
#

Either figure out a way to use unmanaged components, i.e. structs and pointers, or dont burst compile it.

viral sonnet
proper charm
rustic rain
#

then

robust scaffold
#

This is coding 101. If you have an error, copy paste it exactly so everyone knows what's the problem

#

Dont describe it. If you dont know how to solve an error, you most likely dont know how to interpret the error in the first place.

proper charm
proper charm
robust scaffold
# proper charm

Well then, generate heights has a managed variable within it.

proper charm
# proper charm

the commented part is the whole method which I brought there after getting no soln

robust scaffold
#

It either uses arrays or a class variable.

proper charm
robust scaffold
#

Yep, that doesnt work with burst.

#

Use a NativeArray with a flattened structure. Record the width as an int.

proper charm
#

can you give a list of all managed items. Docs gave etc after arrays.

robust scaffold
#

Basically you can only use basic numbers and structures of them. Along with manually allocated memory.

gentle gyro
# light mason used to render my entities but no longer does 😦

Are you not using the Build Configuration Pipeline to compile DOTS projects?

Note:
If you reply to a post or mention people (i.e. @gentle gyro ) they will get a notification. Otherwise your response might not be noticed
Also, do you have your notifications turned on so that you know when someone responds to you?

proper charm
robust scaffold
#

Do your height generation with an indexer of var x = i % width; var y = i / width;

#

And of course the reverse should be self evident.

proper charm
#

yeah this is good but my question was which would be more performant? array of array or flattening of 2d array?

proper charm
robust scaffold
proper charm
rustic rain
#

unless it's readonly primitive

robust scaffold
robust scaffold
rustic rain
#

or this

robust scaffold
#

not array, I dont think arrays get converted.

rustic rain
#

but hid static method have no out or return so

#

oh wait

#

actually he writes to static field

proper charm
robust scaffold
#

Oh yea, writing to static field definitely no.

#

Write to a native array you have passed into the job struct's properties then write that native array to whatever static properties you have around

proper charm
robust scaffold
viral sonnet
#

or don't use a static variable at all and use a singleton entity instead with an IComp that holds the native container

#

makes RW dependencies a lot smoother too

robust scaffold
#

But yea, not threaded though

rustic rain
robust scaffold
#

But rendering upload cant be threaded so static has no downsides.

rustic rain
#

in OnUpdate

robust scaffold
# rustic rain no burst

The marshalling of data is bursted. Upload is a unity engine internal C++ call so burst wont do anything anyways

rustic rain
#

I don't think so..

viral sonnet
rustic rain
#

Scheduling bursted is probably better

#

with singleton

#

rather than unbursted with static

robust scaffold
proper charm
robust scaffold
proper charm
#

Oooo

robust scaffold
#

It's my pride and joy

unknown.png
#

A solid 600 line 8 pass lighting system

proper charm
#

use
/** */

instead of multiple // for diff lines

robust scaffold
#

HLSL. No multiline comments

#

Oh wait, it does have it. Eh

proper charm
#

Oh

#

LOL I made a discovery

robust scaffold
#

I think GLSL doesnt have the multiline. One of the shader langs dont.

proper charm
robust scaffold
#

DOTS is basically C# shader coding

#

I highly recommend you learn how to write HLSL frag shaders.

#

If you master how to write performant shadercode, you practically mastered DOTS

proper charm
#

I use dots only to replace threading and all. it drives my naive mind crazy.

robust scaffold
#

It's so similar, it has the same math and limitations

#

ah, that also works

proper charm
rustic rain
#

I really suggest to just go over manual on Burst and Jobs

#

after reading them both

#

your knowledge will stonks

proper charm
proper charm
robust scaffold
#

Burst and pointers, it's magical

unknown.png
rustic rain
robust scaffold
#

And this is my best section of burst code I've made.

unknown.png
#

Calculate count from a null terminated stack allocated float array

rustic rain
#

jeez, no one will ever need that unless they write their own engine

#

😅

proper charm
robust scaffold
#

And the burst output of that count() is 4 lines of beautiful hyper optimized code. I cried when I saw it.

robust scaffold
#

Every line or two should have a comment. That's my policy.

unknown.png
gusty comet
#

Insanely good

rotund token
#

Maybe over commented 😅

#

Not sure need you need a comment on var x = 0

#

Does look very pretty though

gusty comet
#

Does Component/BufferLookup generate an entirely new data structure to lookup data from entities, or does it use pointer logic so that your reading directly from the rows of data? If the latter, if those entities are grouped into well-utilized chunks, and the Job with the Lookup tends to read from entities in that single chunk, would ComponentLookup have fewer cache misses?

#

Last premature optimization question, I promise

viral sonnet
#

look at code 🙂 it's a lookup into some very low level entities data containers. some of the lookups are cached and it's really fast

#

so no, it doesn't create any new data

#

it goes like entity -> lookup chunk -> get column of data -> return row

gusty comet
#

That’s reassuring. Sorry for freaking out over the efficiency of Component/BufferLookup, but I’ve already tried structuring my entities so its 1 archetype per group of related operations and 1 archetype per group of iteration operations, and yet there are still instances of ComponentLookup

gusty comet
viral sonnet
#

what kind of mechanic are you making? for some it's just inevitable to have some random access

gusty comet
# viral sonnet what kind of mechanic are you making? for some it's just inevitable to have some...

Groups of features where it’s just a few very small data container entities, and many “logic” entities whose sole purpose is to have an X/Y/ZExecute component enables on it to execute an operation on those data container entities. These operations rarely occur on a per-“Thing” basis (a “Thing” would be groups of data and logic entities representing a functioning Soldier or Landmine or Strongpoint or whatever). However, there will be thousands of Things, so on a per-frame basis, there will be lots of component and buffer lookups

#

Basically, each individual logic entity rarely executes its logic, but there are so many that they add up

#

And I’ve tried thinking about it and asking the other people here, but no matter what, there’s always going to be significant number of lookups per frame. And I might even just be worried about something that won’t be an issue

#

But you said that componentLookups are well designed to not be too taxing, so I should just keep trudging along and instead profile and optimize later?

rustic rain
#

if you make your development 10 times longer just to save 0.001ms - then you are probably doing smth wrong

gusty comet
#

Thanks guys. I needed that wake up call

#

I don’t even think my RTS needs 10,000s of soldiers running around if the player can’t even properly command or attack most of them

#

My current design makes designing features easy, but relies a lot of component lookups.

#

I guess that’s okay now

rustic rain
#

you should probably think way more about extensibility of your code

#

luckily ECS is very friendly towards it by default

gusty comet
#

Yeah. I wrote a Stat system that generalizes all arithmetic operations into easy to configure authorings. Instead of writing separate code for Health (Damage, Heal, Invincible) and Morale (Scare, Inspire, Demoralize), even though they’re both just floats with add or subtract operations, I made it so that you can configure that with a Stat entity and an Operation entity, and using Scriptable Objects as enum to differentiate between every possible Stat. Then, every other behavior does a lookup on a referenced Stat. This way, not only can other entities like Run or Retreat read from a generalized Stat, but you can design features like SlowDownRun and HastenRetreat just by doing an Operation on the Stat that those two read from. That’s better than writing an entirely new authoring and system for SlowDownRun and HastenRetreat. It’s definitely less performant and less parallelizable, but also much quicker to development and keeps my motivation up because I’m not writing hundreds of lines of code just to essentially duplicate existing logic

#

A related question would be: is a core tenant of ECS not the ability to squeeze out every last drop of performance, but the ability to write extensible, maintainable, understandable code that is still very performant?

rustic rain
gusty comet
#

yeah

#

a freaking BBQ channel keeps being sponsored by it

rustic rain
#

I heard that devs used ECS approach just to make it easy to maintain and add features to it

gusty comet
#

Interesting

balmy thistle
gusty comet
#

Well, I should do the same then. Burst and the Job system already give us some leeway on performance. That gives us some wiggle room to invest in maintainability and extensibility. Even if the entity design potentially results in cache misses

rustic rain
#

that's what I do

#

my main goal - make code as moddable as possible. I want to make game that is really easy to mod

gusty comet
gusty comet
#

I like to imagine that the end user of the codebase is someone who's used to Unreal Blueprints or Scratch. That minimizes coding time for a features and maximizes productivity

rustic rain
gusty comet
#

Unfortunately that means a more OOP-like design where there are objects like DecisionMaker, Stat, Trait, etc with "methods" like DecisionMakerRestart, StatSubtract, etc

#

but if it helps productivity, then so be it

rustic rain
#

so in the end modders just drop built assembly + asset bundles into mod folder

gusty comet
#

Is that accomplished through Unity-related functions, or are you doing fancy stuff?

rustic rain
#

Unity mostly

#

loading assemblies is native C# thingy

gusty comet
#

So modders would "inject" their own logic?

rustic rain
#

goal is to be able to let modders load their GOs and SubScenes

#

yeah

gusty comet
#

Ah

rustic rain
#

their own systems, code

#

components, everything

gusty comet
#

Hmm, my current goal is to just let potential modders play dressup with Soldier armor and try out different AI combinations. I'm planning a simple script to enable/disable AI-related GOs to limit a subset of behavior entities, and another to determine which subset of randomized equipment the Soldiers in a Unit would have

#

That was why I was obsessed yesterday over trying to gracefully delete GameObjects whose sole purposes are to act as folders

#

But Subscenes convert those too

#

And I'm not sure if using Subscenes as folders has any issues

viral sonnet
#

if you are in the 1k to 10k range don't think abou all this too hard. burst fixes most code issues for that entity range. it's only in the 100k+ range where you need to really think hard about architecture

#

of course lower frametimes are always better but without hands on experience this is too much of a guessing game

rotund token
#

most people are still main thread capped from projects i've looked at

#

and have quite a bit of idle time in their threads

#

so optimizing for more idle time really doesn't benefit you except to save a little power on mobile devices

#

hopefully ISystem is helping reduce this though

robust scaffold
#

These code gen'ed systems have really multiplied my burst compilation duration. Normally of a project my size back during 0.17 had burst completing within seconds. 30 seconds at most. Now, it's going for 3 or 4 minutes.

rotund token
#

burst 1.8?

#

it's interesting because you were probably using burst 1.5/1.6 back then

#

and burst 1.7/1.8 should be quite a bit faster

robust scaffold
#

yea, 1.8.1

#

It's chugging. Then again, I'm burst compiling 10x more things

rotund token
#

but there are a lot more burst things to compile now with ISystem etc

robust scaffold
#

Everything has a burst compile tag these days

rotund token
#

do you have everything in 1 assembly?

robust scaffold
#

Which is fantastic and a big step forward but now i'm really seeing the complaints about compile times

robust scaffold
rotund token
#

yeah i don't think you're getting the benefits of the 1.7+ burst caching

#

probably have to recompile everything on any change

robust scaffold
#

It's not much code anyways. Only a few systems at most. A job for each system pretty much

unknown.png
rotund token
#

thats taking 4 min?

robust scaffold
#

On build yea

rotund token
#

hmm that seems slow

#

turn on burst logging

#

i'm interested what part is taking a while

#

that thing

gusty comet
# rotund token hopefully ISystem is helping reduce this though

Are you saying that scheduling too many jobs will result in the main thread sitting around while waiting worker threads to finish? But wouldn’t using Isystem result in main thread sync points if that isystem writes to certain components? I’m still wondering if ECS runs main-thread systems in a group after all jobified systems in order to prevent this

rotund token
# rotund token

will give you full timings of burst compiling like this

 - All                                      :    1143ms
   - Discover Methods to Compile            :    5039ms
     - Find Entrypoints                     :    4639ms
     - Group Entrypoints                    :      11ms
     - Calculate Hash of C# IL              :     387ms
   - C# IL -> LLVM IR module                :     122ms
     - Parse C# IL from assemblies          :      39ms
     - Transform C# IL                      :      19ms
     - Produce LLVM IR module               :      64ms
   - LLVM IR module transformations         :      93ms
     - Add target CPU info to module        :       0ms
     - Optimizations on module              :      90ms
     - Cleanup module                       :       0ms
     - Verify module is valid               :       3ms
   - Extract used static readonly variables :       0ms
   - LLVM IR module -> DLL                  :     318ms
     - Splice into per-entry-point modules  :      11ms
     - LLVM IR modules -> object files      :     134ms
     - Get objects from object cache        :       0ms
     - Link object files -> DLL             :     303ms
     - Add objects to object cache          :       3ms
   - Dump                                   :       0ms

Counters:
- Number of Functions                      :        54
  - Entry Points (job or function pointer) :         2
  - Declaring Type or Function is Generic  :        30
- Number of Instructions                   :      1764```
robust scaffold
rotund token
#

it's just logged in console

#

it's also great for comparing burst versions

#

most people don't realize the differences

#

but if you run say burst 1.6 with timings

#

grab results

#

then run burst 1.8

#

you can see difference for better/worse

viral sonnet
#

i don't have a log for that

rotund token
#

because their jobs finish much faster than their main thread work

#

so the worker threads are idle while main thread is doing things

#

effectively nullifying any optimizations you make to the jobs

#

i can say for example, my work project which has a lot of jobs

#

is still 50-70% idle time on my pc (though that is a 3900x with 12/24 cores)

#

i could make all our jobs run twice as slow and it would not affect performance

#

in fact we have tested this by doubling NPCs in the world

#

we can significantly scale creatures/player counts without fps performance changing at all

#

(i have a setting to increase counts by 10x for an extreme fun/silly mode)

robust scaffold
#

still chugging, hrm

unknown.png
winter depot
#

Anyone ran into something like this? A component with type:Unity.Transforms.LocalToWorldTransform has not been added to the entity. Entities Journaling may be able to help determine more information Also enabling journaling and recording gives the same exact message, it just cuts off the part where it says to enable it.

robust scaffold
rotund token
#

have you opened the journalling window

#

to see what removed it?

winter depot
#

No physics/netcode

#

I think you have to open journaling window to hit the record button, but the list is massive I am not sure where to even look

gusty comet
rotund token
#

of course

#

in the end all you care about is main thread performance

#

that's your FPS

#

now if your jobs are running slow and your main thread has to wait on them

#

then your jobs are affecting main thread performance

#

therefore you need to optimize your jobs to improve your fps

#

but if your main thread is not waiting on your jobs optimizing your jobs gives you little benefit

#

and you need to focus on that instead

robust scaffold
gusty comet
#

That’s a good insight

rotund token
robust scaffold
#

Oh, okay, i'll recompile

#

Huh, Unity.Logging.Log is extremely chunky:

unknown.png
#

Oh well, it's some netcode stuff. Not my problem.

winter depot
#

Maybe I am going about this all wrong, should I just be using LocalToWorld and manually setting a pos/scale/rot instead of using LocalToWorldTransform?

rotund token
#

Honestly not a lot of people have used new transform system

#

Because it's not compatible with physics / netcode atm

#

But from what I understand no, you should be using the other component

robust scaffold
#

IIRC the new transform aspect updates both the L2WT and the L2W matrix in the same property { set; } operation.

rotund token
#

UniformScaleTransform

winter depot
#

I wasn't getting anything rendering, so I was trying to sort out why that might be. Looking at their example I see this

unknown.png
#

Where they use that instead of LocalToWorldTransform

robust scaffold
#

hrm, doesnt look like any TRS I know of...

winter depot
#

I am thinking for my issue, could it be that I am missing a scale property somewhere?

#

Because this error doesnt appear to be coming from any Job of mine, it seems internal

robust scaffold
#

I dont know what's doing. Where is this system located?

winter depot
#

Library/PackageCache/com.unity.entities@1.0.0-exp.8/Unity.Transforms/TransformHierarchySystem.cs:76)

#

From what I can gather from the error

#

I assume its something with a nested entity transform

robust scaffold
#

Did you add an entity to LinkedEntityGroup manually?

#

Or in this case, Child buffer

winter depot
#

I will take a look through, but I think I used to but removed all of them

robust scaffold
#

Just control shift f search <Child>

winter depot
#

Nothing <Child>, I do think I have a linked somewhere on a prefab so that when its instantiated it carries over children

robust scaffold
#

if it's a GO prefab, that shouldn't be a problem as transforms are automatically converted and added to the entity

winter depot
#

Nah, this is a pure ECS project

robust scaffold
#

Ah, no clue then

#

Might as well copy paste this other cache breaking error. Hopefully this just isn't me.

unknown.png
#

@rotund token Sorry to ping ya but have you seen this error?

rotund token
#

I don't believe so

#

Had plenty of sub scene entity headers breaking

#

But I don't believe I've seen that cast exception

robust scaffold
#

Might be from the sprite renderer companion GO frankly.

#

It's the most iffy thing I have converted (i stopped converting the tilemap when I moved to physics)

#

Sprite render companions also break the build so maybe

viral sonnet
#

did you manage to see the burst timing log?

robust scaffold
viral sonnet
#

kind of expected that on full recompile but don't they have an asmdef which should prevent a full recompile?

robust scaffold
#

Dont really know what's going on. May be some things happening with code gen

#

Code gen may be under NetCode assembly and changing the src structs may invalidate the whole netcode assembly.

#

I have no clue TBF but it just takes a long time every time I finish editing a section of multiplayer code

viral sonnet
#

frustrating. 3-4 minutes is quite absurd

robust scaffold
#

3-4 min at build. In editor it's about 1-2 min

viral sonnet
#

oh, well that's still quite long. damn

robust scaffold
#

And I have a strange feeling that the entity header corruption and editor freezes seems to occur whenever I press play before burst finishes compiling. It's just something I've noticed these past few editor task kill and restarts.

viral sonnet
#

do you have synchronous compilation enabled?

robust scaffold
#

No

#

Actually, maybe it might help?

viral sonnet
#

turn that on!

robust scaffold
#

Well, lets see if it crashes...

#

Well, reloading domain now takes 15 seconds but there's no burst compilation on the bottom

#

And editor stalled on scene switching. Didnt fix

viral sonnet
#

😦

#

just wanted to give this a try to have a working test for it and i found out BakingSystem has no SceneGUID. is that some extension of yours?

rotund token
#

yeah it's internal

viral sonnet
#

of course! 😄 my authoring namespace has no internal access. well damn, so this isn't even possible out of the box.

rotund token
# 
    {
        public static Hash128 SceneGUID(this BakingSystem bakingSystem)
        {
            return bakingSystem.BakingSettings.SceneGUID;
        }
    }```
rotund token
#

i just get the sceneguid off an existing entity for my use case

#

which is basically generating meta data for all my saving prefabs

# 
            for (var index = 0; index < length; index++)
            {
                var sceneSection = (SceneSection)sections[index];
                var recordEntity = state.EntityManager.CreateEntity();
                state.EntityManager.AddSharedComponent(recordEntity, sceneSection);
                state.EntityManager.AddComponent<Savable>(recordEntity);

                var recordBuffer = state.EntityManager.AddBuffer<SubSceneRecord>(recordEntity);

                var values = records.GetValuesForKey(sceneSection);
                while (values.MoveNext())
                {
                    recordBuffer.Add(new SubSceneRecord { Savable = values.Current });
                }
            }```
viral sonnet
#

from an entity? how do you do it?

rotund token
# 
records.Add(sceneSection, savableScene);```
#

i just get the shared component

robust scaffold
rotund token
#

and populate my hash maps

viral sonnet
#

nice, thanks a lot

#

cool, got that working now ```foreach (var (_, entity) in SystemAPI.Query<Add_EntityTest>().WithEntityQueryOptions(EntityQueryOptions.IncludePrefab | EntityQueryOptions.IncludeDisabledEntities).WithEntityAccess())
{
var sceneSection = state.EntityManager.GetSharedComponent<SceneSection>(entity);

            UnityEngine.Debug.Log(sceneSection.SceneGUID);
            
            var newEnt = ecb.CreateEntity();
            ecb.SetName(newEnt, "testEnt");
            ecb.AddComponent(newEnt, new FixedStructTest.FixedStruct());
            ecb.AddSharedComponent(newEnt, new SceneSection { SceneGUID = sceneSection.SceneGUID, Section = 0 });
        }```
#

no idea what to use it for but might be helpful someday 😄

rotund token
#

😄

viral sonnet
#

this is funny

unknown.png
#

thrown because of JournalAddRecordGetComponentDataRW, but still 😄

rotund token
#

yeah i have these highlights turned off these days

#

i feel like with a bit of experience they aren't that useful

#

can produce some funny moments though

robust scaffold
viral sonnet
#

yeah, it's rare. Debug.Log is once of those methods it calls out for being expensive

robust scaffold
#

ugh entity hierarchy tab seems to be really buggy with netcode entities

viral sonnet
#

so this is the boilerplate for enabled comps in IJobChunk? ```public void Execute(in ArchetypeChunk chunk, int unfilteredChunkIndex, bool useEnabledMask, in v128 chunkEnabledMask)
{
var enumerator = new ChunkEntityEnumerator(useEnabledMask, chunkEnabledMask, chunk.Count);

            while(enumerator.NextEntityIndex(out int i))
            {
            }
        }```
robust scaffold
viral sonnet
#

ok. yeah i learned that from you 😄 and a chunk is skipped if everything is disabled, right?

robust scaffold
#

Yep, if all components within it are disabled

#

if there is a single enabled component, then the chunk gets passed in

#

If you want specific enabled bit masks, i have the extensions I threw together and seems to work fine.

#

Or use the enabled mask also provided by unity but doesnt work with the CEE.

viral sonnet
#

hm, gotta be careful on that. right now i just need it to replace any IsAlive calls

robust scaffold
#

If you just need per entity enabled status, use the built in enabled bit mask struct. Works fine for bool checks.

viral sonnet
#

the built in being CEE?

rotund token
#

imo much easier to just stick with IJE if you're using enabled bits

robust scaffold
#

No, there's a bool mask thing

viral sonnet
rotund token
#

you wrote the job, you can delete that line of code 😄

robust scaffold
#

You will tear my boilerplate from my cold dead hands.

unknown.png
rotund token
#

but what's the advantage with your IJobChunk

#

i see no chunk level operations in the job

robust scaffold
#

And the ability to cut down on the .Value wrappers by direct reinterpreting chunk arrays to their true values is worth all the pain IMO.

rotund token
#

oh i guess ispredicted?

robust scaffold
#

Pretty much

viral sonnet
#

but wow, this merged enabled bits thingy seems like a hardcore pitfall

robust scaffold
#

heh, it's great for a single enabled comp in the query. Beyond that, why they didnt provide a per component mask access is beyond me

#

if I can code it, unity definitely can. Laziness i think

rotund token
#

@viral sonnet have you looked at how IJobEntity handles this?

robust scaffold
#

IJE is just CEE and boilerplate.

viral sonnet
rotund token
#

👍