archived-dots

[rotund token]

its an Object not even a MonoBehaviour

it just does native calls

im pretty sure i could detatch it and make it a component/system

[viral sonnet]

maybe that already works, I just re-read and this was about VFX instancing

[rotund token]

i just dont want to put the effort in for something that will eventually happen as i just dont have the time atm

[viral sonnet]

VFX instancing is currently being actively developped. However target version is 2022 not 2021 as I stated in my previous message

[rotund token]

way too many other things i want to tackle first

[viral sonnet]

same, it's like graphics? ha, what do I care

wake me up when we have nanite

[rotund token]

DreamingImLatios

I'm working on a non-hybrid animation solution which I hope to finish in the next month or two. If you are interested in helping test pre-release versions, feel free to reach out to me!

i really dont like relying on others libraries

but at this stage i'll take anything for animation that a) i dont have to write myself b) doesn't have to use hybrid

[viral sonnet]

eh, you are putting salt on that wound

If I ever want to release my asset I basically have to use hybrid. I can't, in good faith make it rely on Dreamings solution even when it's really solid

[rotund token]

oh your asset needs animation?

i totally agree though

main reason i dont like using others libraries

[viral sonnet]

well, not really but I have to make the bindings, otherwise it would be near useless for other people

same goes for sound effects

[rotund token]

all the actual look good parts that dont work in dots 😦

[viral sonnet]

and it's kind of ironic to have something very scalable and then bog it down with gameobjects 😄

[rotund token]

who needs them

[viral sonnet]

that's also the part that makes people so reluctant to switch to dots. I mean, look at the forum. It's pretty much the same guys over years now. Sure, only a fraction writes in the forum but it's very representative how much people actually develop in dots. As non coder I'd stay as far away as possible.

[rotund token]

look at this channel

it's just me and you 😄

[viral sonnet]

yeah, also mostly the same people. I mean, I like the familarity though hehe

I need to ask PhilSA how the difference is between his assets. He's one of the few who has the same asset for MB and ECS.

Must be very represenative

GetBufferAccessor is doing alot. Makes sense that it slows down a lot with more chunks. What's interesting is that CDFE does caching for the indices that are required.

same goes for ComponentTypeHandle

BufferTypeHandle is missing it

[rotund token]

are you looking at 0.50?

because they added this caching thing in for ComponentTypeHandle

you dont have to get it per frame anymore

you can now create it in OnCreate

and just call Update() on it every frame

but BufferTypeHandle hasn't got the same treatment yet (coming soon~)

[viral sonnet]

ohhh, I have to rewrite some stuff then

[rotund token]

i think they wan tall of these

XTypeHandle, XFromEntity etc

to be only created once in OnCreate, cached

then updated/frame with only required data

[viral sonnet]

In this case I mean specifically GetComponentDataRO

[rotund token]

instead of rebuilding it all

[viral sonnet]

you mean GetComponentTypeHandle called from MT right?

[rotund token]

yeah

sorry went a bit off topic

it's not related to what you're looking at

just something that popped into my mind when you were talking about differences

[viral sonnet]

np, still helpful. I still acquire them per-frame

but yeah, I meant the calls inside the job to get the accessor

[white island]

Can someone help me with this?

oops

i didnt realize people were still talking

[viral sonnet]

the BufferTypeHandle could easily be rewritten with this lookupCache

@white island np 🙂

I read it but I also don't know what's going on, sorry 😦

[rotund token]

nodes.IndexOf(start) is the problem

[white island]

yes

[rotund token]

you're using the wrong overload

you're using

    public unsafe static partial class FixedStringMethods
        public static int IndexOf<T>(ref this T fs, byte* bytes, int bytesLen)```

instead of

    public unsafe static class NativeArrayExtensions
        public static int IndexOf<T, U>(this NativeArray<T> array, U value) where T : struct, IEquatable<U>```

basically add IEquatable<Node> to your struct

you need to implement that to use IndexOf

[white island]

oh, okay

i would not have figured that out, thank you

[viral sonnet]

tertle 4tw 😄

[white island]

which namespace is IEquatable in?

[rotund token]

System from memory

[rotund token]

i'm annoyingly good at debugging issues quickly

just means i get assigned all the painful issues to track down

[white island]

ah, yep, thanks

[rotund token]

i have this theory that the more code you do for yourself/own projects, the better you become at debugging

developers who only code at work have the crutch of being able to fall back onto other devs for help when tracking down issues

whereas if you have a problem in your own library, you don't have much choice but to spend hours/days debugging it

a theory with no evidence, but my theory

could also just be my sample size is 7

[viral sonnet]

I can relate to that 😄 devs who code as a hobby have seen so much more than those who only code at work. the experience one can gain can not be compared

[rotund token]

there are obviously huge benefits of coding in a team that can't be overlooked

knowledge sharing etc

[white island]

people who only code at work have One specific job but with hobbyist, you're the architect, the systems programmer, the UX designer, the QA guy, the

[viral sonnet]

the one who brings you coffee

[white island]

and the one that assigns deadlines

[chilly crow]

and the one who criticizes all your work

[rotund token]

im all 3 of those 😄

nothing like some self criticism as i sip a coffee

which i am literally doing atm thinking how i've achieved nothing in past 4 hours except argue on discord

[white island]

[solves bug] i am a god among men. just hand me the crown and grovel at my feet
[solving the bug introduces 5 more bugs like a hydra] i will never amount to anything. i am a blight on my family name

[viral sonnet]

Ok, I'm pretty convinced now that Entities can be sped up a lot with utilizing the same cache mechanics that are used in CDFE and BufferFromEntity

[viral sonnet]

Does my math fail me? Archetype window shows 48 bytes for this buffer. I calculate 13 bytes * 2 = 26. ResourceType enum is a byte

buffer cap 0 and it's 16 bytes

[white island]

ok i'm almost there with implementing an A* algorithm. my lists are indexes of a list containing my graph, but how do I store the nodes I'm currently working on, what with the parents and such

I've gotten this far:

public struct Node : IEquatable<Node>
    {
        public FixedString64Bytes id;
        public FixedString64Bytes cellWorld;
        public float3 coords;
        //TODO: Whitelisted covens / keys

        public bool Equals(Node other)
        {
            return (
                (other.id == this.id) ||
                (
                    (this.cellWorld == other.cellWorld) &&
                    (this.coords.Equals(other.coords))
                )
            );
        }


    }

    struct Connection
    {
        public ushort start; //index of start
        public ushort end; //index of end
        public byte cost; //cost of traversal
        //TODO: Keys on keyring
    }

static NativeArray<Node> nodes; //Nodes in graph
static NativeArray<Connection> connections; //Connections
public static NativeArray<Node> pathFindThroughGraph(Node start, Node end)
        {
            //Queues
            NativeList<ushort> openList = new NativeList<ushort>(nodes.Length, Allocator.Temp); //indexes of open nodes
            NativeList<ushort> closeList = new NativeList<ushort>(nodes.Length, Allocator.Temp); //indexes of closed nodes
            openList.Add(((ushort)nodes.IndexOf(start))); //add first node
            //AStar
            while (!openList.IsEmpty)
            {

            }

            //Dispose
            openList.Dispose();
            closeList.Dispose();

            return new NativeArray<Node>(0, Allocator.Temp);
        }

how do I store my Fs and Gs and Hs

[rotund token]

firstly it's 16 bytes per element not 13 because of padding (4x4 bytes)

and there's 16 bytes overhead per buffer (the cost of the buffer if you use buffercapacity(0))
8 bytes pointer
4 bytes length
4 bytes capacity

16x2 + 16 = 48

ResourceType enum is a byte
still takes 4 bytes

[white island]

also is there a more efficient way to do this

[rotund token]

not use FixedString64Bytes which is huge for an id

[white island]

well yeah im working on that

[rotund token]

public FixedString64Bytes id;
public FixedString64Bytes cellWorld;

128 bytes for something that could be done in 4-8

[white island]

the cell world has to be a string for compatibility reasons but I guess it could be 32

and im not using id

[rotund token]

😬

[white island]

ok. so. the A*. How should I store g, h, f, and parents

just a struct?

[rotund token]

h is usually calculated right

as for g,h if you're using a grid a native array makes sense

and parents the same (assume thats the camefrom map)

[white island]

i never ereally understood the camefrom map? is that the path I'm working on?

It's not really a grid so

[rotund token]

gh are just floats usually right

[white island]

yes

f can be calculated

g can be an int

h is a float because it's distance

[rotund token]

cameFrom is used to reconstruct path isn't it

(it's been a couple of years since i've look at this)

[white island]

i never really understood that part, was it just formatting a path for use

[rotund token]

it's simply points each node to the next node

at end of a* all you end up with is current == goal

you got to rebuild the path backwards using the map

[white island]

ok so what i've come up with so far is:

struct CameFromPath
  ushort correspondingNodeIndex
  int g
  float h

struct Connection
  ushort start
  ushort end
  ushort cost

openlist: list<ushort> = start
closedlist: list<ushort>
cameFrom: list<CameFromPath>


while openlist is not empty:
  //TODO: Figure out how to get lowest F from openList

  workingNode: ushort = openlist.pop
  closedlist.push(workingNode)
  foreach(c:Connection in getConnections(workingNode))
    if(c.isTraversable())
      openlist.add(c.end)
      cameFrom.Add(new CameFromPath(workingNode, c.cost, calculateHeuristics()))

//TODO: Figure out how to get shortest path, store multiple paths, then rebuild and return

by multiple paths i mean work on multiple things at once

not entuirely sure how to do that yet

i have like 10% of an idea of what im doing by the way so yeah

[worthy rampart]

I 100% agree

[pulsar jay]

I recently encountered that accessing components from MonoBehaviours can lead to dependency errors. Is there any way to prevent this as MBs cannot declare dependencies or can they?

[rotund token]

i've been advocating for a while never to have monobehaviours access entity world

just write to monobehaviours from systems if you need to get data on them

otherwise it's a huge nightmare

[pulsar jay]

Till last week I was sure that reading and writing entities from Main thread was totally safe and fine. But now (may be connected to the 0.50) update it started causing dependency errors

No it will be hard to move all the MB code to systems 😬

[rotund token]

entity manager calls should be safe

entities 0.50 has improved safety i think. picks up a few missed things from 0.17

but yeah em calls will always sync to ensure safety

@viral sonnet you might be interested to see the updated test of yours
https://forum.unity.com/threads/bufferfromentity-seems-significantly-slower-than-multihashmap.1272245/#post-8075747
tldr: if you mix the order of adding to the hashmap it becomes significantly slower.

[pulsar jay]

This does not seem to be safe though:

        var query = EntityManager.CreateEntityQuery(
            ComponentType.ReadOnly<CellPosition>(),
            ComponentType.ReadOnly<CellHeight>(),
            ComponentType.ReadOnly<MapComponent>()
        );
        query.SetSharedComponentFilter(map); //Filter by map

        var entities = query.ToEntityArray(Allocator.TempJob);
        var positions = query.ToComponentDataArray<CellPosition>(Allocator.Temp);
        var heights = query.ToComponentDataArray<CellHeight>(Allocator.Temp);

        var position = positions.Select((cellPos, index)
            => new {cellPos, index}).FirstOrDefault(p => p.cellPos.Value.Equals(cell));

        float height = position == null ? 0f : heights[position.index].Value;
        entities.Dispose();
        positions.Dispose();
        heights.Dispose();

This worked fine before but now it gets a dependency error with a system that writes to CellPosition

[rotund token]

Call query complete dependency

The issue is actually just the to component data array

This would error in systems as well

Regression in 0.50

[pulsar jay]

Oh so ToComponentDataArray fails to properly handle dependecies now?

[rotund token]

    I'd recommend you use query.ToComponentDataArrayAsync instead

#9 on my compilation of issues with 0.50

[pulsar jay]

Oh nice. Thx for the info. That was really giving me a headache last week 😬

[viral sonnet]

@rotund token nice work! you found the fishy part in the test

now I wonder why the DB has less trouble with random writes than the NMHM. Was the test still with cap 0?

must be the overhead of iterating through the keys in the NMHM

when the job overhead of DBs could be optimised we would look at a very fast solution

[misty wedge]

So DBs are as fast as a native arrays as long as you use ToNativeArray when reading / writing them, and the actual slowdown was from iterating over the chunks?

[rotund token]

AsNativeArray

and only if you're comparing the native array being on the entity

if you were iterating a native array from a system the array going to be faster because well no entity overhead

[misty wedge]

Yeah, my question pertains more to architecture, e.g. storing the native array on a pointer in the components vs using a DB

Is your DynamicHashMap / DynamicMultiHashMap comparable performance wise to a component with a pointer to a NHM / NMHM?

I've never looked at how exactly they are implemented under the hood

[viral sonnet]

most of the overhead is from the chunk iteration and GetBufferAccessor. the bigger the archetype the more iterations, the more GetBufferAccessor

[rotund token]

it should be a tad faster actually

it has all the thread buffers removed etc

so doesn't need to perform as many checks and i could optimize a few things

[misty wedge]

Interesting, I might just go with storing my grids etc as DBs after all. Even if they were slightly slower I would prefer that to making mistakes with pointers, and stuff like automatic cleanup on entity destruction, etc

[rotund token]

just go with whatever you find the easiest that you think will meet your requirements

and design it in a way it's modular enough to switch out in the future if you need

[misty wedge]

I wish that was so easy for me to say 😅

[rotund token]

if you need to optimize it or take it to some extreme

honestly i think that's the most important thing

[viral sonnet]

DBs might really be the go to for any arrays on entities. Man, that would be something. And again pointing that out, GetBufferAccessor and BufferTypeHandle has no caching at all. ChunkDataUtility.GetIndexInTypeArray has to iterate over the types every time. In ComponentTypeHandle the correct index is cached in a short.

[misty wedge]

What's the fastest way to construct a DB from dynamic data? For example if I am running some algorithm and am adding stuff to a native list, would I just use CopyFrom or something else?

[rotund token]

the 1 major performance gotcha of dynamic buffers is Add()

Add is much slower than Add() on a nativelist

so ideally add as much data in 1 as possible

this isn't too hard if you know how much data you need

at the very least, the max you'll need

[misty wedge]

In 1? Using AddRange or something else?

[rotund token]

yeah

or just resizeuninitialized

asarray

write like a native array

dynamicbuffer actually has a method to regain memory

TrimExcess()

so you can over allocate it without too much worry
then trim it if you did it badly

[misty wedge]

Does resizeUnitiailized drop the old memory or does it use the same location?

[rotund token]

depends if it needs new capacity

any new capacity will need to move it

[misty wedge]

Makes sense

But If I have old code that uses e.g. a native list at the moment, I could just use AddRange and then dispose the list right?

[rotund token]

yeah

[misty wedge]

Awesome, thanks.

I wish this was actually what I was doing for work, would give me much more opportunity to really dig down deep

Out of curiosity, if you don't mind answering, what made your employer want to switch to ECS even while it wasn't close to being finished? Performance requirements?

[rotund token]

i was hired after they had already switched

i guess you could say they were trying to be forward thinking and ran into some performance issues with their existing architecture and wanted to refactor large parts to add multiplayer (to the best of my knowledge, again i wasn't working there at the time so this is all second hand from piece)

unfortunately switching to a new paradigm without any experience can be painful

[misty wedge]

Tell me about it. At least for me it is only a hobby project, so no real stake

[rotund token]

their pain, my gain 😄

[misty wedge]

Must be rough, especially since ECS has changed so much since then

I assume this was back in the [Injection] (was that what it was called?) days

[rotund token]

i dont think that was still around

[misty wedge]

Really looking forward to what 1.0 brings, 0.50 already had a lot of cool stuff you mentioned I didn't even know was in

Also you mentioned yesterday that I need access to some unity internal stuff (for the DynamicHashMap). So just make a local copy of the package and changing encapsulation types? Or is there a better way to do it

[rotund token]

nah much better way to do it, you can just give any library internal access without modifying the package

#archived-dots message

detailed it here

heading to bed now

[misty wedge]

Awesome, thanks!

[viral sonnet]

good night tertle

[haughty rampart]

i gotta say i really like the new build configuration asset, but there's one issue.
why can you only define Major and Minor version? why is Fix missing? i hope that gets added at some point

[viral sonnet]

Exception: IL2CPP error for method 'System.Boolean BovineLabs.Core.Extensions.NativeMultiHashMapExtensions::TryGetNextRefValue(Unity.Collections.LowLevel.Unsafe.UnsafeMultiHashMap2<TKey,TValue>,System.Byte*&,Unity.Collections.NativeMultiHashMapIterator1<TKey>&)' in Plugins\BovineLabs\NativeMultiHashMapExtensions.cs:380
System.InvalidOperationException: Attempting to constrain a value of type 'System.Byte' to type 'Unity.Entities.Entity'.

Oh boy, any ideas?

Entity is used for TKey but there's no byte in the code. Only a byte*

I don't even know what that means and google tells me pretty much this is an il2cpp error that can only be fixed by unity

            this UnsafeMultiHashMap<TKey, TValue> m_MultiHashMapData,
            ref byte* item,
            ref NativeMultiHashMapIterator<TKey> it)
            where TKey : unmanaged, IEquatable<TKey>
            where TValue : unmanaged
        {
            //CheckRead();
            int entryIdx = it.NextEntryIndex;
            it.NextEntryIndex = -1;
            it.EntryIndex = -1;
            item = default;

            var data = m_MultiHashMapData.m_Buffer;
            if (entryIdx < 0 || entryIdx >= data->keyCapacity)
            {
                return false;
            }

            int* nextPtrs = (int*)data->next;
            while (!(*(TKey*) (data->keys + entryIdx * sizeof(TKey))).Equals(it.key))
            {
                entryIdx = nextPtrs[entryIdx];
                if (entryIdx < 0 || entryIdx >= data->keyCapacity)
                {
                    return false;
                }
            }

            it.NextEntryIndex = nextPtrs[entryIdx];
            it.EntryIndex = entryIdx;

            // Read the value
            item = data->values + entryIdx * sizeof(TValue);

            return true;
        }```

[pine plaza]

hi everyone, i have a trouble and dont know how to resolve it. in Gameobject we can get eulerangle easylly, but in Dots i dont find any function to do the same, does anyone know? thanks

[haughty rampart]

that 'functionality' is not available and will never be available because you generally do not need it.
what would you need it for?

[pine plaza]

i need it for movethis code to ecs code

[haughty rampart]

then just manually keep track of transform.eulerangles.y of the entity.
super easy

[pine plaza]

there is no GO in my code, it is pure ecs

i need it because i send camera's quaternion from client to server to handle logic

[haughty rampart]

exactly. that's why you need to track it yourself

[pine plaza]

i can send the tranform.eulerangle.y, but i wanna understand how to resolve this in pure ecs

[haughty rampart]

create a component
add float yAngle
whenever you change the angle of the entity on the y-axis, update yAngle
use yAngle in your code instead of transform.eulerangles.y

[pine plaza]

so basically there is no way how to resolve this with pure ecs, quaternion and math

[haughty rampart]

wdym? that is the pure ecs way

[pine plaza]

i mean in ecs world we cant get eulerangle by tranforms

[haughty rampart]

ecs has no transforms so of course that is not possible

you could of course get back the euler angles from quaternion, but you can only get normalized euler angles back from that, and it's a relatively hefty math operation.

[pine plaza]

how i cannt find that function

[haughty rampart]

of course it's not an integrated method.
but if you really need it you can just google to find an algorithm for that

[pine plaza]

thanks u so much!

[dense crypt]

There's a source code example on wikipedia https://en.wikipedia.org/wiki/Conversion_between_quaternions_and_Euler_angles#Quaternion_to_Euler_angles_conversion

[misty wedge]

If I create a hashmap and get the pointer to it with UnsafeUtility.AddressOf, how do I convert it back?

I'm assuming UnsafeUtility.As but I'm not sure what the correct syntax is

[junior fjord]

Hi, for a new 2D Dots project, how do I set it up? Do I use the hybrid renderer or is that only 3d?

[misty wedge]

You can have it render quads, since that's basically already what the normal spriterenderer is doing (unless you use the non-full rect sprite settings)

[viral sonnet]

UnsafeUtility.AsRef works

you can also cast just with (T)

[misty wedge]

Yeah, I saw that. e.g.
ref var hashmap = ref UnsafeUtility.AsRef<NativeHashMap<int, int>>(p.Ptr);
That leaves me wondering why *(T*) works, but not *(NativeHashMap<int, int>*)

[viral sonnet]

oh that doesn't format -.-

[junior fjord]

Ok, so I would setup a 3D project and just render everyting into one plane using the hybrid renderer? But I probably loose a lot of convenience compared to just using a normal 2D setup right?
Alternatively I could have the positions in DOTS and just copy them over to normal unity GO every frame? Or is that very imperformant?

[viral sonnet]

huh, valid question. It doesn't? Not sure why

[misty wedge]

It's really up to you. I use normal gameobjects linked to the entities and only use gameobject stuff for rendering

[junior fjord]

Can I just link gameobjects to entities nowadays (I used DOTS/Unity the last time when it still was at version 0.08 or something)

why do I need a hybrid renderer then?

that sounds perfect

[haughty rampart]

@rotund token i tried testing your fix. i don't build a lot of my projects so i couldn't figure this out before.
each time i try to build it throws an error: Library\PackageCache\com.unity.collections@1.2.3\Unity.Collections\NativeList.cs(839,24): error CS7036: There is no argument given that corresponds to the required formal parameter 'safety' of 'NativeArray<T>.ReadOnly.ReadOnly(void*, int, ref AtomicSafetyHandle)' did this happen to you as well?

[misty wedge]

Hybrid renderer is an ECS component based renderer. It doesn't use any game objects at all (and is allegedly faster, but I never bothered with it)

[haughty rampart]

hybrid renderer is for entity rendering only. it's 'hybrid' because it transforms the data in a non-ecs way iirc

[junior fjord]

So the alternatives are:

1. Have Gameobjects linked to entities (and they can their position etc. automatically from the entity position now?)
2. Use hybrid renderer which I guess just renders all entities with specific components (like position, mesh, rotation)?

Do I understand that right?

And thanks for the helP!

[misty wedge]

Not automatically, you need to write some code for that yourself.

But you can create a managed component that contains a reference to the associated game object

[junior fjord]

for 1 or for 2?

[misty wedge]

1

[junior fjord]

Ok, gonna try to find docs on that

thanks so much!

[misty wedge]

Here's a small example: https://hastebin.com/usetenusum.hs

[viral sonnet]

I think it's NativeHashMap*<int, int>

[misty wedge]

Wouldn't the data type also be important for the pointer offset?

[viral sonnet]

sorry I was lazy and didn't write out the types, see the edit

[misty wedge]

🤷

[haughty rampart]

it would be NativehashMap<int, int>* not NativehashMap*<int, int>

[misty wedge]

Also doesn't work

It says it's a managed type for some reason

[haughty rampart]

what's the error?

[haughty rampart]

yeah because it IS a managed type

[misty wedge]

Then why does the extension method work?

[haughty rampart]

which one?

[misty wedge]

UnsafeUtility.AsRef

[haughty rampart]

because it's in an unsafe context

[misty wedge]

I marked the method unsafe

[haughty rampart]

please show the entire code

[misty wedge]

Using UnsafeHashmap in the above example works, but I'm still confused why the extension method works with NativeHashMap

[haughty rampart]

probably because it's restrained to struct

[misty wedge]

But shouldn't the extension method throw an error saying something along the lines of "T must be unmanaged to declare a pointer to it"?

[haughty rampart]

no. because it's not contrained to unmanaged only to struct

[misty wedge]

But that would mean the extension method can get a pointer to a managed struct

If I copy the method to my own code I get the same error:

Which makes sense to me

[haughty rampart]

yeah exactly

it should do that

but it does not

[misty wedge]

Sooo why can it do that? I'm confused D:

Basically I don't get why unity's extension method can do it but mine can't

[haughty rampart]

not quite sure. i mean they are using a different roslyn currently. maybe that's just too 'old' to see that

[misty wedge]

Weird

So is it unsafe to get a void* to a NativeHashMap? Or does it not matter because the underlying buffer is unmanaged?

[viral sonnet]

getting a poiner to a NativeHashMap doesn't make much sense because it's just a wrapper around UnsafeHashMap

[haughty rampart]

i'm pretty sure it's unsafe because it is still a managed type. (that won't be the case for much longer though)
you would definitely need to fixed it to get remotely safe

[misty wedge]

If I use UnsafeHashMap I don't need to use any fixed statements right?

[haughty rampart]

prob not

[misty wedge]

I guess an advantage would be that the safety system would inform me if I forget to dispose the native collection?

[haughty rampart]

normal nativehashmap will be an unmanaged type in the future and still inform you

[viral sonnet]

correct, safety handles and helper methods are in it. otherwise you could just use the UnsafeHashMapData* and create a new struct.

[misty wedge]

Ah, good to know

[viral sonnet]

But I'm trying to figure out what you want to achieve with saving a NHM pointer inside an IComp

[misty wedge]

Basically just messing around with pointers to better understand them. How else do I store a reference to a hashmap?

[haughty rampart]

you dont

[viral sonnet]

is it bound to an entity? if not, you can just make a reference in the OnUpdate which will be snapped so it can be used inside the ForEach. In a job you'd just reference it normally, like any other parameter. NativeContainers are handled by pointer anyway, no struct copies involved when used inside a job

[misty wedge]

It is bound to an entity

[viral sonnet]

Ah I see, well I have not tried it but you should be able to use a NativeHashMap inside an IComp without pointers.

[misty wedge]

I'm 99% sure that doesn't work

[viral sonnet]

If not, the fixed keyword inside an IComp does

[haughty rampart]

? you can't store nativearray, nativehashmap, .... in an IComp

[misty wedge]

You can store the unsafe collections inside a component, but I've never tried it

[viral sonnet]

my messy health code where I tried some things

[misty wedge]

I don't think I entirely understand why I can't store a pointer to a collection inside a component though. What's keeping me from doing it?

[haughty rampart]

a pointer should be fine

[viral sonnet]

I think it has to do with only blitable types.

[misty wedge]

This also works:

[haughty rampart]

why do people not use IntPtr anyway?

[misty wedge]

I guess it makes it easier to see what the underlying type is?

[viral sonnet]

I mean, you can store all kinds of struct in there. Maybe Unity explicitly forbids using structs with NativeContainer tag

[haughty rampart]

void* ? could literally be anything but you can't even do pointer arithmetic

[misty wedge]

It's what UnsafeUtility.AddressOf returns

I guess I could stick it in an IntPtr

But like I said, just messing around

[haughty rampart]

yeah

[viral sonnet]

void* is like the most generic form and not a struct unlike IntPtr

[haughty rampart]

i mean i don't mind people using IntPtr, int*, byte*, long* or anything else, but i just don't get void* except for methods

[misty wedge]

It's not a struct?

[haughty rampart]

no. void* is a pointer

[misty wedge]

But so is int*

[haughty rampart]

but IntPtr is a struct

[misty wedge]

Ah okay that's what you meant

A wrapper

[viral sonnet]

Also I think with void* it's expected and clear to the dev that it has to be cast to something else.

[haughty rampart]

and what do you cast it to? it literally gives you no clue

i have that problem with a PInvoke api currently

[misty wedge]

I just think the bottom one is really ugly

where hashmappointer is an IntPtr

[viral sonnet]

Well, the data you are expecting. I mean, you have to know of course otherwise, yeah ... 🙂

[haughty rampart]

why is that one ugly? it's good readable

[misty wedge]

I think it's just personal preference 😅

[haughty rampart]

yeah. otherwise you're stuck like me

[viral sonnet]

I think it looks cool ^^ just a pain to write maybe

[misty wedge]

And you can't accidentally cast it to the wrong type

[haughty rampart]

you can remove some unnecessary paratheses

[misty wedge]

Yeah I know, but looking at just the struct you have no idea what type it is supposed to be

[haughty rampart]

that's why you add xml documentation

that's shown in VS

[misty wedge]

I just don't see why you would use IntPtr over a pointer to the actual type?

[viral sonnet]

I'd take the first method though or write a method in the struct that returns the hashmap csated. there is no point in always writing out the cast

[haughty rampart]

because it's the recommended c# way to handle pointers.
and it does not need an unsafe context

[misty wedge]

Interesting, I hadn't thought about the unsafe part

Although I guess it would still need the unsafe context when you cast it back to the pointer type

[haughty rampart]

yeah true. so maybe in your instance it's not that big of a deal

[misty wedge]

How do I actually pass the pointer to the component though? Since I can't just create it and then pass the pointer address of the struct to it

[haughty rampart]

but still. i don't like void* for anything other than pointing to a method

[misty wedge]

Wouldn't something like this just get the address of the struct on the stack?

Meaning I would need to just store the normal struct inside the component, not a pointer to it

[viral sonnet]

it should work in theory. although you'd miss the helper methods from NHM

[misty wedge]

That seems very odd, since something like this would never work in C / C++. The address of the pointer is invalid as soon as the locally scoped struct goes out of scope (when the method completes). The only way it could work from the way I understand it is if UnsafeUtility.AddressOf somehow grabs a pointer to the actual buffer inside the unsafe collection.

Then the struct could go out of scope, but the buffer address is still valid (if it hasn't been disposed already somehow)

[viral sonnet]

I don't quite follow. The whole struct gets copied, which means the pointer to the UnsafeHashMap also gets copied

I'm not aware of any scoping in C++?

[misty wedge]

Let me check, it's been years since I've done anything in C / C++

[viral sonnet]

at least not in the sense that would invalidate something that you just copied somewhere else

for the local scope okay, but you essentially make 2 copies. one on the stack and one in memory

[misty wedge]

Where is it copied to in memory?

[viral sonnet]

ah, I did misunderstand. yeah, the code above would not work like that.

unless you fixed the pointer

[misty wedge]

Basically something like this:

It even gives you a warning:

[viral sonnet]

best thing is to just try it tbh 🙂 c# compiler is aware of many things

[rotund token]

Oh I've seen this a few times in other projects I forgot what the fix was

[haughty rampart]

frick

[rotund token]

Going to need a bit more context if you haven't fixed it. Is this your ref iterator method?

[rotund token]

1 thing about my fix, it breaks again anytime you add a new package so you have to rebuild

[viral sonnet]

I have taken the code out and it compiled. Have to check why that happens.

[haughty rampart]

siiigh unity. please release 0.51 already. then i can remove the ui packages

[shut pewter]

Hi again. I've been trying to load some game data in burst+jobs code.

My "old" OOP way was to have a lot of Dictionaries in Singletons.
Now I have some NativeHashMaps with my data, but accessing that data ... Singletons don't work cuz it's a managed class. Is there a standard way of injecting other systems in jobs?

[misty wedge]

You can pass the NativeHashMaps to the job

[shut pewter]

Can I ? I use ForEach+ScheduleParralel syntax, not sure how that works in that case?

[viral sonnet]

btw. I have added profile markers to the Buffer test. 40ms iteration loop - 11.7ms Indexor

[misty wedge]

Just make sure to make a local copy of the hashmap before you pass it, otherwise you are doing this.hashMap behind the hood, which is a managed operation and not allowed inside a scheduled job.

[viral sonnet]

Yes but you need to use NativeHashMap<int, int>.ParallelWriter and the hashmap has to be pre-allocated. no dynamic adding/resizing possible in parallel

[misty wedge]

Schedule isn't parallel

[viral sonnet]

I mixed up the question 😄

[shut pewter]

Even when not writing to it ?

[viral sonnet]

you're good when just reading

[shut pewter]

That's good. I load a lot of readonly game data on start, so having easy access to it is essential

[misty wedge]

So just to make sure, I can put a UnsafeHashMap inside a component and my computer will not explode?

[rotund token]

if you're not crashing your machine for bad pointers half a dozen times a day you aren't doing it right

[misty wedge]

@rotund token this wouldn't work right?

[rotund token]

why not just store the actual map instead of wrapping it?

[misty wedge]

Yeah, that was my conclusion too

I'm storing the pointer

But the way I understand it, the pointer is pointing to the stack variable of the struct

[rotund token]

ignore my ignore

[misty wedge]

😅

[rotund token]

yes that is going to break

why not just go
Hashmap = hashmap

?

[misty wedge]

Yeah, like I mentioned that was my conclusion too, just store the struct on the component

The only other alternative that I can see would be to store the struct in the system as a variable, and then pass the pointer

[rotund token]

that doesn't mean it won't move

[misty wedge]

Ah yeah, the fixed keyword I saw for the first time today 😅

Can you even fix a class variable permanently?

[rotund token]

you could GCHandle.Alloc(src, GCHandleType.Pinned) the whole system or something

[misty wedge]

Ok, I definitely like the first version more

So just a non-pointer UnsafeHashMap on the component

and remembering to dispose it, since the safety system won't yell at me

[rotund token]

pretty much why i'm not a fan of this

memory mangement just a pain

[misty wedge]

Yeah I'm just trying to understand what's possible

I like the DB approach more

[rotund token]

that said, it's not like i've never done this

[viral sonnet]

@rotund token have you seen the post from me before with the profile markers? Job overhead is not that much from buffer iteration. Most time is really spent in the iteration

[rotund token]

had a quick glance but haven't read that much

busy providing burst guys with a repo for something

[viral sonnet]

oh, you have a new bug? 😄

[rotund token]

actually its like a 6month old bug i never reported

and forgot about

[misty wedge]

Also something else we weren't able to figure out that you probably already know, is why UnsafeUtility.AsRef can get a pointer to a managed type, but I'm not allowed to

[rotund token]

left them a repo last night and they couldn't repo it with my steps

so just sending across a small project

[rotund token]

error DC0058: All SystemBase-derived classes must be defined with the partial keyword, so that source generators can emit additional code into these classes. Please add the partial keyword to GasSystem, as well as all the classes it is nested within.
is going to be the new ; that I always forget

[robust scaffold]

i wish regular unity c# had source gens

[rotund token]

you can get it working pretty easy

it's not that hard at least from what i saw someone else set it up

[robust scaffold]

I got so many identical data structs differing only by a single constant buffer size

[rotund token]

the tldr is

add com.unity.roslyn

[robust scaffold]

I'm sure it's possible, maybe even easy, but I'll wait until unity rolls it out as officially supported. either my moving to the next .net version or something

[rotund token]

add these 3 labels to the assembly
RunOnlyOnAssembliesWithReference
RoslynAnalyzer
SourceGenerator

done

[robust scaffold]

hrm

https://docs.unity3d.com/Manual/roslyn-analyzers.html

huh

huuuuh

[rotund token]

yeah basically same thing in 2021 just no need for the separate package

[robust scaffold]

Took me a bit to install roslyn package, had to install dotnet 6.0 as well. How do i add these labels to the csharp assemblies?

[rotund token]

click on an asset

bottom of inspector has a label option

[robust scaffold]

I have never noticed this before

[rotund token]

its been around for years

pretty sure it was in unity 3

[robust scaffold]

Yea, never used it. So, what do I do?

[rotund token]

just click the blue icon

and add the label

to the asmdef i believe

i haven't actually set this up myself just second hand report

[robust scaffold]

The global Asmdef or a newly created one for the folder that the source gen scripts are located in?

[rotund token]

the source generator one

[misty wedge]

Should look like this once you are done

[robust scaffold]

Checks out?

Or should those asset labels be on the scripts themselves?

[misty wedge]

Good question. I've only done it with .dlls, and the dll was what I tagged, so I'm not entirely sure.

[robust scaffold]

Does the DLL contain the source gen code? Have you found any examples of unity doing this?

Not the source gen itself, but the input code for the source gen.

[rotund token]

think label only needs to be on the asmdef

[misty wedge]

What do you mean exactly?

Unity's source generators are also in DLLs as far as I'm aware

under Unity.Entities/SourceGenerators or something

https://github.com/needle-mirror/com.unity.entities/tree/master/Unity.Entities/SourceGenerators

[robust scaffold]

Oh, I'm not using entities. Just straight Jobs, Burst, and Collections. Rolling my own ECS system but I wanna try out using source gens as well

[haughty rampart]

you don't need any label besides RoslynAnalyzer btw

[robust scaffold]

more labels wouldnt hurt... right?

[haughty rampart]

but they dont do anything, so as long as you don't need them for something specific yourself, i'd argue yes they do

[haughty rampart]

but what are you doing with source generators anyway? i didn't see anything specific

[robust scaffold]

I just wanna merge my custom fixed buffers into one "easy" source gen'ed script

I have this:

[white island]

dang it, i'm back in the Entities.ForEach Lambda expression captures a non-value type 'pathFinder'. This is only allowed with .WithoutBurst() and .Run() hole again. Is there a way I can like, alias an instanced class call so this doesnt happen.

[robust scaffold]

And so on, until 256

A bunch of files differing only by that capacity value

[haughty rampart]

you need to extract it like the first time and reference the extracted variable

[white island]

well, i did. CellNavigationSystem.CellNodePathfind pathFinder = CellNavigationSystem.CellNodePathfind.instance;

[haughty rampart]

yeah, yeah ok, that would be super easy to source gen

[robust scaffold]

I think source gens would help compress all those into a single source gen file that would be easy to change.

[haughty rampart]

if you want some help with that, i have worked intensively with source generators and analyzers. (although this is a super simple case where you don't really need roslyn api knowledge)

[robust scaffold]

yep yep, hit me up. Here or DM?

[haughty rampart]

whereever

[robust scaffold]

here, for public knowledge. Although it's a lot more popular now that 0.50 dropped. Used to be dead (i liked it back then...)

[white island]

it's not a variable, it's a utility method. It just can't be static

[robust scaffold]

So I just copied the default microsoft hello world example but my IDE is saying it isnt picking up on the source gen structure. Nor is Unity doing so either. But CodeAnalysis exists.

[haughty rampart]

pls post code if you can

[robust scaffold]

Screenshot the entire job.

[violet crest]

Jetbrains

[robust scaffold]

the superior IDE

[violet crest]

yeah

[haughty rampart]

ok, so you can't write them in unity (currently)

you need to create a new .net solution class library targeting .netstandard2.0

[white island]

the function I want to call:

public static CellNodePathfind instance;

private void Awake()
        {
            instance = this;
        }

public NativeArray<Node> pathFindThroughGraph(Node start, Node end)
        {
            //Queues
            NativeList<ushort> openList = new NativeList<ushort>(nodes.Length, Allocator.Temp);
            NativeList<ushort> closeList = new NativeList<ushort>(nodes.Length, Allocator.Temp);
            openList.Add(((ushort)nodes.IndexOf(start)));

            //AStar

the job:

[white island]

obligatory vim comment

[robust scaffold]

oh no. should I instead attempt to copy over what Unity has done in Entities and somehow transfer their source gen implementation?

[haughty rampart]

no. just create the class library

it must target .netstandard2.0

[haughty rampart]

idk about that. never liked it

[robust scaffold]

Only options (in jetbrains) is frameworks. Im assuming v3.5 ?

[haughty rampart]

NOT unity class library. just normal .net class library

[white island]

the reason im doing this is the function I want to call would otherwise reference a static array elsewhere in the class and thats no good

actually i guess. it doesnt need to be a singleton. it just needs to be a class instance stored inside of my system somewhere. but then I get problems about like, this

[robust scaffold]

I generally find that attempting to call functions that are not static will not work. The Main-thread to Job-thread boundary is very picky.

[haughty rampart]

there should be something similar in jetbrains

[robust scaffold]

Yep yep. Installing .net right now...

[white island]

Right, yeah. But I need those static arrays, which it also doesn’t like.

[robust scaffold]

Yea, static arrays are generally a no go. Try using static NativeArrays<> instead.

That's basically what my entire handcrafted ECS is built on. Piles upon piles of static NativeArrays

[white island]

Well yeah that’s what I meant

Huh ok

[robust scaffold]

I think you may need to local capture the native array. Basically:

public static NativeArray<int> GlobalArray;
public void main()
{
  NativeArray<int> localArray = GlobalArray;
  Entities.ForEach((ref something variable) =>
  {
    // Do something
    localArray[variable.value] = 10;
  }).Schedule();
}```

[haughty rampart]

it must target netstandard2.0

[robust scaffold]

There's option for frameworks... gonna have to make me boot up vs arn't ya

[haughty rampart]

i mean.....
either you know how to change it in rider, or you use VS to set everything up and may switch to rider once that's done

sorry

[robust scaffold]

yea, looking to be the latter option. Since you know what ya doing, and I dont.

[haughty rampart]

btw, i can definitely recommend VS 2022. very performant on my hardware, very nice to work with. (maybe not as slick as rider)

[robust scaffold]

Yea. I have a copy since it's required to build IL2CPP in Unity.

[robust scaffold]

Oof. Nevermind. I'll see ya tomorrow

my internet is pain

[haughty rampart]

what? how long does that take for you?

[robust scaffold]

at this rate, probably an hour or two. Might be faster if I walk back to campus, download it there, then come back home

[haughty rampart]

well, i may have an idea to make it work in rider

so just create the class library for the time being

@robust scaffold

[robust scaffold]

under net6.0?

[haughty rampart]

yeah. just keep .net6.0 selected

[robust scaffold]

Done.

[haughty rampart]

ok. rider should still create a .csproj file i hope. can you open that?

[misty wedge]

For some reason VS 2022 becomes unusable for me if I open another large solution (e.g. Unity.Collections) besides assembly first pass. It becomes extremely slow and typing is like 1 second delayed, as well as intellisense basically not working anymore and easily taking 5+ seconds to figure out what is going on. No idea what is happening

[haughty rampart]

how much ram do you have?

[misty wedge]

16 GB, I've also thought that may be the issue

PC is getting old 😦

[robust scaffold]

Collections has a lot of repeated code that really should be under source gen as well.

[haughty rampart]

i mean, it should be good still good i think
1 - unity's structure is bad currently / way worse than .net native
2 - may be a ram issue
3 - VS is unfortunately STILL a .net framework process. (will be upgraded to .net 6+) eventually

[haughty rampart]

ok yeah. change <TargetFramework>... to netstandard2.0

[robust scaffold]

Well, nothing exploded. Kinda

[haughty rampart]

then add

  <ItemGroup>
    <PackageReference Include="Microsoft.CodeAnalysis.CSharp.Workspaces" Version="3.8.0" PrivateAssets="all" />
    <PackageReference Include="Microsoft.CodeAnalysis.Analyzers" Version="3.3.2" PrivateAssets="all" />
  </ItemGroup>

below </propertygroup>

and just add <LangVersion>preview</LangVersion> below <TargetFramework>

[robust scaffold]

Nice, everything unexploded

[haughty rampart]

put a blank line between the itemgroup and propertygroup. then it's correctly formatted

[haughty rampart]

nice

[robust scaffold]

Well, it works. IDE is detecting the source gen

[haughty rampart]

ok. now can you rightclick on the .csproj file in your solution explorer (or whatever)

[solar spire]

I may have missed it in the convo, and haven't tried it myself, but is there a reason not to use the com.unity.roslyn package?

Maybe it's stuck to 2020.3

[haughty rampart]

in my experience that's only needed for 2020

ok, then properties

[haughty rampart]

debug

[haughty rampart]

scroll down please

[robust scaffold]

Just XML and warnings. That's the bottom.

[haughty rampart]

yeah ok. idk if that's even available in rider
so unfortunately you won't currently have a nice debugging experience

[robust scaffold]

dont use the debugger anyways. Debug.Log is all that I need and love.

[haughty rampart]

you can't do that

[robust scaffold]

Uh, then hope and prayer.

[haughty rampart]

File.WriteAllLines() is all you can do

[robust scaffold]

Yea, that also works I think.

[haughty rampart]

pain though

[robust scaffold]

Eh, I've worked well enough with the console print. Jetbrain rider's debugger is extremely unreliable for all these pointer operations as well so I dont use it in the first place.

[haughty rampart]

delete class 1 and just make the generator top level please. no idea why you have an extra class there

[robust scaffold]

That was just default. Done

using Microsoft.CodeAnalysis;

namespace TestSourceGens;

[Generator]
public class HelloSourceGenerator : ISourceGenerator
{
    public void Execute(GeneratorExecutionContext context)
    {
        // Code generation goes here
    }

    public void Initialize(GeneratorInitializationContext context)
    {
        // No initialization required for this one
    }
}```

[haughty rampart]

oh right. cause you created a class library and just pasted it in

[robust scaffold]

Yea. I'll turn my brain on when I need to figure out how to implement this. for now, monkey see monkey do

So how do I couple this with the Unity project? Do I have to keep switching back and forth between the projects if I want something done? So do I run this external solution to create the files required in Unity the paste them over?

[haughty rampart]

a note before going further: ISourceGenerator is terrible and should not be used for new source generators.
use IIncrementalGenerator instead. ....unfortunately that is not available in unity yet

[robust scaffold]

And probably wont be available for a few more years

[haughty rampart]

nah. the next roslyn upgrade on unity's side should fix that hopefully

[robust scaffold]

Oh do i also need to set Unity to target net standard 2.0?

[haughty rampart]

currently unity does not support project referenced source generators. only loading them as dll's.
so you may copy them manually or set up auto copying in the project file

[haughty rampart]

no. it has nothing to do with unity per se

[robust scaffold]

So how is Unity themselves doing it? Their own external DLL?

[haughty rampart]

yeah. all source generators are included as .dll's in the packages

[robust scaffold]

Oh. Ohhh. oh pain.

[haughty rampart]

like i said. you can set up auto copying

to make it less bad

[robust scaffold]

Yea. Hrm. I'll probably keep my current pile of copy pasted structs until Unity integrates source gens into UnityC# (if that ever happens). The ctrl-c ctrl-v and changing the capacity values thankfully isnt too tedious.

[haughty rampart]

it's really not complex setting up the source generator. we are almost done

[robust scaffold]

Compile into a DLL then copy it into unity?

[haughty rampart]

yeah exactly

[robust scaffold]

Alright, let's go all the way to the end then. I'll just copy over Microsoft's Hello World example.

[haughty rampart]

if you also want to set up auto-copying you can add

    <Target Name="AfterBuild">
      <Copy SourceFiles="$(ProjectDir)..\..\Lib\*.dll" DestinationFolder="$(OutDir)Debug\bin" SkipUnchangedFiles="false" />
      <Copy SourceFiles="$(ProjectDir)..\..\Lib\*.dll" DestinationFolder="$(OutDir)Release\bin" SkipUnchangedFiles="false" />
    </Target>```
to the .csproj and just change to the path that's needed

[robust scaffold]

I'm assuming that changing the Console.WriteLine to Debug.Log wouldnt work.

[haughty rampart]

ahm, i'm not sure that that particular hello world example will work. because unity does not have a Main(string[] args) method

[haughty rampart]

no of course it would work. but you'd also have to do using UnityEngine; after using System;

[robust scaffold]

Alright, so that means my main form of debugging is possible? What was the issue for using Debug.Log then?

[robust scaffold]

I'll just reference this partial class in a monobehavior after it's been generated so it runs.

[haughty rampart]

...no. that is the code to be written. that will be like a normal file in unity.
that has nothing to do with the source generator. the generator is anything that happens inside the Execute()

[haughty rampart]

this is what one of my generators looks like

[robust scaffold]

Interesting. And this is run in the editor before builds?

Is this run every time you click play? Is there a way to communicate input from the Unity project into this source generator before the files are created? For example, I present to this source generator a list of ints that indicates what StringXs to create as to not create all 8 or 9 from String 2 to String256 or whatnot.

[haughty rampart]

anything that's in the Execute() of a source generator is run EVERY TIME you make a syntax change in any of your project files, aka e.g. each keystroke.
(that's why ISourceGenerator is pretty much already deprecated)

they are also run during (before) each build.

[robust scaffold]

Oh, I can see how that's a problem.

[haughty rampart]

yes. this is absolutely possible

[robust scaffold]

Yea, I cant imagine Unity's Entities source gens working without that.

[haughty rampart]

well....but unity handles files differently currently than native .net, so........you have to write the generator a bit differently than you would with normal .net
and it might break when unity changes to the csproj workflow (cause then it's standard .net), so......no guarantee XD

[robust scaffold]

I trimmed out the namespace and switched to Debug.Log.

[haughty rampart]

question, do you have a preview version of .net 7 installed?

[robust scaffold]

No. Should I?

[haughty rampart]

no, not needed, but there's a huge new feature in c# 11 that makes the to generate code much nicer to work with

[robust scaffold]

Oh right, with these DLLs, we arent chained to Unity's C# implementation.

[haughty rampart]

indeed

[robust scaffold]

That's a... actually big benefit.

I just wanna get this working first before trying out the shiniest new toys.

[haughty rampart]

btw, i don't really get that point

[robust scaffold]

I was just wondering if there was any way to communicate from Unity to the source generator before the source gen creates the file. If there was no way, how would Unity Entities files actually be source generated?

Because I think I'll be hardcoding the general structure of the StringX buffers and just running it from the loading monobehavior. Unless there was some way in Unity to provide a template that somehow gets interpolated to all 8 buffer sizes.

But that's just a source generator implementation inside unity and we wouldnt be needing a second project to create the DLL in the first place, right?

[haughty rampart]

so, how source generators work:
-they are a form of metaprogramming-
the code that you wrote will be fed to the generator fully, and the generator scans over the code for anything you want it to look for (e.g. some partial class that implements SystemBase). when it finds something, it can get more information about that thing and you can apply certain transformations to it.
but source generators may not only take code files as input, but any files

[robust scaffold]

So going back to what my desired implementation is. I have these set of StringX buffers, all identical save for their constant capacity values hardcoded in. I also have an IntX buffer version as well. Identical to StringX except uses Ints (along with different checks and toString() implementations but otherwise identical within the same "template").

Is there a way I can create, or download, a generic source generator that can recognize these templates and then, with a hardcoded init file or within the loading Monobehavior, create the required buffer capacity values?

[haughty rampart]

download -> no, i don't think anyone has created one yet
create -> absolutely

[robust scaffold]

And how do I get the IDE to recognize that String32 exists before the source generator runs and actually produces the file that contains a StringX buffer with capacity 32?

Unless I have to run the source generator first before coding in any usage of String32?

[haughty rampart]

that is up your implementation. there are many possibilities. i will list a few

[robust scaffold]

And I also have this, a JsonConverter that must implement all possible StringX capacities. It's currently using reflection to obtain the target type constant capacity and then type converts it to the proper StringX buffer type.

Right now, without the source gens, if I need larger capacities than 64, I just copy paste the String16 implementation, rename it to the size, change the constant capacity to the size required, and then add it to that switch statement.

[haughty rampart]

1 - create a text file in your project as input for your source generator. on each line of that text file you put e.g. String32 String64 etc.... and let the source generator generate the code for each of those lines (this approach is actually used by at least 1 microsoft source generator)

2 - have an attribute that let's you specify an argument e.g. StringAttribute(32) or GenAttribute("String32")

3 - do not take any input to the source generator, and just have a for loop in your generator that generates from (e.g. 32......64) all the String / Int code. (not recommended imo)

[haughty rampart]

you could also easily let that me source generated as well

[robust scaffold]

Definitely, and it's very ugly and hacky. But it works.

[robust scaffold]

I like the attribute idea. Seems to be the most intended way.

[haughty rampart]

most often, attributes are a good choice

[robust scaffold]

So. I got this hello world script that may or may not work. Depending if var mainMethod = context.Compilation.GetEntryPoint(context.CancellationToken); works in Unity.
How do I compile this into a DLL (I've only worked with Unity and nothing else) and run it in Unity?

[haughty rampart]

just build it

[haughty rampart]

👍

[robust scaffold]

I havent added the automatic copying yet so that's still in the TestSourceGens file. Do I copy all 3 of these into the unity project assets?

[haughty rampart]

you only need to copy the .dll

[robust scaffold]

Ah, okay

[robust scaffold]

Into Plugins or anywhere is fine?

[haughty rampart]

i recommend plugins.

and don't forget to prepare the dll to actually be registered as a source generator

[haughty rampart]

XD exactly

do you know how to?

[robust scaffold]

Nope.

Do I need to download the CodeAnalysis DLL as well?

[haughty rampart]

no

[robust scaffold]

I had to piece together System.Text.Json using all the DLLs..

[haughty rampart]

click the dll so it appears in the inspector

[haughty rampart]

it must look like this

and have this label

[robust scaffold]

Alright, cleared out the platforms (that removed the error at least) and added the label.

[haughty rampart]

good

[robust scaffold]

So, did it run? Where's the output source gen'ed file?

[haughty rampart]

so, generally source generators do not 'output' any visible files, cause they are not meant to be changed anyway. afaik, unity (because of the bad debugging for unity) does currently output the generated files to Temp/GeneratedCode

[robust scaffold]

Because there's no IDE syntax checking in these raw strings, I just want to confirm that the source gen'ed file is actually printed correctly.

[haughty rampart]

yeah

[robust scaffold]

Are we stuck with notepad (not even ++) levels of code checking for these source generators? Because my code is shit without jetbrains literally typing out half the code for me.

And with those new fangled AI driven code assistants, i think it'll probably jump to 75% of the code sooner or later..

[haughty rampart]

what code checking do you expect with code fragments like this? (that is the most common thing in source generators)

with source generators, you really want to create unit tests and enable the better debugging workflow (which is not available in rider as noted previously)

[robust scaffold]

True. Which is why I wished for these source generators to produce actual files like Unity's GeneratedCode just to run through the linter. Guess I just hope?

[haughty rampart]

they are still verified for syntax of course. source generators will not add invalid code to the compilation.

the source generator code you write is still compiled

[robust scaffold]

Makes sense. So how do I run this from something like a monobehavior?

[haughty rampart]

you don't. source generators are automatically run

1 - with every syntax change
2 - with every compilation

[robust scaffold]

In the example microsoft provided, it made the script that it was generating have a dynamic name. What is that actual name?

context.Compilation.GetEntryPoint(context.CancellationToken).ContainingType.Name

[haughty rampart]

yeah. which maps to Program in standard .net

[robust scaffold]

Should I just hardcode it?

[haughty rampart]

because

public class Program
{
  private static void Main(string args)
  {
  }
}

[haughty rampart]

yeah, probably good idea for the test in unity

[haughty rampart]

yeah

[robust scaffold]

Remove the partials?

[haughty rampart]

you want that to be dynamic in a real source generator because it may output multiple files. so it would get overwritten

[haughty rampart]

no.

uhm, just remove the partial from void HelloFrom()

[robust scaffold]

well I removed both partials and there's no error

[haughty rampart]

yeah ok, i guess that works

[haughty rampart]

yeah, but now you should make HelloFrom() public

so you can actually call it

[white island]

Well. the static arrays are within the class and referenced in the function. Unless you mean, take them out with variables and then pass them to itself?...

[haughty rampart]

and i see you removed the entrypoint thing. yeah, should be absolutely fine this way

[robust scaffold]

No errors within the IDE. Time to run...

[robust scaffold]

Tada, it works.

[haughty rampart]

👏

[robust scaffold]

That is a lot of roundabout effort but it's good to know

[robust scaffold]

Yea, try localizing them with local variables and then pass the local var into the function

[haughty rampart]

ok. if you need help with the actual source generator at some point, feel free to reach out.

[robust scaffold]

Yep. I'll take a look at microsoft's docs as well. But now I got it to work so it's only up from here.

I just wish Unity's C# could do this in one single project...

[haughty rampart]

not yet :/
(technically it will never be 1 singular project though. you still need to create the class library and set a project reference, but that is definitely much nicer to work with)

[robust scaffold]

There is that Roslyn package. Maybe????

I dont have it installed right now but the DLL is functioning so it isnt required.

[white island]

okay! that seems to get the errors to go waway, thank you.

[haughty rampart]

i don't think that has anything to do with it really. i think it's just a newer roslyn implementation. but that won't change any generator workflow

[robust scaffold]

NP.

[robust scaffold]

Yea. Microsoft.CodeAnalysis is valid without the package. Just the [Generator] and the rest isnt.

And plus, if it was valid, Unity would be doing that instead.

[haughty rampart]

:/ idk what unity is doing. maybe the roslyn package actually get's you a 'better' workflow, but i have no idea. if you find anything in that regard, let me know though

[gentle harness]

Is it possible to schedule a System.Action command with an ECB? I'm trying to add a created entity to hashmap, but adding the deferred entity is giving me errors

[pine plaza]

i believe it is impossible to do that

does anyone know about physic constraint ?

gg it and found key word " joint component" but there isnt many documents there

[karmic basin]

Did you try exploring and playing with the official physics samples scenes ?

[pastel field]

**Hum... 2021.3.1f1 changelog **
Packages com.unity.mathematics was updated from 1.2.5 → 1.2.6

Which silently upgrades :
com.unity.collections from "version": "0.15.0-preview.21" → "version": "1.2.3",

Throwing a nice

Library\PackageCache\com.unity.jobs@0.8.0-preview.23\Unity.Jobs\IJobParallelForDefer.cs(77,85): error CS8377: The type 'U' must be a non-nullable value type, along with all fields at any level of nesting, in order to use it as parameter 'T' in the generic type or method 'NativeList<T>'
(error that can be found all over the place on the forum)

I guess it's fixed in the new DOTS 0.50 but since it doesn't work on 2021.3.1f1, I have to go back to 2021.3.0f1.
Reverting packages does not work when they are forcedly upgraded this way (at least not this time for me).

Soo... be careful when updating ?
Don't make the same mistake I did of thinking this is a minor update 🙄

[dense crypt]

ECS 0.50 only supported on 2020.3.30f1, you're using 2021 so don't be surprised that changes will break the heck out of it

[haughty rampart]

they are not using ECS

[dull copper]

wait what... https://github.com/needle-mirror/com.unity.physics/releases/tag/0.50.0-preview.43 this date can't be right

so they changed out of 9999-12-31 and now they are stuck on 2021-09-17

why even have dates there if can't bother updating them? 🤔

[safe lintel]

seems like we're so far behind whatever they have internally

i get that they want to have more polished package releases but it feels like even their attempts for refined packages are really buggy and would benefit from faster releases for more feedback

[covert lagoon]

I have an HDRP project with a bunch of DOTS entities that each have a RenderMesh component set by my code (not automatically converted by the built-in GameObject conversion systems) with a material and programmatically generated mesh, but they don't render unless I change the value of their RenderMesh component's "Layer Mask" field to something else than the default of 0 in the inspector, how should I fix it in my code?

I set the RenderMesh components like this in a conversion system I made:

DstEntityManager.SetSharedComponentData(worldChunkEntity, new RenderMesh
{
    mesh = new Mesh(),
    material = authoring.material,
});

[safe lintel]

you should use the utility inside the hybrid renderer package for setup, its in the hybrid renderer docs

[covert lagoon]

This?
https://docs.unity3d.com/Packages/com.unity.rendering.hybrid@0.50/manual/runtime-entity-creation.html#rendermeshutility---addcomponents

[safe lintel]

yep

[covert lagoon]

Thanks

How do I recalculate the value for the RenderBounds components after generating the meshes?

[rustic rain]

What is correct way of getting ECB?

Do I create it every update or?

Can I reuse it somehow?

[safe lintel]

you just create one each update

[haughty rampart]

EcbSystem = GetOrCreate()
ecbsystem.CreateECB()

[rustic rain]

so I do create it every update

ok

[hot basin]

I create them in OnCreate() and store in a field

[rustic rain]

and then?

you simply use same ecb?

[hot basin]

public class SpawnerSpawnSystem : SystemBase
    {
        private EndSimulationEntityCommandBufferSystem entityCommandBufferSystem;

        protected override void OnCreate()
        {
            entityCommandBufferSystem = World.GetOrCreateSystem<EndSimulationEntityCommandBufferSystem>();
        }

        protected override void OnUpdate()
        {
            var commandBuffer = entityCommandBufferSystem.CreateCommandBuffer().AsParallelWriter();
        }
}

as an example

as a curiosity, I will add that I'm using ECB in MonoBehaviours also, as systems don't change so it's best inject-point for OOP

Did someone tackle utility ai yet? or something similar?

[rotund token]

I have be a really complex Ai system thats kind of a alternate utility implementation

Also we have be a full utility implementation at work (I didn't write it though)

[hot basin]

I wonder how to gather inputs

like best access patern ect

[rotund token]

If you are implementing an utility Ai system

I'd look at infinite axis utility ai

It kind of maps really nice to entities

[hot basin]

yeah I'm looking at it right now

but I wonder how it should be implemented in ECS as it cant be just on entity

as there is possibility of multiple same considerations/axis

[rotund token]

The one at work is purely entities

No unsafe code

[hot basin]

are they using entity per consideration?

[rotund token]

The guy who e wrote it was actually a new hire with 1 day of unity entities experience

[hot basin]

probably you can't say much 😄

[rotund token]

Yep

[hot basin]

in this case it's simple but is it efficient?

[rotund token]

I make this point just to show how naturally it can map to entities

He did have a lot of dod experience

[rotund token]

It's extremely good at going wide

It has had some issues on old consoles though

Had to dramatically merge number of systems

[hot basin]

I wonder if dynamic buffer can be used in this context somehow

[hot basin]

btw

## [0.50.1-preview.1] - 2022-04-07

### Added

* Documentation on EntityCommandBuffer public functions including ParallelWriter and EntityCommandBufferManagedComponentExtensions.
* Hybrid assemblies will not be included in DOTS Runtime builds.
* `[WithAll]` Attribute that can be added to a struct that implements IJobEntity. Adding additional required components to the existing execute parameter required components.
* `[WithNone]` Attribute that can be added to a struct that implements IJobEntity. Specifying which components shouldn't be on the entity found by the query.
* `[WithAny]` Attribute that can be added to a struct that implements IJobEntity. Specifying that the entity found by this query should have at least one of these components.
* `[WithChangeFilter]` Attribute that can be added to a struct that implements IJobEntity, as well as on component parameters within the signature of the execute method. This makes it so that the query only runs on entities, which has marked a change on the component specified by the `[WithChangeFilter]`.
* `[WithEntityQueryOptions]` Attribute that can be added to a struct that implements IJobEntity. Enabling you to query on disabled entities, prefab entities, and use write groups on entities.
* Diagnostic suppressor to ignore specific generation of CS0282 warnings due to codegen.
* SystemBase.GetBuffer takes an optional isReadOnly parameter.

IJobEntity seems more reasonable to use now

sorry it's my fault

I've looked into

https://github.com/needle-mirror/com.unity.entities/commit/6129e474c5d51c52548fe7bfa3b13f886ecc7e26

[viral sonnet]

Release preparations, no functional changes.

oh boy, do we get Unity2021 ECS??

[viral sonnet]

damn, using my own keys and values array in a NMHM goes a little over my head :/

[viral sonnet]

            where TKey : unmanaged, IEquatable<TKey>
            where TValue : unmanaged
        {
            var keys = hashMap.m_MultiHashMapData.m_Buffer->keys;
            var values = hashMap.m_MultiHashMapData.m_Buffer->values;
             
            keysList = default(NativeList<TKey>);
            UnsafeList<TKey> unsafeKeyList = new UnsafeList<TKey>();
            unsafeKeyList.Ptr = (TKey*) keys;
            unsafeKeyList.m_length = 0;
            unsafeKeyList.m_capacity = hashMap.Capacity;

            valuesList = default(NativeList<TValue>);
            UnsafeList<TValue> unsafeValuesList = new UnsafeList<TValue>();
            unsafeValuesList.Ptr = (TValue*) values;
            unsafeValuesList.m_length = 0;
            unsafeValuesList.m_capacity = hashMap.Capacity;

            keysList.m_ListData = (UnsafeList<TKey>*)UnsafeUtility.AddressOf(ref unsafeKeyList);
            valuesList.m_ListData = (UnsafeList<TValue>*)UnsafeUtility.AddressOf(ref unsafeValuesList);

#if ENABLE_UNITY_COLLECTIONS_CHECKS

            //keysList.m_Safety = hashMap.m_Safety;
            //valuesList.m_Safety = hashMap.m_Safety;
            DisposeSentinel.Create(out keysList.m_Safety, out keysList.m_DisposeSentinel, 2, hashMap.m_MultiHashMapData.m_AllocatorLabel.ToAllocator);
            DisposeSentinel.Create(out valuesList.m_Safety, out valuesList.m_DisposeSentinel, 2, hashMap.m_MultiHashMapData.m_AllocatorLabel.ToAllocator);
            keysList.m_DeprecatedAllocator = hashMap.m_MultiHashMapData.m_AllocatorLabel;
            valuesList.m_DeprecatedAllocator = hashMap.m_MultiHashMapData.m_AllocatorLabel;
#endif
        }``` anyone able to help me extract the key/values array from a hashmap to a nativelist?

getting null errors when writing to the list

[viral sonnet]

fixed it with using hashMap.m_MultiHashMapData.m_AllocatorLabel.Allocate(default(UnsafeList<TKey>), 0);

@rotund token the speed up is INSANE. writing to the nativelists (reserved) is fast as expected and calculating the keys in a single threaded burst job is 0.8ms for 500k keys

[rotund token]

you're liking this whole new world of hash map optimizations ^_^

[viral sonnet]

yeah 😄

only 1 problem to go

for some reason clearing the NHM takes up a lot and runs faster without burst

well, that question will be postponed. Reducing the capacity makes it only take 0.4ms which is okay. Alltogether, I can run this PhilSA test in 4.90ms - 5.00ms. The interlocked method takes 3.00ms on my machine

[viral sonnet]

funny, going through the chunks, looking for an entity key in a NMHM and applying events is around 1.70ms while going through the NMHM and applying events to random entities is 0.6ms. That's .... kind of dumb

I'm at a point where I have done so much testing that I came to the conclusion that it hardly matters if you use a ComponentTypeHandle or a CDFE. The CTH would only be faster when burst is able to vectorize

[viral sonnet]

down to 4 ms now. 😄

do you understand why this runs at 0.8ms while IJobNativeMultiHashMapVisitKeyValue takes upwards of 3ms?

[rotund token]

why does this all need to run on main thread?

[viral sonnet]

just testing, I should move it as schedule to the end

[rotund token]

not sure it looks like it's doing practically the same thing

[viral sonnet]

lol, well I can explain it now. The job code from Phil is bugged. No health values get decremented

[rotund token]

😅

[viral sonnet]

no wonder it runs so fast ... shit

then I have to reverse my above statement. iterating through chunks and looking up keys is 2 times faster

What's just frustrating is that I don't get even close to Interlocked timings. And that's even with the totally useless building of a NativeStream with the events which could be just erased

In really high performance code, I think Interlocked is king and shouldn't even be questioned if it's "good". I read so many sketchy things about it why it should not be used

[rotund token]

i agree to an extent

but it does kind of depend

you've seen the cost of interlocked first hand with NativeList.ParallelWriter

when you have so many tiny interlocks

[viral sonnet]

yeah, and it's only really useful when really nothing else is going on. In my case, I've so much other stuff to calculate. Sure, Interlocked is king in one small part of the code (the health reduction) but all the other stuff doesn't profit in the slightest. Triggermaps, IsCombat flag, having simply no race conditions.

With the testing complete, I can get rid of the 2 other NativeArrays and just use the NMHM keys/values. I've a really good speedup with that method.

ah, and I forgot about the spell pushback on casting targets. man that took quite a lot of cpu time. taking the hit sometimes to build an acceleration structure is really worth it sometimes to speed the rest up

especially when you can stay in the chunk iteration and have no random access to other entities

[white island]

Man I hope Dots gets updated to 2021 soon because my 2020 project is becoming increasingly unstable …

[rotund token]

Happy the patch came out today

Because it at least gives me some confidence they'll hit their schedule got 0.51

That said, outside of runtime ui elements was there anything else causing you instability?

[rustic rain]

Hmm. Is there any way to determine entity in world space, just by coordinates?

without physics package

And without going through literally all entities

Basically, I want to determine what I click on

[deft ridge]

what version in your DOTS

in 0.50, you can change them to preview in SceneView

[rustic rain]

nnnah, I mean during runtime

my game has no place to 3D physics
It's tycoon kind of strategy

but I still need colliders I guess

[deft ridge]

wath caculating point to point distance by circle range

just simple circle range colliders

[rustic rain]

well, I guess for that I'd need to have my own colliders system

and while keeping it that simple I'd only have circles

I wonder what would be better solution

have companian game objects with 2D colliders

or install physics package

just for this

[deft ridge]

emmmm

https://github.com/Jellybit/UnityAABB

this was simple one

[rustic rain]

well, for one

this is 8 yo obsolete system kek

which is already simply replaced with 2D physics

[deft ridge]

has otehr case

[rustic rain]

of Unity

and what's more importantly

I am choosing between all entity

and hybrid here

so doesn't matter what kind of GO system I choose

it's still GO system

compared to all Entity system

but I do lean towards GO

seems just easier

when it comes to player controls

[deft ridge]

I think pure ecs can get better performance in Entity system

you hybrid

E.g: pure ecs physics

and your gameObject Anchoring there data

[rotund token]

https://medium.com/@philippchristoph/pixel-perfect-selection-using-shaders-16070d3094d

based off graphics repo updates, i believe a similar technique is actually how unity is adding picking in 1.0 in the editor

[rustic rain]

huh

[rotund token]

(this could change or be wildly off)

but yeah it's a very interesting technique

gives you pixel perfect selection

that said, this is for selecting objects in world from view port i guess not world space

if you want to have entities find another entity in world space well yeah going to need some type of spatial map

whether it's physics our you roll your own system

[rustic rain]

nah, this is it pretty much

since I need it for player selection

with mouse

[rotund token]

yeah then this type of technique is great

the tldr anything you want to be selectable you render everything with a slightly different color in an extra pass

then you can just read the texture, get the color at the point and look up what entity it is

i had an implementation of this way way back in entities

[rustic rain]

basically you mean

I use ID of entity

as color

[rotund token]

yeah effectively

that works

[rustic rain]

hm

[rotund token]

https://www.youtube.com/watch?v=Y7-AYa5s2QQ

how i did it in this demo way way ago i think

[rustic rain]

I had troubles implementing custom rendering before tho

[rotund token]

the issue is the article i linked

works out of box for old render pipeline

with a simple replacement shader

from memory this doesn't really work with new srp

you would need to add a custom pass or something

(i'm not super familiar with it)

[rustic rain]

yeah, sounds like pain

So, could you elaborate a bit more about this:
Basically I render another screen with unique colors
Then when I need, I simply read pixel I want with color

is that it?

that rendered screen doesn't have to be render player sees?

[rotund token]

yeah

you just render it to a texture

then read the pixel at a point

and that's the entity

pretty much imagine how you look at a screen and determine different objects

its working that way!

[rustic rain]

yeah, sounds fun ngl

Technical part is scary tho

I have no idea how to do anything about it

xD

Yeah, here's the first problem
In order to add new render pass, you'd need to change game objects that are rendered

meaning Hybrid renderer is not really allowing me to do that

to be precise, you need to select object layer

for which it will be applied

Hmmmm

[deft ridge]

Wouldn't it be that the larger the resolution, the greater the amount of computation

[rustic rain]

Can I somehow change internal class CompanionLink to public?

[rustic rain]

it will

but it's GPU side computation

which is not a bottleneck at all

[rotund token]

there's no reason to make it public, just access it internally

that said, why

[rustic rain]

how?

oh, you mean use Entities namespace?

[rotund token]

#archived-dots message

[rustic rain]

wait, is that even a thing in C#?

[rotund token]

you can give yourself internal access to any package

that said, i'm pretty sure you will regret touching this specific component 😅

[rustic rain]

well, all I want it for

is to simply add new components to same game objects

without creating new systems

and even more game objects

Idk tho, how changing layers can affect it

[rustic rain]

soo, regarding this
I'm confused

[rotund token]

on step 1 or 2

[rustic rain]

step 1

is that one?

[rotund token]

yes

put that in an empty directory

put your AssemblyInfo.cs in that same directory

done

[rustic rain]

Do I need this?

[rotund token]

yes

[rustic rain]

hmm, I'm not sure I'm following step 2

[rotund token]

SpaceTycoon is the name of whatever assembly of yours you want to give internal access to

that assembly will then be able to access any internal members of the Unity.Entities package

effectively what you've done is inject this AssemblyInfo.cs file into the Unity.Entities assembly

and InternalVisibleTo simply provides your assembly internal access

you could inject anything you wanted

[rustic rain]

I don't have any assembly reference yet

for my namespace

[rotund token]

sorry what do you mean

[rustic rain]

Do I need it?

[rotund token]

assembly reference points to the assembly you're injecting into i.e. unity.entities

you don't need to inject into your own assembly you have access to just put files into it ^_^'

[rustic rain]

oh, you mean I need to put files with ECS system

into this folder and add this code into it?

[rotund token]

this is the only thing in the folder

you create

you're done

whatever assembly you put in the asmref file will be accessible by whatever assembly you put in the assemblyinfo file

[rustic rain]

ok, so I did this.
That code with unaccesible is inside PlayerController folder

[rotund token]

well your issue is

companion link isn't in unity.entities

it's in unity.entities.hybrid

[rustic rain]

oh

it still is

[rotund token]

dont know what to say

works fine

if you tab into unity

do you actually get a compile error

(i.e. is it just visual studios being dumb)

[rustic rain]

yep

maybe I just need assembly definition?

[rotund token]

wait

is your code not in SpaceTycoon asmdef?

because yes you need an amsdef

[rustic rain]

nah, I didn't have one

[rotund token]

didn't know people still did this 😐

[rustic rain]

I mean, it's literally just 5 scrips in a folder

kek

seems way easier to manage for low scale project

[rotund token]

i have asmdef with 1 script

what you need is my 1 click asmdef builder 😄

auto reference and setup

[rustic rain]

wha

how can I get it

doing this is literally the only thing that bothers me lol

[rotund token]

all yours for the low price of $449/month for 4 months

[rustic rain]

🙃

[rotund token]

this was one of the first things i wrote

when i started using asmdef

because ah yeah

on about the third i setup i said fuck it

look at how many asmdef i have

screw setting that up by hand!

[rustic rain]

yep

[rotund token]

anyway releasing core library soon~ and its in there

but if you really want it now

[rustic rain]

thank you, sir

[rustic rain]

hmmm, I'm having trouble installing it
I dropped it into folder like this. But I don't get any additional menus anywhere.

[rotund token]

        private static void ShowWindow()```

if it's compiling

it should exist there