Do you know of any bugs that would ONLY

[hallow basin]

Not directly, but IL2CPP had a regression in 2022.2.0b8 where the size of components with BlobAssetReferences in them could be miscalculated and thus you could crash in deserialization and/or potentially later by reading what looks like a garbage blob asset data pointer.

This bug has been fixed and should be in 0b10 (but I'm not 100% that release, it could be in b11 as the fix landed really close to the cut-off)

[gritty pike]

So this is a Hello World project. One GameObject, one SubScene and one "GameObject" in the subscene... That wouldn't hit this bug would it? Also, do you know anyone who has made a Quest 2 build with 0.5, 0.51, or 1.0?

No systems at all...

[hallow basin]

I see, you're not using DOTS at all in this case -- it's just a crash using il2cpp?

[gritty pike]

Well, yes. The "GameObject" in the DOTS subscene (1.0 in this case) is generating an entity that gets rendered using Entities Graphics... In fact it only crashes when the entity gets spawned...

[hallow basin]

Do you have a player.log for the crash or a callstack?

[gritty pike]

No callstack since the crash only happens on device. Not sure where the player.log is...

Just showing the editor set up for DOTS...

[gritty pike]

Is it related to Android Logcat?

[hallow basin]

Yes: https://docs.unity3d.com/Manual/android-debugging-on-an-android-device.html#view-android-logs

[fickle zinc]

we also know about major issues where required stuff gets stripped in il2cpp, and i could imagine stuff crashing for that reason

[gritty pike]

OK, yes I did a logcat last night but forgot to capture it.
I'll get it now...

[gritty pike]

I'm sure it's related to the 'entities graphics' lib because it crashes as soon as the entity get's spawned and there is a render function error. Getting it now...

Got the logcat but too big to send in discord...

Here it is:

[fickle zinc]

seems like that has no symbols, but we know about sufficiently many issues in il2cpp in this first version that i am honestly not surprised. i would personally wait for a patch release before making il2cpp builds.

[hallow basin]

One thing you could ensure in the meanwhile is that IL2CPP stripping is set to minimal

[gritty pike]

Unfortunately I need to do a stress test of DOTS on Quest 2 for a client so I have to have IL2CPP for arm64...

No symbols...oops...

[gritty pike]

would setting to minimal avoid the crash possibly?

[fickle zinc]

helps with some crashes and not others

[gritty pike]

Any ballpark time frame on this?

[gritty pike]

let me try...

[hallow basin]

The error looks more like it's due to some corruption though. Hard to tell without debug symbol names to know if this crash is in scene loading or blob asset code etc... But strangely the faulting address isn't presented so it's hard to know if this is dreferencing null (which could be done via an exception trying to fail/kill the app) but it could also be due to misinterpreting a data as a ptr and jumping to invalid memory causing the SEGV_MAPPERR

[gritty pike]

My settings:

Am I missing symbols because C++ is set to release?

Or do I maybe need Script Debugging turned on?

[fickle zinc]

i doubt it. kind of seems like a separate android bug that symbols are missing from the stacktrace? script debugging is not related

anyway, we aren't allowed to say anything about when because we're just always wrong about that, but i can tell you what needs to happen before you can have all the fixes we know about for the patch release:

• we have to land all the fixes that we know about in the dots repository for ones that need to be fixed there;
• we have to land (or wait for other teams to land) all the fixes that we know about in the main unity repository
• we need to backport the dots fixes to the dots release branch
• we need ourselves or other teams to backport the main unity fixes to the 22.2 branch
• we need the release management team to release another beta for 22.2 after all our backports have landed
• we need to coordinate with our own dots release people, who work on announcements and stuff, and who i believe only release on tuesdays, once all our dots-side backports have landed
• if we find more major issues after all this has happened, we have to decide whether to hold the release to get those new fixes or send this one out and start the cycle again for the next set of fixes

[fallen vine]

hehe, this should be pinned because I was often wondering why something simple can take quite a while. 😄 this perfectly explains it

i could get dizzy just from reading

[gritty pike]

😱

[hallow basin]

The stripping setting is this one:

[gritty pike]

This?

[gritty pike]

Yes, set to minimal...

[fickle zinc]

don't think that should be related

[gritty pike]

😦

I can't tell, does this have to symbols your looking for?

[hallow basin]

Thanks much better. Looks like it is indeed the recent IL2CPP regression that happened in 2022.2.0b8

same one mentioned here: #archived-dots message

This is not specific to Quest. It's an IL2CPP issue for all platforms

[gritty pike]

Cool! But I had the same crash, Quest Build Only, on 0.51.1...

[gritty pike]

I only build IL2CPP on Quest 2!

[hallow basin]

Yes, the regression was pushed to all 202x.x LTS versions

I imagine your testing on 0.51 was using 2021.3.9f1 or later

[gritty pike]

Great news!
At least I (Probably) know what the issue is!

[gritty pike]

Yep!

[hallow basin]

If you're blocked, one last thing you could try (it's a stetch) is using 2022.2.0b7 That version should not have the IL2CPP issue. However I do know there are other fixes for DOTS 1.0 specifically in 2022.2.0b8 so you might still run into issues but you might not 🙂 🤞

https://unity3d.com/unity/beta/2022.2.0b7

[gritty pike]

Thanks!!! I'll check these out when I'm able.

I have to go, but thanks for ALL you help!!

[hallow basin]

No problemo. Take care 🙂

[gritty pike]

And you 🙂

[gritty pike]

Well...2022.2.0b7 didn't crash but it also doesn't seem compatible with the 'entities graphics 1.0-exp.8' 😦
Here is the last of the logcat:

Note the Error after the IL2CPP... (line 23)

Maybe this is the bug they were trying to fix when the new bug got introduced...

[gritty pike]

Should I be able to build 0.51 with 2021.2.2.f1? Doesn't seem to be working...

[hallow basin]

You need to use 2021.3.4+ when using DOTS 0.51 (although I know 2021.3.9f1 contains the same il2cpp regression, so you'd want to try 2021.3.8f1)
https://forum.unity.com/threads/experimental-entities-0-51-is-available.1281233/
https://unity3d.com/unity/whats-new/2021.3.8

Regarding

ArgumentException: Cannot find TypeIndex for type hash 7881543491081840453. Check in the debug file ExportedTypes.log of your project Logs folder (<projectName>/Logs) the corresponding Component type name for the type hash 7881543491081840453. And ensure your runtime depends on all assemblies defining the Component types your data uses.```
You should look at the log file to see what that hash matches to. I suspect you built to an directory with a build from 2022 (or some other version of Unity) and the `.entities` asset files were re-used instead of regenerated. This error happens when you load components that don't exist in your game so we won't deserialize them since we will misinterpret the data. I suspect your .entities data wasn't built / stale files are being used.

[gritty pike]

That does seem probable. In fact I think I switched Unity versions down a couple of notches and rebuilt without deleting the Library dir... DOTS seems especially sensitive to this...

I'm off tomorrow so I should have time to try again. Though I went back to some earlier projects and they seem to have mysteriously broken. Maybe an update to DOTS broke the older projects?

[gritty pike]

2021.3.8f1 doesn't crash but it doesn't render entities either...
Here's the logcat:

The errors are near the end...

[gritty pike]

My Quest 2 Dots 0.51.1 build worked on 2021.3.8f1!
I finally have a working Build!

Thanks @hallow basin!!!!

[crisp mango]

This was discussed a bit elsewhere, is the regression in question responsible for this or something different? I'm on 2021.3.6f1 which seems to be before this?
https://forum.unity.com/threads/il2cpp-crash-with-entities-0-51.1311858/

[gritty pike]

I'm about %99.9999 certain it's the regression at this point. The evidence is VERY STRONG.
Try a 2021.3.8f1 Hello World with DOTS 0.51 and see if it works. I bet it will...

I think yours is another issue...

I tried 2021.3.7 and it didn't work either...

Or, maybe the regression missed 2021.3.8f1. Not sure...

[crisp mango]

Yeah I think so too, a hello world works fine, everything's fine until a shared component is written to in an ecb

[gritty pike]

Ah...I have a super simple system. Just moves a sphere around...

[crisp mango]

Couldn't even repro it in a clean project, no idea what triggers it, and it's only on arm IL2CPP builds

[gritty pike]

When you say Hello World, are you using any systems at all?

[crisp mango]

I'll try 2021.3.8f1 when I get a chance, but I think it's a different issue

[gritty pike]

Yah, this is the first IL2CPP I have ever successfully made. Even just to render. But this has a simple system even.

[gritty pike]

For you it may be. But maybe not

As yours is no doubt much more complicated and bringing in more possibilities...

[crisp mango]

My "hello world" was just a new scene in the project with all the systems running normally, so yes systems work

[gritty pike]

Strange, I wonder why my 2021.3.7 didn't work...
Oh well, I guess I should like actually go to sleep now. It's a bit late... 🙂

[crisp mango]

Yeah, go :D I'll wait and see if kevin has any insights

[gritty pike]

Alright, have a good night! 🙂

[hallow basin]

Great to hear 🙂

[hallow basin]

I don't think I'd seen this thread before. Looking at your crash the callstack says we are trying to dispose an ECB and it's crashing when freeing the internally allocated managed chain

Looking at the code in entities, it looks like there is a bug to me where we are grabbing a managed heap pointer however we are not pinning the memory. This means the Garbage Collector is allowed to move the memory which would be a problem here since we could start accessing bad memory.

You could try the following:

• Make a local copy of entities and refer to that local copy in your manfiest.json for your project
• Make the following edit to the com.unity.entities\Unity.Entities\EntityCommandBuffer.cs

        internal static void AddEntityComponentCommandFromMainThread<T>(EntityCommandBufferData* ecbd, int sortKey, ECBCommand op, Entity e, T component) where T : class, IComponentData
        {
            var typeIndex = TypeManager.GetTypeIndex<T>();
            var sizeNeeded = EntityCommandBufferData.Align(sizeof(EntityManagedComponentCommand), 8);

            var chain = &ecbd->m_MainThreadChain;
            ecbd->ResetCommandBatching(chain);
            var data = (EntityManagedComponentCommand*)ecbd->Reserve(chain, sortKey, sizeNeeded);

            data->Header.Header.CommandType = op;
            data->Header.Header.TotalSize = sizeNeeded;
            data->Header.Header.SortKey = chain->m_LastSortKey;
            data->Header.Entity = e;
            data->Header.IdentityIndex = 0;
            data->Header.BatchCount = 1;
            data->ComponentTypeIndex = typeIndex;

            if (component != null)
            {
                data->GCNode.BoxedObject = GCHandle.Alloc(component);
                // We need to store all GCHandles on a cleanup list so we can dispose them later, regardless of if playback occurs or not.
                data->GCNode.Prev = chain->m_Cleanup->CleanupList;
                chain->m_Cleanup->CleanupList = &(data->GCNode);
            }
            else
            {
                data->GCNode.BoxedObject = new GCHandle();
            }
        }

should be changed to:

        internal static void AddEntityComponentCommandFromMainThread<T>(EntityCommandBufferData* ecbd, int sortKey, ECBCommand op, Entity e, T component) where T : class, IComponentData
        {
            var typeIndex = TypeManager.GetTypeIndex<T>();
            var sizeNeeded = EntityCommandBufferData.Align(sizeof(EntityManagedComponentCommand), 8);

            var chain = &ecbd->m_MainThreadChain;
            ecbd->ResetCommandBatching(chain);
            var data = (EntityManagedComponentCommand*)ecbd->Reserve(chain, sortKey, sizeNeeded);

            data->Header.Header.CommandType = op;
            data->Header.Header.TotalSize = sizeNeeded;
            data->Header.Header.SortKey = chain->m_LastSortKey;
            data->Header.Entity = e;
            data->Header.IdentityIndex = 0;
            data->Header.BatchCount = 1;
            data->ComponentTypeIndex = typeIndex;

            if (component != null)
            {
                data->GCNode.BoxedObject = GCHandle.Alloc(component, GCHandleType.Pinned); /// <------- changed line

                // We need to store all GCHandles on a cleanup list so we can dispose them later, regardless of if playback occurs or not.
                data->GCNode.Prev = chain->m_Cleanup->CleanupList;
                chain->m_Cleanup->CleanupList = &(data->GCNode);
            }
            else
            {
                data->GCNode.BoxedObject = new GCHandle();
            }
        }

In fact, you likely need to change that for all the GCHandle.Allocs done in that file

[crisp mango]

This is promising! Thanks for taking a look at it, I'm not certain on the viability of updating this project to Entities 1.0 for future fixes from there, the possibility of a local fix for 0.51 is great news :)

[hallow basin]

Let me know if this works out for you -- I can see this hasn't been fixed in 1.0 but I can fix that for our next release as the code is wrong as is. I suspect you see this because on android you likely have much less memory available than desktop machines so the GC will be more pressured to repack memory as allocations start building up

[crisp mango]

Hmm it crashed on the first frame with a single entity, so pretty light allocations I'd think, but you never know. Hope I get the time to try this out soon!

I think I was loading a few meshes and stuff in to memory aside from that one entity, so that might be pretty heavy, the Quest is a paperweight after all

[hallow basin]

Hmm that is curious then. The other possibility is something is corrupting the ecb (or it's being double disposed) as the crash is due to a bad memory access caused by trying to free what looks like a old GCHandle but could even be a corrupt one.

[crisp mango]

Why does it only manifest with IL2CPP, not with Mono?

Am I incorrect assuming they use the same underlying GC?

That might be worded incorrectly, I'm sure there's a plethora of differences, and that's exactly what I'm trying to understand 😅 How does the same allocation/dispose C# code work on Mono but not on IL2CPP

[hallow basin]

While I'm not certain this bug is the cause of your crash, this bug of not pinning GC memory will likely fail in any GC implementation. I'm not certain that mono and IL2CPP use the same GC implementation (but they might). However if they do use the same one, a crash caused by the GC moving some memory that should have been pinned relies on many factors to be deterministic so I wouldn't be surprised by mono working and il2cpp breaking. This bug should be causing issues on all platforms in mono but as far as I know it hasn't actually been reported yet

[crisp mango]

Hmm fair enough, makes me think double dispose and the like are ruled out if those should fail in Mono too. I'll still try the fix and see if anything changes. Good thing I don't absolutely require IL2CPP, would just be nice :)