Developer Experience with Burst-Compatible C-Sharp

[late sedge]

We're finding Unity ECS, DOTS and Burst to be really useful and crucial to our current project (Thank you! 😁). One issue we've been facing is that occasionally we encounter memory corruption issues that take considerable time to diagnose and fix. At its worst, there are concerns that burst utilises C# in ways it was not designed for, and as such the language lacks the information/awareness to aid the developer in avoiding certain unsafe behaviours. In some ways it is actually less safe than modern C++ for which common practices & tooling exists, preventing such issues.

An example is the Colliders in Unity Physics, which use a form of native, unsafe polymorphism (casting pointers to memory). One of the issues we had was tracking down misbehaving raycasts, only to eventually find that we had omitted a ref keyword from one local variable, resulting in a copy-by-value of a Collider type (not the CapsuleCollider that was actually stored at that memory location), resulting in a partial-copy. This then led to code accessing this "truncated" value in memory as if it was a full CapsuleCollider, at which point it accessed the random values adjacent to it in memory, causing nonsense results. We have encountered this particular (missing ref keyword) issue twice in our project, both times causing a fair amount of angst to debug.

We're curious to know what Unity's long-term plans are for mitigating this, to reduce the "foot-gun" nature of Burst-compatible C#? Tooling for diagnosing memory corruption? Better static analysis? I'm curious to know what Unity's vision for a glorious burst-compiled future is! 😛

[compact basin]

Just noting that the terminology for burst-compatible C# is High-Performance C# or HPC#
(https://blog.unity.com/technology/on-dots-c-c). Great question though, I'm looking forward to the answer, maybe the upgrade to newer c# and dotnet versions might give some features that help.

[late sedge]

@compact basin If I recall correctly, Unity has an ongoing initiative to migrate toward using a more mainstream .net runtime (I don't recall exactly, perhaps .net core?). I wonder where Burst/HPC# fits into the picture when that comes to pass, as I believe there is Microsoft + Open Source backed ahead-of-time (AOT) compilation of C# functionality there already. Perhaps one day it could replace Unity's proprietary IL2CPP/burst code? Unfortunately, even if that was a path forwards, it wouldn't actually help with these corruption issues very much. I don't think there are a great deal of C# features in the pipeline that stand to actively help with this stuff, although obviously it would be good if I'm wrong!

[compact basin]

They are moving towards .NET 7/8, and C# 10/11+ will come with it. It's less .NET that I imagine being helpful, but the keywords/modifiers newer versions of C# have been adding that have allowed for some things to be enforced that couldn't be previously. Not saying I know they'll help, as I've honestly not yet dived into ECS proper, but stuff like this: https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/proposals/csharp-11.0/low-level-struct-improvements seems to me to be relevant to improving HPC#

[late sedge]

@compact basin Aye, I figured that's what you meant - I look forward to hearing Unity's thoughts! 😄

Oops, thinking about it - the fact they are named refs in C# should've been a clue to me - but they're obviously analogous to C++'s references, which I think actually do have the same issue. So I'd just like to retcon some of what I said above, as it is uncharitable of me to suggest HPC# compares unfavourably to C++ in terms of safety! Apologies!

It's still very scary that a missing ref keyword can cause memory corruption in C#, though, so still curious to hear Unity's dreams for the future 😛

[compact basin]

Most upvotes, least answers 😛

[late sedge]

I'm still remaining hopeful for now, as I believe they said elsewhere that their intention was to answer all questions from the blitz day, but not necessarily answer them all during the blitz day, so 🤞

I appreciate it's a tough question to answer, but also I could do with some reassurance from Unity that this will get better 😅

[next prawn]

Thanks for asking this question and sorry for the delay in answering. Questions about long-term plans are always the hardest to answer because we talking about features basically until the release is already cooked and almost out the door.

This is a big question. Here's my attempt to summarize and comment on the points below.

Memory corruption errors are more frequent in HPC# (High Performance C#) than normal C#, and the tooling to diagnose and fix these errors is less mature than C++, leading to a worst-of-both worlds scenario.

We have seen this from early adopters. Working as we do with an existing language and toolchain, there are sometimes seams in language support and DOTS features that we must address after the fact. We intend to improve static analysis using Rosyln analyzers (catching things that we can only do in IL PostProcessors now, which are difficult enough to write and slow enough to run that we don't do nearly as much as we could) as well as simple things like improved preprocessor-style defines to validate bounds checking across targets. We currently use ILPPs to ensure correct ref handling for blob assets for example.

There are longer term ideas for isolating player memory for better protection but they are extremely extremely far off.

[late sedge]

Thanks for the reply, I appreciate it was a tough question to answer!

This issue affects us quite strongly as for various reasons we're developing our game almost entirely within the DOTS ECS ecosystem of tooling, with the occasional dip into Game Objects for presentation (UI & general rendering). This makes us quite sensitive to any workflow issues or burst compiler bugs, etc!