Awaitables and Jobs, awaiting hangs the program

[tired girder]

I have the following code for a Job:
https://ghostbin.axel.org/paste/v8nwb

I am scheduling and Awaiting it in the following code:

            var metadataBlobResults = new NativeList<BlobAssetReference<ContentPackMetadataBlob>>(allDirectories.Count, Allocator.TempJob);

            var jobHandle = ContentLoadJobs.ScheduleMetadataBlobsFromPetitionDirectoryPathAsync(
                ref allDirectories, ref allDirectoriesPetitions, ref metadataBlobResults,
                out var inputDirectories, out var inputPetitions, out var errors);

            while (!jobHandle.IsCompleted)
                await Awaitable.NextFrameAsync();

This in turn is also in another Awaitable (thats why of the Awaitable.NextFrameAsync(), which gets called like so: ```cs
var loadTask = ContentLoadAsyncOrchestrator.LoadContentAsync(petitions, Manager);
while (!loadTask.GetAwaiter().IsCompleted)
World.Update();

I'm running and debugging the code through tests, and the issue is that it hangs permanently and I have to kill Unity's process. Debugging and putting a breakpoint in the jobHandle / await Awaitable.NextFrameAsync(); shows that it only gets hit once. 

I've put breakpoints inside the job itself and the exception block never gets hit either, so it might not be a "exception gets swallowed" case as no exception gets thrown.

No breaks due to exception happen and no log messages appear in console.

Putting a breakpoint at the Test's class ``World.Update()`` shows that the World is properly getting updated, getting hit multiple times, so the code is still running.

I have absolutely no idea what is making the code hang, I previously had another step before this one where I `await` a Task for reading files async and that hanged as well, but as that task was meaningless I just removed it. This is to say that I believe there's something wrong with `await` and how I'm scheduling it. The Editor hangs as well so I cannot use the job profiler to see what's going on.

[tulip notch]

ContentLoadJobs.ScheduleMetadataBlobsFromPetitionDirectoryPathAsync
This doesn't seem to be any normal unity job..?
Didn't realize you put the job code in the pastebin.

var loadTask = ContentLoadAsyncOrchestrator.LoadContentAsync(petitions, Manager);
while (!loadTask.GetAwaiter().IsCompleted)
  World.Update();

It would also help if you share all the relevant code. Or at least the method this is running in. Including it's signature.

[tired girder]

https://ghostbin.axel.org/paste/2cnok Here's the entire test fixture, including the base class

and here's the Awaitables https://ghostbin.axel.org/paste/cxv3x

Its an awaitable awaiting another awaitable

its complicated

[tulip notch]

LoadSimpleJson_ComponentsHaveCorrectData is a sync method, so you basically have an infinite loop:

while (!loadTask.GetAwaiter().IsCompleted)
{
    World.Update();
}

I think the main thread needs to be able to run to sync jobs.

[tired girder]

Did I understand multithreading wrong? I thought the sync (this, main thread) would just check the boolean meanwhile the Awaitable just loads in a secondary thread

[tulip notch]

First, you're confusing async/await and jobs. Async/await is single threaded in unity by default.

Jobs are multithreaded, but they likely need to sync their results to main thread at certain point in the update loop.

[tired girder]

They do yes, by the end when they have to move Entities from the Staging World to the main World

[tulip notch]

If it was a simple background thread toggling IsCompleted it would have worked, but unity jobs system is a lot more complicated and needs to take care of a lot of stuff, so it's no guaranteed that the state of the job would be set to Completed until the next frame(or later in this frame)

Basically, you need to let unity run.

[tired girder]

By debugging the break point of the while on the jobHandle does get hit earlier than the code running the job does

If that's what you mean

[tulip notch]

I'm not sure what you're trying to say with that.

[tired girder]

Agreeing with what you say "later in this frame", the job does run later in the frame

[tulip notch]

It doesn't matter when the job runs. What matters is when that property is set to true. And that might never happen since you're blocking unity from doing that

[tired girder]

So because the test method itself is not async it doesnt matter that I'm using Awaitable and async methods, its sync and it runs on the same thread which I am blocking with an infinite loop?

[tulip notch]

Yep.

[tired girder]

That's bad as I wanted to execute this from a System, which are unable to be made async

Like the idea of the loop was because of that

[tulip notch]

Well, in a system, you'd just put it in it's update loop(without your while loop)

void Update()
{
  if(!loadTask.GetAwaiter().IsCompleted)
  {
      World.Update();
  }
}

[tired girder]

Right something like this I had

[tulip notch]

Aah, that might block too

If it's a Task

If it's Awaitable, it should be fine

It doesn't seem like you're even supposed to use GetAwaiter

This method is intended for compiler use rather than use directly in code.

https://learn.microsoft.com/en-us/dotnet/api/system.threading.tasks.task.getawaiter?view=net-10.0

Task.IsCompleted should work

https://learn.microsoft.com/en-us/dotnet/api/system.threading.tasks.task.iscompleted?view=net-10.0#system-threading-tasks-task-iscompleted

[tired girder]

I'm using the generic Awaitable so it only has Cancel https://docs.unity3d.com/6000.5/Documentation/ScriptReference/Awaitable_1.html

If I were using Awaitable as is it does have a IsCompleted

https://docs.unity3d.com/6000.5/Documentation/ScriptReference/Awaitable.html

[tulip notch]

Well, I don't see it having GetAwaiter method

[tulip notch]

Yeah, well the questions is what is it? It could be the same thing as Task awaiter that you shouldn't use

If it's not documented better not touch it

[tired girder]

It actually has the ExcludeFromDocs attribute on purpose even

[tulip notch]

Yep. You should be able to use Awaitable.IsCompleted just fine, according to the docs.

[tired girder]

The Awaiter it returns with GetAwaiter is a Unity Awaitable

though

Man this is complicated

[tulip notch]

I don't see why that would matter at all.

It's excluded from docs, and a parallel implementation in C# explicitely says not to use it

[civic minnow]

For the most part, you should never use GetAwaiter yourself https://learn.microsoft.com/en-us/dotnet/api/system.threading.tasks.task.getawaiter?view=net-10.0

Remarks
This method is intended for compiler use rather than use directly in code.

If I'm understanding your goal correctly:

• You have a LoadContentAsync which internally uses the job system to do some wrok, and returns a task when it's done. To return a task, you check for job's completion frame by frame.
• You have a caller of LoadContentAsync, that wants to do something every frame while the task is ongoing.

There are two ways you can approach it:

• If you are okay with changing LoadContentAsync, then simply make it return the job handle rather than a task, then the caller can just do the frame by frame check itself.
• If changing LoadContentAsync is not acceptable and it must return a task, then the caller can run also run a frame by frame check on the returned task and perform logic if it's not completed yet.

[wintry glacier]

rather than touch the awaiter i think something simple like this should work for your case?

    async void StartDoingThing() {
        await DoThingAsync();
        m_doneThing = true;
    }
    
    void Update() {
        if (!m_doneThing) {
            // whatever
        }
    }

[civic minnow]

I don't use Unity Awaitable (also would recommend UniTask instead), but it seems like the issue you are running into is that Awaitable<T> does not have an IsCompleted property which honestly seems weird to me, but there are plenty other ways you can approach it.
Using a GO's Update like others have pointed out above is one way, but that might be problematic since now an execution must also spawn a GO to accompany it.

[wintry glacier]

i'm pretty sure it's due to the fact that awaitables are pooled, so doing anything other than awaiting them is very sketchy

a tradeoff of convenience for performance i suppose

UniTask has some of the same issue with reused tasks last i checked!

[civic minnow]

Awaitable has IsCompleted, but Awaitable<T> doesn't, so that doesn't seem to be the reason why.

[wintry glacier]

well either way, i'd advise against touching the awaiter directly 😄

[civic minnow]

If you don't want that, you can also just run another task alongside the returned task, here's how that looks like with UniTask, but you should be able to write the same with Awaitable:

var isCompleted = false;

UniTask.Void(async () =>
{
    while (!isCompleted)
    {
        DoFrameByFrameLogic();
        await UniTask.NextFrame();
    }
});

var result = await LoadContentAsync(...);
isCompleted = true;

Or use cancellation mechanism (which unfortunately Awaitable doesn't support the standard CancellationToken unlike UniTask which does, so you have to do something with .Cancel) instead of the isCompleted flag above.

[wintry glacier]

you should probably never use Cancel with awaitables, you can use a cancellation token just fine 😄

[civic minnow]

Oh it does seem like Awaitable supports CancellationToken, TIL.

(You can tell how much I don't use Awaitable 🤣 )

[tired girder]

I could change it to a Job, though my main concern is dealing with managed data (from what I've learnt, passing managed data in/out of a Job is no-go) though I think its only about whenever it can be burstable or not

In fact if you check my job I convert everything into NativeArray and NativeList when passing it in

[tired girder]

Some are jobs and some others are awaitables for this reason I've said

[civic minnow]

Well in that case you can just do the second approach, since you can't change what LoadContentAsync does which rules out approach 1.

[tired girder]

So for now I should try getting rid of the generic and try to use Awaitable directly so I can check its IsCompleted without using GetAwaiter, and the alternative is creating another task that works on content meant to be for the second task which is the one being await(ed) (?)

[civic minnow]

If you want to change LoadContentAsync to non generic then sure that would allow you to use IsCompleted, but it would also mean LoadContentAsync can't return a result anymore.

[civic minnow]

Running a parallel task to do the frame by frame logic like this is completely fine, and doesn't require messing with LoadContentAsync.

[tired girder]

That's not a big deal since I do have a "Staging Context" object where I store the result, it's just that on the first validation steps I do not have it instantiated because why should I instantiate it at the start when Im not even sure it'll get used

For example

[civic minnow]

I'd honestly go with the parallel task approach

The lack of return value is what makes coroutines suck compared to tasks.

[tired girder]

Also yeah this is just for a test suite, in practice it'll be put in a System where it'll have a Update method

[tired girder]

Though the method that spawns it having to be async worries me because you can't make System methods async

Schedule jobs

All system events run on the main thread. It's best practice to use the OnUpdate method to schedule jobs to perform most of the work. To schedule a job from a system, use one of the following:

IJobEntity: Iterates over component data in multiple entities, which you can reuse across systems.
IJobChunk: Iterates over data by archetype chunk.

[civic minnow]

Well, more reason to not bend your implementation code for testing code 😄

[tired girder]

The alternative would be creating the system and running it there as the real implementation so maybe I should do that

Its just that this is so ridiculously large I want tests so any changes that break it are instantly noticed

And also I actually haven't run it on its own, I finished programming it and went to test with a test suite directly

If i were to turn it into an awaitable without the generic I think there's potential for me to turn it into a Job, but I have no idea if Job -> awaitables inside would even work

[quick coral]

I want to point out you can use threads yourself and therefore use managed types as normal. You are free to use async there too and even Task.

And again I'll point out UniTask is just better Vs awaitable if you want a full Task replacement

[tired girder]

Okay yeah I'm using UniTask now, it does seem better.
Also changed the tests to Async so it no longer hangs the main thread
Still having an issue with awaiting, though

var metadataBlobResults = new NativeList<BlobAssetReference<ContentPackMetadataBlob>>(allDirectories.Count, Allocator.TempJob);

var jobHandle = ContentLoadJobs.ScheduleMetadataBlobsFromPetitionDirectoryPathAsync(
ref allDirectories, ref allDirectoriesPetitions, ref metadataBlobResults,
                out var inputDirectories, out var inputPetitions, out var errors);
            
await jobHandle.WaitAsync(PlayerLoopTiming.Update);

the job somehow takes longer than 4 frames and doesn't complete itself until I stop it from running, this makes the native list get deallocated because it lasted longer than 4 frames, and so everything inside the job that uses allocations gets leaked for lasting more than 4 frames

And i'm pretty much confused

wait

You can only call Schedule from the main thread

So i'm doing it all wrong? This is not the main thread after all

[wintry glacier]

if you want to write async code that deals explicitly with threads in any way, including starting from threads other than the main thread, the job system is probably not what you want

[tired girder]

Yeah I just wrote with a job because its what you use in ECS systems usually

I see yeah

Haven't written async code ever other than a couple of couroutines and stuff you see

[wintry glacier]

it should end up somewhat simpler than the equivalent job system code because you don't have to deal with any of the restrictions or converting the data, but in exchange you don't get any builtin safety checks

[quick coral]

Using UniTask you can await the job handle directly so I'd advise you do that instead of a custom attempt

They have many extension methods for this as well as others that allow conversion to a UniTask.

UniTask also has functions to run code on worker threads so these may be the way to go

[tulip notch]

Is this a play mode test?
4 frames for a job is really a lot of time. Unless it's an extremely heavy job. At which point I'll consider breaking it down or not using jobs at all.

[tired girder]

This is indeed a playmode test

But there's something really wrong I am doing

For example now I am doing ```cs
await UniTask.RunOnThreadPool(() => ContentLoadSystemUtils.GetMetadataBlobsInTopologicalDagOrder(
ref sortedMetadata, ref sortedIds, Allocator.TempJob));

And this does work correctly, but never continues executing code until it times out or I cancel it

So idk if the previous Job issue I had is also the same here

[tired girder]

jobhandle.WaitAsync() is a UniTask extension method

But again I think the issue there wasnt the awaiting but the scheduling as again documentation says that schedule can only be called in the main thread

[quick coral]

Yes I know that read my message again 🤣

It should have thrown an exception if this was the case and it's possible for this to not be logged to the unity console

[tired girder]

well I'm debugging and Rider didn't break due to exceptions

one reason might be because I'm using NUnit tests and the test itself is using a Task...

The call chain is:
Test (async Task)
-> awaits LoadContentAsync (returns UniTask<T>)
-> awaits ExecuteStep2... (returns UniTask)
-> awaits UniTask.RunOnThreadPool(...)

and I don't know if it has to do something with unity and UniTask given that this isn't a UnityTest

After all this is what they say about unit testing

and UniTask does use the player loop for updating

[quick coral]

UniTask works fine in edit mode and I have used it with unity tests as they show fine.
You can use the unitask tracker to see what tasks never "finish" as well.

The task type should not matter either but its easier for exceptions to go un noticed with Task

UniTask offers .Forget() to counter this issue

[tired girder]

Do you have them as NUnits or actual UnityTest

[quick coral]

Tbh we can just do what unity say and use Task because in edit mode we dont care
https://docs.unity3d.com/6000.2/Documentation/Manual/test-framework/reference-async-tests.html

Your shared code can still use UniTask but the test function can use Task (i forget if it will accept UniTask or Awaitable too)

and ofc you can still utilise unitask apis within the test function itself

[tired girder]

Okay it seems it works correctly in playmode testing too

then I am doing something wrong

[quick coral]

Task works just fine its just not a good choice for unity hence UniTask + Awaitable

anyway hopefully you can debug things a bit more

[tired girder]

Oh yeah there they are

This doesn't give me much information though? I do know that its the run on thread pool await hanging right now

like I already knew that

Like I can remove the run on thread pool as its instantly awaited there because its a single method call but that's kicking the ball down as I have no idea whats still happening

[tulip notch]

What happens when you step through the code in the debugger?

Does it still use unity jobs? Becaus I'm not sure you can use start them from a background thread.

[tired girder]

It just runs as normal, then UniTask runs await UniTask.Yield();

         /// <summary>Run action on the threadPool and return to main thread if configureAwait = true.</summary>```

maybe this is the issue right now, by default RunOnThreadPool has configureAwait true so its expecting to return to the main thread, when im not running on the main thread

[tulip notch]

Or your main thread is blocked for some reason. If it stop on the yield, you should check what other threads are doing at that time. Especially the main thread.

[tired girder]

It stops on yield but I can continue stepping over

as in, it totally shouldnt be blocked

the editor is also responsive, so its totally not blocked. Shouldn't be

Anyways I changed configureawait to false and now it doesnt stay there waiting forever so that's something

[quick coral]

Make sure to use .Forget() on unawaited UniTasks as it will ensure exceptions get reported strait away (and makes your ide happy too)

[tired girder]

Can I use UniTasks inside a job?

I cannot find anything

[quick coral]

No point using unitys job system in this case its too restrictive.
Make use of the job system with burst, otherwise just use unitasks thread pool

[tired girder]

I need to use a job due to allocation restrictions

I guess i'll break up my task orchestration and schedule everything manually from the main thread

Actually UniTask has SwitchToMainThread, maybe I can do that and schedule jobs???

If I can do that that's super evil and that'll solve my problem

[quick coral]

Yea that would return you to the main unity thread so that would let you then schedule a job and wait for it

[tired girder]

Oh My God

I love UniTask now

I cannot post a gif on my thread... sob sob

[quick coral]

it actually has the features we want

it has lots of useful extensions and even provided a cancellation token for on destroy before unity did

[tired girder]

Ok I don't know how to use SwitchToMainThread because it hangs permanently after trying to await it

Maybe it never runs because I'm already in a thread? Like reading docs it seems it posts an event that requests it to be switched to the main thread

I really don't get UniTask

[quick coral]

There should also be ReturnToMainThread() which should work always

[tired girder]

Oh thats not awaitable

I'll try with that

[tulip notch]

I recommend trying to use the debugger more: placing breakpoints in different places, inspecting different threads and their callstacks. Reading the unitask source code also help to understand how it works.
Then you can easily investigate issues like "it's stuck". Or at leadt provide better/more info for the community to help you.

[tired girder]

I kinda don't really know how to read other threads when stopping with breakpoints

The most I've gotten so far is actually recognizing that the thread has stopped for whatever reason

Which at least its something and I'm progressing

[tulip notch]

Then it's a great opportunity to learn. In VS you have parallel callstacks(or something like that) and threads windows that can show you all the threads and their callstacks and you can switch between them. Vs code has a threads list with callstack, where you can switch between threads as well.

[tired girder]

There's this on Rider

Now that I see, the "thread pool worker" is showing me that a move next is not complete, so theres a task that hasn't finished yet..?

Rider also has this marker at the rightmost side of the IDE that shows which thread is it on

I think that's what it means

[tulip notch]

MoveNext just means that it's executing a coroutine or an async method line by line. To see what exactly it is doing you need to see the code. Just double click that entry in the callstack.

[tulip notch]

Is that the last frame in the callstack? What does the normal callstack window show?

[tulip notch]

If it is, then it doesn't seem to be blocked. You could probably "step in" or "step over" to step through the code.

[tired girder]

It's not blocked right now but its not behaving as expected, which is returning to the main thread

gonna undo the code change to the await, which does get blocked

[tulip notch]

What makes you think that it should return to the main thread? Where exactly does this "missbehaving" happens in the code?

[tired girder]

Calling UniTask.ReturnToMainThread()

[tulip notch]

Well, what happens in the debugger when that call is reached? As well as after that?

[tired girder]

Still on a thread pool it seems

[tulip notch]

Well, how are you using it? It seems like it's supposed to be used in a using scope.

    async UniTaskVoid Start()
    {
        try
        {
            Debug.Log($"実行前のスレッド={Thread.CurrentThread.ManagedThreadId}");

            // UniTask.ReturnToMainThread を await using する
            // ブロックを抜けたタイミングでメインスレッドに戻る
            await using (UniTask.ReturnToMainThread())
            {
                // 処理をスレッドプールへ切り替え
                await UniTask.SwitchToThreadPool();
                SampleMethod();
            }

            Debug.Log($"実行完了後のスレッド={Thread.CurrentThread.ManagedThreadId}");
        }
        catch (Exception e)
        {
            Debug.Log($"例外処理のスレッド={Thread.CurrentThread.ManagedThreadId}");
            Debug.LogException(e);
        }
    }

That's the only example I found online. It's in japanese but that doesn't matter. Point is, it's a bit confusing. The idea is that the Start method is starting on the main thread.

That's why I'm sort of against libraries like UniTask. At least for beginners. They hide a lot of what's actually happening and often have poor documentation.

[tired girder]

worked correctly right now after aborting because the debugger is not connecting to the editor, so there's probably something wrong with the await as per usual
Regarding the example I'm following

async UniTask ProcessOnThreadPool()
{
    // Start on main thread
    Debug.Log($"Thread: {Thread.CurrentThread.ManagedThreadId}");
    
    await UniTask.SwitchToThreadPool();
    // Now on thread pool
    var result = PerformHeavyComputation();
    
    await UniTask.SwitchToMainThread();
    // Back on main thread - safe to call Unity APIs
    UpdateUI(result);
}

It's also on their first example on the readme

[tulip notch]

Yep. This one is documented properly. I was talking about UniTask.ReturnToMainThread().

[tired girder]

Oh, no idea, according to its summary its

    /// Return to mainthread(same as await SwitchToMainThread) after using scope is closed.

[tulip notch]

Yes. That's why I said that it's supposed to be used with a using scope

[tired girder]

there's just probably something wrong with my awaits overall, though I am doing earlier in the code something like:

            for (var i = 0; i < contentPackIdLength; i++)
            {
                var index = i;
                tasks[i] = UniTask.RunOnThreadPool(() => CreateSingleResolvedJson(
                    context, allFilesGroups[index]), false);
            }

            var results = await UniTask.WhenAll(tasks);
``` and this one works fine

I'll just continue tomorrow, it's late

[tulip notch]

The first step is to understand(and explain) what exactly is wrong, how do we know that? Are we interpreting the issue correctly? Code execution should be the ultimate source of truth. Step through the code and make sure all the values are what you expect at the right time.

[quick coral]

I would use UniTask.RunOnThreadPool here instead (Similar to Task.Run) instead of trying to change threads twice

[tired girder]

That's an example from UniTask

[tired girder]

Well there might be a difference between the usage of both lines of code, that one might be working because it posts on the thread pool, the await to the main thread switch might not work because it posts a request to be executed next Update and that isn't working because of Tests or whatever

I've had this issue testing ECS before where ECS would not have update cycles because the Tests simply do not do that

So that's a possibility

Would also explain all other Await issues i've had because it tries to wait for the next player loop cycle, which never comes

[quick coral]

It should work the same in and out of play mode but perhaps there are differences with these things in edit mode/tests

[tired girder]

In fact I have this Test base for testing with ECS, otherwise I couldn't get it to work

    public abstract class EcsTestsFixture
    {
        (...)

        [SetUp]
        public virtual void Setup()
        {
            // unit tests preserve the current player loop to restore later and start from a blank slate.
            _previousPlayerLoop = PlayerLoop.GetCurrentPlayerLoop();
            PlayerLoop.SetPlayerLoop(PlayerLoop.GetDefaultPlayerLoop());

            _previousWorld = World.DefaultGameObjectInjectionWorld;
            World = World.DefaultGameObjectInjectionWorld = new World("Test World");
            World.UpdateAllocatorEnableBlockFree = true;
            Manager = World.EntityManager;
            ManagerDebug = new EntityManager.EntityManagerDebug(Manager);

            // Many ECS tests will only pass if the Jobs Debugger enabled;
            // force it enabled for all tests, and restore the original value at teardown.
            _jobsDebuggerWasEnabled = JobsUtility.JobDebuggerEnabled;
            JobsUtility.JobDebuggerEnabled = true;

#if (UNITY_EDITOR || DEVELOPMENT_BUILD) && !DISABLE_ENTITIES_JOURNALING
            // In case entities journaling is initialized, clear it
            EntitiesJournaling.Clear();
#endif
        }

        [TearDown]
        public virtual void TearDown()
        {
            // Clean up systems before calling CheckInternalConsistency because we might have filters etc
            // holding on SharedComponentData making checks fail
            while (World.Systems.Count > 0)
            {
                World.DestroySystemManaged(World.Systems[0]);
            }

            ManagerDebug.CheckInternalConsistency();
            World.Dispose();
            World.DefaultGameObjectInjectionWorld = _previousWorld!;

            JobsUtility.JobDebuggerEnabled = _jobsDebuggerWasEnabled;

            PlayerLoop.SetPlayerLoop(_previousPlayerLoop);
        }
    }

Which is just taken from somewhere in #1064581837055348857

I wouldn't be surprised if UniTask is injecting itself on the default player loop which gets overriden by the test-specific player loop

In fact the awaits I've posted that do work do work because I've forced them to not configure the awaitable, which iirc it posts an event to be configured next loop

There it is

[tired girder]

Wow that fixed everything

[quick coral]

Post play mode tasks will continue executing. Only when we press play and a domain reload happens are they cleared out

Its important all async code uses the exit token, destroy token or a custom cancellatio token to end