#Rayon randomly get a execution from `tokio-runtime-worker` thread

8 messages · Page 1 of 1 (latest)

fluid yew
#
pub fn block_on<F: core::future::Future>(mut runtime: Option<Runtime>, f: F) -> F::Output {
        if runtime.is_none() {
            runtime = Handle::try_current()
                .is_err()
                .then(|| Runtime::new().unwrap());
        }
    
        match runtime {
            Some(runtime) => runtime.block_on(f),
            None => futures::executor::block_on(f), // <= DeadLock here where it get called from tokio-runtimte-worker
        }
    }

let me try to explain the problem, im using tokio as my async lib, and rayon to do some CPU task par_iter().filter_map

when block_on called from cli.exe!std::sys.... threads (which is should always do be cause its Rayon threads) it works normally. but when called from tokio-runtime-worker is case a deadlock, the real problem is why it even get called from tokio-runtime-worker its same code and same execution path, but some times it get called from tokio-runtime-worker.

Here is the execution path:
Async tokio task tokio::spawn -> async task -> async task that contains rayon call par_iter().filter_map -> inside filter_map calls a function that call block_on

i know i can't explain the problem in a good way so if u have a question pls ask.

There a two images that show the stacktrace

#

Rayon randomly get a execution from tokio-runtime-worker thread

warm heron
#

The nature of Rayon's execution strategy is that any thread participating in Rayon task execution, which includes a thread calling par_iter(), may end up executing code from any other Rayon task.

#

This works fine for all of Rayon's own operations, because they are all organized in "structured concurrency" fashion; the only thing that is waited for is a child task

#

It breaks if you block on waiting for some other task to finish.

#

Blocking also interferes with Rayon being able to efficiently execute tasks, by occupying part of the thread pool.

#

To avoid these issues, follow this rule: never block in a Rayon task

#

There are patterns which could be sound, but it is best to keep things simple