🔒 Help with understanding coroutines and how they work with blocking C functions

[frosty lotus]

Hello, I'm currently working on a library that provides an asynchronous interface for creating game servers (for a specifc game). My library uses a C-extension to make the networking library used by the game accessible in python (so C bindings).

So the question is, if I were to await a coroutine that calls one of the blocking C functions (let's say something like recv) would the coroutine block the thread or would it pass control back to the event loop and then come back to it once the C function is done?

I'm fairly new to async and concurrency in general. Thanks!

[covert thicket]

@frosty lotus

Python help channel opened

Remember to:

• Ask your Python question, not if you can ask or if there's an expert who can help.
• Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
• Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

Closes after a period of inactivity, or when you send !close.

[frosty lotus]

Help with understanding coroutines and how they work with blocking C functions

[rancid wing]

block im pretty sure but im confused with the question are you saying it calls upon the blocking function

[foggy hemlock]

if the C function would block a regular thread, then it would block the event loop as well in async code - as async in python is single-threaded

[rancid wing]

now i am new to all the asynchronous stuff too but any help is good help hopefully

i was right thanks for confirming that haha

[frosty lotus]

I see, thanks a lot. Though could you please tell me why? I'm struggling to understand how async coroutines pass control back to the event loop (like when you send a request using aiohttp)

[foggy hemlock]

await does that.

in the olden times (python 3.4) the await keyword still didn't exist, and it was implemented using yield from instead.
when you await something, similar to yield/yield from the current functions execution gets suspended and gives control back to the calling loop (for-loop with yield/yield from)

[frosty lotus]

I see. I'm really sorry, but could you tell me why awaiting a coroutine that calls a blocking C function wouldn't achieve the same thing? Is it that something has to be done internally to make the blocking C function not really blocking (like yield)?

[foggy hemlock]

awaiting the coroutine will suspend the function that calls the await, and at some point the event loop decides to coll the coroutine. and when that coroutine gets called, and it in turn calls an underlying C function and that C function itself cannot be awaited, it wil block until the C function returns

async def mycoro():
    my_c_function()

async def main():
    await mycoro()

asyncio.run(main())
``` awaiting mycoro may suspend `main`, and the event loop decides to call `mycoro` at some point... but once mycoro runs, it will just call the c function and *that* function is blocking and not giving control back to the event loop.
So for as long as the c function runs, the event loop won't have a chance to run anything else

[frosty lotus]

aha I see

[foggy hemlock]

a common way to avoid that is to just run the c function in a thread

async def mycoro():
    await asyncio.to_thread(my_c_function)

async def main():
    await mycoro()

asyncio.run(main())
``` or similar (forgot the exact way)

[frosty lotus]

amazing thanks a lot

truly appreciate it

[foggy hemlock]

you can also always ask in #async-and-concurrency

[frosty lotus]

yep I did but I thought it was dead so I asked here 😭

[foggy hemlock]

people don't look there nearly as often, but generally are experts

[frosty lotus]

I see I'll make sure to ask there next time :-)

!close

[covert thicket]

Python help channel closed

This help channel has been closed and it's no longer possible to send messages here. If your question wasn't answered, feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.