🔒 Using asyncio.gather to download multiple videos at the same time

[ember rose]

I am trying to do that but with my code it looks like download is skipped and that files are empty.
Did I fail my function ?
code : https://paste.pythondiscord.com/P6PA

[modest iron]

@ember rose

Python help channel opened

Remember to:

• Ask your Python question, not if you can ask or if there's an expert who can help.
• Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
• Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

Closes after a period of inactivity, or when you send !close.

[polar kindle]

loop.run_in_executor is outdated, I believe - nowadays you can use the simpler asyncio.to_thread.

[ember rose]

thanks I will have a look rn !

[ember rose]

I'm getting the same problem, really feels like it's skipping the tasks as it has 0 delay

[polar kindle]

The code you posted overall looks correct, though. Perhaps something's wrong with e.g. download_video_from_url, and so all of your tasks immediately fail with exceptions?

i'd temporarily unset return_exceptions=True, so that you can see the exceptions if that's the case.

[ember rose]

this is the function :

async def download_video_from_url(url, temp_file_path):
    loop = asyncio.get_running_loop()
    
    def _download_instagram():

    if "tiktok" in url:
        return await asyncio.to_thread(_download_tiktok)
    elif "instagram" in url:
        return await asyncio.to_thread(_download_instagram)
    elif "youtube" in url:
        return await asyncio.to_thread(_download_youtube)
    elif "pinkbike" in url:
        return await _download_pinkbike_async()

    return "Invalid URL"

didn't put the in between code for each site but the idea is here

a synchrone function

[ember rose]

I see

[polar kindle]

Ah, well, that explains it. run_in_executor (and so is to_thread) is for running non-async functions (by putting them into another thread, so that waiting for their execution doesn't block the asyncio event loop).

Since download_video_from_url is async, just have download_and_convert await it.

[ember rose]

That's it ! thanks so much
So basically what is happening when using run_in_executor on an asynchrone function?

[polar kindle]

Yeah, I believe the reason nothing happens is just that all your coroutines are called by run_in_executor... and then not awaited, so as it normally goes with unawaited coroutines, they don't actually get to run. So it finishes very quickly and does nothing.

[ebon cape]

!e I'd like to add that async functions in python are much different than in javascript, if you have some background knowledge about them
for example:

async def async_function():
    print('Hello')

import time
async_function()
time.sleep(2)
print('Finished')```

[modest iron]

@ebon cape :white_check_mark: Your 3.12 eval job has completed with return code 0.

001 | /home/main.py:5: RuntimeWarning: coroutine 'async_function' was never awaited
002 |   async_function()
003 | RuntimeWarning: Enable tracemalloc to get the object allocation traceback
004 | Finished

[ebon cape]

By calling the function you create a coroutine, but it never actually runs, until you await it or use some function from asyncio which accepts coroutines, for example asyncio.run or asyncio.create_task

[ember rose]

thanks so much for the explanation 🙂

[modest iron]

Python help channel closed

This help channel has been closed and it's no longer possible to send messages here. If your question wasn't answered, feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.