#🔒 Fetching data from API and writing to DB concurrently

11 messages · Page 1 of 1 (latest)

sharp shuttleBOT
#

@azure birch

Python help channel opened

Remember to:

  • Ask your Python question, not if you can ask or if there's an expert who can help.
  • Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
  • Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

Closes after a period of inactivity, or when you send !close.
azure birch
#

Why was original message deleted?

median girder
#

Because we do not allow .py attachments.

#

!paste -- The bot should've sent you a DM, but anyway use this.

sharp shuttleBOT
#
Pasting large amounts of code

If your code is too long to fit in a codeblock in Discord, you can paste your code here:
https://paste.pythondiscord.com/

After pasting your code, save it by clicking the Paste! button in the bottom left, or by pressing CTRL + S. After doing that, you will be navigated to the new paste's page. Copy the URL and post it here so others can see it.

azure birch
#

Hi, I'm new to async programming and trying to fetch data from an api and save to a landing table concurrently.

But the issue is it's waiting for all concurrent api requests to complete before starting concurrent DB writes which is unnecessary and loading all data into memory. I've been reading about asyncio.Queue and all but don't know how to make it work or is the right approach. Can anyone please advise?

#

Can you see my code now? Sorry first timer here

grand flint
#

the indentation doesnt seem right here, can you double check

azure birch
#

Thanks for pointing it out. Should be good now

sharp shuttleBOT
#
Python help channel closed

This help channel has been closed and it's no longer possible to send messages here. If your question wasn't answered, feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.