concurrent api requests

[marsh pivot]

heyo everyone! i’m building a discord bot that runs using GPT-4. currently, the bot only uses a single api key from my account. if more than one user attempts to query the bot before the bot has completed delivering an output, it returns an error. i was curious if anyone could point me in the direction for how to solve for this? ideally, theoretically infinitely many users would be able to query the bot at the ‘same’ time— cost for managing this type of scalability is a non-issue

[fading loom]

Hey!

Are you using python?

you can simply use asyncio 🙂

Each request should be an asyncio function

Some example code from my own project: https://github.com/Kav-K/GPTDiscord/blob/main/models/openai_model.py