#RPM Rate Limit

23 messages · Page 1 of 1 (latest)

halcyon thorn
#

So im using the new chat gpt api model in my discord bot (using discord.js library) with openai node.js library. i coded the main function in messageCreate event limiting it to a specific channel but after triggering it about 4 times, the bot came with an error from the api that you are rate limited (RPM). i didnt have this problem using text-davinci and now i have no idea why it got rate limited.

here is the code:

const question = message.content
let conversation = [{role: "system", content: instructions}, {role: "user", content: question}]
// instructions already defined (string)
openai.createChatCompletion({
             model: "gpt-3.5-turbo",
             messages: conversation,
             max_tokens: 2048,
             temperature: 0.5
            }).then(async (response) => {
             message.reply(`${response.data.choices[0].message.content}\n\n\`Used ${response.data.usage.total_tokens} tokens\``)

             }).catch(async (error) => {
              console.error(error);

              if (error.response) await message.reply({ content: error.response.data.error.message });
              else if (error.message) await message.reply({ content: error.message });

            });
image.png
#

any ideas?

unkempt jewel
#

it just has a much more restricted ratelimit

#

i asked it to solve that for me btw and it did

#

here is the code 1 sec

# 
async def handle_rate_limit_error(channel, messages):
    while True:
        try:

            completion = openai.ChatCompletion.create(
                model="gpt-3.5-turbo",
                messages=messages,
                temperature=1.2

            )

            return completion
        except openai.error.RateLimitError as e:
            msg = await channel.send(f"Got rate-limited; retrying after 5 seconds.")
            time.sleep(5)
            await msg.delete()
halcyon thorn
unkempt jewel
#

yes this is python

halcyon thorn
#

from what i understand, this code sends a request and if the api returned rate limit error, it will retry after 5 secs

#

well, i dont think thats a good solution... not sure

unkempt jewel
#

theres only 2 solutions, retry if ratelimited after a certain time or handle the request timing on your end making sure to add things into a queue and ask the api in time intervals long enough not to be ratelimited

#

i just retry after 5 seconds, which works well for me

halcyon thorn
unkempt jewel
#

the rate limit hits me, then my bot types "Ratelimited, trying again in 5 seconds". Then it retries, and deletes the previous message

halcyon thorn
unkempt jewel
#

yeh

#

do a queue with sleep in your code then before u send it

halcyon thorn
halcyon thorn
#

@unkempt jewel kinda forget to say thanks
A specific delay between requests worked for me

#

👍

abstract dune
#

@halcyon thorn

image.png
halcyon thorn