Getting rate limit reached error while running assistant API, RPM.

[shadow stream]

const startPolling = async (runId: string, threadId: string) => {
  if (!threadId) return;
  console.log("Start polling");

  let baseTimeout = 30000; // Initial timeout duration
  let currentTimeout = baseTimeout; // Current timeout duration

  while (true) {
    try {
      const response = await openai.beta.threads.runs.retrieve(threadId, runId);
      console.log(response);
      if (
        ["cancelled", "failed", "completed", "expired"].includes(
          response.status
        )
      ) {
        const completion = await openai.beta.threads.messages.list(threadId);
        console.log(completion.data);
        return completion.data[0].content;
      }

      // If status is "in_progress", double the timeout duration
      if (response.status === "in_progress") {
        currentTimeout *= 2;
      } else {
        // Reset the timeout duration to the base value if not "in_progress"
        currentTimeout = baseTimeout;
      }
      // Wait for 5 second before checking the status again
      await new Promise((resolve) => setTimeout(resolve, currentTimeout));
    } catch (error) {
      console.error("Error polling run status:", error);
      return "Error polling run status";
    }
  }
};

Even with an initial delay of 30s and incrementing the delay by doubling it after every last call, I am still getting this error.

Rate limit reached for gpt-3.5-turbo-1106 in organization org-aG27TwnZI6bnQGOJHrooU6LF on requests per min (RPM): Limit 3, Used 3, Requested 1. Please try again in 20s. Visit https://platform.openai.com/account/rate-limits to learn more. You can increase your rate limit by adding a payment method to your account at https://platform.openai.com/account/billing.

How can I get through this error? Also to note, I am on free tier plan of OpenAI.

[open gorge]

Hi @shadow stream. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:

---

The error message you're seeing indicates that you've hit the rate limit for your API requests. This means you've submitted too many requests in a short period of time and have exceeded the number of requests allowed for your current usage tier. This is common when you're using a loop or a script that makes frequent or concurrent requests, or if you're on a free or low-tier plan that has a lower rate limit [source (https://help.openai.com/en/articles/6891829-error-code-429-rate-limit-reached-for-requests)].

Here are some steps you can take to resolve this issue:

1. Pace your requests: Avoid making unnecessary or redundant calls. If you're using a loop or a script, make sure to implement a backoff mechanism or a retry logic that respects the rate limit and the response headers.

2. Check your usage: If you're sharing your organization with other users, note that limits are applied per organization and not per user. It's worth checking the usage of the rest of your team as this will contribute to this limit.

3. Upgrade your plan: If you're using a free or low-tier plan, consider upgrading to a pay-as-you-go plan that offers a higher rate limit.

4. Increase your usage tier: If none of the previous steps work and you're consistently hitting a Rate Limit Error, you can increase your rate limits by increasing your usage tier. You can view your current rate limits, your current usage tier, and how to raise your usage tier/limits in the Limits section of your account settings [source (https://help.openai.com/en/articles/6891753-rate-limit-advice#h_857f48cf2f)].

Remember, the rate limit error will occur when API requests are sent too quickly. If you're using the OpenAI Python library, the error message will look something like this:

RateLimitError: Rate limit reached for default-codex in organization org-{id} on requests per min. Limit: 20.000000 / min. Current: 24.000000 / min. Contact [e-mail] if you continue to have issues or if you’d like to request an increase.

[source (https://cookbook.openai.com/examples/how_to_handle_rate_limits#example-rate-limit-error)]

If you'd like your organization's rate limit increased, you can fill out the OpenAI Rate Limit Increase Request form [source (https://cookbook.openai.com/examples/how_to_handle_rate_limits#default-rate-limits)].

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

@shadow stream kapa.ai is still learning and improving, please let us know how it did by reacting below

[shadow stream]

@open gorge

const uploadFile = async (
  url: string,
  format: FormatType,
  questions: number
) => {
  try {
    const file = await openai.files.create({
      file: await fetch(url),
      purpose: "assistants",
    });
    if (file.id) {
      return createAssistant(file.id, format, questions);
    }
  } catch (error) {
    console.error(error);
    return "Error while uploading file to OpenAI.";
  }
};

const createAssistant = async (
  fileId: string,
  format: FormatType,
  quesitons: number
) => {
  try {
    const assistant = await openai.beta.assistants.create({
      name: "Quiz Generator",
      instructions: `You are a quiz generator`
      model: "gpt-3.5-turbo-1106",
      tools: [{ type: "retrieval" }],
      file_ids: [fileId],
    });
    if (assistant.id) {
      return createThreadAndRun(format, fileId, assistant.id, quesitons);
    }
  } catch (error) {
    console.error(error);
    return "Error while creating an assistant.";
  }
};

const createThreadAndRun = async (
  format: FormatType,
  fileId: string,
  assistantId: string,
  quesitons: number
) => {
  try {
    const run = await openai.beta.threads.createAndRun({
      assistant_id: assistantId,
      thread: {
        messages: [
          {
            role: "user",
            content: `Generate ${quesitons} ${format} type questions. And format the response as the example format given - ${JSON.stringify(
              JSONFormat(format)
            )}.`,
            file_ids: [fileId],
          },
        ],
      },
    });
    if (run.id) {
      return await startPolling(run.id, run.thread_id);
    }
  } catch (error) {
    console.error(error);
    return "Error while creating thread.";
  }
};

These are the API calls I have one after the other, should I call the next API after waiting for a certain period? Or these have no effect on the rate limit?

[open gorge]

Let me check the knowledge base for you. One moment! :books:

---

The rate limits apply to the number of API requests you make within a certain period of time, regardless of the type of request. This means that all API calls, including those to create files, assistants, and threads, count towards these limits.

If you're making these calls one after the other in quick succession, it's possible that you could hit the rate limit, especially if you're on a lower-tier plan with a lower limit.

To avoid hitting the rate limit, you could implement a delay between each API call. However, the exact amount of delay needed would depend on your specific rate limit and the frequency of your API calls.

It's also worth noting that the rate limit is typically higher for paid plans, so if you're consistently hitting the rate limit, you might want to consider upgrading your plan.

Remember, the rate limit error will occur when API requests are sent too quickly. If you're using the OpenAI Python library, the error message will look something like this:

RateLimitError: Rate limit reached for default-codex in organization org-{{id}} on requests per min. Limit: 20.000000 / min. Current: 24.000000 / min. Contact [e-mail] if you continue to have issues or if you’d like to request an increase.

[source (https://cookbook.openai.com/examples/how_to_handle_rate_limits#example-rate-limit-error)]

If you'd like your organization's rate limit increased, you can fill out the OpenAI Rate Limit Increase Request form [source (https://cookbook.openai.com/examples/how_to_handle_rate_limits#default-rate-limits)].

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon: