#How to add a polling interval for run object?

1 messages · Page 1 of 1 (latest)

cobalt ibex
# 
        const run = await openai.beta.threads.runs.create(thread.id, {
          assistant_id: assistant.id,
        });

          const completedRun = await openai.beta.threads.runs.retrieve(
            thread.id,
            run.id
          );

How can I call the retrieval run to check for status? I want to poll after every 500ms and timeout after 15000 ms. How to implement it?

reef iceBOT
#

Hi @cobalt ibex. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:

#

You can implement a polling interval and timeout using JavaScript's setTimeout and Promise.race functions. Here's how you can modify your code:

const run = await openai.beta.threads.runs.create(thread.id, {
  assistant_id: assistant.id,
});

let completedRun = await openai.beta.threads.runs.retrieve(
  thread.id,
  run.id
);

const timeout = new Promise((_, reject) =>
  setTimeout(() => reject(new Error('Request timed out')), 15000)
);

while (completedRun.status === 'in_progress') {
  await Promise.race([
    new Promise(resolve => setTimeout(resolve, 500)),
    timeout
  ]);
  completedRun = await openai.beta.threads.runs.retrieve(thread.id, run.id);
}

In this code, Promise.race is used to either wait for 500ms or reject the promise if 15000ms (15 seconds) have passed, whichever happens first. If the timeout rejects the promise first, an error will be thrown.

Please note that you should handle the error thrown when the request times out. This is just a basic example and might need to be adjusted based on your specific use case.

This approach is based on the information provided in the extracts (https://community.openai.com/t/513815) from the OpenAI community forum.

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

#

@cobalt ibex kapa.ai is still learning and improving, please let us know how it did by reacting below

cobalt ibex
#

@reef ice
Why am I running completedRun initially? If I remove it what will happen? Any side effects?

reef iceBOT
#

Let me check the knowledge base for you. One moment! :books:

#

The initial call to completedRun is made to get the current status of the run. This is important because it's possible that the run could have completed even before you start your polling loop.

If you remove the initial completedRun, your code will enter the while loop without knowing the current status of the run. This could potentially lead to unnecessary polling requests if the run has already completed, or even an infinite loop if the run status is something other than 'in_progress' before entering the loop.

So, it's recommended to keep the initial completedRun to ensure that your polling logic works correctly and efficiently.

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

#

@cobalt ibex kapa.ai is still learning and improving, please let us know how it did by reacting below

cobalt ibex
#

@reef ice , I am using the free tier with model: "gpt-3.5-turbo-1106".
I am always get rate limit exceeded

  last_error: {
    code: 'rate_limit_exceeded',
    message: 'Rate limit reached for gpt-3.5-turbo-1106 in organization org-aG27TwnZI6bnQGOJHrooU6LF on requests per min (RPM): Limit 3, Used 3, Requested 1. Please try again in 20s. Visit https://platform.openai.com/account/rate-limits to learn more. You can increase your rate limit by adding a payment method to your account at https://platform.openai.com/account/billing.'
  },

I am polling after every 5seconds and have set timeout as 30s and yet I am getting this

        const run = await openai.beta.threads.runs.create(thread.id, {
          assistant_id: assistant.id,
        });

        let completedRun = await openai.beta.threads.runs.retrieve(
          thread.id,
          run.id
        );

        const timeout = new Promise((_, reject) =>
          setTimeout(() => reject(new Error("Request timed out")), 30000)
        );

        while (["in_progress", "queued"].includes(completedRun.status)) {
          await Promise.race([
            new Promise((resolve) => setTimeout(resolve, 5000)),
            timeout,
          ]);
          completedRun = await openai.beta.threads.runs.retrieve(
            thread.id,
            run.id
          );
          console.log(completedRun);
        }

How to get through this error?

reef iceBOT
#

Let me check the knowledge base for you. One moment! :books:

#

The error message you're seeing indicates that you've exceeded the rate limit for your account. The rate limit for free trial users is 20 requests per minute (RPM) and 40000 tokens per minute (TPM) as per the extract (https://community.openai.com/t/498517) from the OpenAI community forum.

In your case, the error message shows that your limit is 3 RPM, and you've already used up those 3 requests. This is why you're seeing the rate limit exceeded error.

Your current polling interval is set to 5 seconds (or 12 requests per minute), which exceeds your limit of 3 RPM. To avoid hitting the rate limit, you need to adjust your polling interval to fit within your rate limit. For example, if your limit is 3 RPM, you should poll no more than once every 20 seconds.

Here's how you can adjust your code:

const run = await openai.beta.threads.runs.create(thread.id, {
  assistant_id: assistant.id,
});

let completedRun = await openai.beta.threads.runs.retrieve(
  thread.id,
  run.id
);

const timeout = new Promise((_, reject) =>
  setTimeout(() => reject(new Error("Request timed out")), 30000)
);

while (["in_progress", "queued"].includes(completedRun.status)) {
  await Promise.race([
    new Promise((resolve) => setTimeout(resolve, 20000)), // Adjusted to 20 seconds
    timeout,
  ]);
  completedRun = await openai.beta.threads.runs.retrieve(
    thread.id,
    run.id
  );
  console.log(completedRun);
}

Remember, the rate limit is there to prevent abuse and ensure fair usage among all users. If you need a higher rate limit, consider upgrading your account or contacting OpenAI support.

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

#

@cobalt ibex kapa.ai is still learning and improving, please let us know how it did by reacting below

cobalt ibex
#

@reef ice ,

       const file = await openai.files.create({
          file: fileStream,
          purpose: "assistants",
        });
        console.log(file.id);

        const assistant = await openai.beta.assistants.create({
          instructions:
            systemPrompt + " And only make use of the provided file.",
          model: "gpt-3.5-turbo-1106",
          tools: [{ type: "retrieval" }],
          file_ids: [file.id],
        });
        console.log(assistant.id);

        const thread = await openai.beta.threads.create({
          messages: [
            {
              role: "user",
              content: `Generate ${quiz?.questionNumber} ${quiz?.format} type questions, and provide a suitable title for the quiz, excluding the word Quiz.`,
              file_ids: [file.id],
            },
          ],
        });
        console.log(thread.id);

        const run = await openai.beta.threads.runs.create(thread.id, {
          assistant_id: assistant.id,
        });

        let completedRun = await openai.beta.threads.runs.retrieve(
          thread.id,
          run.id
        );

        const timeout = new Promise((_, reject) =>
          setTimeout(() => reject(new Error("Request timed out")), 30000)
        );

        while (["in_progress", "queued"].includes(completedRun.status)) {
          await Promise.race([
            new Promise((resolve) => setTimeout(resolve, 5000)),
            timeout,
          ]);
          completedRun = await openai.beta.threads.runs.retrieve(
            thread.id,
            run.id
          );
          console.log(completedRun);
        }

        const completion = await openai.beta.threads.messages.list(thread.id);
        let message = completion.data[0].content;

Which of these requests will be considered as request in that rate limit? Does it only apply for retrieval run requests or for all the requests (creating assistant, creating thread)?

reef iceBOT
#

Let me check the knowledge base for you. One moment! :books:

#

The rate limit applies to all API requests made to OpenAI, not just the retrieval run requests. This includes creating files, assistants, threads, and any other API calls you make. Each of these operations counts as a separate request.

In your code, the following operations would each count as a request:

  1. Creating a file: openai.files.create()
  2. Creating an assistant: openai.beta.assistants.create()
  3. Creating a thread: openai.beta.threads.create()
  4. Creating a run: openai.beta.threads.runs.create()
  5. Retrieving a run: openai.beta.threads.runs.retrieve()
  6. Listing messages: openai.beta.threads.messages.list()

Each time you call openai.beta.threads.runs.retrieve() in your polling loop, it also counts as a separate request. So if your polling loop runs 10 times, that's 10 requests.

Remember to take all these into account when considering your rate limit. If you're hitting the rate limit frequently, you may need to adjust your polling frequency or consider upgrading your account for a higher rate limit.

For more details on rate limits, you can refer to the OpenAI documentation (https://platform.openai.com/docs/guides/rate-limits?context=tier-free).

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

#

@cobalt ibex kapa.ai is still learning and improving, please let us know how it did by reacting below

cobalt ibex
#

@reef ice ,
Can you set up a function for me which takes into consideration the rate limit? My RPM is 3, and my use case is to upload a file, create an assistant, create a thread, create a run and retrieve that run till status is completed.

All of these requests is called by the user on one button click. I have already sent you my code.

reef iceBOT
#

Let me check the knowledge base for you. One moment! :books:

#

Sure, I can help you adjust your code to take into account the rate limit. Given that your rate limit is 3 requests per minute (RPM), you need to ensure that your code doesn't make more than one request every 20 seconds (60 seconds / 3 RPM = 20 seconds per request).

Here's how you can adjust your code:

async function createAndRetrieveRun() {
  const file = await openai.files.create({
    file: fileStream,
    purpose: "assistants",
  });
  console.log(file.id);
  await new Promise(resolve => setTimeout(resolve, 20000)); // Wait 20 seconds

  const assistant = await openai.beta.assistants.create({
    instructions:
      systemPrompt + " And only make use of the provided file.",
    model: "gpt-3.5-turbo-1106",
    tools: [{ type: "retrieval" }],
    file_ids: [file.id],
  });
  console.log(assistant.id);
  await new Promise(resolve => setTimeout(resolve, 20000)); // Wait 20 seconds

  const thread = await openai.beta.threads.create({
    messages: [
      {
        role: "user",
        content: `Generate ${quiz?.questionNumber} ${quiz?.format} type questions, and provide a suitable title for the quiz, excluding the word Quiz.`,
        file_ids: [file.id],
      },
    ],
  });
  console.log(thread.id);
  await new Promise(resolve => setTimeout(resolve, 20000)); // Wait 20 seconds

  const run = await openai.beta.threads.runs.create(thread.id, {
    assistant_id: assistant.id,
  });

  let completedRun = await openai.beta.threads.runs.retrieve(
    thread.id,
    run.id
  );

  const timeout = new Promise((_, reject) =>
    setTimeout(() => reject(new Error("Request timed out")), 30000)
  );
#

while (["in_progress", "queued"].includes(completedRun.status)) {
await Promise.race([
new Promise(resolve => setTimeout(resolve, 20000)), // Adjusted to 20 seconds
timeout,
]);
completedRun = await openai.beta.threads.runs.retrieve(
thread.id,
run.id
);
console.log(completedRun);
}

const completion = await openai.beta.threads.messages.list(thread.id);
let message = completion.data[0].content;
return message;
}


In this code, I've added a 20-second delay after each API request using `setTimeout`. This ensures that you don't exceed your rate limit of 3 RPM. Please note that this is a basic example and might need to be adjusted based on your specific use case.
----------------------
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
----------------------