Hi guys. I have implemented a api that takes pdf as input and giving the output as json. I am using the Open AI 3.5 turbo model passing the text as input in prompt with specified instructions and asking the desired response as json in output. But recently I have noticed that in the paid plan the response from the model is very slow generally 35-40 seconds and that will be a very bad user experience.The output that I expect from Open AI generally has 800 tokens minimum as output. Any suggestions what can I do to improve the response time from Open AI
#3.5 Turbo model is giving slow responses for parsing text need help to improve timings
hard pilot
rapid pecan
This may happen when the server load is high. You can subscribe this to know whether the server is encounting issues. https://status.openai.com/
Maybe things will go better when Open AI release its SLA
crisp lily
Thank you. This is a valuable ressource! Since latency is also a matter of distance... Can anyone share if
**a) **creating an API Key determines which infrastructure is used? **a1 **if a = Yes) is there some kind of GEO Filtering when creating API Key?
b) do we know WHERE openai is hosting their (API-)infrastructure?
haughty ledge
Anyone else experiencing an issue when you are trying to retrieve a very large amount of tokens, it will fail halfway through with a Cloudflare error? Been seeing it more and more over the past week or two.