Currently getting an issue when I'm using the API.
Model is overloaded with other requests.
Any idea what's going on?
#model overload?
somber glen
spark mango
Probably the same issue many others (myself included) are having with constant timeouts (even after 10 retries) and huge latency increases. No word at all from the OpenAI team.
fair temple
I have same issue, constant timeout and it 2 request within short period of time, also lead to fail, I m using 3.5 turbo. Tried renew API key, upgrade server network, still same.
spark mango
I have same issue, constant timeout and it 2 request within short period of time...
Sadly, it will probably take 10+ days before enough people complain for OpenAI to look into the problem. Last time, a Discord thread had to get 100+ likes, and there had to be a lot of outrage on the community forums before they took it seriously.
It looks like until the issue is more widely noticed by users, we're pretty much screwed.
upbeat wyvern
OpenAICEO told Congress "I wish subscribers would use less compute. We don't have enough GPUs." So unless user demand has gone down... OpenAPI users are maybe getting something like rolling blackouts in the GPU service? Except the Que is possibly getting so full that they have to terminate some pending prompts? Maybe it is time to investigate whether the alternative open source GPTs and local GPU are adequate for your needs. GPT-J? Alpaca ...
split jay
It has gotten really bad. There is this 24/7 stream that has prompts taking even hours to generate. API users are getting a lot of failures to try communicate with it... Infinite Chronicles twitch stream... Other stream called "How is it Manifested" also had recently issues with this to the point they decided to switch to GPT-4 temporally just to get prompts through
somber glen
Well they keep releasing more to the public so it will only get worse. Moving to a custom model helps a bit I think
spark mango
I increased timeout 60 -> 120, with 5-10 retries with a 5 second sleep in between. It seems to work… but the outputs are insanely slow because it times out and errors so much.
fair temple
the text-davinci-003 so far work better the gpt-3.5-turbo