open mist
Am I the only one receiving request and socket timouts with requests against the OpenAI API? Even setting timeouts as high as 2 minutes for both building up socket connection and the request itself doesn't help in 90% of the time.
For me only roughly every 10th request comes through. I'm on a paid plan.
cursive crater
There is a max. amount of requests per minute or so, maybe this is why?
open mist
nah, that can't be it. I'm just testing locally and firing 1 request every 2 mins tops.
Just got this though instead of a timeout:
statusCode=503, body='{
"error": {
"message": "That model is currently overloaded with other requests. You can retry your request, or contact us through our help center at help.openai.com if the error persists. (Please include the request ID <redacted> in your message.)",
"type": "server_error",
"param": null,
"code": null
}
}
so there seems to be an issue indeed 🤔
steel dune
Hey! There was an outage earlier so this may have been that issue. If you still get this issue, try regenerating your API key and deleting old keys, and you can also try making sure you are under your usage limits
open mist
Usage limits are no issue, but the outage might have been the reason indeed. Didn't know that regenerating API keys fixes such issues as well though, thanks for that tip! 🙏
hm, no, even with a new API key I'm still experiencing the timeouts. It's a bit frustrating tbh. 😓
Alright, I increased timeouts both for socket and for request to 240s. Works now. But is the API intended to be that slow? 😵💫
steel dune
Alright, I increased timeouts both for socket and for request to 240s. Works now...
Is your account a paid account?
open mist
Is your account a paid account?
yes
steel dune
Hmm that's odd, maybe try reaching out to the helpcenter and you can also try requesting a quota/rate limit increase
open mist
I'm really starting to wonder how reliable the API is, even for paying customers. 🤔 I'm still getting socket exception with timouts of 240s. I can't possibly offer a product built on this to any user, not mentioning charging for it.
Am I just doing something fundamentally wrong here?
steel dune
It may be some issues with the way you're making the request
I can make API requests roughly every half a second and they work fine
and they only fail when the API has latency
open mist
yes, must be my prompts
But on the other hand: why would I sometimes get a socket timeout then?
request timeout, yes, but socket timeout?
maybe it's because I'm trying to access the API from Europe (Germany) and there is a routing issue? Just a wild guess 😅
steel dune
Can I see the type of prompts you're sending, and how you're sending the request programatically?
open mist
I'm using https://github.com/Aallam/openai-kotlin
It uses ktor under the hood, so pretty standard stuff in kotlin world
steel dune
if you make the same request using python or using something like PostMan, do you still get timeout errors? I have a feeling that it may be an issue with that kotlin library if the rest of your request looks good, that library doesn't look well tested or popular
open mist
will check, thanks for your support!
open mist
checked it out with Insomnia, taking ages there as well. It might really be the complexity of the prompt...
If the request succeeds I'm getting exactly what I want. But alas, in most cases it doesn't succeed or only after a super long time.