#calling chatCompletionAPI extremely slow

19 messages · Page 1 of 1 (latest)

shell frigate
#

Hello,

I am building an app that uses chatgpt (specifically gpt-3.5-turbo) currently I have it hosted on heroku 25/month shared tier. Locally the API call is snappy and responds quick. On the cloud it can take anywhere from 2-6 seconds for the api to kick off. I’ve identified that it is the initial api call itself that is slow, once streaming starts it is okay. My question is, where do you guys host your back ends for calling chatgpt?

Other info:

Running on node js using OpenAI official node client

Using websockets

Api is using stream=true

unique iron
shell frigate
#

So I finally figured out the issue. I have two OpenAI accounts. My personal one and my company one. I discovered my personal accounts api key is fast. The company account (just created the other day) is VERY slow. The company one also shows zero usage even tho we have been using it. So there seems to be an issue with that account. Maybe it is still using the free credits?

#

Thanks for the reply btw

unique iron
#

I am facing the same problem

#

they shouldn't have done it like this

#

without telling anyone...

#

it took me 3-4 hours to debug the problem

#

💀 💀 💀

#

can you check if it's the same for gpt-3 ? in the playground it's fast (with the free account)

shell frigate
#

Yeah, I spent around the same time figuring it out 😭😭😭

#

Yeah, playground it works fine. The model I’m calling is got-3

#

Gpt *

unique iron
#

humm, can you try using the api to call gpt-3?

#

maybe they whitelisted the playground endpoints

shell frigate
#

Yeah I’ll try it out once I get back to my computer. Hopefully this issue resolves itself once I blow through the free credits since I don’t want to be using my personal account for everything haha

visual solar
#

I am also experiencing very high response times. The api is down I think

unique iron