Hi, I'm testing the text-davinci-002 model and it's really slow. I'm testing it via postman. Anyone has the same feeling?
The performance of this model was good yesterday but today seems to be really bad. I have the feeling that they are using batches to provide the answer...
#Davinci API performance
spice nebula
spice nebula
Looks like the issue is only in the API as if I'm using the same request in playground, it goes faster
magic fjord
I am not from their team, but experienced as follow :
Seems most of current models and servers capacity are not for production level, the simpler the model is, the faster it is :
For example, ada answer mostly instantely, but barely don't know anything.
Cury answer relatively fast in English, slow in other languages, and know minimum for common sense.
Da vinci answer nicely but not as good as GPT3.5 version, but response time can reach up to more than 10 seconds, which lock most of the "realtime" chatbot use cases.
Online 3.5 version on "chat.openai" or on page like playground are demo versions with a very specific server capacity with the strong help of microsoft backing up.
Since it's a demo, "i guess" they currently do not have the capacity to deploy for API with correctly scaling dedicated cloud infra and a clear pricing and engagement on response time, etc...
So our best is to wait for a market proposal product 🙂 Which will probably come soon or later with more service specs and SLA 🙂
( Again, it's my guess and feedback since i met the same issue, but hope it can guide you a little... )
spice nebula
Thank you so much. I have the same feeling as you but I have to say that davinci-002 was quite better in terms of performance before deploying the new chatGPT