How can I reduce the time of getting an answer from the API to 1-2 seconds? | OpenAI | Page 1

sturdy sapphire Aug 16, 2023, 9:48 PM

#

Hi!
I have a chat using openAI API, and it's taking a lot of time to get an answer, like 8-9 seconds.
This is slow!!!

How can I reduce the time to 1-2 seconds?
Thanks

fading yoke Aug 17, 2023, 2:15 PM

#

The time depends on the current load of the server. There's nothing you can do but shorten your prompt to reduce the time.

gleaming kelp Aug 24, 2023, 10:28 PM

#

I've read lowering max tokens can help. I agree, the API is slow.

mortal jewel Aug 25, 2023, 12:02 AM

#

streaming the answer helps it feel faster.

graceful sentinel Aug 25, 2023, 5:18 PM

#

Since yours is a chat, I would suggest to go with streaming like webhead suggested, it’s not fast but it gives you the feeling of faster response the same thing you see on the UI of chat gpt. If you don’t think it works for you see if this helps in your use case #1143773908139515965 message

#How can I reduce the time of getting an answer from the API to 1-2 seconds?