API Response Time | Mistral AI | Page 1

twilit ravine Feb 7, 2024, 3:46 PM

#

Hello guys, I'm currently using the api to request the mistral-small model. But I'm seeing a fairly long response time (around 30 seconds), knowing that the expected response is in the form of a JSON and contains around 1500 characters.

First of all, is it normal to have such a long response time? And if not, do you know of anything that could help me reduce this time? If need be, I'd change the way I request the api by splitting it into several requests, but I'd like, as far as possible, not to have to do this.

Thanks for your help !

near canyon Feb 7, 2024, 3:56 PM

#

you can try streaming and see if you get the next tokens in a reasonable time

twilit ravine Feb 7, 2024, 7:37 PM

#

near canyon you can try streaming and see if you get the next tokens in a reasonable time

how could i do that on this model ?

#API Response Time