Seems that when sending multiple requests in parallel by API to paid models they are handled sequentially! but this is not useful for our app since we have high traffic and we cannot serve users sequentially!
fading summit
Seems that when sending multiple requests in parallel by API to paid models they are handled sequentially! but this is not useful for our app since we have high traffic and we cannot serve users sequentially!
spice rose
Why do you think they're being handled sequentially?
That isn't really the case. how are you making the API call?
fading summit
we sent multiple requests by API but the response time increased when we had multiple requests in comparison to only one request