Previously, we were seeing latency around 500ms, but this has recently risen to an average of 1000ms, with occasional spikes tripling that time. We’ve tried experimenting with different prompt sizes, but even with small prompts, we are still experiencing the increased latency. Has anyone else been experiencing this?
#API Latency Increase
harsh rivet