gilded raft
I'm experiencing significant latency issues while using client.tts.websocket() on AWS EC2 instances (t2.medium type) :
500ms latency in us-west-1 (California) region
300ms latency in us-east-2 (Ohio) region
I've already implemented the recommendations from the help section posts regarding latency reduction, but I'm still unable to achieve a latency below 300ms.
Any insights or suggestions would be greatly appreciated. Thanks
lavish kettle
Thanks for reporting this! Could you share how you’re measuring the latency?
gilded raft
@lavish kettle using time.time() just before ws = client.tts.websocket() and time.time()again when the first chunk from ws.send() is received
Don't get me wrong, I'm still happy with this latency (pretty low in comparison with other TTS services). I just don't understand why I can't achieve a latency of less than 150ms like on the playground, when the playground is also using the WebSocket endpoint.
merry ivy
I’m not familiar with the js library, but the WS should stay open. You should not measure WS opening, as this is always slow and meant to be done once.
Once you open the WS, you should keep it and reuse it while it is open.
lavish kettle
+1 to @merry ivy, thanks!
Even if you're using the Python client, you should measure before ws.send() and then after the first chunk is received
gilded raft
Indeed, it was a mistake on my part to include the websocket opening latency; thank you for pointing out this error. After correction, I am getting an average latency of 200ms, which is very satisfactory. Thanks again @merry ivy @lavish kettle
lavish kettle
Awesome!
Let us know if there's anything else we can help with, excited to see what you build
gilded raft
I am ready to use Cartesia for my AI agent in production @lavish kettle , despite the multilingual model still being in alpha. The various tests conducted are conclusive; the issue of audio cutting off at the end of dictation is problematic but not a blocker. Most importantly, after various optimizations to minimize the latency of different services (Vonage, Deepgram, GPT-3.5-turbo, and Cartesia), my AI agent achieves a total latency between 800ms and 1s, which was the main objective. Thank you again for this amazing service that is Cartesia!
lavish kettle
Super exciting to hear that, glad you're enjoying the product!