@compact steeple Please provide more information related to your query. One of our moderators will help you out soon.
#How to play the audio received from elvenlabs websocket api continuusly without delays or chunks in
robust anvilBOT
compact steeple
Ok so basically elevenlabs websocket sends the audio in chunks via websocket. I can get it work using the python demo, where you play straight from the stream straight from the listen function here https://docs.elevenlabs.io/api-reference/text-to-speech-websockets
But I'm struggling when:
I need to send the audio recieved over websocket to a client, and then have the client play it
Whether that client is in python or javascript. I can hear the little delay between playing the consecutive audio chunks, if I do play the audio (bytes) I receive from elevenlabs. Sometimes that delay is in the middle of the words. Any advice on this? 🙂
gleaming barn
I have a client on Swift and the same problem, a lot of people facing it too, here are two more reports on this #1166469137548189706 message
Elevenlabs just ghosting all of this, they don’t seem to be interested in any support or feedback on this
compact steeple
Super sad as it's a big challenge to using the audio in realtime 😦
gleaming barn
@compact steeple Good news that openai now have tts with $0.015 per 1K charecters, so we can forget this websockets bs
compact steeple
No voice cloning yet though 😦 @gleaming barn
Think of me stuck in the weeds when you're rocking it with openai xD
dense sable
@compact steeple When I was implementing playback in JavaScript, I had gaps when using MP3 response. These gaps were not there when using MediaSource API but since MediaSource is not available on IOS, I had to use Web Audio API and there I used PCM audio response instead.
compact steeple
I'll give it a whirl thank you 🙂
viscid hare
<@456226577798135808> Good news that openai now have tts with $0.015 per 1K char...
OpenAI TTS doesn't have any API for streaming input. If you don't need that, why were you using the WebSocket API?
dense sable
@gleaming barn I created a capacitor plugin for IOS to play base64 chunks. Maybe you can use the code to see if it helps. I am using PCM_44100 output tho.
Go to https://npmjs.com/package/capacitor-streaming-audio then go to code and click on 'ios/Plugin' folder. There you will see 'StreamingAudio.swift' file which has playback implementation. I am not iOS developer so code might not be good quality but maybe it can help you.
gleaming barn
OpenAI TTS doesn't have any API for streaming input. If you don't need that, why...
We initially used WebSockets to optimize the voiceover delay for real-time conversations with LLM. Eventually, we transitioned to feeding it sentence by sentence through another endpoint, which offered similar delay and enhanced stability. Therefore, we are now migrating to OpenAI, utilizing the same principle.
dense sable
@gleaming barn I had to add initial buffer of 2-3 chunks to start playback. It adds small delay but then audio is seamless. Although slow network can sometime cause gaps which is inevitable.
gleaming barn
<@694924217845678180> I had to add initial buffer of 2-3 chunks to start playbac...
Thanks for sharing! We might need some elevenlabs customization features in the future so might give it a shot!