Sequential Audio Response | Cartesia | Page 1

feral saffron Jun 27, 2024, 7:47 PM

#

Hey guys, I'm building a real-time application where I'm streaming chunks/sentences to Cartesia Websocket.

I'm sending the data to the Cartesia websocket in sentences, but the responses are not in the order of what I send them.

For e.g., if I send "How's it going?" and then send "Hope you are well" in two ws messages, I kind of get both audio responses together, instead of coming through "in order". Basically the audio for "how's it going" should come first then "hope you are well" come later.

Wondering if I'm missing something here? Or if I have to build some buffering observing the "done" field and context_id.

Note: I'm keeping the context_id same for the two messages above.

Overall, Cartesia is great guys and the latency has been amazing for me so far, let me know if there's something I've missed, thanks!

wild whale Jun 27, 2024, 8:21 PM

#

I observed the same. I just built my own queue to keep track and then order them properly myself.

hoary void Jun 27, 2024, 8:22 PM

#

For e.g., if I send "How's it going?" and then send "Hope you are well" in two ws messages, I kind of get both audio responses together, instead of coming through "in order". Basically the audio for "how's it going" comes first then "hope you are well" comes later.

Are you using the same context_id for the messages sent to WS or different ones?

feral saffron Jun 27, 2024, 8:33 PM

#

I kpet the context_id same for both sentences above, but the sequential order wasn't followed when the audio was sent back in the response

hoary void Jun 27, 2024, 8:37 PM

#

Currently you'll need to specify different context_ids for the two sentences and then order them yourself.

#

That being said, we're going to be releasing a feature in the coming week that allows you to stream in inputs on the same context_id and pass in a continue flag that conditions audio generations upon previous requests!

#

Stay tuned to the Discord for this release but happy to help with anything else in the meanwhile.

#

Hope this helps!

feral saffron Jun 27, 2024, 8:39 PM

#

It does! I just wanted to check if I was missing something. And gotcha, I'll move to build my own queue then for now and track the context_ids.

Appreciate the response, and waiting eagerly for the next release!

feral saffron Jun 27, 2024, 11:04 PM

#

Hey @hoary void , sorry but I was building out what you had mentioned prior to building my own queue. I'm just trying to figure if the audio associated to a particular context_id has finished processing. I was hoping the done flag would be a good indicator for this. However, I am getting it to be false consistently. Any thoughts or advice you would have for me?

hoary void Jun 27, 2024, 11:13 PM

#

The done flag should only be false as long as audio is being received. Could you clarify the behavior you're seeing?

feral saffron Jun 27, 2024, 11:18 PM

#

Hey, thanks for the reply. So weird thing here but, around 10-15 mins ago or so, even after I got the complete audio, the done flag was still false on the last websocket message (the last message with audio).

But trying again now, it seems like true is actually being returned consistently? I haven't really changed anything from then to now...

hoary void Jun 27, 2024, 11:19 PM

#

That's unexpected, we'll keep an eye out. But hope this helps in the meanwhile!

#

Thanks for reporting!

feral saffron Jun 27, 2024, 11:19 PM

#

Yeah, seems to be working as intended now. I'll let you know if I see that pop up again. Thanks!

hoary void Jun 27, 2024, 11:24 PM

#

Awesome

paper lintel Jun 27, 2024, 11:45 PM

#

hi @hoary void , can you clarify the context_id you're talking about here? AFAIK, the websocket response does not contain context_id - how does ordering work in this case?

#

I guess what I'm trying to say here is that, during a long running conversation with multiple ws.sends, how do I ensure that generations do not overlap, OR, if the generations overlap, how do I sort them so that I do not play them at the same time?

hoary void Jun 27, 2024, 11:55 PM

#

WebSocket responses do contain context_id and it'll always be the context_id passed in on the WS request.

#

It temporarily got removed from the docs, pushing a fix ASAP

paper lintel Jun 28, 2024, 12:23 AM

#

Awesome, thanks Sauhard!**

haughty ermine Jul 1, 2024, 5:14 PM

#

Also, check your code to do not expect audio for when the done flag is true. Is by Cartesia design that the message with done flag true will not contain audio.

#

That used to be documented, but I can’t find it now. <@&1199024793194868776> would you please confirm or correct me?

tidal fulcrum Jul 1, 2024, 5:20 PM

#

The chunk with the done flag will not contain audio, correct.

haughty ermine Jul 1, 2024, 5:22 PM

#

Cool, it would great to get that back in the docs for new comers.

#Sequential Audio Response