Output format mulaw/8000 takes x3 times longer

[grave lotus]

Hello guys.

We noticed that output format mulat/8000 takes ~3 times longer in comparasing with pcm_f32le/44100 for example

Here is a timings when we measure how long the request itself takes:

tts pcm_f32le: 412.313ms
tts pcm_mulaw: 1.742s

Everything, absolutely the same, the only difference in the output format.

Is that an issue or this is by design because of this format?

Here is quick code example:

const mainOptions = {
  method: 'POST',
  headers: {
    'Cartesia-Version': '2024-06-10',
    'X-API-Key': SECRET_KEY',
    'Content-Type': 'application/json'
  },
};

const body = {
  transcript: 'Welcome to our company, how can I help you today?',
  model_id: 'sonic-english',
  // https://docs.cartesia.ai/api-reference/endpoints/list-voices#list-voices
  voice: {
    mode: 'id',
    id: 'b7d50908-b17c-442d-ad8d-810c63997ed9',
  },
}

const options = {
  ...mainOptions,
  body: JSON.stringify({
      ...body,
      output_format: {
        container: 'raw',
        encoding: 'pcm_f32le',
        sample_rate: 44100,
      },
    })
};

console.time('tts pcm_f32le');
const resPcmf32le = await fetch('https://api.cartesia.ai/tts/bytes', options)
console.timeEnd('tts pcm_f32le');


const optionsMulaw =  {
  ...mainOptions,
  body: JSON.stringify({
    ...body,
    output_format: {
      container: 'raw',
      encoding: 'pcm_mulaw',
      sample_rate: 8000,
    },
  })
};

console.time('tts pcm_mulaw');
const resMulaw = await fetch('https://api.cartesia.ai/tts/bytes', optionsMulaw)
console.timeEnd('tts pcm_mulaw');

Thanks.

[valid cloak]

Hi @grave lotus - hm that’s not expected thanks for flagging it. We’ll take a look and follow up!

[grave lotus]

Thank you.

[valid cloak]

Hi @grave lotus - we've identified it as likely a network buffering issue. Working on a resolution here, appreciate your patience 🙏

[grave lotus]

Thank you guys.

[grave lotus]

Hey guys. Any estimates when this could be fixed?
Thanks.

[mint temple]

We're actively investigating! We'll get you an update on this by EOD

[grave lotus]

Hey guys, any update on this one?

[mint temple]

cc @valid cloak

[valid cloak]

Hi @grave lotus apologies for the delayed follow up here. We resolved this issue on SSE and would recommend switching to that in the interim - unfortunately bytes has been more of an issue and we’re actively working with cloudflare to resolve it.

[grave lotus]

@valid cloak thank you for the update. We will try that.

Btw, we tried the SSE initially, but it looks like the connection is closing after very short time of idle, so in our case it's works like a regular request. Is there any way to increase the timeout, to keep thee SSE active?

[valid cloak]

Hm @grave lotus to clarify wdym by closing after very short time of idle?

SSE is intended to function akin to bytes but with more of a structured output

[grave lotus]

@valid cloak I mean that SSE connection should be established once, and next call should reuse it, right?

But when we tried the connection was closed after a very short time. Or it’s by design?

[valid cloak]

Ah @grave lotus for that type of behavior we’d recommend web sockets. The SSE endpoint we have closes once the audio generation is complete

[grave lotus]

@valid cloak Got it. Thank you!

“SSE is intended to function akin to bytes but with more of a structured output”

I just have one more question, could you please clarify what do you mean by that?

[valid cloak]

The underlying backend logic behind bytes and sse is functionally similar, but bytes just returns raw bytes while sse returns structured fields https://docs.cartesia.ai/reference/api-reference/rest/stream-speech-server-sent-events

[grave lotus]

Thank you for clarification, we will try to use SSE for now.