Streaming data format | Deepgram | Page 1

floral wren Aug 1, 2024, 3:24 AM

#

When streaming data in JS SDK, what format should it be streamed as? is Uint8Array ok?

muted crowBOT Aug 1, 2024, 3:24 AM

#

Hi @floral wren. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:

#

Yes, streaming data in the form of a Uint8Array is acceptable when using the Deepgram JavaScript SDK. In the example provided in the Deepgram documentation (https://developers.deepgram.com/docs/js-sdk-text-to-speech#chunked-text-source-payload), the audio data is read from the stream and stored in chunks, which are then combined into a Uint8Array. This array is then converted into a Buffer for further processing.

Here's the relevant code snippet:

const getAudioBuffer = async (response) => {
  const reader = response.getReader();
  const chunks = [];

  while (true) {
    const { done, value } = await reader.read();
    if (done) break;

    chunks.push(value);
  }

  const dataArray = chunks.reduce(
    (acc, chunk) => Uint8Array.from([...acc, ...chunk]),
    new Uint8Array(0)
  );

  return Buffer.from(dataArray.buffer);
};

In this code, `chunks` is an array of `Uint8Array` objects, each representing a chunk of audio data. These chunks are then combined into a single `Uint8Array` (`dataArray`), which is then converted into a `Buffer`.

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

#

@floral wren kapa.ai is still learning and improving, please let us know how it did by reacting below

floral wren Aug 1, 2024, 3:25 AM

#

@muted crow So when actually doing connection.send(data), the data has to be in a buffer format?

muted crowBOT Aug 1, 2024, 3:25 AM

#

Let me check the knowledge base for you. One moment! :books:

#

Yes, when using the send function of the LiveClient instance in Deepgram's SDK, you should send the data as a Buffer. This is evident from the Deepgram documentation (https://developers.deepgram.com/docs/node-sdk-streaming-transcription#functions) and the GitHub discussion (https://github.com/orgs/deepgram/discussions/502) where the data is sent to the connection in the form of a Buffer.

Here's the relevant code snippet:

if (twilioMessage["event"] === "media") {
  const media = twilioMessage["media"];
  const audio = Buffer.from(media["payload"], "base64");
  connection.send(audio);
}

In this code, the `audio` data is created as a `Buffer` from the `payload` of the `media` object, and then sent to the connection using `connection.send(audio)`.

#Streaming data format

In this code, chunks is an array of Uint8Array objects, each representing a chunk of audio data. These chunks are then combined into a single Uint8Array (dataArray), which is then converted into a Buffer.

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

In this code, the audio data is created as a Buffer from the payload of the media object, and then sent to the connection using connection.send(audio).

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

In this code, `chunks` is an array of `Uint8Array` objects, each representing a chunk of audio data. These chunks are then combined into a single `Uint8Array` (`dataArray`), which is then converted into a `Buffer`.

In this code, the `audio` data is created as a `Buffer` from the `payload` of the `media` object, and then sent to the connection using `connection.send(audio)`.