#How to properly play audio/webm;codecs=opus buffers to a voice channel sent via WebSocket?

14 messages · Page 1 of 1 (latest)

somber sinew
#

Trying to broadcast a stream of audio buffers being recorded as audio/webm;codecs=opus using MediaRecorder to a voice channel via a WebSocket,

I know that the recording works because i can save the recording to a file and it plays in the browser.

Here's my code:

// join channel if name contains radio
    if (newVC?.name.toLocaleLowerCase().includes("radio")) {
      if (newVC.members.size >= 1) {
        // defining the voice connection
        const connection = joinVoiceChannel({
          channelId: newVC.id,
          guildId: newVC.guild.id,
          adapterCreator: newVC.guild.voiceAdapterCreator,
          selfMute: false,
          debug: true,
        });

        const audioPlayer = createAudioPlayer();
        // websocket receiveing buffer as audio/webm;codecs=opus
        // listener is a just custom subprotocol to broadcast audio to this client.
        const ws = new WebSocket("ws://localhost:3000", "listener"); 


        ws.addEventListener("open", () => {
          console.log("Connection opened");
        });

        connection.subscribe(audioPlayer);

        let stream = new Readable({
          objectMode: true,
          highWaterMark: 1024 * 1024 * 20,
          read() {},
        });

        connection.on(VoiceConnectionStatus.Ready, () => {
          ws.addEventListener("message", (event) => {
            // pushing buffer to the stream  
            stream.push(event.data); 

          });
          // creating resouce with stream
          const audio = createAudioResource(stream); 

          // playing the buffers pushed to the stream
          audioPlayer.play(audio); 

          audioPlayer.on("error", console.error);
          audioPlayer.on("debug", console.log);

          connection.on(VoiceConnectionStatus.Destroyed, async () => {
            stream.destroy();
            ws.close();
          });
        });
      } // endif
    } // endif

raw cypressBOT
#
  • What's your exact discord.js npm list discord.js and node node -v version?
  • Not a discord.js issue? Check out #1081585952654360687.
  • Consider reading #how-to-get-help to improve your question!
  • Explain what exactly your issue is.
  • Post the full error stack trace, not just the top part!
  • Show your code!
  • Issue solved? Press the button!
somber sinew
#

How to properly play audio/webm;codecs=opus buffers to a voice channel sent via WebSocket?

autumn elm
#

I imagine you'll at least want to specify inputType: StreamType.WebmOpus for your resource
beyond this if it's still not properly playing audio, please share the logs from debugging your AudioPlayer and VoiceConnection

#

and just to make absolutely sure, might as well generate a dependency report

woven socketBOT
#

guide Getting Started: Introduction - Debugging Dependencies
read more

#

To debug your voice connection and player:

  • Use debug: true when creating your VoiceConnection and AudioPlayer
  • Add an event listener to the <VoiceConnection> and the <AudioPlayer>:
// Add one for each class if applicable
<AudioPlayer | VoiceConnection>
    .on('debug', console.log)
    .on('error', console.error)
  • Add an error listener to the stream you are passing to the resource:
<Stream>.on('error', console.error)

Note: The <> represents classes that need to be adapted to their respective name in your code

somber sinew
#

Hi @autumn elm thanks for replying, here's what happened:

After I set inputType:StreamType.WebmOpus it gave me the following error which i've encountered before, the audioPlayer debug info is included in this log.

44 |     let result;
45 |     while (result !== TOO_SHORT) {
46 |       try {
47 |         result = this._readTag(chunk, offset);
48 |       } catch (error) {
49 |         done(error);
     ^
error: Did not find the EBML tag at the start of the stream
      at onwrite (native:1:1)
      at native:1:1
      at _transform (/Users/Main/Desktop/weaver/node_modules/prism-media/src/core/WebmBase.js:49:9)
      at native:1:1
      at writeOrBuffer (native:1:1)
      at native:1:1
      at ondata (native:1:1)
      at emit (native:1:1)
      at addChunk (native:1:1)
      at readableAddChunk (native:1:1)

state change:
from {"status":"buffering","resource":true,"stepTimeout":false}
to {"status":"idle","resource":false,"stepTimeout":false}

Dependency report

--------------------------------------------------
Core Dependencies
- @discordjs/voice: 0.17.0
- prism-media: 1.3.5

Opus Libraries
- @discordjs/opus: 0.9.0
- opusscript: not found

Encryption Libraries
- sodium-native: not found
- sodium: 3.0.2
- libsodium-wrappers: 0.7.14
- tweetnacl: not found

FFmpeg
- version: 6.0
- libopus: yes
--------------------------------------------------
#

One more thing, when i save my recording as an audio.webm file and i play it through discord, using createAudioResource("path/to/audio.webm") it plays normally...

#

WebSocket server code

Basically it receives the recorded audio from a WS client (called insider) and then forwards it to other WS clients (the listeners) like in the first snippet.

// websocket server
const server = Bun.serve({
  fetch(req, server) {

    // checking for "listener" in  headers
    const headers = Object.fromEntries(req.headers.entries())
    const isListener = headers["sec-websocket-protocol"] === "listener"

    // upgrade the request to a WebSocket
    // add data to the client so it can be referenced
    if (server.upgrade(req, {
      data: {
        headers,
        listener: isListener,
        id: isListener ? listeners.length + 1 : 0
      }
    })) {
      return; 
    }
    return new Response("Upgrade failed", { status: 500 });
  },

  websocket: {
    // if client is a listener, subscribe it to "listeners" topic, else is the insider client.
    open: (ws) => {
      if (ws.data.listener) {
        listeners.push(ws.data)
        ws.subscribe('listeners')
      } else {
        insider = ws.data
      }
      updateConsole()
    },
    close: (ws, code, reason) => {
      // a listener disconnected
      if (ws.isSubscribed("listeners") || ws.data.listener) {
        ws.unsubscribe('listeners')
        listeners = listeners.filter(l => l.id !== ws.data.id)
      }
      else {
       // terminate script
        end("Insider disconnected")
      }
    },
    async message(ws, data) {
      try {
        // if this option is enabled, write recording to a file
        if (writer) { 
          writer.write(data)
          writer.flush()
        }

        if (listeners.length > 0) {
          // broadcast recording
          server.publish("listeners", data)
        }
        
        // for debugging
        buffer = data
        totalBytes += data.length

        updateConsole()
      }
      catch (e) {
        console.error(e)
        end("Error in server message")
      }
    }
  }
});
balmy warren
#

Why exactly do you implement an empty read() method to your Readable? How do you expect AudioPlayer to read that stream then?

somber sinew
somber sinew
#

How can i fix this particular error?

error: Did not find the EBML tag at the start of the stream
      at onwrite (native:1:1)
      at native:1:1
      at _transform (/Users/Main/Desktop/weaver/node_modules/prism-media/src/core/WebmBase.js:49:9)
      at native:1:1
      at writeOrBuffer (native:1:1)
      at native:1:1
      at ondata (native:1:1)
      at emit (native:1:1)
      at addChunk (native:1:1)
      at readableAddChunk (native:1:1)
balmy warren
#

https://github.com/discordjs/discord.js/blob/defb083528ef31383778187a04ced8b00d886242/packages/voice/src/audio/AudioPlayer.ts#L600 is what‘s getting frames to be played from your AudioResource. Which internally will call https://github.com/discordjs/discord.js/blob/defb083528ef31383778187a04ced8b00d886242/packages/voice/src/audio/AudioResource.ts#L161 so if your Readable has an empty read() function it will never return anything to be read/played

GitHub

A powerful JavaScript library for interacting with the Discord API - discordjs/discord.js

GitHub

A powerful JavaScript library for interacting with the Discord API - discordjs/discord.js