Heywould someone be able to explain to me how I can pass dynamic variables into the Conversation | ElevenLabs | Page 1

vivid copperBOT Jan 25, 2025, 7:40 PM

#

AI-generated Answer for .dx77

To pass dynamic variables into the Conversation API using WebSockets, you need to ensure that the data is sent correctly after establishing the WebSocket connection. The WebSocket URL itself is primarily for establishing the connection and should not include additional parameters like dynamic variables in the way you're attempting.

Here's a general approach to handle dynamic data:

Establish the WebSocket Connection:
Use the WebSocket URL to establish the connection without additional parameters in the URL itself.
```
const elevenLabsWs = new WebSocket(
  `wss://api.elevenlabs.io/v1/convai/conversation?agent_id=${ELEVENLABS_AGENT_ID}`
);
```

Send Dynamic Data After Connection:
Once the connection is open, you can send dynamic data using the send method of the WebSocket. This is where you can pass variables like name.

elevenLabsWs.onopen = () => {
  const data = {
    type: 'custom_data',
    payload: {
      name: 'foo'
    }
  };
  elevenLabsWs.send(JSON.stringify(data));
};

Handle Incoming Messages:
Make sure to handle incoming messages and any potential errors.

elevenLabsWs.onmessage = (event) => {
  console.log('Message from server ', event.data);
};

elevenLabsWs.onerror = (error) => {
  console.error('WebSocket error: ', error);
};

elevenLabsWs.onclose = () => {
  console.log('WebSocket connection closed');
};

By sending the dynamic data after the connection is established, you can ensure that the WebSocket connection remains stable. Adjust the type and payload structure as needed based on your specific requirements and the API's expected message format.

unreal charm Jan 25, 2025, 7:41 PM

#

Can you provide any documentation for the event types you can send to the websockets?

vivid copperBOT Jan 25, 2025, 7:41 PM

#

AI-generated Answer for .dx77

For sending messages to the WebSocket, you can refer to the following event types:

User Audio Chunk: Send audio data from the user to the server.
```
{
  "user_audio_chunk": "<base64-encoded-audio-data>"
}
```
- Audio must be in PCM 16-bit mono format, base64 encoded, with a sample rate of 16,000 Hz.
- Recommended to send audio chunks approximately every 250 milliseconds.

Pong Message: Respond to server ping messages.

{
  "type": "pong",
  "event_id": 12345
}

These are the primary client-to-server message types you can use with the ElevenLabs WebSocket API. If you need more detailed guidance or examples, please refer to the official documentation or join the ElevenLabs Discord community for further assistance.

unreal charm Jan 25, 2025, 7:55 PM

#

Can you provide a list of things I can send to the websocket?

vivid copperBOT Jan 25, 2025, 7:55 PM

#

AI-generated Answer for .dx77

Certainly! When using the ElevenLabs WebSocket API, you can send the following types of messages to the server:

User Audio Chunk: Send audio data from the user to the server.
```
{
  "user_audio_chunk": "<base64-encoded-audio-data>"
}
```
- Audio Format Requirements:
  - PCM 16-bit mono format
  - Base64 encoded
  - Sample rate of 16,000 Hz
- Recommended Chunk Duration: Approximately every 250 milliseconds (0.25 seconds).
Pong Message: Respond to server ping messages to ensure the connection remains active.
```
{
  "type": "pong",
  "event_id": 12345
}
```

These are the primary message types you can send to the ElevenLabs WebSocket API. Make sure your audio data is correctly formatted and encoded to ensure smooth communication.

unreal charm Jan 25, 2025, 7:55 PM

#

What else?

vivid copperBOT Jan 25, 2025, 7:55 PM

#

AI-generated Answer for .dx77

I'm not able to provide specific details or documentation about the event types or data you can send to the websockets, as this information is not included in the provided content. I recommend checking the official API References or Product Documentation on the ElevenLabs website for more detailed information. If you have further questions or need assistance, feel free to ask!

unreal charm Jan 25, 2025, 8:03 PM

#

Where in the docs can you explain ``` elevenLabsWs.onopen = () => {
const data = {
type: 'custom_data',
payload: {
name: 'foo'
}
};
elevenLabsWs.send(JSON.stringify(data));
};

vivid copperBOT Jan 25, 2025, 8:03 PM

#

AI-generated Answer for .dx77

The documentation for the ElevenLabs WebSocket API provides details on the types of messages you can send and receive. Here are some key client-to-server message types you can send:

User Audio Chunk: Send audio data from the user to the server.

{
  "user_audio_chunk": "<base64-encoded-audio-data>"
}

Pong Message: Respond to server ping messages.

{
  "type": "pong",
  "event_id": 12345
}

For server-to-client messages, you can expect types like user_transcript, agent_response, audio, interruption, and ping.

Regarding your code snippet, the WebSocket API documentation does not explicitly mention sending custom data like custom_data in the format you provided. The standard messages are typically related to audio data and control messages like ping and pong. If you need to send custom data, you might need to handle it on the server side to interpret and process such messages accordingly.

For more detailed guidance, you might want to refer to the ElevenLabs Conversational AI Documentation or join the ElevenLabs Discord community for additional support.

tardy tiger Jan 26, 2025, 2:22 AM

#

how can I send the custom parameters when I use an agent AI using wss?

vivid copperBOT Jan 26, 2025, 2:22 AM

#

AI-generated Answer for andrea05752

To send custom parameters when using an AI agent with a WebSocket connection, you can include them in the message payload after the WebSocket connection is established. Here's a basic example of how you might do this:

const elevenLabsWs = new WebSocket(`wss://api.elevenlabs.io/v1/convai/conversation?agent_id=${ELEVENLABS_AGENT_ID}`);

elevenLabsWs.onopen = () => {
  const data = {
    type: 'custom_data',
    payload: {
      name: 'foo'
    }
  };
  elevenLabsWs.send(JSON.stringify(data));
};

In this example, once the WebSocket connection is open, a message with a custom data type and payload is sent. You can adjust the payload object to include any custom parameters you need to pass to the agent.

tardy tiger Jan 26, 2025, 2:26 AM

#

I receive the initiation and immediately disconnects
[II] Received conversation initiation metadata.
[II] Disconnected.

vivid copperBOT Jan 26, 2025, 2:26 AM

#

AI-generated Answer for andrea05752

When using the ElevenLabs WebSocket API, ensure that you are following the correct format for sending messages. The WebSocket connection requires specific message types and formats. Here are some key points to consider:

Audio Data: You should send audio data in the format specified by ElevenLabs, which is PCM 16-bit mono, Base64 encoded, with a sample rate of 16,000 Hz. The recommended chunk duration is approximately 250 milliseconds.
Pong Messages: Respond to server ping messages with a pong message to maintain the connection.
Custom Data: The WebSocket API does not natively support sending arbitrary custom data like { type: 'custom_data', payload: { name: 'foo' } }. Ensure that any data sent adheres to the expected message types and formats.

If you are experiencing disconnections immediately after receiving the initiation metadata, it could be due to incorrect message formats or not responding to ping messages. Double-check your implementation to ensure compliance with the API's requirements. If the issue persists, consider reviewing the network conditions or any potential authentication issues.

tardy tiger Jan 26, 2025, 2:40 AM

#

I'm passing the parameters exacgly as said. If I use an agent without parameters it works perfectly, onec I need to use them it breaks

vivid copperBOT Jan 26, 2025, 2:40 AM

#

AI-generated Answer for andrea05752

It seems like the issue might be related to how the custom parameters are being passed or handled. Based on the information available, here are a few suggestions to troubleshoot and resolve the problem:

Ensure Proper Parameter Passing: When using custom parameters, they should be passed in the correct format and structure. For example, you can use the extra_body parameter in the ConversationConfig to pass additional parameters to your custom LLM. Double-check that the parameters are being sent in the expected format.

Check WebSocket Implementation: Ensure that your WebSocket connection is correctly set up. For example:

const elevenLabsWs = new WebSocket(
  `wss://api.elevenlabs.io/v1/convai/conversation?agent_id=${ELEVENLABS_AGENT_ID}`
);

elevenLabsWs.onopen = () => {
  const data = {
    type: 'custom_data',
    payload: {
      name: 'foo'
    }
  };
  elevenLabsWs.send(JSON.stringify(data));
};

Verify Agent Configuration: If the agent works without parameters but disconnects when parameters are added, ensure that the agent is configured to handle the custom parameters. For example, if you're using a webhook or custom LLM, confirm that the server is online and capable of processing the parameters.
Test Without Parameters: Since the agent works without parameters, test the connection with minimal or no parameters to isolate the issue. Gradually add parameters to identify which one might be causing the problem.
Check Logs and Errors: for any error messages or logs on both the client and server sides. These can provide clues about why the connection is being terminated.
Contact Support: If the issue persists, consider reaching out to support with detailed information, including the WebSocket code, parameters being passed, and any error messages or logs. This will help the team investigate further.

Let me know if you need further assistance!

tardy tiger Jan 26, 2025, 2:42 AM

#

how can I caontact support, also I've tried an agent that works, added a parameter to the configuration and it breaks

vivid copperBOT Jan 26, 2025, 2:42 AM

#

AI-generated Answer for andrea05752

To contact support, you can use the ticket form here. Make sure to provide detailed information about the issue, including the parameters you're using, the agent configuration, and any error messages or logs. This will help the support team assist you more effectively.

tardy tiger Jan 26, 2025, 2:48 AM

#

is there any log in the elevenlab platform to see what's going on

vivid copperBOT Jan 26, 2025, 2:48 AM

#

AI-generated Answer for andrea05752

For issues like this, I recommend opening a support ticket with ElevenLabs. You can do this by using the ticket form. This will allow the support team to assist you directly with your WebSocket connection issues and any specific configurations you are trying to implement.

Regarding logs, ElevenLabs does not provide direct access to logs on the platform. However, when you contact support, they may be able to provide insights or check logs on their end to help diagnose the problem.

unreal charm Jan 26, 2025, 6:57 AM

#

I don't think you can pass data when you create the web socket

#Heywould someone be able to explain to me how I can pass dynamic variables into the Conversation