What is the difference between using websockets and webrtc with Agents Plarform? | ElevenLabs | Page 1

tribal flumeBOT Sep 5, 2025, 8:53 AM

#

AI-generated Answer for umanema

Short answer:

WebRTC is the recommended, browser‑native transport for Agents (formerly Conversational AI) with built‑in echo cancellation, background‑noise removal, automatic network adaptation, and connection recovery. Switch to it in the SDK by setting connectionType: 'webrtc'. For authenticated sessions you obtain a conversation token via GET /v1/convai/conversation/token. (elevenlabs.io)
WebSockets are the lower‑level, direct API option: you open wss://api.elevenlabs.io/v1/convai/conversation?agent_id=... to stream audio in/out. Public agents can connect with just agent_id; private agents require a signed URL. (elevenlabs.io)

Details:

Audio/quality: WebRTC unlocks best‑in‑class echo cancellation and background noise removal, improving conversation quality vs traditional streaming. (elevenlabs.io)
Browser integration/reliability: WebRTC works natively in modern browsers and provides automatic network adaptation and connection recovery. (elevenlabs.io)
How to enable in SDKs: Available in the npm package and Swift SDK; flip a single parameter: connectionType: 'webrtc'. Public agents need only agentId; for authenticated conversations, generate a token via the REST endpoint above. (elevenlabs.io)
WebSocket auth patterns: For private agents use a server‑generated signed URL (GET /v1/convai/conversation/get-signed-url); never expose your API key client‑side. (elevenlabs.io)
Positioning: The WebSocket docs target developers “integrating directly” with the Agents WebSocket API; WebRTC was rolled out to improve stability/performance and is used in 11.ai. (elevenlabs.io)

If you want, I can show a minimal example for each path (WebRTC via SDK with a token, and raw WebSocket) based on the docs. (elevenlabs.io)

Sources:
ElevenLabs Conversational AI now supports WebRTC | ElevenLabs
WebSocket — ElevenLabs Documentation
ElevenLabs Conversational AI now supports WebRTC | ElevenLabs
ElevenLabs Conversational AI now supports WebRTC | ElevenLabs
ElevenLabs Conversational AI now supports WebRTC | ElevenLabs
WebSocket | ElevenLabs Documentation
WebSocket — ElevenLabs Documentation
Get conversation token | ElevenLabs Documentation

nimble hill Sep 5, 2025, 8:55 AM

#

Is there an easy way of integrating webrtc to cpp code?

tribal flumeBOT Sep 5, 2025, 8:55 AM

#

AI-generated Answer for umanema

Short answer: not today.

ElevenLabs documents WebRTC for the JavaScript/React SDK (startSession with connectionType: "webrtc") and for the Swift SDK. There is also an API to mint a WebRTC conversation token. But there is no documented C++ SDK or C++ example for WebRTC. (elevenlabs.io)
Officially supported client libraries are Python and Node.js; for languages outside those, the documented real-time path is the WebSocket API (which you can use from any language, including C++). (elevenlabs.io)

If you need C++ today, use the WebSocket Conversational AI API from C++. If you specifically require WebRTC in C++, ElevenLabs does not currently provide a C++ SDK or guide in the public docs. (elevenlabs.io)

Relevant docs:

React SDK with WebRTC: startSession({ connectionType: 'webrtc' }) (elevenlabs.io)
JavaScript client SDK overview (elevenlabs.io)
Swift SDK overview (elevenlabs.io)
Get WebRTC token (REST) (elevenlabs.io)
Supported languages (official bindings) and WebSocket usage (elevenlabs.io)
WebRTC announcement (availability in npm and Swift SDK) (elevenlabs.io)

nimble hill Sep 5, 2025, 8:57 AM

#

Does that mean that i lose sound quality when using websockets?

tribal flumeBOT Sep 5, 2025, 8:57 AM

#

AI-generated Answer for umanema

Short answer: no, not inherently.

With WebSockets, audio quality depends on how you stream. ElevenLabs notes that sending more text per chunk improves quality, while very small chunks or frequent flushes reduce quality (a latency–quality trade‑off you control). (elevenlabs.io)
WebRTC adds built‑in echo cancellation, background‑noise removal, and adaptive networking that “weren’t possible with traditional streaming approaches,” which improves perceived conversation quality without extra work on your side. (elevenlabs.io)

So you don’t “lose” quality by choosing WebSockets, but you may need more buffering/chunking to match the clarity you’ll get out‑of‑the‑box with WebRTC for live, two‑way audio. (elevenlabs.io)

Sources:
Generate audio in real-time | ElevenLabs Documentation
ElevenLabs Conversational AI now supports WebRTC | ElevenLabs
Generate audio in real-time | ElevenLabs Documentation

nimble hill Sep 5, 2025, 9:00 AM

#

Do websockets have noise cancellation and echo cancellation?

tribal flumeBOT Sep 5, 2025, 9:00 AM

#

AI-generated Answer for umanema

No. In ElevenLabs, noise and echo cancellation are provided when you use WebRTC; the WebSocket connection does not include these built-in audio processing features. (elevenlabs.io)

ElevenLabs’ WebSocket docs cover real-time streaming, buffering, and voice settings but do not mention any built-in echo or noise suppression. (elevenlabs.io)

Sources:
ElevenLabs Conversational AI now supports WebRTC | ElevenLabs
Generate audio in real-time | ElevenLabs Documentation

light kayak Sep 6, 2025, 8:30 PM

#

I’d like the agent to have background audio (e.g. office ambience), so that when the agent is silent (because it’s using MCP/tools), the caller still hears something and has a more natural experience — instead of dead silence.

Any guidance or examples would be really appreciated 🙏

#What is the difference between using websockets and webrtc with Agents Plarform?