#I can see documentation for integrating the TTS and STT APIs, but I’m not sure how to embed or conn

1 messages · Page 1 of 1 (latest)

distant socketBOT
#
AI-generated Answer for joffersadraw

Short answer: Yes — you can use a full ElevenLabs Agent directly in your own web app without rebuilding it from the low‑level TTS/STT APIs. You have three supported ways to integrate it, and supported methods to give it access to your data.

How to embed/connect your existing Agent

  • Drop‑in website widget (quickest)

    • Add the web component to your site and point it at your agent_id. Note: the widget currently requires a public agent (auth disabled). You can restrict access with an allowlist of permitted domains. The widget supports voice, voice+text, or chat (text‑only) and accepts dynamic variables and runtime overrides.
    • For domain restrictions and security options (signed URLs + allowlists), see Auth.
  • Client SDKs (recommended for custom UIs)

    • Use @elevenlabs/react’s useConversation hook and startSession with connectionType set to "webrtc" (recommended) or "websocket". For private agents use a server‑generated token/URL; for public agents pass agentId only. You can send text via sendUserMessage (chat UIs) and handle audio/text events.
    • WebRTC tokens (server-side) are issued via GET /v1/convai/conversation/token; clients connect with that token.
    • Background on WebRTC support and how to switch from WebSocket with one parameter.
  • Direct WebSocket API (lowest-level client control)

    • Connect to your agent at wss://api.elevenlabs.io/v1/convai/conversation?agent_id={agent_id}. For private agents, your server requests a signed URL via GET /v1/convai/conversation/get-signed-url and returns it to the browser. Events include user_transcript, agent_response, audio, client_tool_call, etc.

Text‑only (“chat mode”) without audio

  • Configure text_only at agent creation or enforce it at runtime via overrides. Then send user text messages programmatically and read agent_response events. This is intended for embedding chat experiences.

Letting the Agent use your knowledge and data

  • Knowledge Base (KB)

    • Continue programmatically adding your summaries via POST /v1/convai/knowledge-base, then attach/update on the agent with the Update Agent API. Limits and RAG details are documented.
  • Server Tools (recommended to reach MongoDB/S3 via your backend)

    • Define webhook tools in your agent that call your APIs. Use tool parameters/headers (with Secrets or workspace auth) to pass identifiers, then have your API read from MongoDB/S3 and return JSON the agent can use. Supported auth methods include OAuth2, Bearer tokens, and custom headers.
  • MCP (Model Context Protocol) integration (optional)

    • Connect the agent to an MCP server to expose tools/data sources. ElevenLabs supports SSE and HTTP streamable MCP servers with approval controls; you can host your own MCP server that queries MongoDB/S3.
  • Dynamic variables (per‑user context)

    • Inject runtime values (e.g., user_id) into prompts and tool parameters via SDKs or the widget’s dynamic-variables attribute. Tool responses can also update variables.

Security notes

  • Never expose your xi‑api‑key in the browser. For private agents, have your server fetch a signed WebSocket URL or WebRTC token and return it to the client. You can also allowlist domains. Signed URLs expire; the conversation must start within the validity window.

Bottom line

  • You do not need to re‑implement the agent using raw TTS/STT. Use the widget, SDKs (WebRTC/WebSocket), or the WebSocket API to run your existing Agent inside your site, and connect it to your KB plus your MongoDB/S3 via Server Tools or MCP.

If you want, I can point you to the minimal SDK snippet for your preferred stack (React/vanilla) and a server-tool schema example that queries your MongoDB and returns S3 URLs.

ornate pilot
#

Can I get my agent to push parts of the conversation (when the user tells a story) to the MongoDB on its own, or do I need to add a feature to my website for the user to type out their story?