#how to use Ai agents on elevenLabs

1 messages · Page 1 of 1 (latest)

keen oarBOT
#
AI-generated Answer for smen2

To use AI agents on ElevenLabs, follow these steps:

1. Configuring Your Agent

  • Go to the Conversational AI section.
  • You can start with a blank template or choose a preset template. Preset templates come with predefined first messages and system prompts tailored to specific personas.

2. Setting Up Your Agent

  • Choosing Your LLM: Select from leading models like Anthropic, OpenAI, or Google. Soon, you’ll also be able to integrate your custom LLM via a server.
  • System Prompt: Define your agent’s behavior, style, and tone using a system prompt. For example, you can set it up as a customer support agent or a librarian.
  • Knowledge Base: Add additional information to your agent. Non-enterprise users can upload up to 5 files or links, with a total limit of 20MB and 300,000 characters.

3. Choosing a Voice

  • Use any default or library voice, or create a custom voice clone. For conversational use cases, it’s recommended to use Default Voices like Chris or Jessica.

4. Testing Your Agent

  • Test your agent by interacting with it in the dashboard. You can review call recordings and transcripts in the History tab.

5. Deploying the Agent

  • Deploy your agent to your site by customizing the widget and embedding the code.
  • Alternatively, use the JavaScript or Python SDKs to build a custom interface.

For more details, you can explore the Conversational AI overview.

tawdry adder
#

How to use it via api

keen oarBOT
#
AI-generated Answer for smen2

To use AI agents on ElevenLabs via the API, you can follow these steps:

1. Set Up Your Agent

  • Create and configure your AI agent on the ElevenLabs platform. You can customize the agent's behavior, system prompt, and knowledge base. For more details, refer to the Conversational AI Overview.

2. Establish a WebSocket Connection

  • ElevenLabs provides a WebSocket API for real-time, interactive voice conversations. Use the following endpoint to connect:
    wss://api.elevenlabs.io/v1/convai/conversation?agent_id=<your-agent-id>
  • For private agents, you’ll need to generate a signed URL for authentication. Use the API key to obtain the signed URL:
    curl -X GET "https://api.elevenlabs.io/v1/convai/conversation/get_signed_url?agent_id=<your-agent-id>" \
     -H "xi-api-key: <your-api-key>"

3. Send and Receive Messages

  • Client-to-Server Messages: Send audio data in chunks to the server in the required format (PCM 16-bit mono, Base64 encoded, 16,000 Hz sample rate). Example:
    {
  "user_audio_chunk": "<base64-encoded-audio-data>"
}
  • Server-to-Client Messages: The server will respond with messages such as transcriptions, agent responses, and audio chunks.

4. Handle Real-Time Communication

  • Implement logic to manage the conversation flow, including handling interruptions, turn-taking, and latency optimization. Use ping-pong messages to monitor latency and ensure a smooth experience.

5. Test and Deploy

  • Test your agent using the ElevenLabs dashboard or your application. Once satisfied, deploy it to your desired platform.

For more technical details, you can explore the API Reference for Conversational AI.

tawdry adder
#

how to add tools

keen oarBOT
#
AI-generated Answer for smen2

To use AI agents on ElevenLabs via the API, you can follow these steps:

  1. Set Up Your Agent: Start by creating an AI agent in the ElevenLabs platform. You can configure the agent with a system prompt, knowledge base, and voice settings. For more details on setting up agents, refer to the Conversational AI overview.

  2. Obtain API Credentials: You’ll need an API key and Agent ID to interact with the API. These can be found in your ElevenLabs account.

  3. Use the SDK or API: ElevenLabs provides SDKs (e.g., JavaScript or Python) to simplify API integration. You can also directly use the API endpoints to interact with your agent. For example, you can retrieve a signed URL for secure communication with the agent.

    Example of creating a signed URL endpoint in your backend:

    app.get('/api/signed-url', async (req, res) => {
    try {
        const response = await fetch(
            `https://api.elevenlabs.io/v1/convai/conversation/get_signed_url?agent_id=${process.env.AGENT_ID}`,
            {
                method: 'GET',
                headers: {
                    'xi-api-key': process.env.XI_API_KEY,
                }
            }
        );

        if (!response.ok) {
            throw new Error('Failed to get signed URL');
        }

        const data = await response.json();
        res.json({ signedUrl: data.signed_url });
    } catch (error) {
        console.error('Error:', error);
        res.status(500).json({ error: 'Failed to get signed URL' });
    }
});

  4. Start a Conversation: Use the signed URL to initiate a session with the agent. You can send and receive audio or text data through the API.

Adding Tools to AI Agents

AI agents on ElevenLabs can have up to five tools added across all standard subscription plans. If you need more than five tools, this is only possible with an Enterprise plan. Tools can be configured to enhance the agent's functionality, but the exact process for adding them depends on your use case and setup.

If you need further assistance or want to explore Enterprise options, you can contact the Enterprise team.

tawdry adder
#

can you have multiple knowledge bases on 1 agent

keen oarBOT
#
AI-generated Answer for smen2

Yes, you can have multiple knowledge bases for a single AI agent However, non-enterprise users are limited to 5 files or links with a combined size of 20MB or 300,000 characters. If you need to exceed these limits, you may want to consider upgrading to an Enterprise plan, which offers expanded capabilities.

If you're looking to dynamically adjust the knowledge base for specific conversations, you can use the "Agent Conversation Overrides" feature. This allows you to temporarily customize the agent's knowledge base for a particular session without permanently altering it. The agent will still retain access to its original knowledge base.

tawdry adder
#

can you show domunetion on agent conversation overides

keen oarBOT
#
AI-generated Answer for smen2

Yes, you can use conversation overrides with the API to dynamically adjust the behavior of your AI agent. This allows you to pass specific context, such as prompts or first messages, directly when starting a session, without needing to configure these settings manually in the UI.

For example, you can dynamically set the system prompt, first message, or even override the voice ID during a session. Here's a sample code snippet:

const conversation = await Conversation.startSession({
  overrides: {
    agent: {
      prompt: {
        prompt: `The customer's bank account balance is ${customer_balance}. They are based in ${customer_location}.`
      },
      first_message: `Hi ${customer_name}, how can I help you today?`,
    },
    tts: {
      voiceId: "" // override the voice ID dynamically
    }
  },
  ...
});

This approach allows you to create a single agent and provide dynamic context for each interaction at runtime.

If you need further details, you can explore the Conversational AI documentation.

tawdry adder
#

how to acces to secrets

keen oarBOT
#
AI-generated Answer for smen2

The content provided does not include any information about AI agents, tools, knowledge bases, or conversation overrides. If you're looking for details on how to use ElevenLabs' services via API, you can refer to the API References for guidance on authentication and usage.

For further assistance or specific questions, please provide more details or clarify your request.

tawdry adder
#

How to use secrets in ai agents

keen oarBOT
#
AI-generated Answer for smen2

The provided content does not include information about using secrets in AI agents or how to access them. If you need assistance with this, I recommend checking the API References or the Product Documentation for more details. If you have a specific question, feel free to ask!

tawdry adder
#

can you open links with ai agents to get context from them

keen oarBOT
#
AI-generated Answer for smen2

Currently, ElevenLabs AI Agents cannot directly access external websites or open links to gather context. While you can add URLs to the knowledge base, the AI will not be able to navigate or extract information from those links. Instead, you can upload relevant documents or provide structured information in the knowledge base to guide the agent's responses.

If you need more advanced functionality, such as processing external links, you might consider building a custom solution using the API.

tawdry adder
#

how to do it:

const conversation = await Conversation.startSession({
  agentId: '<your-agent-id>',
});