Hello. Does anyone know how to integrate with evenlabs & twilio, what is in the documentation does | ElevenLabs | Page 1

quartz dewBOT Dec 15, 2024, 10:02 AM

#

AI-generated Answer for vlad06611

To integrate ElevenLabs with Twilio, you can follow these general steps:

Set Up Your Environment: Ensure you have an ElevenLabs account with a configured Conversational Agent and a Twilio account with an active phone number. You'll also need Python 3.7+ or Node.js 16+ and ngrok for local development.
Configure Your Agent: Set the audio format to "μ-law 8000 Hz" in both the TTS Output and Input Format settings of your agent.
Implement the Integration: You can use either JavaScript or Python for the backend setup. If you're using Node.js, initialize your project, install necessary dependencies, and create project files. Use the ElevenLabs API to convert text to speech and send the audio to Twilio via a WebSocket connection.
Set Up Twilio: Use ngrok to create a public URL for your local server. Configure Twilio to use this URL for incoming calls by setting the webhook in the Twilio Console.
Test the Setup: Make a call to your Twilio number to ensure the integration works and the AI voice is correctly played.

If you encounter issues, ensure your ngrok URL is correctly set in Twilio settings, your server is running and accessible, and your firewall isn’t blocking WebSocket connections. For more detailed guidance, you can refer to the relevant documentation or examples on GitHub. If problems persist, consider reaching out to developers familiar with both platforms for further assistance.

wanton saddle Dec 15, 2024, 10:06 AM

#

I want use nodejs

quartz dewBOT Dec 15, 2024, 10:06 AM

#

AI-generated Answer for vlad06611

To integrate ElevenLabs with Twilio using Node.js, you can follow these steps:

Set Up Your Environment: Ensure you have an ElevenLabs account with a configured Conversational Agent and a Twilio account with an active phone number. You'll also need Node.js 16+ and ngrok for local development.
Initialize Your Project: Create a new folder for your project and initialize it:
```
mkdir elevenlabs-twilio
cd elevenlabs-twilio
npm init -y
```
Install Dependencies: Install the necessary packages:
```
npm install elevenlabs express express-ws twilio
```

Create Your Files: Set up your application files. For example, create a src/app.ts file with the following code:

import 'dotenv/config';
import express, { Response } from 'express';
import ExpressWs from 'express-ws';
import VoiceResponse from 'twilio/lib/twiml/VoiceResponse';
import { ElevenLabsClient } from 'elevenlabs';
import { type WebSocket } from 'ws';
import { Readable } from 'stream';

const app = ExpressWs(express()).app;
const PORT: number = parseInt(process.env.PORT || '5000');

const elevenlabs = new ElevenLabsClient();
const voiceId = '21m00Tcm4TlvDq8ikWAM';
const outputFormat = 'ulaw_8000';
const text = 'This is a test. You can now hang up. Thank you.';

function startApp() {
  app.post('/call/incoming', (_, res: Response) => {
    const twiml = new VoiceResponse();
    twiml.connect().stream({
      url: `wss://${process.env.SERVER_DOMAIN}/call/connection`,
    });
    res.writeHead(200, { 'Content-Type': 'text/xml' });
    res.end(twiml.toString());
  });

  app.ws('/call/connection', (ws: WebSocket) => {
    ws.on('message', async (data: string) => {
      const message: {
        event: string;
        start?: { streamSid: string; callSid: string };
      } = JSON.parse(data);

      if (message.event === 'start' && message.start) {
        const streamSid = message.start.streamSid;
        const response = await elevenlabs.textToSpeech.convert(voiceId, {
          model_id: 'eleven_turbo_v2_5',
          output_format: outputFormat,
          text,
        });

        const readableStream = Readable.from(response);
        const audioArrayBuffer = await streamToArrayBuffer(readableStream);

        ws.send(
          JSON.stringify({
            streamSid,
            event: 'media',
            media: {
              payload: Buffer.from(audioArrayBuffer as any).toString('base64'),
            },
          }),
        );
      }
    });

    ws.on('error', console.error);
  });

  app.listen(PORT, () => {
    console.log(`Local: http://localhost:${PORT}`);
    console.log(`Remote: https://${process.env.SERVER_DOMAIN}`);
  });
}

function streamToArrayBuffer(readableStream: Readable) {
  return new Promise((resolve, reject) => {
    const chunks: Buffer[] = [];

    readableStream.on('data', (chunk) => {
      chunks.push(chunk);
    });

    readableStream.on('end', () => {
      resolve(Buffer.concat(chunks).buffer);
    });

    readableStream.on('error', reject);
  });
}

startApp();

Set Up ngrok: Use ngrok to create a public URL for your local server:
```
ngrok http 5000
```
Configure Twilio: In the Twilio Console, set the webhook for incoming calls to your ngrok URL with the path /call/incoming.
Test the Setup: Make a call to your Twilio number to ensure the integration works and the AI voice is correctly played.

For more detailed guidance, you can refer to the examples on GitHub. If you encounter issues, ensure your ngrok URL is correctly set in Twilio, and your server is running and accessible.

wanton saddle Dec 15, 2024, 10:09 AM

#

This code you provide does not work

#Hello. Does anyone know how to integrate with evenlabs & twilio, what is in the documentation does