#Need some help to understand the twilio deepgram documentation

1 messages · Page 1 of 1 (latest)

mellow girderBOT
#

Thanks for asking your question. Please be sure to reply with as much detail as possible so we can assist you efficiently. Such as:

  • Provide the request_id if you've a question about a transcription response.
  • The options you used or the api.deepgram.com URL you sent your request to, including parameters.
  • Any code snippets you can include.
  • Any audio you can include, or if you can't share it here please email it to us at [email protected] and provide a link to this thread.
cinder rover
#

Just some more info of what I was trying to do. I wanted to keep the call open and only transcribe what the person who called was saying. I looked into some other threads on the server and could not figure out what I was doing differently

cinder rover
#

I ran the client using websocat to see the outputs and this is what occurred after the call ended. https://pastebin.com/nHQV8iBR (this is server output). The client is correctly grabbing the callsid but no messages get printed out like what was shown in the example. It is straight from the tutorial but here it is for reference. https://pastebin.com/G4uDs3Uz

Pastebin
client connection incomingtwilio connection incomingdeepgram_sender...

Pastebin.com is the number one paste tool since 2002. Pastebin is a website where you can store text online for a set period of time.

Pastebin
import asyncioimport base64import jsonimport sysimport websocketsim...

Pastebin.com is the number one paste tool since 2002. Pastebin is a website where you can store text online for a set period of time.

cinder rover
#

fixed it

fallow wigeon
#

Would you mind sharing how? I am trying to do the same in twilio. Thank you in advance!

cinder rover
fallow wigeon
#

Certainly. Let me finish cleaning up my cloud instance. I'll send after

fallow wigeon
cinder rover
fallow wigeon
fallow wigeon
fallow wigeon
cinder rover
#

I use the websocket

#

do you have a twilio bin?

fallow wigeon
#

I do

cinder rover
#

send it

#

i want to confirm you have a pause inside

fallow wigeon
cinder rover
#

okay everything looks good but you are missing this

<Pause length="50"/>

The 50 is the amount of seconds you are pausing for. Now let me explain why

After you connect its going to play the next segment and if there is nothing the call will just hang up

#

That is the conclusion I came to when I ran into the issue

#

if this is your problem you can extend the call by sending a response back to twlio with an extended pause duration so that the call does not hang up

fallow wigeon
#

are you using a cloud based wss?

cinder rover
# fallow wigeon are you using a cloud based wss?

im hosting it on my computer with ngrok. my twilio bin looks like this

<?xml version="1.0" encoding="UTF-8"?>
<Response>
<Connect>
<Stream url="wss://67c5-24-151-91-112.ngrok-free.app" track="inbound_track"/>
</Connect>
<Pause length="50"/>
</Response>

fallow wigeon
#

are you not getting both side of the conversation?

cinder rover
#

I am only using it to record what is being said into the phone call. I am not re-routing the call to another number. I am probably just using it for something different but the same concept for the disconnection should still apply.

fallow wigeon
#

I am connecting the lead to an AI voice Agent and streaming the full audio

cinder rover
#

ah I see

#

and right now are you just figuring out how to get the transcript from what the user is saying?

#

I mean is that your goal

fallow wigeon
#

I need the complete conversation so I can monitor transcript chunks for keywords that trigger an action, like sending a scheduling link or purchase link

cinder rover
#

okay i understand, you do not need to change your track type like I have.

#

I just sent what I have as an example

fallow wigeon
#

Thank you. What files do you use for your wss and connection to deepgram?

#

I feel like I have way overcomplicated this

cinder rover
#

okay let me take a step back and lets take another look at the issue. Here is what I think happens

  1. You call the twilio number
  2. it picks up and speaks the automated message
  3. hangs up

is that correct?

#

This portion here

<Dial>
    <Number>{{leadPhoneNumber}}</Number>
</Dial>

you want to connect to an operator?

fallow wigeon
#

Negative

Survey form submitted -> Pipedream workflow -> Twilio Outbound call request made to Studio Flow -> Call_lead widget calls the lead -> Connect Voice AI Agent widget. This worked (did not have deepgram involved)

now, when the outbound call is triggered, no call happens to the lead and it fails at the function or twiml OR with some corrections, it gives a 200 response from the wss but never calls the lead, it just hangs up in that stage of execution

#

the Twiml I provided was my most recent attempt

cinder rover
#

oh okay sounds like most of it is working. So you are saying that everything works but calling the lead phone number?

fallow wigeon
#

AFAIK

#

I still don't get any logs from the wss though

#

I need to wipe my cloud deployment and start from scratch with the recommended deployment for this simple task

cinder rover
#

okay you do not need to pause then since you want to just connect it to another call. I was mis interpreting that part. What is the format of the number?

fallow wigeon
#

+1918xxxxxxx

cinder rover
#

okay I took a look at the documentation and here is the format present.

<?xml version="1.0" encoding="UTF-8"?>
<Response>
    <Dial>415-123-4567</Dial>
</Response>

The number is in xxx-xxx-xxxx form

fallow wigeon
#

ok.

#

What do you suggest then for the wss side for connecting to deepgram?

#

Also, do you recommend placing the twiml as the last step of the stuio flow?

cinder rover
#

Why are you connecting google cloud to deepgram to transcribe the audio?

fallow wigeon
#

I thought I needed the wss in the cloud to connect

#

I created a wss in the cloud for scalability

cinder rover
#

well if you are able to connect to the twilio web socket then that is fine

fallow wigeon
#

wdym?

#

the websocket in the twiml I shared is to the google cloud wss

cinder rover
#

I did not use google cloud so i was just saying that if it works it works lol. if you went through the trouble to set it up I would not worry about it

fallow wigeon
#

Am I able to connect directly to deepgram from the twiml?

cinder rover
#

what happens is this. you will connect to the twilio websocket in the backend, get the audio, send the audio through deepgram websocket and then return the json with the transcript

fallow wigeon
#

I am way off base then

cinder rover
#

okay, was twilio giving you any other problems?

fallow wigeon
#

not until I tried to stream to deepgram

#

Would you be so kind to elaborate on this: what happens is this. you will connect to the twilio websocket in the backend, get the audio, send the audio through deepgram websocket and then return the json with the transcript?

cinder rover
#

yeah so that would be the work flow, you set a proxy to connect to twilio and then you are able to connect to the websocket and get the audio from the call. Then when you get that audio returned you send that to the deepgram websocket and then it will give you a json with the returned information. From there you can get the transcript.

fallow wigeon
#

Is this for post call or livestream?

cinder rover
#

this occurs during the phone call

fallow wigeon
#

Thank you for your help