#Deepgram Disconnect While Streaming

1 messages · Page 1 of 1 (latest)

shell condor
#

This is crazy issue because I just don't understand!
I read the documentation and I am spamming the "keep-alive" but Deepgram Still Closes!
BRO Connection closed for stream MZ7b6ca693f016d1a1ed3d3393055ad9eb. Status code: 1011, Reason: Deepgram did not receive audio data or a text message within the timeout window. See https://dpgr.am/net0001. Trying to re-establish.

Keep Alive Code:
def keep_alive(ws): """Send a KeepAlive message every interval seconds.""" keep_alive_msg = json.dumps({"type": "KeepAlive"}) try: ws.send(keep_alive_msg) # Send the KeepAlive message except Exception as e: print(f"Error sending KeepAlive: {e}")

How I spam it:
def send_audio_to_deepgram(audio_chunk, ws): if ws: try: if not ws.sock or not ws.sock.connected: raise websocket.WebSocketConnectionClosedException("Connection is closed") ws.send(audio_chunk, opcode=websocket.ABNF.OPCODE_BINARY) keep_alive(ws)

idle jacinthBOT
#

Thanks for asking your question. Please be sure to reply with as much detail as possible so we can assist you efficiently. Such as:

  • Provide the request_id if you've a question about a transcription response.
  • The options you used or the api.deepgram.com URL you sent your request to, including parameters.
  • Any code snippets you can include.
  • Any audio you can include, or if you can't share it here please email it to us at [email protected] and provide a link to this thread.
forest eagle
#

hey, I appreciate your excitement, but you literally have 4 other threads going

#

can we just consolidate everything to this one thread from this point forward because it's really actually getting in the way of helping you

#

there are a lot of things to keep track of and when you create 2-3 threads a day for the same issue, it literally creates more overhead to try and figure out who needs help, which ones are unresolved/resolved, and time and energy go into sorting it all out

leaden pivot
#

If I’m reading that code correctly, you send the keep alive message every time you send audio? If so then this isn’t taking advantage of keep alive since sending the audio has the same effect, and since the error is reporting at some point you stopped sending audio, it looks like you may have stopped sending keep alive messages too. I would spin up a separate async task for sending keep alive messages not embedded in the send audio function

forest eagle
#

yea, if that's the case, that wont work

#

I also forgot to ask if you are using the threaded/sync or async version of the live client. if you are using the async, the keepalive is not implemented and you need to implement that yourself

#

minimally, if you want to make sure to keep the connection alive, you should send the keepalive on a different thread independent of the send thread and send the keepalive message every 5ish seconds

#

which I believe is just slightly less than 1/2 the timeout time

#

basically if you lose the first one, you have just enough time to get a second keepalive out before the connection gets terminated

shell condor
#

Looks like it was a noob mistake!?!
Ok Ill update

shell condor
#

around 0.9s after sending keep alive it closes

#

also sent 1.5s ago

image.png
#

Even with asyncio, threading, etc.. I'm still getting this err!

shell condor
#

'request_id': 'ccf328da-281b-417d-b36a-26f75471e207'

#

^this is reqeust_id for reference

leaden pivot
#

Can you share more of the code? Something I’ve seen happen before is users sending the keep alive message as a binary message instead of a text message, but I don’t remember off the top of my head Python ws behavior

shell condor
#

def keep_alive(): global prev_keep_alive keep_alive_msg = json.dumps({"type": "KeepAlive"}) current_time = datetime.datetime.now() if (current_time - prev_keep_alive).total_seconds() > 1: for streamSid in deepgram_ws_connections.keys(): ws = deepgram_ws_connections[streamSid] try: if ws.sock and ws.sock.connected: ws.send(keep_alive_msg) # Send the KeepAlive message print(f"Keep Alive Sent to Stream {streamSid}") else: print(f"Keep Alive Socket Not Connected for Stream {streamSid}!") except Exception as e: print(f"Error sending KeepAlive for Stream {streamSid}: {e}") prev_keep_alive = current_time

#

literally called every 50ms lol

#

removed it from teh send_audio_to_deepgram and now its in main lambda function

#

To be honest If I could send a query parameter at the start to just set the timeout to 30s, that would be enough time to speak, respond, and if the user stops, then I can use the "timeout" to say "goodbye" and end the call

leaden pivot
#

I’ll try this in the morning! I want to understand the use case a bit more - this is Twilio audio right? Shouldn’t your function to send audio to Deepgram always be sending audio? What is happening in the application when audio is not being sent to Deepgram? Lastly is this using Python’s “websockets” library or “ws” library (there are a couple)

shell condor
shell condor
#

Will On-Prem make it faster + not close the connection like this?

#

Since I'm using serverless aws lambda?

forest eagle
#

if you are always sending audio... you shouldn't need to send the keepalive. that's why I think there is something else going on

#

going to see about getting an environment setup at some point

shell condor
#

It may not be deepgram but something with my setup?!?! I random 18s delay occurs where nothing is streamed for some reason!!!

image.png
leaden pivot
#

is it saying that only 1~2 ms of audio was processed?

shell condor
#

Its written as:
(04T){hr}:{min}:{sec}.{ms}

shell condor
#

Its a consistent 18s delay where anything I say is not sent to aws lambda

Im not sure if its some sort of throttling

image.png
forest eagle
#

I think I found your problem...
https://docs.aws.amazon.com/apigateway/latest/developerguide/limits.html

Message payload size 128 KB **

** Because of the WebSocket frame-size quota of 32 KB, a message larger than 32 KB must be split into multiple frames, each 32 KB or smaller. This applies to @connections commands. If a larger message (or larger frame size) is received, the connection is closed with code 1009.

Amazon API Gateway quotas and important notes - Amazon API Gateway

Lists the quotas and important notes for Amazon API Gateway.

#

The upshot is don't use lambda with websockets where you are sending even a modest amount of data

#

You are probably hitting that limit and AWS is terminating that connection

#

Oh that's ec2 gateway...

#

Sigh I specifically searched for lambda

#

But I bet there is probably a limit there too

#

Seems like there is a 6MB limit on data transfer

AWS has kept the payload max limit to 6 MB for synchronous flows.
#

It seems like there is definitely a limit happening if it's consistent.

shell condor
#

We need to find the SOURCE!

shell condor
shell condor
#

OK...

#

It is...

#

Solved 😎

#

Basically:

  1. Run Multiple WS Connections per Invocation of Lambda
  2. Store Connection Info in DynaboDB and retrieve if doesnt exist in Lambda Instance
  3. Seperate LLM Generation & Voice Synthesis in Seperate Lambda Call
  4. Handle SSL Errors w/ Auto-Reconnect < 1s
#

Also Introduced AWS Polly
Give it a try please:
+1 256 588 3299

#

If its good let me know any bugs/issues
It should technically be scalable since its fully serverless!

leaden pivot
#

^ as scalable as Amazon allows it to be!

#

will try in a bit