I can't connect with Voice Agents (Unity C# Websockets)

[nova knot]

Hi! My name is Juan Miguel Rochina.

I'm trying to integrate Voice Agent in Unity (using WebSockets in C#), but I can't establish a connection.

I am able to connect to the WebSockets for Speech to Text, but not for Text to Speech or Voice Agent (even when including the authentication Token and using the WebSocket connection, etc).

Is there any special requirement for the websocket? I know .NET 6.0 is required, but it works for TTS and I don't understand why it's not working for Voice Agent or STT.

Here is a simplified version of the function I use to connect, focusing on the connection and event structure:

private IEnumerator InitAgentWebSocket()
{
    if (string.IsNullOrEmpty(apiKey) || string.IsNullOrEmpty(agentId))
        yield break;

#if UNITY_WEBGL && !UNITY_EDITOR
    // WebGL cannot send Authorization headers.
    yield break;
#endif

    string wsUrl = "wss://agent.deepgram.com/v1/agent/converse";
    var headers = new Dictionary<string, string> {
        { "Authorization", $"Token {apiKey}" }
    };

    ws = new WebSocket(wsUrl, headers);

    ws.OnOpen += () =>
    {
        isConnected = true;
        // Send config here
    };

    ws.OnError += (e) =>
    {
        isConnected = false;
        // Handle error
    };

    ws.OnClose += (e) =>
    {
        isConnected = false;
        // Handle close
    };

    var connectTask = ws.Connect();
    while (!connectTask.IsCompleted)
        yield return null;

    if (connectTask.IsFaulted)
    {
        // Handle failed connection
    }
    else if (!isConnected)
    {
        // OnOpen not triggered
    }
}

Thank you in advance for your help!

[trail kayak]

Thanks for asking your question. Please be sure to reply with as much detail as possible so the community can assist you efficiently.
-# If you haven't done so, ensure your Discord and Github profiles are linked to Deepgram so you can earn points to redeem on cool stuff just by being active!

Hey there! You don't seem to have your Discord account connected to your Deepgram account. Not sure how you posted in here, but you should go to https://community.deepgram.com and link your accounts!

It looks like we're missing some important information to help debug your issue. Would you mind providing us with the following details in a reply?

• A request ID that triggered your error or issue.

[nova knot]

I am unable to provide a request ID because the WebSocket connection to wss://agent.deepgram.com/v1/agent/converse fails before any message or response is received from the server.
The only information I get is the error: "Unable to connect to the remote server".

Is there anything else I can provide to help debug this issue?

[frozen crystal]

thanks for sending in. could you provide your project ID? you should be able to see this from console (https://console.deepgram.com/)

[nova knot]

Hello! My project-id is 48bd41af-9d01-4d86-aa0c-3bbd11a73744