#How can I give my voice agent web_search access by tool calling?

1 messages · Page 1 of 1 (latest)

tiny bison
#

How can I give my voice agent web_search access by tool calling?

hidden pelicanBOT
#

Hi @tiny bison. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:

#

You can give your Voice Agent “web search” access by defining a function (tool) in agent.think.functions and then implementing that function either:

  • Server-side via an HTTP endpoint (Deepgram calls it for you), or
  • Client-side where your app receives a FunctionCallRequest and performs the web search itself.

1. Define a web_search function in the Settings

In your Voice Agent Settings message, add a function definition under agent.think.functions. For a server-side (HTTP) tool:

{
  "type": "Settings",
  "agent": {
    "think": {
      "provider": {
        "type": "open_ai",
        "model": "gpt-4o-mini",
        "temperature": 0.7
      },
      "prompt": "You are a helpful assistant that can search the web when needed.",
      "functions": [
        {
          "name": "web_search",
          "description": "Search the web for up-to-date information.",
          "parameters": {
            "type": "object",
            "properties": {
              "query": {
                "type": "string",
                "description": "The search query to look up on the web"
              }
            },
            "required": ["query"]
          },
          "endpoint": {
            "url": "<https://your-api.com/web-search",>
            "method": "post",
            "headers": {
              "authorization": "Bearer {{token}}"
            }
          }
        }
      ]
    }
  }
}

This follows the same pattern as the check_order_status example: the LLM decides when to call web_search, Deepgram sends an HTTP request to your endpoint, and uses the response in its reply.[Function calling; Configure agent.think]

#

2. Implement the web search endpoint

On your server (any language), implement POST /web-search to:

  1. Parse the JSON body (it will include the arguments the LLM produced).
  2. Run your actual web search (e.g., call a search API).
  3. Return a concise text/JSON result that the LLM can read back to the user.

Deepgram’s docs don’t prescribe the exact body/response schema beyond the example pattern, so you’ll align it with how your LLM/tooling expects to see function results. The check_order_status example shows the same style of function endpoint.[Configure example]

3. (Alternative) Client-side web search tool

If you don’t want Deepgram to call an HTTP endpoint, omit the endpoint field. Then the function is “client-side”:

"functions": [
  {
    "name": "web_search",
    "description": "Search the web for up-to-date information.",
    "parameters": {
      "type": "object",
      "properties": {
        "query": {
          "type": "string",
          "description": "The search query to look up on the web"
        }
      },
      "required": ["query"]
    }
  }
]

When the LLM decides to call web_search, the Voice Agent server sends your client a FunctionCallRequest with client_side: true. Your client must:

  1. Receive the FunctionCallRequest.
  2. Run the web search locally (e.g., call a search API from your app).
  3. Send back a FunctionCallResponse with the result.[FunctionCallRequest; Function calling flow; Client-side example]
#

This is the same pattern as the get_weather client-side example; you just replace the weather API with your web search API.

In summary: define a web_search function in agent.think.functions and either attach an endpoint (server-side tool) or handle FunctionCallRequest messages in your client (client-side tool). The LLM will then autonomously invoke web_search whenever it needs live web information.

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

#

@tiny bison Kapa.ai is still learning and improving, please let me know how I did by reacting below.

tiny bison
#

@hidden pelican i know that but open ai provides web_search natively, so it should work.

hidden pelicanBOT
#

Let me check the knowledge base for you again. One moment! :books:

#

Deepgram’s docs don’t currently mention OpenAI’s native web_search / tools feature or any special handling for it, so I can’t confirm that it “just works” when you set agent.think.provider.type = "open_ai". The documented way to give a Voice Agent tool access is still via agent.think.functions (with optional endpoint for server-side tools) or client-side FunctionCallRequest handling.[Function calling; Agent config]

What is documented:

  • You choose OpenAI as the provider with:

    "think": {
  "provider": {
    "type": "open_ai",
    "model": "gpt-4o-mini",
    "temperature": 0.7
  }
}

    and Deepgram treats it as an OpenAI Chat Completions–style LLM.[LLM models; Settings example]

  • Tool / function calling for Voice Agents is configured explicitly via agent.think.functions (with optional endpoint for server-side execution, or omitted for client-side execution).[Function calling; Full example]

The knowledge sources do not state that Deepgram will automatically expose OpenAI’s own tools / web_search capability without you defining a corresponding function in agent.think.functions, nor do they show a way to pass through an arbitrary OpenAI tools array.

Given that limitation in the docs, the safe, supported pattern today is:

#
  1. Use OpenAI as the LLM (provider.type = "open_ai", model = "gpt-4o-mini" or similar).[Supported LLM providers]
  2. Define a web_search function in agent.think.functions and implement it via:

If Deepgram adds first-class support for OpenAI’s native web_search tools, it isn’t reflected in the provided knowledge sources, so I can’t describe or recommend that path based on the docs you’ve shared.

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

#

@tiny bison Kapa.ai is still learning and improving, please let me know how I did by reacting below.