openai real time voice tool calling

[vapid shell]

i need help in tool calling

[fickle phoenix]

Hi @vapid shell. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:

---

To implement tool calling with OpenAI's Realtime API, you can define custom functions as tools that the model can use. The Realtime API allows for low-latency, multi-modal experiences, supporting both text and audio inputs and outputs, as well as function calling capabilities.

Defining Tools

You can define tools in the session configuration. Here's an example of how to define a tool for getting weather information:

{
  "tools": [
    {
      "name": "get_weather",
      "description": "Get the weather at a given location",
      "parameters": {
        "type": "object",
        "properties": {
          "location": {
            "type": "string",
            "description": "Location to get the weather from"
          },
          "scale": {
            "type": "string",
            "enum": ["celsius", "fahrenheit"]
          }
        },
        "required": ["location", "scale"]
      }
    }
  ]
}

Handling Function Calls

When the server decides to call a function, it will send a function_call item. You can handle this by responding with a conversation.item.create message with type: "function_call_output".

Example of Tool Calling

Here's an example of how you might set up a tool call using the OpenAI API:

const tools = [
    {
        type: "function",
        function: {
            name: "get_weather",
            strict: true,
            parameters: {
                type: "object",
                properties: {
                    location: { type: "string" },
                    unit: { type: "string", enum: ["c", "f"] }
                },
                required: ["location", "unit"],
                additionalProperties: false
            }
        }
    }
];

const response = await openai.chat.completions.create({
model: "gpt-4o",
messages: [
{
role: "user",
content: "Can you tell me the weather in Tokyo?"
}
],
tool_choice: "required",
tools
});

console.log(response);

For more detailed information, you can refer to the **Realtime API documentation** (<https://platform.openai.com/docs/guides/realtime#tool-calling>) and the **Function Calling guide** (<https://platform.openai.com/docs/guides/function-calling#advanced-usage>).

If you need further assistance, please let me know!
----------------------
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
----------------------

@vapid shell Kapa.ai is still learning and improving, please let me know how I did by reacting below.