voice-call open ai real time tts

[calm pilot]

I move from streaming to real time to lower latency. It's better now at ~1s but real time doesn't have access to tool calls. Any way to get around this so my voice-call agent can handle scheduling and other tasks?

[regal panther]

To help others find answers, you can mark your question as solved via Right click solution message -> Apps -> ✅ Mark Solution

[tepid thunder]

Yes — on current OpenClaw, the intended pattern is Realtime for low-latency speech + openclaw_agent_consult for tool-backed actions.

OpenAI Realtime itself supports function calling, and OpenClaw’s voice-call realtime mode exposes a bridge tool called openclaw_agent_consult that can hand off to the normal OpenClaw agent for deeper reasoning/current info/tools. (platform.openai.com)

For scheduling/write actions, the important bit is toolPolicy:

realtime: {
  enabled: true,
  provider: "openai",
  instructions: "Speak briefly. If the caller asks to schedule, check calendars, or use tools, call openclaw_agent_consult before answering.",
  toolPolicy: "owner",
  providers: {
    openai: { apiKey: "${OPENAI_API_KEY}" },
  },
}

• safe-read-only = consult tool exists, but the regular agent only gets read/search/memory tools.
• owner = consult tool exists and the regular agent can use the normal tool policy, which is what you’d need for scheduling-style tasks.
• none = disables the consult bridge. (raw.githubusercontent.com)

Docs: https://docs.openclaw.ai/plugins/voice-call

If you’re on a build where realtime appears to have no tool access at all, first check:

openclaw --version
openclaw voicecall setup

and update/restart if needed. I’d also keep calendar writes behind an explicit spoken confirmation — realtime voice is great, but accidental “yeah sure” scheduling is a very real footgun.

[calm pilot]

how much latency can I expect by adding a tool policy? Can I just build a tool which returns free/busy for calendar (read only) and a calendar write tool (write only to calendar). I think that abstraction would be enough to secure this surface area

[tepid thunder]

Yep, that abstraction is exactly the right security shape.

Latency-wise:

• Just enabling toolPolicy: basically no extra latency until the realtime model actually calls a tool.
• When openclaw_agent_consult fires: expect an extra realtime tool-call roundtrip + a normal OpenClaw agent run. For simple read-only checks, I’d budget roughly 2–6s; for write flows / calendar APIs / confirmation, 3–10s isn’t surprising.
• A direct purpose-built realtime calendar tool would be faster — often closer to sub-second to ~2s over your current path, depending on calendar API latency.

Important nuance: realtime.tools can expose custom tool schemas, but unless code registers a handler for that tool, the call comes back as unavailable. Current built-in path registers openclaw_agent_consult; custom realtime tools need plugin/code integration, not just config.

For your use case I’d do:

1. calendar_freebusy(start, end, attendees?) — read-only, no sensitive event details unless needed.
2. calendar_prepare_event(...) — returns a proposed event + confirmation phrase.
3. calendar_commit_event(proposalId) — write-only, only after explicit spoken confirmation.

That’s much safer than giving the whole voice agent broad “calendar access”. If you want it through today’s consult path, use toolPolicy: "owner" with a dedicated voice-call agent whose available tools are only those calendar wrappers. If you want the lower-latency version, implement them as direct realtime tool handlers.