Use ElevenLabs Agents (Conversational AI) with the native Twilio integration from the Dashboard for inbound taxi calls, and use the Agents/ConvAI API where you need programmatic control (e.g., outbound calls, custom routing, tool calls). This combination gives you the lowest latency, best PSTN audio compatibility, and fastest path to production. (elevenlabs.io)
Why this meets your two requirements
Works in noisy environments
ElevenLabs’ real‑time speech recognition (Scribe v2 Realtime) is designed for low‑latency transcription and explicitly calls out robustness on low‑quality and noisy audio; use it with Agents for phone calls. (elevenlabs.io)
For PSTN calls, configure the agent’s input and output to μ‑law 8 kHz (Twilio’s native codec) to avoid transcoding artifacts that worsen noise; this is the recommended setting in the Twilio/Agents docs. (elevenlabs.io)
Lowest possible latency (avoid 2+ second pauses)
Choose the Flash TTS model for agents or API (“eleven_flash_v2” or “eleven_flash_v2_5”)—it’s optimized for ultra‑low latency (starts speaking in ~75 ms + network). Avoid v3 for real‑time; it’s higher‑latency. (elevenlabs.io)
Tune conversation flow: enable interruptions (barge‑in), set Turn Eagerness to “Eager,” and shorten silence timeouts—these settings reduce perceived pauses on calls. (elevenlabs.io)
If you build with the raw streaming API, use the WebSockets TTS endpoint and send flush: true at turn ends; this is documented to minimize TTFB/latency. (elevenlabs.io)
Recommended architecture
Inbound calls on Twilio numbers
Use the native Twilio integration in the Dashboard. It auto‑configures webhooks, supports inbound and outbound, and is the fastest, most reliable path for PSTN. (elevenlabs.io)
Optionally add a Twilio personalization webhook to fetch caller/context during ringing so the agent can start immediately with the right info. (elevenlabs.io)
External tools (availability/dispatch)
Add Tools to your agent as Webhooks (server tools) so the agent can call your availability and dispatch APIs mid‑call; Tools are first‑class in Agents. You can also use MCP if you standardize tools behind that protocol. (help.elevenlabs.io)
Use tool‑call sounds to mask multi‑second external API waits without dead air. (elevenlabs.io)
Outbound calls and custom routing (optional)
For programmatic outbound, call the ConvAI Twilio “outbound‑call” API. (elevenlabs.io)
If you must keep custom Twilio logic, use the “register‑call” flow; set the agent to μ‑law 8 kHz for both input and output. Note: some transfer features aren’t available in this mode. (elevenlabs.io)
Notes on alternatives
ElevenLabs recently introduced WebRTC for agent sessions (great echo cancellation/background noise removal), but that applies to WebRTC channels, not PSTN via Twilio. For phone
empty cipher
marble sandal