#I don't know if this is some know-how or

1 messages · Page 1 of 1 (latest)

thick smelt
#

what would the use case for this agent handoff be/ what example? Would the purpose just be to feed back purely text information to the original agent a user is speaking to? Or would it be to transfer calls to a specialized agent?

light elbow
#

This is something I would try as a direct solution to your problem.

Basically you create an entirely separate agent, and supply your conversational agent with a tool that would allow it to "call" this separate agent.

Since with the separate agent you are basically not limited by anything, you can give him any amount of tools you want and at any time.

Tool calls from the conversational agent would go to this helper agent, helper agent does it's own tool calling based on the tools you give to it, and returns a response to the conversational agent. Kind of like a proxy

#

If you know ahead of time that you can group up some possible tools into some kind of categories, you could even create multiple helper agents based on those categories.

This would create a Mixture-of-Experts type of situation and increase the quality/reliability of the whole setup

#

OpenAI's Python Agents SDK natively supports MCP. If you don't use Python you can still use the Responses API, but you would need to pass the tools list to the API request (shouldn't be too hard)

4.1-mini, 4.1-nano, 4o-mini would be okay for this (good balance of quality and latency)

#

So yeah this secondary agent would be text-only, since all you need is extended tool calling capability

#

If latency is a problem you can opt-in for Realtime API with text-only modality

thick smelt
#

But dont these secondary agent's exist in elevenlabs land as well? meaning i'd have the same problem where i cant provide runtime tools?

#

or do they exist on the client side?

light elbow
#

What exists in elevenlabs land is just another conversational agent with the same set of issues, yes.

What I'm proposing is a client-side system that you would create yourself, and I imagine this would only work if you are communicating with 11labs agent through their SDKs or WS API directly (may be terribly wrong here)

thick smelt
#

iiuc, you are proposing a client side tool that is actually just a client side agent

light elbow
#

Yeah

thick smelt
#

i see i see

#

latency will probably be interesting, but i will look into it!

#

thanks

#

Just to be clear though, the agents handoff you originally mentioned is to transfer calls entirely to a specialized agent?

#

where the agent is another elevenlabs based agent

light elbow
#

No, by "handoff" i meant exactly proxying tool calling through a single tool to a separate client-side agent

thick smelt
#

i understand what you mean though.

light elbow
#

Yeah sorry for confusion. This wouldn't do anything though because this is essentially the same agent that doesn't support dynamic tools