customise functions per agent

[elfin pumice]

Hi guys,
I just reviewed that example https://github.com/microsoft/autogen/blob/main/notebook/agentchat_langchain.ipynb

My question, why functions map is defined for user_proxy not for assistant? How to split the tools which should be used by different agent? Lets say agent1 has access to func1, func2. Agent2 - func3, func4 etc. Same with knowledge base.

Register the tool and start the conversation

user_proxy.register_function(
function_map={
custom_tool.name: custom_tool._run,
read_file_tool.name: read_file_tool._run,
}
)

chatbot = autogen.AssistantAgent(
name="chatbot",
system_message="For coding tasks, only use the functions you have been provided with. Reply TERMINATE when the task is done.",
llm_config=llm_config,
)

[round wigeon]

The natural language definition is needed for LLM to suggest a function call.
The programming language definition is needed for function execution.
In this example, the AssistantAgent uses LLM to suggest function call; the UserProxyAgent takes the suggested function call and execute it.

[elfin pumice]

@round wigeon thanks, how about multiple AssistantAgents? and we need that each of them will have own list of allowed functions to use?

[round wigeon]

Each of them can have their own list of functions.

[elfin pumice]

@round wigeon can you share some doc link where i can read more about functon_map? still not clear why functon_map should be part of confic for assistant and for user_agent.

[round wigeon]

@lament forum could you share some doc? Or add them if not available, and explain the answer here?

[elfin pumice]

In this example, the AssistantAgent uses LLM to suggest function call; the UserProxyAgent takes the suggested function call and execute it.
@round wigeonI just reviewed a couple of examples. Maybe I'm wrong, but it's weird that the agent which asks should know the function the responding agent should use. It's the same between us. Just imagine if I had to inform you of the topic you should use to reply to my question

[round wigeon]

I tell you that I can execute functions. I ask you to suggest the right functions and bindings to use for a specific question

[elfin pumice]

Why should the responsibility be placed on the 'asking agent'? When the agent asks, the LLM should understand which function satisfies the request and suggest using the responding agent. For example, if we have three agents, each with a unique function to perform its own tasks, the mapping would look like:

Agent1 function map:
func1 - own
func2 - from Agent2
func3 - from Agent3

Agent2 function map:
func1 - from Agent1
func2 - own
func3 - from Agent3

Agent3 function map:
func1 - from Agent1
func2 - from Agent2
func3 - own"

Does this help clarify things?

[teal arrow]

I wanted to follow up on this thread as I am running into a similar problem and trying to figure it out. @elfin pumice did you end up getting it to work? If so could you share the code?

[brittle surge]

Hi @elfin pumice I spent some time with function_call and I think I got some reasonable understanding.

I even created an agent that can self execute its own function call
https://gist.github.com/bonadio/96435a1b6ccc32297aa8cc1db7cfc381

Usually we have the user_proxy -> assistant

In this case assistant has the function_call definition in the llm_config and user_proxy has the function_map so it knows what "program" to run when the assistant returns the function_call named function.

At this point I was asking myself "why should user_proxy runs the function and not the assistant?". At first I thought it was a bad design and I created the self execution agent. But then I found this example that made me think different
https://github.com/sugarforever/LangChain-Advanced/blob/main/Integrations/AutoGen/autogen_langchain_uniswap_ai_agent.ipynb

Here the assistant receives the result of the function_call from the user_proxy at each iteration and it can "reason about this result" so this back and forth gets richer.

For a simple use case the assistant could execute the function_call by itself and return the result, but having the user_proxy executing the code and passing the result back to assistant gives the opportunity of the assistant to "think about the result" because the assistant is the one with access to the LLM

Hope that make sense

[teal arrow]

So I got a follow up question here about my own code 🙂 sorry to hijack this thread.... So here is what I got

    "functions": GoogleCalendar_functions,
    "config_list": config_list,
    "request_timeout": 120,
}

# Get current date
current_date = datetime.datetime.now().date().strftime('%Y-%m-%d')  # Format as 'YYYY-MM-DD'

# Customize system message to include current date
system_message = f"For handling the user's Google Calendar on {current_date}, only use the functions you have been provided with. Reply TERMINATE when the task is done."

Google_Calendar_bot = autogen.AssistantAgent(
    name="Google Calendar Assistant",
    system_message=system_message,
    llm_config=llm_config,
)



# Create a UserProxyAgent instance named "user_proxy"
user = autogen.UserProxyAgent(
    name="user",
    max_consecutive_auto_reply=0,  # terminate without auto-reply
    human_input_mode="ALWAYS",
)

def ask_planner(message):
    user.initiate_chat(Google_Calendar_bot, message=message)
    # return the last message received from the planner
    return user.last_message()["content"]

# Register the functions
user.register_function(
    function_map={
        "fetch_google_calendar_events": gcal_agent.fetch_google_calendar_events,
        "create_google_calendar_events": gcal_agent.create_google_calendar_events,
        "update_google_calendar_event_by_criteria": gcal_agent.update_google_calendar_event_by_criteria,
        "delete_google_calendar_events_based_on_criteria": gcal_agent.delete_google_calendar_events_based_on_criteria,
    }
)

assistant = autogen.AssistantAgent(
    name="assistant",
    llm_config={
        "temperature": 0,
        "timeout": 600,
        "seed": 42,
        "config_list": config_list,
        "functions": [
            {
                "name": "ask_planner",
                "description": "ask planner has access to my google calendar.",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "message": {
                            "type": "string",
                            "description": "question to ask planner. Make sure the question include enough context, such as the code and the execution result. The planner does not know the conversation between you and the user, unless you share the conversation with the planner.",
                        },
                    },
                    "required": ["message"],
                },
            },
        ],
    }
)


# create a UserProxyAgent instance named "user_proxy"
user_proxy = autogen.UserProxyAgent(
    name="user_proxy",
    human_input_mode="TERMINATE",
    max_consecutive_auto_reply=10,
    is_termination_msg=lambda x: "content" in x and x["content"] is not None and x["content"].rstrip().endswith("TERMINATE"),
    code_execution_config={"work_dir": "planning"},
    function_map={"ask_planner": ask_planner},
)

# Start the conversation
# the assistant receives a message from the user, which contains the task description
user_proxy.initiate_chat(
    assistant,
    message="""What is on my schedule for today?""",
)```

The problem I am running into is that it passes the right function name and arguments, but then my user agent doesn't know how to use the functions even though I have the functions mapped?

--------------------------------------------------------------------------------
assistant (to user_proxy):

***** Suggested function Call: ask_planner *****
Arguments: 
{
  "message": "What is on my schedule for today?"
}
************************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...

>>>>>>>> EXECUTING FUNCTION ask_planner...
user (to Google Calendar Assistant):

What is on my schedule for today?

--------------------------------------------------------------------------------
Google Calendar Assistant (to user):

***** Suggested function Call: fetch_google_calendar_events *****
Arguments: 
{
  "start_time": "2023-11-09T00:00:00",
  "end_time": "2023-11-09T23:59:59"
}
*****************************************************************

--------------------------------------------------------------------------------
Provide feedback to Google Calendar Assistant. Press enter to skip and use auto-reply, or type 'exit' to end the conversation:  

>>>>>>>> NO HUMAN INPUT RECEIVED.
user (to Google Calendar Assistant):



--------------------------------------------------------------------------------
Google Calendar Assistant (to user):```

So the question here is how do I get the user agent to actually take the suggested function call and arguments and run them with the functions I am calling 🤨

[brittle surge]

My suggestion is make only "user" and "Google_Calendar_bot" to work then add "user_proxy" and "assistant"

[teal arrow]

I think that is the right idea, I am trying to make write the code so I can add as many agents as I want tthat are managed by the assistant. Maybe I am going in the wrong direction about this?

[round wigeon]

The function is registered to the "user" agent.

[teal arrow]

Should it be registered to the assistant?

[round wigeon]

no, it's correct.

It seems a bug. Are you using openai models?

[teal arrow]

I reverted this morning becaues the newest version of openai was breaking auto gen

What version of openai should I be on?

[round wigeon]

Check this: https://microsoft.github.io/autogen/docs/Installation#python

The latest release is v0.2.0b3, which works w/ openai v1

[teal arrow]

I'll get working on this and keep ya'll updated 🙂 thank you!!! @round wigeon appreciate your help!