does the assistants API support calling multiple parallel tool calls?
#Does the assistants API support parallel tool calls?
proud hill
stark oarBOT
Hi @proud hill. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:
Yes, the Assistants API does support parallel tool calls. This feature allows the model to perform multiple function calls together, which can be especially useful if the functions take a long time to process. The results of these function calls can be resolved in parallel, reducing round trips with the API.
For example, the model may call functions to get the weather in 3 different locations at the same time, which will result in a message with 3 function calls in the tool_calls array, each with an id. To respond to these function calls, you add 3 new messages to the conversation, each containing the result of one function call, with a tool_call_id referencing the id from tool_calls [source (https://platform.openai.com/docs/guides/function-calling)].
However, it's important to note that not all models support parallel tool calls. For instance, only 1106 models support parallel tool calls of retrieval functions [source (https://community.openai.com/t/640916)].
Here's an example of how to use parallel function calling with the chat models:
messages = []
messages.append({"role": "system", "content": "Don't make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous."})
messages.append({"role": "user", "content": "what is the weather going to be like in San Francisco and Glasgow over the next 4 days"})
chat_response = chat_completion_request(
messages, tools=tools, model='gpt-3.5-turbo-1106'
)
assistant_message = chat_response.choices[0].message.tool_calls
assistant_message
[source (https://cookbook.openai.com/examples/how_to_call_functions_with_chat_models#parallel-function-calling)]
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
proud hill
@stark oar are you sure, i think you are referring to the chat api not the assistants api
stark oarBOT
Let me check the knowledge base for you. One moment! :books:
I apologize for the confusion. You're correct. The information I provided earlier was about the Chat API, not the Assistants API.
In the context of the Assistants API, it is indeed capable of accessing multiple tools in parallel. These can be both OpenAI-hosted tools like Code Interpreter and Knowledge Retrieval, or tools that you build or host via Function Calling. This allows the assistant to perform more complex tasks and provide more comprehensive responses to user queries [source (https://platform.openai.com/docs/assistants/how-it-works)].