Can you call custom functions with streaming enabled, if so how does that work?
#How to call custom functions with streaming in node
topaz swallow
royal fjordBOT
Hi @topaz swallow. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:
Based on the knowledge sources provided, there isn't a direct example of calling custom functions with streaming enabled in Node.js. However, there are some insights that can be inferred from the sources.
In the Forum Post: Streaming with recursive function / tools calling (https://community.openai.com/t/687313), a user is trying to call functions with streaming enabled but is facing issues. The user is trying to build a drive-thru app and do recursive calls based on order single item or multiple items. The user's code shows how they are defining their functions and trying to call them.
In the Forum Post: Help for function calls with streaming (https://community.openai.com/t/627170), there's a Python example of how to handle function calls with streaming. The example shows how to accumulate JSON fragments of tool call arguments and handle new tool calls.
In the Forum Post: Streaming with function calling (https://community.openai.com/t/550829), a user is struggling with rebuilding the chat completion message from the chunks in the stream object, which is required for feeding the function output back to the chat.
In the Forum Post: Streaming with recursive function / tools calling > Official Answer (https://community.openai.com/t/687313), a user suggests turning off the experimental_streamData: true and removing all references to data.
From these sources, it seems that calling custom functions with streaming enabled can be complex and may require handling data in chunks and managing function calls carefully. However, without a direct Node.js example, it's hard to provide a definitive guide on how to do this.
For more detailed guidance, you may want to refer to the OpenAI API documentation or ask for help in the OpenAI community forum.
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
topaz swallow
@royal fjord in my code how can I call this get_current_weather function?
export async function POST({ request }) {
const { messages } = await request.json();
console.log(messages, 'request')
const tools = [
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Get the current weather in a given location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA",
},
"unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
},
"required": ["location"],
},
}
}
];
const initialMessage = {
role: "system",
content: "You are a real estate assistant. Your name is Tina, please keep your responses succint when possible."
};
const updatedMessages = [initialMessage, ...messages];
// Ask OpenAI for a streaming chat completion given the prompt
const response = await openai.chat.completions.create({
model: 'gpt-3.5-turbo', //gpt-4-1106-preview gpt-3.5-turbo gpt-4-0125-preview // does not work gpt-4-1106-vision-preview
stream: true,
messages:updatedMessages,
tools: tools,
tool_choice: "auto",
});
async function get_current_weather() {
// Here you would typically call a weather API to get the current weather
// For simplicity, let's return a static response
return {
"weather_response": "sunny and muddy",
"temperature": 21,
};
}
const stream = OpenAIStream(response);
console.log(response, 'choice 0')
console.log(JSON.stringify(response), 'choice 1')
// Respond with the stream
return new StreamingTextResponse(stream);
}
royal fjordBOT
Let me check the knowledge base for you. One moment! :books:
In your code, you have defined the get_current_weather function, but it's not being called anywhere. To call this function when the model requests it, you need to handle the tool calls in the response from the model.
Here's a simplified example of how you might do this in Python, based on the OpenAI Cookbook (https://cookbook.openai.com/examples/azure/functions#3-call-function-with-arguments-from-the-response) and OpenAI Community (https://community.openai.com/t/486260) examples:
function_call = chat_completion.choices[0].message.tool_calls[0].function
print(function_call.name)
print(function_call.arguments)
if function_call.name == "get_current_weather":
response = get_current_weather(json.loads(function_call.arguments))
In this example, the function call name and arguments are extracted from the model's response. If the function name is "get_current_weather", the function is called with the arguments.
However, your code is in JavaScript, and the streaming response from the model is handled differently. You'll need to process the streaming response, check for tool calls, and call the appropriate function. Unfortunately, the provided knowledge sources do not contain a direct example of how to do this in JavaScript.
For more detailed guidance, you may want to refer to the OpenAI API documentation or ask for help in the OpenAI community forum.
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
topaz swallow
@royal fjord
what does this mean
error: {
message: "An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'. The following tool_call_ids did not have response messages: call_TJHIITSyVkVXPFzHqFfk06R4",
type: 'invalid_request_error',
param: 'messages.[3].role',
code: null
},
code: null,
param: 'messages.[3].role',
type: 'invalid_request_error'
}
royal fjordBOT
Let me check the knowledge base for you. One moment! :books:
This error message is indicating that there's an issue with the way tool calls are being handled in your code.
In the OpenAI API, when a model makes a tool call (like calling a function), it includes a tool_call_id in the response. Your code is then expected to handle this tool call and respond with a message that includes the same tool_call_id. This is how the API keeps track of which tool calls have been responded to.
The error message is saying that a tool call was made with the id call_TJHIITSyVkVXPFzHqFfk06R4, but no response message was found for this tool call. This means that your code needs to handle this tool call and provide a response.