#Trouble with a tool call

57 messages · Page 1 of 1 (latest)

manic dew
#

I testing tool calling and I have a simple question.

Inthe docs the response you are supposed to send to the ai is:
...
"messages": [{
"role": "user",
"content": "What is the weather like in Boston?"
}],
"tools": [{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Get the current weather in a given location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA"
},
"unit": {
"type": "string",
"enum": [
"celsius",
"fahrenheit"
]
}
},
"required": [
"location"
]
}
}
}],

Question and problem I gave is. Do I have to send the triage of responses, 1 my request. Second the API's response and then a tool resoult from me. Currently I am sending conversation history and tool response but then the code exectution stops right at the get. When I change the actual "role" from "tool" to "user" the tool responds fine

FOR more info here's bits of my code.

The test function
def phony_tool(query, call):
return{"role": "tool",
"name": "search_data",
"tool_call_id": f"{call}",
"content": f"No data on:{query}"
}

#

I can send the api function if nesccecary

#

When the role in the function is changed to user it responds

#

If nobody responds since I can't see if this was seen or not I will @ somebody.

#

<@&1094455453599137872> Yo, if you mind responding.

#

If none respond will @ others. Cannot know if this was seen or not.

tight gale
#

Which model is this?

mental scaffold
#

@manic dew What model are you using? What happends if you try Qwen 2.5 72b?

manic dew
#

I left the model tab empty so whatever the api picks sometimes it switches them around. But the model isn't the problem. As I said I don't know weather I need to send the conv history and the tool or all 3 like the docs

tight gale
manic dew
#

U positive? I am sending all 3 but they are just in conv history

#

Outside of it I send the tool response

tight gale
#

The tool response should also be in history

manic dew
#

[Conv history] {tool response}

tight gale
#

Yeah they should all be in messages

manic dew
#

If that's the case I'll try it

#

But wait I want to make sure

tight gale
#

(NOT the tools array btw, that needs to be a sibling of messgaes)

manic dew
#

The messages section. Has 3 things here's the code
message1 = {"role": role, "content": input_message} if role == "user" else input_message
Input message is a function var

tight gale
#

Can you share the full final JSON body that you're sending to us? That'd help a lot

manic dew
#

If role is user then the user input gets put in content. If not then the function return def phony_tool(query, call):
return{"role": "tool",
"name": "search_data",
"tool_call_id": f"{call}",
"content": f"No data on:{query}"
} this one gets sent

manic dew
#

@lab here's a message BEGIN OF Message:[
{
"role": "system",
"content": "You will use tools to search for queries. When you perform a function call, the next user response will be the function result. If the result is not what you expect, deem it inconclusive and respond with: 'I am unable to receive the data.'"
},
{
"role": "user",
"content": "h"
},
{
"role": "assistant",
"content": "I notice you've just sent "h" as your message. I'm here to help, but I'll need a more specific question or request to assist you properly. Could you please:\n\n1. Ask a complete question, or\n2. Explain what information you're looking for, or\n3. Provide a detailed request\n\nThis will help me use the available tools effectively to give you the most relevant and helpful response.",
"refusal": ""
},
{
"role": "user",
"content": "search hello"
},
{
"role": "assistant",
"content": "I'll search for information related to "hello".",
"refusal": null,
"tool_calls": [
{
"id": "toolu_01P1QGZ9XNVBuRCBb98yC59q",
"index": 0,
"type": "function",
"function": {
"name": "search_data",
"arguments": "{"query": "hello"}"
}
}
]
},
{
"id": "toolu_01P1QGZ9XNVBuRCBb98yC59q",
"index": 0,
"type": "function",
"function": {
"name": "search_data",
"arguments": "{"query": "hello"}"
}
},
{
"role": "tool",
"name": "search_data",
"tool_call_id": "toolu_01P1QGZ9XNVBuRCBb98yC59q",
"content": "No data on:{"query": "hello"}"
},
{
"role": "user",
"content": "search hello"
}
]
END

drowsy iris
# 

[
  {
    "role": "system",
    "content": "You will use tools to search for queries. When you perform a function call, the next user response will be the function result. If the result is not what you expect, deem it inconclusive and respond with: 'I am unable to receive the data.'"
  },
  {
    "role": "user",
    "content": "h"
  },
  {
    "role": "assistant",
    "content": "I notice you've just sent \"h\" as your message. I'm here to help, but I'll need a more specific question or request to assist you properly. Could you please:\n\n1. Ask a complete question, or\n2. Explain what information you're looking for, or\n3. Provide a detailed request\n\nThis will help me use the available tools effectively to give you the most relevant and helpful response.",
    "refusal": ""
  },
  {
    "role": "user",
    "content": "search hello"
  },
  {
    "role": "assistant",
    "content": "I'll search for information related to \"hello\".",
    "refusal": null,
    "tool_calls": [
      {
        "id": "toolu_01P1QGZ9XNVBuRCBb98yC59q",
        "index": 0,
        "type": "function",
        "function": {
          "name": "search_data",
          "arguments": "{\"query\": \"hello\"}"
        }
      }
    ]
  },
  {
    "id": "toolu_01P1QGZ9XNVBuRCBb98yC59q",
    "index": 0,
    "type": "function",
    "function": {
      "name": "search_data",
      "arguments": "{\"query\": \"hello\"}"
    }
  },
  {
    "role": "tool",
    "name": "search_data",
    "tool_call_id": "toolu_01P1QGZ9XNVBuRCBb98yC59q",
    "content": "No data on:{\"query\": \"hello\"}"
  },
  {
    "role": "user",
    "content": "search hello"
  }
]
#

@manic dew Can you post the the full JSON you’re calling with? looks like that’s just your messages array?

manic dew
#

Yes wait a bit

drowsy iris
#

tools key should be it’s own array

manic dew
#

@drowsy iris what json do you want the api response or my request

drowsy iris
#

your request

manic dew
#

I can just send you the important bits of the code

#

Is that fine?

drowsy iris
#

Yeah

manic dew
#

This is just a play around script

#

import requests
import json
import time

messages_history = []
val = 0
last = None
tool_use = False

def print_nested(Name,data, indent=0):
print(f"\nBEGIN OF {Name}:{json.dumps(data, indent=2)}\nEND")

def phony_tool(query, call):
return{"role": "tool",
"name": "search_data",
"tool_call_id": f"{call}",
"content": f"No data on:{query}"
}

def send_api_message(input_message,ex = last, role = None):
global messages_history
global val
global last
print(val)
val += 1

url = "https://openrouter.ai/api/v1/chat/completions"
headers = {
    "Authorization": "Bearer API KEY REMOVED BUT ITS HERE",
    "Content-Type": "application/json",
}

sy = {"role": "system", "content": "You will use tools to search for queries. When you perform a function call, the next user response will be the function result. If the result is not what you expect, deem it inconclusive and respond with: 'I am unable to receive the data.'"}

if role == "user":
    message1 = {"role": role, "content": input_message} 
    print(role)
    print(f"Role in if:{role}")
    last = message1
    print(f"Last message now is:{last}")
    message = [sy, *messages_history,message1]
    print(f"Ex:{ex}")
elif role == "tool":
   print(f"Ex:{ex}")
   message1 = input_message
   message = [sy, *messages_history,message1]


print_nested("Message" ,message)
tools = [{"type": "function",
            "function": {
                "name": "search_data",
                "description": "Search for information",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "query": {
                            "type": "string",
                            "description": "Search query"
#

}
},
"required": ["query"]
}
}
}]

data = {
    "model": "",
    "messages": message, 
    "tool_choice": "auto",
    "tools": tools,
    "top_p": 1,
    "temperature": 0.67,
    "frequency_penalty": 0,
    "presence_penalty": 0,
    "repetition_penalty": 1,
    "top_k": 0
}

print("Sending")

print(f"\nUrl:{url}\n",f"\nHeaders:{headers}\n",f"\nData:{data}\n DataComp:{data['messages']}\n")

print_nested(url)

print_nested(headers)

print_nested(data)

response = requests.post(url, headers=headers, json=data)
print("sent")

messages_history.append(message1)

json1 = response.json()
print_nested("Json",json1)

ai_response = json1.get('choices', [{}])[0].get('message', {}).get('content', '')

if 'choices' in json1 and 'message' in json1['choices'][0]:
    messages_history.append(json1['choices'][0]['message'])
else:
    print("Message not found")

print_nested("Message history",messages_history)
#print(f"\nThis is json:{json1}")   
tool_calls = json1.get('choices', [{}])[0].get('message', {}).get('tool_calls', [])

if tool_calls:
    tool_call = tool_calls[0]  # Assuming only one tool call for simplicity

    # Access the function name and arguments from the tool call
    function_name = tool_call.get('function', {}).get('name', 'Unknown Function')
    function_arguments = tool_call.get('function', {}).get('arguments', '{}')

    tool_info = {
        "role": "tool",
        "name": function_name,
        "id": tool_call.get('id', 'Unknown ID'),
        "content": function_arguments
    }

messages_history.append(tool_call)

    #print(f"\nTool info: {tool_info}")

    response = send_api_message(phony_tool(function_arguments, tool_info['id']),last,"tool")
    return response
#

Ensure the recursive call returns a value

else:
    print("\nNo tool request made by AI.")

return ai_response

while True:
user_input = input("Enter your message (or 'exit' to quit):\n")

if user_input.lower() == 'exit':
    break
    
response = send_api_message(user_input, None, "user" )
print(f"AI: {response}\n")
#

@drowsy iris it took a while to fit into 3 messages.

#

It does work but I want to know how should the memory for the ai be implemented whilst using tools. In what order should the user request for a tool be placed. Then the apis response and then the tool response

drowsy iris
#

it looks like your second call to send_api_message is using this tool_info var that has different data than your original tool

#

user -> assistant -> tool -> assistant response using tool data -> user is the sequence

manic dew
drowsy iris
#

yeah

manic dew
manic dew
# drowsy iris yeah

Look at the oppen router docs ...
"messages": [
{
"role": "user",
"content": "What is the weather like in Boston?"
},
{
"role": "assistant",
"content": null,
"tool_calls": [
{
"id": "call_9pw1qnYScqvGrCH58HWCvFH6",
"type": "function",
"function": {
"name": "get_current_weather",
"arguments": "{ "location": "Boston, MA"}"
}
}
]
},
{
"role": "tool",
"name": "get_current_weather",
"tool_call_id": "call_9pw1qnYScqvGrCH58HWCvFH6",
"content": "{"temperature": "22", "unit": "celsius", "description": "Sunny"}"
}
],

manic dew
drowsy iris
#

yeah? that’s the same as what you were saying no?

manic dew
#

Yes however I don't know in witch order to place them.
I placed them like this:
User message, ai response, tool response, same user message

manic dew
#

So is there a documented order

#

I tested variations and some worker some didnt

#

Worked*

drowsy iris
# manic dew Look at the oppen router docs ... "messages": [ { "role": "user", "con...

you’d take this array and call the model again to generate a completion based on tool call data, that is sent to user.

User Message, AI Response, Tool Response (these 3 are shown in snippet)

make another api call to get another AI Response so you’re at:

User Message, AI Response, Tool Response, AI Response

then get the user’s next message ( don’t use the same as original, just keep going now):

User Message, AI Response, Tool Response, AI Response, User Message

manic dew
drowsy iris
#

any time your model makes a tool call you will have in Messages:

AI Response, Tool Response, AI Response

drowsy iris
#

You can maybe remove extra tokens by trimming tool data? if you want

#

like you don’t need old tool data to generate the most recent response

manic dew