thorny frost
Hello, am wondering if anyone is running into issues with OpenAI messing-up the responses with tool call definitions.
Our use case is a chat that gets some information after defining multiple API calls that need to be made. Therefore, sometimes the first or second api call will generate multiple result matches that then you expect the LLM to define consecutive tool calls with each of those matches. So far it did a good job at defining individual tool calls definitions, but suddenly it's having issues.
Issues:
- Arguments properties
- often arguments include multiple objects without comma separation; all in one toll call ID (it seems its trying to make a parallel function call).
- arguments sometimes include properties from a different function (doesnt happen too often)
- Function name:
- sometimes (once in a blue moon) is a wrong non-existing function name
- also once in a blue moon will look like something that doesnt meet the expected function name pattern. For example "multi_tool_use.parallel" or something like "functions,my_function". In the next chat run, this crashes.
So far we are filtering and validating so when multiple objects are sent in arguments, we extract the first one, run that and tell the LLM we only ran that one.
We use Azure OpenAI, GPT-4o ( version 2024-05-13 which accepts parallel function calling), API version 2024-06-01, Python sdk 1.35.0 and only the chat completions API. Per Azure, Parallel function calling is ON by default. We tried telling the LLM to use parallel function calling too (to see if arguments comes back with one set per tool ID but it didnt work). We tried with good and bad examples in the system prompt and function definitions but still didnt work.
Anyone ran into this issue? What did you do to prevent it from happening?
Any help is greatly appreciated.
hoary timber