Function calling performs poorly compared to other methods of extracting structured data

[digital current]

I'd like to submit a report of the function calling API performing very poorly compared to other methods of extracting structured data. Here is a notebook showing three different ways of answering a boolean question about a piece of text. The function calling API gets it right 0/10 times while asking for JSON the old way gets it right 9/10 times, and asking for a natural language "yes" or "no" answer gets it right 10/10 times. https://gist.github.com/kylemcdonald/abfead2e780196840dff327c6264295a

[hardy hill]

You need to put the pet_clause in the system message instead the user message. The function api is tuned for system messages, when it comes to knowledge bases. And lower to temperature to 0.

[digital current]

i appreciate your help. i tried moving it into the system message, but that wasn't enough. i had to spend some time tuning the prompty before i got the right answer. changing the function description by one word (which doesn't really change the meaning of the function imo) will change the output results. i updated the gist to show this.

[hardy hill]

Nice, that it works now. GPT-3 has become very sensitive there when it comes to the prompts.