#How to do Few Shot learning with function args?

36 messages · Page 1 of 1 (latest)

rapid heart
#

I'm using the new function call methods in the june versions of GPT-3 . How are you supposed to do few shot learning with them? I can put a description field in the JSON schema. but if i put example JSON args in my prompts, it seems like it gets confused and returns JSON as content, instead of a function call.

I want to give multiple examples of different API calls to use when executing the funciton to improve the performance but am unsure the optimal way to do it and still get the JSON function call back.

plain tinsel
#

You mean separating out different functions or perhaps varying whether specific parameters are required or not?

rapid heart
#

i mean giving examples for what the args should be.

#

for instance, I have a field that's called customer_reply where the bot gives a reply to the question that's being asked. i want it in the function args though, not in the contents of a string/chat reply.

#

and i want to give some examples of good customer_reply strings for different questions. usually i would just write some examples in the system message and then get a pretty good reply in the context object that GPT returns. however, i want it structured as a JSON object so that I can get other things back too like "sentiment" or "entities" or "tags"

#

so i would ideally want to give examples of several different function args where the fields display non-trivial examples of the kinds of responses that should be generated for different questions

plain tinsel
rapid heart
#

i don't see any example JSON for save_car_to_db in there.

plain tinsel
#

That's what this is:

Then you would respond with a make of Honda, a model of Civic, and a summary something like "Efficient and reliable compact car"

Except you don't need to format it as JSON because the function output will already do that

rapid heart
#

have you used the function args functionality and done few shot prompting with it? It's been my experience that the more context i put in the system prompt around what goes into the args, the more likely it is to return a content string in the reply instead of a function call.

plain tinsel
#

Yes, but generally I find it easier (and cheaper at scale) to tune the prompt rather than provide examples. The longer the prompts get, the worse GPT will perform.

The bigger issue I run into with functions is not returning all parameters that are marked as required

rapid heart
#

the function args i need returned are quite extensive. without examples of what they are supposed to be they are generally low quality.

#

i suspect at some point, i will have to do fine tuning instead. however i do not have enough examples to do it well yet.

plain tinsel
#

We'll need more realistic examples for your use case if you want to dig into it more. One option would be splitting the different parameters into different API calls. Getting too long (including too many function parameters) will have a negative effect on your output

rapid heart
#

to me, it feels like you should be able to somehow provide multiple examples of the function calls you expect to help guide the model. and to provide them in a way that the api does not think that the examples were delivered in the content string of the reply

plain tinsel
rapid heart
plain tinsel
plain tinsel
rapid heart
#

what i'm hearing is that there is no known way to do this and I should work on a work-around.

plain tinsel
#

More or less yeah, we're all trying to figure out the best ways to use it, still a really new feature

rapid heart
#

agreed 100%. your suggestion is to just put things in the system prompt and possible simplifythe function call and send off multiple completion requests and then reassemble them.

#

so, how do you include the args from the function call in subsequent calls to the API? in a list of messages to and from the API some of the assistant replies have no content.

plain tinsel
#

So I think we're talking about a couple different problems here:

  1. As context length expands, the output gets unpredictable or completely muddled.

Solution idea - break the calls up into different parts, if different parameters can be separated (like in my car details example).

  1. Function calls don't always return all the data, even if they are tagged as "required".

Solution idea - you need some concept of defaults or "null" values. Then write a function that takes the GPT output and applies these, so that all parameters exist.

If you can't make do with a default, perhaps try calling the endpoint up to a capped number of times with slightly varying parameters, e.g., increase temperature by 0.25 each call.

rapid heart
#

thank you for your take! I'd love to hear other folks experience and ideas as well.

light carbon
#

We're still not sure what kind of output you expect from the model. Perhaps if you give some examples, we'd be able to provide a better advice.

rapid heart
#

I must be miscommunicating then. I want JSON output from the model in the function_call args field, not the content of the reply.

i'm asking how to include multiple examples of what you want the args output for your functions to be. In the "old days" I would include several examples of what I wanted the output to look like in the system prompt. That would improve the quality of the output greatly. The output would be in the content field of the assistant reply.

Now, using function calls, I could imclude several examples of the args JSON in the prompt to demonstrate what I want the output to be. It seems like, when I do that, GPT is more likely to return the output as the content of a reply, as opposed to a function_call reply.

What I'm hearing is that you folks are just rewriting the contents of the JSON in a semi-narrative form and hoping that GPT takes that and formats it correctly in the JSON.

#

the examples show how to include results from function calls in the message stream (as parts of turns in the conversation), but not really examples of inputs.

        # Step 4: send the info on the function call and function response to GPT
        messages.append(response_message)  # extend conversation with assistant's reply
        messages.append(
            {
                "role": "function",
                "name": function_name,
                "content": function_response,
            }
        )
rapid heart
#

In that example the messages.append(response_message) has the function_call args. So I see how to include the args as we take turns in the conversation.

#

i'm not sure if folks have experimented with adding the function_call args as an example in the system prompt. or perhaps a fake conversation turn as an example.

fiery thorn
rapid heart
#

I have a JSON Schema with well defined fields. It hasn't been my experience that that's enough.

fiery thorn
#

If you can give me the access to that JSON file, I am happy to try it on my environment to find out what is going on.

rapid heart
#

for instance, one of the fields is the text string i want to use in reply to the user. That field in particular should vary based o nthe contents of the other items.

#

I appreciate the offer for help!

#

I think the conversation here has been enough for me to see that there is not a way to do the thing I was trying to do. Other folks have other ways of dealing with it or don't think it's an issue.