#Few shot in API: include examples in system message or as user and assistant pairs?

1 messages · Page 1 of 1 (latest)

restive trout
#

What is the consensus in terms of Few-Shot examples in the API? Do you include them in the System Prompt it self (perhaps in XML tags) or do you add them as user and assistant pairs?

I can't seem to find a "conclusion". I have found that both works just fine, but I am curious what people think at scale.

Using Anthropic's prompt helper, I notice that it defaults to writing it all out in a user prompt, although you can ask it specifically to include a system prompt. This may be nothing, but had me wondering once again.

bold nexusBOT
#

Hi @restive trout. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:

#

The consensus on how to design few-shot prompts with the OpenAI API seems to lean towards including the examples in the system message. According to a forum post (https://community.openai.com/t/656727) on the OpenAI community, you can achieve few-shot prompting by providing examples directly in the system message. This is where you add the examples you want the model to refer to when answering the user message. You are still sending only one system message and one user message with the question, but the system message contains additional examples. There is no need to add additional system or user messages with replies.

Here's an example from the forum post:

System:
> "You answer questions about Ice cream. Answer with ‘Yes’ or ‘No’ only.
>
> When answering user questions follow these examples:
>
> Question: Is ice cream cold?
>
> Answer: Yes.
>
> Question: Does Ice cream make children happy?
>
> Answer: Yes
>
> Question: Can I fix my car’s overheating problem with ice cream?
>
> Answer: No.

And then you have the User message which will be answered according to your examples (few-shots).

> Can you write a 500 word paragraph about the advantages of eating ice cream?

However, it's also possible to include examples as user and assistant pairs, as shown in the OpenAI Cookbook (https://cookbook.openai.com/examples/how_to_format_inputs_to_chatgpt_models#few-shot-prompting). This approach might be more suitable for certain use cases, especially when you want to demonstrate a specific interaction pattern between the user and the assistant.

#

In conclusion, both methods can work, and the choice between them might depend on the specific requirements of your application.

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon: