#Issue with model answers when not enough information is provided

1 messages · Page 1 of 1 (latest)

humble karma
#

I'm using GPT-4o for my RAG model. I run into a problem with the model answering to questions of no clear context.
Example:
Any restaurant around here? => There is no location provided in this question nor in the conversation history, thus vague information about location => cannot answer
Can I visit the doctor now? => There is no specific time provided, thus vague information about time => cannot answer

I'm using this system prompt but it doesn't work:
IF a question contains spatial references ("around here", "near me", "in my area") or temporal references ("now", "in 2 hours", "soon") AND there is no specific time or location in the conversation history, THEN ask for clarifications

Help me improve that system prompt to resolve my issue

zinc spadeBOT
#

Hi @humble karma. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:

#

Based on the knowledge sources provided, you can improve your system prompt by making it more explicit and instructive. Here's an example of how you can modify your system prompt:

[
    {"role": "system", "content": "You are a helpful assistant. Answer the following question based on the context only. If the question contains spatial references ('around here', 'near me', 'in my area') or temporal references ('now', 'in 2 hours', 'soon') and there is no specific time or location in the conversation history, ask for clarifications. If you don't know the answer, say 'I don't know'."},
    {"role": "user", "content": "Your question here"}
]

This prompt instructs the model to answer the question based on the context only and to ask for clarifications when there is not enough context. It also instructs the model to say 'I don't know' when it doesn't know the answer.

You can also consider using a smaller, fine-tuned model like GPT-3.5, which can process tokens faster and might be more efficient for your task (source (https://platform.openai.com/docs/guides/latency-optimization)).

Remember to test your changes systematically to ensure they improve the performance of your model (source (https://platform.openai.com/docs/guides/prompt-engineering)).

For more tips on prompt engineering, you can refer to the OpenAI prompt engineering guide (https://platform.openai.com/docs/guides/prompt-engineering).

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

#

@humble karma Kapa.ai is still learning and improving, please let me know how I did by reacting below.

humble karma
#

@zinc spade any other ways of wording for spatial references and temporal references to better capture my idea?

zinc spadeBOT
#

Let me check the knowledge base for you again. One moment! :books:

#

Certainly, you can expand your system prompt to include a wider range of spatial and temporal references. Here are some examples:

Spatial references:

  • "close by"
  • "within walking distance"
  • "a short drive away"
  • "in this city"
  • "in this neighborhood"
  • "in this vicinity"

Temporal references:

  • "later"
  • "in a moment"
  • "in a while"
  • "this afternoon"
  • "this evening"
  • "tomorrow"

You can modify your system prompt to include these additional phrases:

[
    {"role": "system", "content": "You are a helpful assistant. Answer the following question based on the context only. If the question contains spatial references ('around here', 'near me', 'in my area', 'close by', 'within walking distance', 'a short drive away', 'in this city', 'in this neighborhood', 'in this vicinity') or temporal references ('now', 'in 2 hours', 'soon', 'later', 'in a moment', 'in a while', 'this afternoon', 'this evening', 'tomorrow') and there is no specific time or location in the conversation history, ask for clarifications. If you don't know the answer, say 'I don't know'."},
    {"role": "user", "content": "Your question here"}
]

Remember, the goal is to capture as many variations of spatial and temporal references as possible to ensure your model can handle a wide range of user inputs.

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon: