#Proper use of Model/Character Settings "System Prompt"

1 messages · Page 1 of 1 (latest)

hasty snow
#

How do I properly use the Model/Character Settings "System Prompt"?

rose flaxBOT
#

Hi @hasty snow. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:

#

The "System Prompt" in the Model/Character Settings is used to provide an initial instruction for the model. This is particularly useful when you want to set a specific context or role for the model during a chat session.

In the GPT4All Python API, you can use the chat_session(system_prompt='', prompt_template='') method to set the system prompt. Here's an example of how to use it:

from gpt4all import GPT4All
model = GPT4All("model_name")
with model.chat_session("You are a cyber security expert who can find vulnerabilities in source code"):
    response = model.generate('who are you?', temp=0)
    print(response)

In this example, the system prompt is "You are a cyber security expert who can find vulnerabilities in source code". This sets the context for the model's responses.

Please note that changing the system prompt effectively changes the model settings. If you want to work with multiple system prompts, one way to do this is to clone the model in the Model/Character settings for each different system prompt you want to use.

For more information, you can refer to the GPT4All Python API documentation (https://docs.gpt4all.io/gpt4all_python.html#gpt4all.gpt4all.GPT4All.chat_session).

Sources:

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

#

@hasty snow kapa.ai is still learning and improving, please let us know how it did by reacting below

hasty snow
#

@rose flax I have written a list of instructions into the system prompt, it seems like there is an unknown limit to the amount of instructions it will follow. It appears to forget some of the rules I set.

rose flaxBOT
#

Let me check the knowledge base for you. One moment! :books:

#

The issue you're experiencing might be related to the length of the system prompt. If the system prompt is too long, the model might not be able to process all the instructions effectively. This could be due to the model's context window size (n_ctx), which determines how much information the model can consider at once.

In one of the GitHub issues (https://github.com/nomic-ai/gpt4all/issues/668) for GPT4All, a user experimented with different n_ctx values and found that increasing the context window size could lead to longer loading times or even cause the program to hang, especially if the model wasn't trained with a larger context window size.

Therefore, if your system prompt is very long, you might want to try shortening it or breaking it up into smaller parts. Also, keep in mind that the model might not always follow all the instructions perfectly, especially if they are complex or contradictory.

If you continue to experience issues, it might be worth raising a new issue on the GPT4All GitHub page (https://github.com/nomic-ai/gpt4all/issues) to get further assistance.

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

#

@hasty snow kapa.ai is still learning and improving, please let us know how it did by reacting below

hasty snow
#

@rose flax I'm testing the setting "system prompt" today. I need to figure out how many instructions I can expect the model to follow, and figure out what limits will get good results. Do you have any tips?

rose flaxBOT
#

Let me check the knowledge base for you. One moment! :books:

#

The "System Prompt" setting in GPT4All is used to provide initial instructions to the model. However, the number of instructions the model can follow and remember is not explicitly defined and can depend on several factors:

  1. Model's Capacity: Larger models generally have a better ability to remember and follow more instructions compared to smaller models.

  2. Instruction Complexity: The model's ability to follow instructions can also depend on how complex these instructions are. Simple and clear instructions are generally easier for the model to follow.

  3. Context Window Size: The model's context window size (n_ctx) can also affect its ability to remember instructions. However, increasing the context window size can lead to longer loading times or even cause the program to hang, especially if the model wasn't trained with a larger context window size.

Here are some tips for setting the system prompt:

  • Keep it Concise: Try to keep your instructions concise and clear. Too many instructions or overly complex instructions might confuse the model.

  • Prioritize Important Instructions: If you have many instructions, prioritize the most important ones to increase the likelihood that the model will follow them.

  • Experiment: The best way to find out how many instructions the model can follow is by experimenting. Start with a few instructions and gradually add more to see how the model's responses change.

Remember, changing the system prompt effectively changes the model settings. If you want to work with multiple system prompts, one way to do this is to clone the model in the Model/Character settings for each different system prompt you want to use.

For more information, you can refer to the GPT4All Python API documentation (https://docs.gpt4all.io/gpt4all_python.html#gpt4all.gpt4all.GPT4All.chat_session).

#

@hasty snow kapa.ai is still learning and improving, please let us know how it did by reacting below

hasty snow
#

@rose flax When does a change to the system prompt take effect?

rose flaxBOT
#

Let me check the knowledge base for you. One moment! :books:

#

A change to the system prompt takes effect when you start a new chat session with the model. This is because the system prompt is used to provide an initial instruction for the model at the start of a chat session.

Here's an example of how to change the system prompt in the GPT4All Python API:

from gpt4all import GPT4All
model = GPT4All("model_name")
with model.chat_session("Your new system prompt here"):
    response = model.generate('Your message here')
    print(response)

In this example, "Your new system prompt here" is the new system prompt. This will be the initial instruction for the model in this chat session.

Please note that changing the system prompt effectively changes the model settings. If you want to work with multiple system prompts, one way to do this is to clone the model in the Model/Character settings for each different system prompt you want to use.

For more information, you can refer to the GPT4All Python API documentation (https://docs.gpt4all.io/gpt4all_python.html#gpt4all.gpt4all.GPT4All.chat_session).

Sources:

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon: