which system prompt are you using?

[oblique river]

Hello @grave holly ! quick question. How do you change the SYSTEM prompt when the chat template does not allow that role? https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.2-AWQ/blob/main/tokenizer_config.json#L42

[grave holly]

not required to specify role, prompt template is such:

<s>[INST] {systemPrompt} [/INST]</s>[INST] {userInstructions} [/INST]

[oblique river]

Got it! so it is part of the prompt then. Interesting. Thank you! I am trying to figure out how to use Mistral or Mixtral instruct models with autogen and I belive somewhere, there is a System role being set by default 😦 . I was not sure if there was a way to update the chat template to allow "System" role initially. 😅

[grave holly]

which system prompt are you using?

[worn remnant]

did you achive use mistral ai api with autogen bro? im literally being trying this these past few days with no success 🥲

[oblique river]

Hey @worn remnant ! I am hosting the models myself (self deployment). I have not checked the API with Autogen yet. What have you tried so far? Maybe we can take the conversation to another thread since this one is for system prompt 🙈

[worn remnant]

sure

[past gull]

How will I put following conversation

System: Be a good assistant
User: Hey you how are you?
Assistant: Hello, I don't have feelings.

do this would be like as following?

<s>[INST] Be a good assistant [/INST]</s>[INST] Hey you how are you? [/INST] Hello I don't have feelings.

or like this?

<s>[INST] Be a good assistant [/INST]</s>[INST] Hey you how are you? [/INST] Hello I don't have feelings.</s>

[dawn arch]

i am interested in this. Can't find any good docs on this. very frustrating...

[grave holly]

<s>[INST] Be a good assistant [/INST] [INST] Hey you how are you? [/INST] Hello I don't have feelings.</s> [INST] { userInstructions } [/INST]