robust bramble
I have a Gemma prompting question - in the case of having a context passed in as a starting prompt for a RAG chatbot, I would think using the instruct version with the chat prompting format would be the way to go.
However, not sure what exactly this starting prompt would include. Since the template is strictly alternating user/model/user/model I have a template that includes a model "introduction" looking like this:
"""
<start_of_turn>user
You are a helpful assistant answering questions about a specific topic. Answer user questions with the provided context: {context}. Answer "I don't know" if not present in the documents.
<end_of_turn>
<start_of_turn>model
Nice to meet you! I am [secret name], and I would love to answer any questions you may have for ————. [extra sentence here to provide dialog style]
<end_of_turn>
<start_of_turn>user
"""
(user input value as prompt would follow this)
Would this even work? For the RAG architecture I am using LlamaIndex and UI w/ chat is Streamlit.
Hopefully this illustrates my intent here, so the main question is, if my template WOULD work for "2b-it" would I go with it, or would ditching the format and go with base "2b" be better? @golden flare or anybody?