I have no prompt-response, just information I would like to train the model on. How do I do it?
#How do I fine-tune on just information?
pseudo shore
hearty geodeBOT
Hi @pseudo shore. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:
Fine-tuning an AI model on just information, without specific prompt-response pairs, can be achieved by structuring your data in a way that the model can learn from. Here are some steps you can follow:
-
Prepare Your Dataset: Once you've determined that fine-tuning is the right solution, you'll need to prepare data for training the model. You should create a diverse set of demonstration conversations that are similar to the conversations you will ask the model to respond to at inference time in production. Each example in the dataset should be a conversation in the same format as OpenAI's Chat Completions API (https://platform.openai.com/docs/api-reference/chat/create), specifically a list of messages where each message has a role, content, and optional name (https://platform.openai.com/docs/api-reference/chat/create#chat/create-chat/create-messages-name). At least some of the training examples should directly target cases where the prompted model is not behaving as desired, and the provided assistant messages in the data should be the ideal responses you want the model to provide. source (https://platform.openai.com/docs/guides/fine-tuning)
-
Use Few-Shot Prompts: You can use a few-shot prompt to extend or modify a fine-tuning dataset. This idea can be generalized beyond explanation writing. For example, if you have large quantities of unstructured text that you want to train on, you may find opportunities to use a prompt to extract a structured dataset from your unstructured text, and then fine-tune a custom model on that structured dataset. source (https://cookbook.openai.com/articles/techniques_to_improve_reliability#implications-2)
- Include Separator Sequences: It can be helpful to include separator sequences (e.g.,
->or###or any string that doesn't commonly appear in your inputs) to tell the model when the prompt has ended and the output should begin. Without separator sequences, there is a risk that the model continues elaborating on the input text rather than starting on the answer you want to see. source (https://cookbook.openai.com/articles/how_to_work_with_large_language_models#fine-tuned-prompt-example)
Remember, fine-tuning data provides models with examples of how it should respond to a given conversation. You'll want these examples to match the input that the model will see in production as closely as possible. source (https://help.openai.com/en/articles/6811186-how-do-i-format-my-fine-tuning-data-for-the-openai-api#h_085717bb68)