What are the things I should consider while preparing my data set for fine tuning to ensure I get the best output?
#preparing data set for fine tuning
stray wyvern
novel bloomBOT
Hi @stray wyvern. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:
When preparing your dataset for fine-tuning, there are several important steps to consider:
-
Determine the need for fine-tuning: Fine-tuning is the right solution when you've optimized your prompt as far as it can take you and identified problems that the model still has.
-
Create a diverse set of demonstration conversations: These should be similar to the conversations you will ask the model to respond to at inference time in production. Each example in the dataset should be a conversation in the same format as the Chat Completions API, specifically a list of messages where each message has a role, content, and optional name. Some of the training examples should directly target cases where the prompted model is not behaving as desired, and the provided assistant messages in the data should be the ideal responses you want the model to provide. source (https://platform.openai.com/docs/guides/fine-tuning)
-
Check data formatting: Before you create a fine-tuning job, it is important to check the data formatting. OpenAI provides a simple Python script which you can use to find potential errors, review token counts, and estimate the cost of a fine-tuning job. source (https://platform.openai.com/docs/guides/fine-tuning)
-
Upload a training file: Once you have the data validated, the file needs to be uploaded using the Files API in order to be used with a fine-tuning job. Here's a sample code snippet for this:
from openai import OpenAI client = OpenAI()
client.files.create(
file=open("mydata.jsonl", "rb"),
purpose="fine-tune"
)
```
Note that the maximum file upload size is 1 GB, though it's not suggested to fine-tune with that amount of data since you are unlikely to need that large of an amount to see improvements. source (https://platform.openai.com/docs/guides/fine-tuning)
-
Crafting prompts: It's recommended to take the set of instructions and prompts that worked best for the model prior to fine-tuning, and include them in every training example. This should let you reach the best and most general results, especially if you have relatively few (e.g. under a hundred) training examples. source (https://platform.openai.com/docs/guides/fine-tuning)
-
Train and test splits: After collecting the initial dataset, it's recommended to split it into a training and test portion. When submitting a fine-tuning job with both training and test files, OpenAI will provide statistics on both during the course of training. These statistics will be your initial signal of how much the model is improving. source (https://platform.openai.com/docs/guides/fine-tuning)
Remember, it may take more training examples to arrive at good results, as the model has to learn entirely through demonstration and without guided instructions.
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
stray wyvern
@novel bloom while training how do I also submit my custom data files so that the model is trained basis that data set and does not hallucinate
novel bloomBOT
Let me check the knowledge base for you. One moment! :books:
To train your model with custom data, you need to upload your data file to OpenAI using the Files API. Here's a sample code snippet for this:
from openai import OpenAI
client = OpenAI()
client.files.create(
file=open("mydata.jsonl", "rb"),
purpose="fine-tune"
)
source (https://platform.openai.com/docs/guides/fine-tuning)
This file will be used as the basis for the fine-tuning process. The model will learn from the examples provided in this file, which should help guide its responses and reduce the likelihood of it generating inappropriate or "hallucinated" content.
However, it's important to note that even with custom training data, the model may still occasionally generate unexpected or off-topic responses. This is a known limitation of the technology. To mitigate this, you can use techniques like prompt engineering, or adjusting the temperature and max tokens settings during inference.
Also, remember to split your data into training and test portions. This allows you to monitor the model's performance and improvement during the training process. source (https://platform.openai.com/docs/guides/fine-tuning)
Finally, it's important to note that the model does not "learn" in the traditional sense over time or from feedback. It does not retain information between different conversations or instances. The only way for it to truly "learn" is through fine-tuning. source (https://community.openai.com/t/605180)
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
mystic dune
novel bloomBOT
Let me check the knowledge base for you. One moment! :books: