#Why does a 1115 length fine-tuning model file costs 1,520 trained tokens?

1 messages · Page 1 of 1 (latest)

pearl oracle
#

I wrote a post in the online community forum https://community.openai.com/t/why-does-a-1115-length-fine-tuning-model-file-costs-1-520-trained-tokens/125970.

I create a fine-tuning model to train GPT to remove useless and expired feature flags. I follow the fine-tuning instructions from fine-tuning documentation. I create a jsonl file which containing the code below (which contains 1115 characters).

I tokenized the above code using the online tool OpenAI API Tokenizer. It told me that the above code will consume 388 tokens.

But after the fine-tuning was completed with the code above, it said that this fine-tuning cost 0.05 USD and used 1,520 trained tokens.

I want to know why does a 1115 length fine-tuning model file costs 1,520 trained tokens.