Fine tuning with text dataset!

[short crystal]

I want to fine-tune a model, but is it possible to fine-tune using only a text dataset?

[lean hound]

Sure. Base models (not instruct) are pretty much entirely trained on raw text.

Make sure to break it up into chucks smaller than the context window.

[short crystal]

can you show the example?

[desert tiger]

https://docs.unsloth.ai/

Start here.