How incremental learning is done in LLMs? | Mistral AI | Page 1

Hi!
I’m a junior developer and I’m playing with LLMs. I have been doing a fine tunning with llama, mistral… to make text classifying, Is there any way to add new samples that I’m recollecting without retraining the with the full dataset?

I’m concerned that there are technics that allow to respect original weights in order not to loose the learned things, but I don’t know if in these foundational models is a typical practice.

Thanks a lot

#How incremental learning is done in LLMs?