Does using the standard HF/PEFT method of combining the LoRA weights from a checkpoint into the base model (doing a "merge") acceptable for Unsloth-trained checkpoints?
#Merging lora weights from checkpoint into base model
candid pebble
dry cosmosBOT
-# 🦥 Unsloth's Auto Bot might have mistakes
Yes, using the standard Hugging Face/PEFT method of combining LoRA weights from a checkpoint into the base model (doing a "merge") is acceptable for Unsloth-trained checkpoints. The unsloth_zoo/saving_utils.py file provides helpers for creating Hugging Face repositories and merging LoRA weights from a checkpoint into the base model.
candid pebble
@dry cosmos How do I prevent unsloth export from going OOM?
dry cosmosBOT
Use the unsloth export command with the --merged flag to prevent Out-Of-Memory (OOM) errors during export. This flag merges the LoRA weights from the checkpoint into the base model before saving.
unsloth export --merged
@dry cosmos That flag doesn't exist!