limpid etherIt seems likely that this error is caused by in-place modification of a tensor returned from a custom autograd function (UnslothFusedLossBackward), which PyTorch forbids for correctness. The recommended fix is to clone the output tensor before any in-place operation, as the error message suggests. This issue can occur with both Trainer and UnslothTrainer, as both may perform in-place operations on the loss tensor in distributed/multi-GPU settings. There is no explicit mention in the data sources of a built-in fix for this in Unsloth, so manual cloning may be necessary in your training loop or by patching the relevant code.
Regarding which trainer to use: UnslothTrainer is designed to provide Unsloth-specific optimizations and bug fixes (such as improved gradient accumulation and embedding learning rate control), and is generally recommended for Unsloth-based fine-tuning, especially for continued pretraining or when using Unsloth's advanced features. Example usage of UnslothTrainer can be found in the official documentation and notebooks.
Would you like more detail on how to patch your code to avoid this error, or on the differences between Trainer and UnslothTrainer?
Sources:
clever stratus