#installing unsloth on M1 mac laptop

10 messages · Page 1 of 1 (latest)

trail sparrow
#

Instead of using Colab, I want to use my M1 Mac to fine-tune. I have small dataset (about 300 question/answer dataset to use for fine-tuning an instruct-model). Before I begin choosing a small-enough instruct-model for my laptop, I attempted to install unsloth on my mac and it generated the following error. Any help?

(ft_1) Sri@MacBookPro ft_1 % uv add unsloth
Resolved 121 packages in 519ms
error: Distribution bitsandbytes==0.45.5 @ registry+https://pypi.org/simple can't be installed because it doesn't have a source distribution or wheel for the current platform

hint: You're on macOS (macosx_15_0_arm64), but bitsandbytes (v0.45.5) only has wheels for the following platforms: manylinux_2_24_x86_64, win_amd64
(ft_1) Sri@MacBookPro ft_1 %

#

installing unsloth on M1 mac laptop

sudden tiger
#

support for mac is not there yet. it's being worked on

trail sparrow
#

Do you know of any timeframe? I am trying to begin small...

sudden tiger
#

I'll have to ask

#

I am not the one handling that part

#

although I run a mac redbharold

daring void
#

Specifically for the bitsandbytes dependency, I'm working on that, but the best I can say for macOS is soon ™️ . We're simultaneously working on enablement for ROCm, Intel XPU, Intel Gaudi (HPU), x86-64/aarch64 CPU. macOS is a little further out on the roadmap.

I've had some success with porting for CPU on macOS so far, and have also been able to run LLM.int8() on MPS device, but it's not release quality yet.

I can't speak to timelines on the rest of Unsloth core or other deps, but just offering this as evidence that people are working on this. I think Unsloth right now relies pretty heavily on Triton and torch.compile, for example, which I think isn't super great yet on macOS support either, so there's going to be a lot to work around for that.

golden shuttle
golden shuttle