Gemma 3n Colab Notebook Not Working | Unsloth AI | Page 1

sharp raven Jul 7, 2025, 3:31 PM

#

The Notebook: https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Gemma3N_(4B)-Conversational.ipynb#scrollTo=-Xbb0cuLzwgf

Cell Three Error:

Please restructure your imports with 'import unsloth' at the top of your file.
from unsloth import FastModel

ImportError Traceback (most recent call last)
/tmp/ipython-input-24-3770780297.py in <cell line: 0>()
----> 1 from unsloth import FastModel
2 import torch
3
4 fourbit_models = [
5 # 4bit dynamic quants for superior accuracy and low memory use

4 frames
/usr/local/lib/python3.11/dist-packages/unsloth_zoo/temporary_patches/misc.py in patch_CsmDepthDecoderForCausalLM_forward()
204
205 from transformers.modeling_outputs import CausalLMOutputWithPast
--> 206 from transformers.models.csm.modeling_csm import Cache, Unpack, KwargsForCausalLM
207 from transformers.loss.loss_utils import ForCausalLMLoss
208

ImportError: cannot import name 'KwargsForCausalLM' from 'transformers.models.csm.modeling_csm' (/usr/local/lib/python3.11/dist-packages/transformers/models/csm/modeling_csm.py)

NOTE: If your import is failing due to a missing package, you can
manually install dependencies using either !pip or !apt.

To view examples of installing some common dependencies, click the
"Open Examples" button below.

Google Colab

glacial bramble Jul 7, 2025, 3:40 PM

#

there are issues being solved as we speak, so give it a few hours or a day

sharp raven Jul 7, 2025, 4:15 PM

#

This one doesn't work either: https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Gemma3_(4B).ipynb#scrollTo=-Xbb0cuLzwgf

Google Colab

#

🦥 Unsloth: Will patch your computer to enable 2x faster free finetuning.
🦥 Unsloth Zoo will now patch everything to make training faster!

ImportError Traceback (most recent call last)
/tmp/ipython-input-2-1326964708.py in <cell line: 0>()
17 ] # More models at https://huggingface.co/unsloth
18
---> 19 model, tokenizer = FastModel.from_pretrained(
20 model_name = "unsloth/gemma-3-4b-it",
21 max_seq_length = 2048, # Choose any for long context!

2 frames
/usr/local/lib/python3.11/dist-packages/unsloth_zoo/temporary_patches/gemma.py in patch_Gemma3ForConditionalGeneration_causal_mask()
161 try: import transformers.models.gemma3.modeling_gemma3
162 except: return
--> 163 from transformers.models.gemma3.modeling_gemma3 import (
164 StaticCache,
165 HybridCache,

ImportError: cannot import name 'StaticCache' from 'transformers.models.gemma3.modeling_gemma3' (/usr/local/lib/python3.11/dist-packages/transformers/models/gemma3/modeling_gemma3.py)

unsloth (Unsloth AI)

glacial bramble Jul 7, 2025, 4:18 PM

#

temp solution !pip install -U transformers==4.52.4 in the installation cell

#

ugh 😄

sharp raven Jul 7, 2025, 4:29 PM

#

temp solution for the first notebook or the second or both 🙂

glacial bramble Jul 7, 2025, 4:31 PM

#

gemma3

sharp raven Jul 7, 2025, 8:47 PM

#

that worked 🙏

arctic delta Jul 8, 2025, 12:46 PM

#

Once I am downgrading to 4.52.4 the FastVisionModel library is not getting installed from unsloth

sharp raven Jul 9, 2025, 1:04 AM

#

@glacial bramble any update on this?

mossy bay Jul 9, 2025, 4:16 AM

#

Fixing it asap sorry!

#

The goal is in a few hours

#

Sorry!

sharp raven Jul 9, 2025, 1:14 PM

#

Thank you Daniel!

high charm Jul 9, 2025, 6:52 PM

#

Thanks @mossy bay as I am also having the same issues with the 3n notebook and tried all of the temporary workarounds to no avail.

glacial bramble Jul 9, 2025, 7:40 PM

#

it's already solved. a release will be made later today

mossy bay Jul 10, 2025, 2:52 PM

#

@high charm @sharp raven @arctic delta I just fixed it + reduced VRAM usage by 25% + made it faster + fixed vision! Please update Unsloth via pip install --upgrade --force-reinstall --no-cache-dir --no-deps unsloth unsloth_zoo

high charm Jul 11, 2025, 12:31 PM

#

mossy bay <@1315659315008376923> <@1284296794066518157> <@797484761093636156> I just fixed...

Awesome sauce! Thanks @mossy bay So the rest of the 3n notebook requires no modification and we only have to update Unsloth via pip install --upgrade --force-reinstall --no-cache-dir --no-deps unsloth unsloth_zoo

sharp raven Jul 11, 2025, 1:25 PM

#

@mossy bay trying to run this notebook on a A100 and get this :
CUDA error: device-side assert triggered\nCUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.\nFor debugging consider passing CUDA_LAUNCH_BLOCKING=1\nCompile with TORCH_USE_CUDA_DSA to enable device-side assertions.\n"

📎 message.txt 📎 message.txt

#

This was just testing the image inference with : model_name = "unsloth/gemma-3n-E4B-it",

glacial bramble Jul 11, 2025, 1:31 PM

#

guys don't tag in every single message 🙏🏻 . we can see the thread

#

I just ran the notebook https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Gemma3N_(4B)-Conversational.ipynb?authuser=2#scrollTo=9jGeSb9bWe0k

Google Colab

#

and it works

#

if you're on a local machine, make sure to update your current install of unsloth

#

and make sure that it did update

sharp raven Jul 11, 2025, 1:39 PM

#

same. worked for me too. 🤔

glacial bramble Jul 11, 2025, 1:40 PM

#

yes can be one of several reasons (notebook cache , etc..)

sharp raven Jul 11, 2025, 1:40 PM

#

Understood. Thanks.

glacial bramble Jul 11, 2025, 1:40 PM

#

✅

sharp raven Jul 11, 2025, 3:12 PM

#

I created a new workstation with a t4 attached and same result.
unknown:0: unknown: block: [55,0,0], thread: [384,0,0] Assertion index out of bounds: 0 <= tmp6 < 128 failed.

glacial bramble Jul 11, 2025, 4:07 PM

#

sharp raven I created a new workstation with a t4 attached and same result. unknown:0: unk...

on colab?

#

and we're still talking about gemma3-n , correct?

sharp raven Jul 11, 2025, 4:43 PM

#

gemma3-n yes. not colab. downloaded the colab nb as a ipynb and am running on a gcp cloud workstation with a t4 attached.

glacial bramble Jul 11, 2025, 4:51 PM

#

in this case can you try one thing for me (if you got th etime):
in a new environment

#

avoid installing into the system wide python

#

install from main repo instead of pypi

#

wait no too much hustle

sharp raven Jul 11, 2025, 4:52 PM

#

yep... using a venv

#

always in fact 🙂

elfin musk Jul 13, 2025, 4:42 PM

#

the same CUDA error. win, docker, 4070/5090 ... and ```hard_emb = self.embedding(input_ids - self.vocab_offset)

#

📎 message.txt

mossy bay Jul 14, 2025, 5:30 AM

#

Oh that's a weird issue

#

Wait

#

please update Unsloth, timm, unsloth_zoo

#

pip install --upgrade --force-reinstall --no-deps --no-cache-dir unsloth unsloth_zoo timm transformers

arctic delta Jul 14, 2025, 8:32 AM

#

I am using H100 gpu machine I also tried the same code as given in Collab notebook but still the same issue I am getting.

#

CUDA error : device-side assert triggered

arctic delta Jul 14, 2025, 8:34 AM

#

mossy bay please update Unsloth, timm, unsloth_zoo

The input tensor are not going to GPU even after upgrading all the libraries

mossy bay Jul 14, 2025, 10:31 AM

#

@arctic delta is it the same issue as above? or another issue

elfin musk Jul 14, 2025, 1:55 PM

#

updating libraries did not help. log for 5090, same problem. unsloth/Mistral-Small-3.2-24B-Instruct-2506-unsloth-bnb-4bit works with the same code

📎 message.txt

arctic delta Jul 14, 2025, 3:45 PM

#

mossy bay <@797484761093636156> is it the same issue as above? or another issue

Same issue Daniel

mossy bay Jul 14, 2025, 3:57 PM

#

@glass lagoon could you investigate these issues!

hot flame Jul 14, 2025, 4:16 PM

#

is unsloth support gemma-3n vision-to-text finetune ?

glass lagoon Jul 14, 2025, 4:20 PM

#

I suspect one way to solve this is using torch2.6 and triton 3.2.
triton 3.2 should get installed with torch 2.6 if you install torch like

pip install torch==2.6.0 torchvision==0.21.0 torchaudio==2.6.0 --index-url https://download.pytorch.org/whl/cu126

#

When running the colab notebooks locally you should also run the notebooks in a fresh environment so that pip installs don't override any existing installs.

elfin musk Jul 14, 2025, 4:31 PM

#

glass lagoon I suspect one way to solve this is using torch2.6 and triton 3.2. triton 3.2 sh...

but it doesn't support 5090

glass lagoon Jul 14, 2025, 4:34 PM

#

should work on the 4000 series, could you try?

glass lagoon Jul 14, 2025, 4:34 PM

#

elfin musk the same CUDA error. win, docker, 4070/5090 ... and ```hard_emb = self.embedding...

yea it would be helpful to narrow down the issue if you have access to different hardware like this

elfin musk Jul 14, 2025, 4:36 PM

#

glass lagoon yea it would be helpful to narrow down the issue if you have access to different...

it helped as workaround

os.environ["UNSLOTH_COMPILE_DISABLE"] = "1"
os.environ["UNSLOTH_DISABLE_FAST_GENERATION"] = "1"

glass lagoon Jul 14, 2025, 4:39 PM

#

got it, but it would be greatly appreciated if you could test the suggestion on the 40xx card without disabling compile.

elfin musk Jul 14, 2025, 5:10 PM

#

glass lagoon got it, but it would be greatly appreciated if you could test the suggestion on ...

📎 message.txt

glass lagoon Jul 14, 2025, 5:14 PM

#

oh intersting. this is without DISABLE_COMPILE?

elfin musk Jul 14, 2025, 5:14 PM

#

yes

glass lagoon Jul 14, 2025, 5:18 PM

#

ok so at least 1 step worked. progress!

#

there are some things im noticing like the python version warning for xformers. then just want to ask how you are installing transformers and timm?

elfin musk Jul 14, 2025, 5:20 PM

#

glass lagoon oh intersting. this is without DISABLE_COMPILE?

with

os.environ["UNSLOTH_COMPILE_DISABLE"] = "1"
os.environ["UNSLOTH_DISABLE_FAST_GENERATION"] = "1"

OOM

📎 message.txt

glass lagoon Jul 14, 2025, 5:22 PM

#

yes torch compile does help with vram usage as it will fuse ops together if possible

#

but i'm wondering if this still stems from the xformers warning

elfin musk Jul 14, 2025, 5:23 PM

#

I built a test container like this

FROM nvidia/cuda:12.8.0-devel-ubuntu24.04

ENV DEBIAN_FRONTEND=noninteractive
ENV PYTHONUNBUFFERED=1

RUN --mount=type=cache,target=/var/cache/apt \
    apt-get update && \
    apt-get install -y \ 
        python3.12 python3.12-venv python3.12-dev pip \ 
        supervisor rsync git wget mc nano \
        cmake pkg-config libcurl4-gnutls-dev build-essential && \ 
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

# libcairo2 libcairo2-dev 

RUN python3.12 -m venv /opt/venv

ENV TORCH_CUDA_ARCH_LIST="8.9 12.0"
ENV CUDA_HOME=/usr/local/cuda-12.8
ENV PATH=$CUDA_HOME/bin:$PATH
ENV NVIDIA_VISIBLE_DEVICES=all
ENV NVIDIA_DRIVER_CAPABILITIES=video,compute,utility

ENV PATH="/opt/venv/bin:$PATH"

ENV MAX_JOBS=16

RUN pip install --upgrade pip setuptools wheel ninja
RUN pip install torch==2.6.0 torchvision==0.21.0 torchaudio==2.6.0 --index-url https://download.pytorch.org/whl/cu126
RUN pip install --no-deps psutil regex rich bitsandbytes accelerate peft trl==0.15.2 cut_cross_entropy unsloth_zoo
RUN pip install transformers sentencepiece protobuf "datasets>=3.4.1" huggingface_hub hf_transfer
RUN pip install --no-deps unsloth
RUN pip install --no-deps triton
RUN pip install --no-deps xformers
RUN pip install --no-deps --upgrade timm

RUN echo "[supervisord]\n\
nodaemon=true\n\
logfile=/dev/null\n\
logfile_maxbytes=0\n" > /etc/supervisor/conf.d/supervisord.conf

WORKDIR /app

CMD ["/usr/bin/supervisord"]

glass lagoon Jul 14, 2025, 5:35 PM

#

oh you're using a cuda12.8 image with cuda 12.6 torch. is at least one issue I see. Although in practice not exactly sure what happens.

that's also an older trl version and I don't think you need to specify a version anymore

#

i'm not sure that torch 2.6 ships with cuda12.8 tbh

#

you can try setting a specific version on xformers because it seems like the no-deps is casuing issues on the 12.8 image

#

but to make it easier after installing torch i think you could just do
pip install unsloth, then do the transformers and timm upgrades

#

but for this test i would use a 12.6 image instead of 12.8 if possible

elfin musk Jul 14, 2025, 5:42 PM

#

cuda 12.6 image: works without errors and without xformers

#

   \\   /|    Num examples = 9,337 | Num Epochs = 3 | Total steps = 7,005
O^O/ \_/ \    Batch size per device = 2 | Gradient accumulation steps = 2
\        /    Data Parallel GPUs = 1 | Total batch size (2 x 2 x 1) = 4
 "-____-"     Trainable parameters = 42,270,720 of 5,481,708,992 (0.77% trained)
  0%|                                                                                                                                     | 0/7005 [00:00<?, ?it/s]`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`.
Unsloth: Will smartly offload gradients to save VRAM!
{'loss': 7.0253, 'grad_norm': 3990330.25, 'learning_rate': 0.0, 'epoch': 0.0}                                                                                      
{'loss': 7.3983, 'grad_norm': 4439897.5, 'learning_rate': 4.27960057061341e-08, 'epoch': 0.0}                                                                      
{'loss': 4.45, 'grad_norm': 207282.3125, 'learning_rate': 8.55920114122682e-08, 'epoch': 0.0}                                                                      
{'loss': 3.7255, 'grad_norm': 116172.859375, 'learning_rate': 1.283880171184023e-07, 'epoch': 0.0}                                                                 
{'loss': 3.6248, 'grad_norm': 47774.57421875, 'learning_rate': 1.711840228245364e-07, 'epoch': 0.0}                                                                
{'loss': 3.4556, 'grad_norm': 25535.576171875, 'learning_rate': 2.139800285306705e-07, 'epoch': 0.0}                                                               
{'loss': 3.8237, 'grad_norm': 24524.2890625, 'learning_rate': 2.567760342368046e-07, 'epoch': 0.0}                                                                 
{'loss': 3.472, 'grad_norm': 16085.9140625, 'learning_rate': 2.9957203994293864e-07, 'epoch': 0.0}                                                                 
{'loss': 3.5032, 'grad_norm': 16694.767578125, 'learning_rate': 3.423680456490728e-07, 'epoch': 0.0}  ```

glass lagoon Jul 14, 2025, 5:43 PM

#

ok great at least that works

#

so now if I rewind a bit, if you use a 40xx machine but run a 12.8 image, with
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128 and not changing anything else, will it fail?

elfin musk Jul 14, 2025, 6:02 PM

#

glass lagoon so now if I rewind a bit, if you use a 40xx machine but run a 12.8 image, with ...

i use 5090, but i have 4070 super ti 16gb and 4060 ti 16gb installed in my system so i can check both... if I install this version the issue comes back

glass lagoon Jul 14, 2025, 6:05 PM

#

elfin musk i use 5090, but i have 4070 super ti 16gb and 4060 ti 16gb installed in my syste...

ok so just to confirm the 12.6 config works on 40xx and 12.8 does not. and 5090 doesn't work for 12.6 bc not supported and doesn't work for 12.8?

elfin musk Jul 14, 2025, 6:05 PM

#

yes

elfin musk Jul 14, 2025, 6:15 PM

#

glass lagoon ok so just to confirm the 12.6 config works on 40xx and 12.8 does not. and 5090 ...

when I try to run training for this model on 5090 it sometimes crashes nvidia driver

glass lagoon Jul 14, 2025, 6:21 PM

#

yea thing with the 5090 is that it's a newer arch, so i'm unsure how it ends up compiling

#

i need to investigate. could be that certain ops aren't yet supported.

elfin musk Jul 14, 2025, 6:45 PM

#

glass lagoon i need to investigate. could be that certain ops aren't yet supported.

strange thing. cuda image 12.8:
with pip install torch==2.6.0 torchvision==0.21.0 torchaudio==2.6.0 --index-url https://download.pytorch.org/whl/cu126 -> not working on 4070
or with pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128 -> not working on 4070

but when i downgraded libs to:
pip install torch==2.6.0 torchvision==0.21.0 torchaudio==2.6.0 --index-url https://download.pytorch.org/whl/cu126 -> working on 4070 with xformers (no warning)

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128 --force-reinstall -> not working again

pip install torch==2.6.0 torchvision==0.21.0 torchaudio==2.6.0 --index-url https://download.pytorch.org/whl/cu126 --force-reinstall -> working on 4070 without xformers ( xFormers can't load C++/CUDA extensions)

glass lagoon Jul 14, 2025, 6:54 PM

#

yea that's interesting, the reinstall must also change some of the reqs that were initially satisified.

elfin musk Jul 14, 2025, 7:22 PM

#

b -> not working on 4070, cuda error; a-> working

📎 requirements-a.txt 📎 requirements-b.txt 📎 message.txt

glass lagoon Jul 14, 2025, 7:28 PM

#

https://www.diffchecker.com/UvfKdUMD/

a vs b diff - Diffchecker

a vs b diff - accelerate==1.8.1
aiohappyeyeballs==2.6.1
aiohttp==3.12.14
aiosignal==1.4.0
attrs==25.3.0
bitsandbyt

elfin musk Jul 14, 2025, 8:39 PM

#

Working docker container for 40xx, torch > 2.6.0 - cuda error

FROM nvidia/cuda:12.6.0-devel-ubuntu24.04

ENV DEBIAN_FRONTEND=noninteractive
ENV PYTHONUNBUFFERED=1

RUN --mount=type=cache,target=/var/cache/apt \
    apt-get update && \
    apt-get install -y \ 
        python3.12 python3.12-venv python3.12-dev pip \ 
        supervisor rsync git wget mc nano \
        cmake pkg-config libcurl4-gnutls-dev build-essential && \ 
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

# libcairo2 libcairo2-dev 

RUN python3.12 -m venv /opt/venv

ENV TORCH_CUDA_ARCH_LIST="8.9 12.0"
ENV CUDA_HOME=/usr/local/cuda-12.6
ENV PATH=$CUDA_HOME/bin:$PATH
ENV NVIDIA_VISIBLE_DEVICES=all
ENV NVIDIA_DRIVER_CAPABILITIES=video,compute,utility

ENV PATH="/opt/venv/bin:$PATH"

ENV MAX_JOBS=16

RUN pip install --upgrade pip setuptools wheel ninja
RUN pip install torch==2.6.0 torchvision==0.21.0 torchaudio==2.6.0 xformers==0.0.29.post3 --index-url https://download.pytorch.org/whl/cu126
RUN pip install --no-deps psutil regex rich bitsandbytes accelerate peft trl cut_cross_entropy unsloth_zoo
RUN pip install transformers sentencepiece protobuf "datasets>=3.4.1" huggingface_hub hf_transfer
RUN pip install --no-deps unsloth
RUN pip install --no-deps triton
RUN pip install --no-deps --upgrade timm

RUN echo "[supervisord]\n\
nodaemon=true\n\
logfile=/dev/null\n\
logfile_maxbytes=0\n" > /etc/supervisor/conf.d/supervisord.conf

WORKDIR /app

CMD ["/usr/bin/supervisord"]

glass lagoon Jul 15, 2025, 2:24 AM

#

ok you could try torch 2.7.0 with triton 3.2.0. just put the following triton line after installing torch 2.7.0
RUN pip install --no-deps triton==3.2.0

karmic crystalBOT Jul 15, 2025, 2:24 AM

#

Torch 2.7.0 is not officially released with CUDA 12.6 or 12.8 wheels as of now, and Unsloth's compatibility matrix and install scripts do not list torch 2.7.0 as supported for CUDA 12.6/12.8. Additionally, using triton==3.2.0 with torch 2.7.0 may cause incompatibility, as torch 2.7.0 expects triton>=3.3.0 for Blackwell/RTX 50xx support and newer CUDA versions. This combination is not documented as supported in Unsloth's official install guides or troubleshooting resources. Attempting this may result in import or runtime errors due to mismatched dependencies and unsupported CUDA/torch/triton combinations. For 40xx cards, torch==2.6.0 with CUDA 12.6 and triton==3.2.0 is the recommended, tested configuration; for 50xx/Blackwell, torch==2.7.0 with triton>=3.3.1 and CUDA 12.8 is required, as per the official Blackwell install guide and Unsloth documentation.

Would you like a step-by-step breakdown of the correct install sequence for your specific GPU and CUDA version?

Sources:

glass lagoon Jul 15, 2025, 2:42 AM

#

lol these are good points unsloth bot

karmic crystalBOT Jul 15, 2025, 2:42 AM

#

I'm here to help with any questions or troubleshooting related to Unsloth, including installation, compatibility, and fine-tuning! If you have a specific Unsloth setup, error, or workflow you want to discuss, just let me know the details and I'll do my best to assist. If you need step-by-step guidance for a particular GPU, CUDA, or library version, feel free to ask!

wise frost Jul 16, 2025, 9:06 PM

#

sharp raven I created a new workstation with a t4 attached and same result. unknown:0: unk...

Same problem running locally

glass lagoon Jul 16, 2025, 11:17 PM

#

for now gemma3n will not work with torch 2.7 /triton 3.3. It will only work with torch 2.6 / triton 3.2. I am working on a fix.

mossy bay Jul 17, 2025, 2:13 PM

#

just fixed it!!! sorry on the delay

#

pip install --upgrade --force-reinstall --no-cache-dir --no-deps unsloth unsloth_zoo

keen mantle Jul 17, 2025, 3:52 PM

#

now i get

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
/usr/local/lib/python3.11/dist-packages/unsloth_zoo/loss_utils.py in _unsloth_get_batch_samples(self, epoch_iterator, num_batches, device, *args, **kwargs)
    315                 if "token_type_ids" in x:
--> 316                     token_type_ids = kwargs["token_type_ids"]
    317                     mark_static (token_type_ids, 0)

KeyError: 'token_type_ids'

During handling of the above exception, another exception occurred:

RuntimeError                              Traceback (most recent call last)
/usr/local/lib/python3.11/dist-packages/accelerate/utils/memory.py in decorator(*args, **kwargs)
    166             try:
--> 167                 return function(batch_size, *args, **kwargs)
    168             except Exception as e:

/usr/local/lib/python3.11/dist-packages/unsloth_zoo/compiler.py in _fast_inner_training_loop(self, batch_size, args, resume_from_checkpoint, trial, ignore_keys_for_eval)

/usr/local/lib/python3.11/dist-packages/unsloth_zoo/loss_utils.py in _unsloth_get_batch_samples(self, epoch_iterator, num_batches, device, *args, **kwargs)
    327         except Exception as exception:
--> 328             raise RuntimeError(exception)
    329     pass

RuntimeError: 'token_type_ids'

in kaggle

glass lagoon Jul 17, 2025, 4:03 PM

#

during training?

#

dang I checked colab earlier and it was working. wonder what's different in kaggle

#

oh yea its inside the training loop

#

it might be a quick fix. I have a branch you could test if you have capacity. pip install --no-deps git+https://github.com/mmathew23/unsloth-zoo.git@gemma3nx

#

@keen mantle

keen mantle Jul 17, 2025, 4:15 PM

#

glass lagoon it might be a quick fix. I have a branch you could test if you have capacity. `p...

trying this rn

#

yea it works thanks

keen mantle Jul 17, 2025, 4:24 PM

#

glass lagoon it might be a quick fix. I have a branch you could test if you have capacity. `p...

📎 message.txt

glacial bramble Jul 17, 2025, 4:24 PM

#

my bad i was wrong

keen mantle Jul 17, 2025, 4:24 PM

#

randomly happen when training

glacial bramble Jul 17, 2025, 4:24 PM

#

they did release on pypi but didn't add to the releases page

#

😄

hot flame Jul 17, 2025, 4:26 PM

#

i got the same error about recompilation as well , it happen at around step 100-200 randomly , Gemma-3n


   1752         result = None

/content/unsloth_compiled_cache/unsloth_compiled_module_gemma3n.py in forward(self, input_ids, inputs_embeds)
   1430         inputs_embeds: Optional[torch.Tensor] = None,
   1431     ) -> torch.Tensor:
-> 1432         return Gemma3nMultimodalEmbedder_forward(self, input_ids, inputs_embeds)
   1433 
   1434 

/usr/local/lib/python3.11/dist-packages/torch/_dynamo/eval_frame.py in _fn(*args, **kwargs)
    572 
    573             try:
--> 574                 return fn(*args, **kwargs)
    575             finally:
    576                 # Restore the dynamic layer stack depth if necessary.

RuntimeError: Recompilation triggered with skip_guard_eval_unsafe stance. This usually means that you have not warmed up your model with enough inputs such that you can guarantee no more recompilations.

keen mantle Jul 17, 2025, 4:28 PM

#

unsloth-zoo==2025.7.4 was working fine in kaggle so i am downgrading to this

glass lagoon Jul 17, 2025, 5:42 PM

#

hot flame i got the same error about recompilation as well , it happen at around step 100-...

what sort of sequence lens are you running? i wonder if padding to multiple of would alleviate this

mossy bay Jul 17, 2025, 9:51 PM

#

oh the recompilation error is normal - you can ignore it

#

i might have to disable some things - the goal was to make it faster 😦

#

does it just error out and thats all?

#

ie it stops running?

hot flame Jul 18, 2025, 2:26 AM

#

mossy bay ie it stops running?

Frankly i had both case. one time its error and stop, another time its just display error and continue.

hot flame Jul 18, 2025, 2:27 AM

#

glass lagoon what sort of sequence lens are you running? i wonder if padding to multiple of w...

i finetuned with 512 sequence length

#

i currently reverted to unsloth==2025.7.3 unsloth_zoo==2025.7.4 and the problem is disappear, please let me know if you want test again !

mossy bay Jul 18, 2025, 3:32 AM

#

oh ok ok

#

ill fix the recompilation issue asap!

#

its mainly my fault for trying to make things faster

#

in the end it was more problematic

mossy bay Jul 18, 2025, 12:50 PM

#

Fixed the recompilation issue!

#

Please do pip install --upgrade --force-reinstall --no-cache-dir --no-deps unsloth unsloth_zoo

hot flame Jul 18, 2025, 3:25 PM

#

After install latest the recompilataion issue is not appear again, thank you Unsloth slothhearts

#Gemma 3n Colab Notebook Not Working

Please restructure your imports with 'import unsloth' at the top of your file. from unsloth import FastModel

🦥 Unsloth: Will patch your computer to enable 2x faster free finetuning. 🦥 Unsloth Zoo will now patch everything to make training faster!

Please restructure your imports with 'import unsloth' at the top of your file.
from unsloth import FastModel

🦥 Unsloth: Will patch your computer to enable 2x faster free finetuning.
🦥 Unsloth Zoo will now patch everything to make training faster!