SmolVLM2: Unexpected Indentation Error | Unsloth AI | Page 1

somber helm Mar 24, 2025, 10:24 AM

#

Hello, I try to use unsloth to load smolvlm2 for inference but I got this error.

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 995, in exec_module
  File "<frozen importlib._bootstrap_external>", line 1133, in get_code
  File "<frozen importlib._bootstrap_external>", line 1063, in source_to_code
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/home/hermeschen/Repo/taiwan-license-plate-recognition/unsloth_compiled_cache/unsloth_compiled_module_smolvlm.py", line 481
    = loss_fct(shift_logits, shift_labels)
IndentationError: unexpected indent

Then I looked up "unsloth_compiled_cache/unsloth_compiled_module_smolvlm.py" and found it generate something wrong.
I supposed that loss_fct(shift_logits, shift_labels) should be assign to loss and try to fix it manaully but it still being overwritten. Is there any solution for this?

worn vigil Mar 26, 2025, 7:56 AM

#

is there any fix for this yet? did you manage to fix this?

placid whale Mar 26, 2025, 11:49 AM

#

somber helm Hello, I try to use unsloth to load smolvlm2 for inference but I got this error....

what is your code. Here it loads fine

#

Please always provide your environment, a code snippet, along with the error.

somber helm Mar 26, 2025, 1:31 PM

#

worn vigil is there any fix for this yet? did you manage to fix this?

Actually, I don’t really know what part of code in unsloth causes this.

somber helm Mar 26, 2025, 1:33 PM

#

placid whale Please always provide your environment, a code snippet, along with the error.

Sorry, I totally forget it. But I just follow the notebook provide by unsloth and add specify the attention implementation (flash attention), dtype(torch.bfloat16), and device_map(auto) as I remembered.

placid whale Mar 26, 2025, 1:34 PM

#

somber helm Sorry, I totally forget it. But I just follow the notebook provide by unsloth an...

can you link me to the specific notebook you're using if you don't mind?

somber helm Mar 26, 2025, 2:18 PM

#

placid whale can you link me to the specific notebook you're using if you don't mind?

https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Qwen2_VL_(7B)-Vision.ipynb

Google Colab

#

I think is this one.

#

It breaks when loading model

#

And my unsloth import is on the top of all other imports but it's before transformers

placid whale Mar 26, 2025, 10:33 PM

#

yes i'll check this out. the reason the file changes is that it's compiled just in time using a triton compiler

#

that's the case for all the filees under the folder "unsloth_compiled_cache" . That's what the unsloth_zoo compiler.py module does

somber helm Mar 26, 2025, 11:40 PM

#

placid whale that's the case for all the filees under the folder "unsloth_compiled_cache" . T...

I already checked compiler.py in unsloth zoo. But the related part looks fine. Maybe it’s triton’s problem?

placid whale Mar 26, 2025, 11:51 PM

#

actually i just ran

#

it

#

you should be fine from the latest updated version from today. They pushed changes

#

so in your system

#

pip install --upgrade --force-reinstall --no-cache-dir --no-deps git+https://github.com/unslothai/unsloth.git
pip install --upgrade --force-reinstall --no-cache-dir --no-deps git+https://github.com/unslothai/unsloth-zoo.git

#

you should be good to go

somber helm Mar 27, 2025, 12:32 AM

#

Yeah, I use the latest version. But still failed.

#

I will try if it works on newer commit

placid whale Mar 27, 2025, 6:09 AM

#

also:

are you running this on your device in a jupyter notebook or running on google colab. The notebook you shared is for Qwen..
if on your device, what is your GPU.
check you have cuda available inside the python environment

nvcc --version

and

nvidia-smi

which exact smolvlm model are you running, share the huggingface repo.
what versions of transformers, xformers, trl and torch do you have on your system

somber helm Mar 27, 2025, 7:44 AM

#

#

#

https://huggingface.co/HuggingFaceTB/SmolVLM2-500M-Video-Instruct
I use this as I remembered.

HuggingFaceTB/SmolVLM2-500M-Video-Instruct · Hugging Face

placid whale Mar 27, 2025, 8:07 AM

#

somber helm

nice usage of uv 😄

#

I am not getting an error at all on a 3060 which is odd

#

either:
1- share your code
or
2- let me know when can you hop on a discord call and share your screen so i can see what's going on

#

all the packages seem the latest

somber helm Mar 27, 2025, 8:23 AM

#

Actually, I already removed the code after found this problem because I just testing if it can works. It looks like the one below

import os
from argparse import ArgumentParser
from typing import Any, Dict, List

import datasets
import evaluate
import torch
import yaml
from unsloth import FastVisionModel
from datasets import load_dataset
from dotenv import load_dotenv
from transformers import GenerationConfig, TextStreamer

model, processor = FastVisionModel.from_pretrained(
    options["model_name"],
    load_in_4bit=False,  # Use 4bit to reduce memory use. False for 16bit LoRA.
    use_gradient_checkpointing="unsloth",  # True or "unsloth" for long context
    _attn_implementation="flash_attention_2",
    use_cache=True,
    dtype=torch.bfloat16,
    device_map="auto",
    low_cpu_mem_usage=True,
)
FastVisionModel.for_inference(model)

placid whale Mar 27, 2025, 10:08 AM

#

somber helm Actually, I already removed the code after found this problem because I just tes...

ok i just reproduced your error.

placid whale Mar 27, 2025, 11:04 AM

#

@somber helm do you mind opening a github issue with this. I think it might be a bug somewhere (potentially maybe in another package) and It might help resolve it faster if its on github issues as it will get more eyes .
just include the code snippet you shared with me.
mention that you're trying to load https://huggingface.co/HuggingFaceTB/SmolVLM2-500M-Video-Instruct
the fact that you're running on your local machine which has an RTX 3080
and that your versions of unsloth, torch,transformers, trl and xformers are up to date

#

let me know when you open it

somber helm Mar 27, 2025, 11:19 AM

#

Yeah, I’ve already opened one on GitHub.

#

https://github.com/unslothai/unsloth/issues/2179

GitHub

Generated unsloth_compiled_cache file cause Indentation Error when ...

I try to use unsloth with smolvlm2 but it keep throwing out "unexpected indentation error". The cause as the error message tells is in 481th line of the generated file unsloth_compiled_ca...

#

Here

placid whale Mar 27, 2025, 11:21 AM

#

thank you

placid whale Mar 27, 2025, 1:57 PM

#

solved it.

somber helm Mar 27, 2025, 2:02 PM

#

placid whale solved it.

Really!? So what’s the problem?

placid whale Mar 27, 2025, 2:04 PM

#

it's in the transformer's library implementation for the model. give me 10 mins i will add details to the github issue , push a PR to transfromers and a monkey patch PR to unsloth_zoo then i'll come back and tell you how to apply a hack until one of the PRs is approved

somber helm Mar 27, 2025, 2:05 PM

#

Okay, thank you very much!

placid whale Mar 27, 2025, 2:06 PM

#

it was tricky. i thought i was hallucinating 😂 sloththumbs

#

also don't turn on

unless you're training or finetuning.
It's not used during inference

somber helm Mar 27, 2025, 2:16 PM

#

Oh, you’re right. I forgot to remove it.

placid whale Mar 27, 2025, 7:19 PM

#

just pushed an unsloth-zoo patch pull request to solve this

#

let's give it a couple of hours to see if it gets merged to main 😄

#SmolVLM2: Unexpected Indentation Error