#Can't proceed past downloading model

2 messages · Page 1 of 1 (latest)

light oyster
#

I'm running unsloth in Jupyer, and after this code:

from unsloth import FastLanguageModel
import torch
max_seq_length = 8192 # Choose any! Unsloth auto supports RoPE Scaling internally!
dtype = None
load_in_4bit = True

# 4bit pre quantized models we support for 4x faster downloading + no OOMs.
fourbit_models = [
    "unsloth/mistral-7b-bnb-4bit",
    "unsloth/mistral-7b-instruct-v0.2-bnb-4bit",
    "unsloth/llama-2-7b-bnb-4bit",
    "unsloth/llama-2-13b-bnb-4bit",
    "unsloth/codellama-34b-bnb-4bit",
    "unsloth/tinyllama-bnb-4bit",
    "unsloth/gemma-7b-bnb-4bit",
    "unsloth/gemma-2b-bnb-4bit",
] # More models at https://huggingface.co/unsloth

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = MODEL, # Choose ANY! eg teknium/OpenHermes-2.5-Mistral-7B
    max_seq_length = max_seq_length,
    dtype = dtype,
    load_in_4bit = load_in_4bit,
    # token = "hf_...", # use one if using gated models like meta-llama/Llama-2-7b-hf
)

MODEL is "unsloth/mistral-7b-instruct-v0.2-bnb-4bit"
Which when ran outputs this:

==((====))==  Unsloth 2025.2.4: Fast Mistral patching. Transformers: 4.46.1.
   \\   /|    GPU: NVIDIA GeForce RTX 3090. Max memory: 24.0 GB. Platform: Windows.
O^O/ \_/ \    Torch: 2.6.0+cu126. CUDA: 8.6. CUDA Toolkit: 12.6. Triton: 3.1.0
\        /    Bfloat16 = TRUE. FA [Xformers = 0.0.29.post3. FA2 = False]
 "-____-"     Free Apache license: http://github.com/unslothai/unsloth
Unsloth: Fast downloading is enabled - ignore downloading bars which are red colored!

Download starts, as observed in task manager, then when it is completed (around 15mins judging by file size and download speed) nothing happens. It just hangs there, with Jupyter reporting busy kernel status and 2/15 cells executed. It's just stuck idling. Any help?

vital quail
#

restart the notebook and try again. the file is likely cached now. also check vram usage in msi afterburner