accuracy while training

[warm roost]

To print accuracy alongside validation loss during training, define a compute_metrics function that calculates accuracy, and pass it to the SFTTrainer via the compute_metrics argument. For binary classification (yes/no), you can use scikit-learn's accuracy_score as shown below:

from sklearn.metrics import accuracy_score

def compute_metrics(eval_pred):
    logits, labels = eval_pred
    preds = logits.argmax(axis=-1)
    return {"accuracy": accuracy_score(labels, preds)}

Then, add compute_metrics=compute_metrics to your SFTTrainer initialization. This approach is demonstrated in Unsloth's official BERT classification examples (see here).

Would you like a full code example or more details?

Sources:

• Unsloth BERT classification example
• Unsloth BERT classification notebook

💡 **Hint:** Mention @RunLLM in the thread for followups.

[soft lance]

per_device_train_batch_size=4,
gradient_accumulation_steps=4,

        eval_strategy="steps",
        eval_steps=60,

        save_strategy="steps",
        save_steps=60,

        warmup_steps=10,
        # max_steps = 150,
        num_train_epochs=3,  # Set this instead of max_steps for full training runs
        learning_rate=4e-5,
        logging_steps=1,
        optim="adamw_8bit",
        weight_decay=0.01,
        lr_scheduler_type="cosine",
        seed=3407,
        output_dir=output_dir,
        report_to="none",  # For Weights and Biases

        # You MUST put the below items for vision finetuning:
        remove_unused_columns=False,
        dataset_text_field="",
        dataset_kwargs={"skip_prepare_dataset": True},
        max_length=2048,

        metric_for_best_model="eval_loss",
        greater_is_better=False,

finetune_vision_layers=True, # False if not finetuning vision layers
finetune_language_layers=True, # False if not finetuning language layers
finetune_attention_modules=False, # False if not finetuning attention layers
finetune_mlp_modules=False, # False if not finetuning MLP layers

    # This is the key change to precisely match the paper
    target_modules=["q_proj", "v_proj"],

    r=8,  # The larger, the higher the accuracy, but might overfit
    lora_alpha=16,  # Recommended alpha == r at least
    lora_dropout=0,
    bias="none",
    random_state=3407,
    use_rslora=False,  # We support rank stabilized LoRA
    loftq_config=None,  # And LoftQ

With these settings i dont get good results at all, the VLM should learn to decide if an image has a good cropped box overlayed ontop of the image. So its binary classification, 2000 image dataset. help me pick better variables or what to change@warm roost