solemn rapids
https://unsloth.ai/docs/basics/fine-tuning-llms-with-nvidia-dgx-spark-and-unsloth
I m running this same notebook in my DGX but it says need 42H to finish the training, but the doc says will get over in 4H
Can anysome help me tune the parameters to get the training quickly ? Also the only 28GB of VRAM is getting used while I have 128GB available
fiery hornet
Hey @solemn rapids ,
Here’s what I would change first, in order:
training_args = GRPOConfig(
temperature = 1.0,
learning_rate = 5e-5,
weight_decay = 0.001,
warmup_ratio = 0.1,
lr_scheduler_type = "linear",
optim = "adamw_8bit",
logging_steps = 10, # less logging overhead
per_device_train_batch_size = 2, # since num_generations is 2 anyway
gradient_accumulation_steps = 1,
num_generations = 1, # biggest speed win
max_prompt_length = max_prompt_length,
max_completion_length = 256, # or 384, cap generation length
max_steps = 200, # test speed first before full 1000
save_steps = 100,
report_to = "none",
output_dir = "outputs",
)