actually is not related to the padding, my bad
#Qwen2.5 14B Instruct giving padding error
foggy verge
the 14B not Instruct: ```---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Cell In[19], line 1
----> 1 trainer_stats = trainer.train()
File <string>:41, in train(self, resume_from_checkpoint, trial, ignore_keys_for_eval, **kwargs)
File /opt/conda/lib/python3.11/site-packages/torch/utils/_contextlib.py:116, in context_decorator.<locals>.decorate_context(*args, **kwargs)
113 @functools.wraps(func)
114 def decorate_context(*args, **kwargs):
115 with ctx_factory():
--> 116 return func(*args, **kwargs)
File /opt/conda/lib/python3.11/site-packages/unsloth/tokenizer_utils.py:944, in fix_untrained_tokens(model, tokenizer, train_dataset, eps)
941 pass
942 pass
--> 944 raise ValueError(
945 f'Unsloth: Untrained tokens of [{list(set(final_bad_items))}] found, but embed_tokens & lm_head not trainable, causing NaNs. '
946 'Restart then add embed_tokens & lm_head to '
947 'FastLanguageModel.get_peft_model(target_modules = [..., "embed_tokens", "lm_head",]). '
948 'Are you using the base model? Instead, use the instruct version to silence this warning.',
949 )
950 pass
952 # Count all the possible bad tokens
ValueError: Unsloth: Untrained tokens of [[151644, 151645]] found, but embed_tokens & lm_head not trainable, causing NaNs. Restart then add embed_tokens & lm_head to FastLanguageModel.get_peft_model(target_modules = [..., "embed_tokens", "lm_head",]). Are you using the base model? Instead, use the instruct version to silence this warning.```
narrow hemlock
foggy verge
seems so. the 14B base on the other hands need to additional target modules. the instruct issue is the above one
no issues on the 7b instruct ( did not try the base one 7b)
i am running an instance with pythorc 2.4.0 and cusa 12.1 and the installation cmd provided in docu, but i don't think the root cause is hiding here since i have no issue with other notebook/models
weary hatch