fierce ether
Hello All! I am really excited to be presenting my new dataset constructor to you all! I have decided to open source the project but im keeping it secret until tomoroow, please help me fix this bug so that i can have a smooth launch https://tenor.com/view/im-sorry-smile-dragon-ball-anime-scratch-head-gif-17395103
i have currently setup my tool to utilize 3 datasets, train, validate, and test, but i think this is wrong, i am also using the unsloth-cli.py file and so this may be causing me and issue
i have the repo here as well but i am hesitating on sharing until it full works i really want the launch to be good
fierce ether
i guess its also good for me to share the dataset I used! its 3 variations of this dataset where the test set has the output column removed
https://huggingface.co/datasets/Borcherding/OARC_Commander_v001
claude may be coming to the rescue
after now calling unsloth-cli.py its saying it has not argument validation dataset
fierce ether
fierce ether
my confusion is that seb told me there is a train dataset, a validation dataset and a test dataset, but im comparing to these scripts and i dont see how to hook up the multiple datasets:
https://github.com/sebdg/SebLLama-Notebooks/blob/main/Emotions/ollama_llama3_finetuning.py
@hollow moon 😭
so from the colab for llama3.1
should i be merging the 3 datasets into 1?
oooh
i think im getting it
claude is helping
the validation is a trick
you seed the train set with it randomly no?
would love to hear from someone if this is correct xD im gonna work on implementing it since it seems like it will work!
fierce ether
@dry barn this modification seems to need to edit the unsloth-cli.py file for now i will make a test file and see if i can extract it out to my tool, but if this is of interest i can provide this as a second option to unsloth-cli.py and push to the main repo
fierce ether
at first i thought my models were bad but after re downloading them a few times im not sure why its saying the header is too large
🍲Agent Chef🥘: Header Too Large
fierce ether
git lfs was not enabled and i was missing my c++ compiler lol
and now its done 🙂
the model was still learning when it finished too!
but this is amazing! it was done seamlessness all through my AgentChef Webui"
fierce ether
==========================================================================================================
Okay! new issue time!
I have successfully generated the adapter_model.safetensors, gonna try and fix that to have it save with a better name,
but now i have build in a merge feature to merge the lora with the base model, and its throwing and error when merging
Here is the error when running the merge
INFO:cutlery.UnslothTrainer:ERROR:main:Error loading adapter: '<=' not supported between instances of 'str' and 'int'
INFO:cutlery.UnslothTrainer:ERROR:main:Failed to load adapter: '<=' not supported between instances of 'str' and 'int'
======================================================================
this is utilizing my update for unsloth-cli.py currently named unsloth-cli-2.py
🍲Agent Chef🥘: LORA MERGE ISSUE
hasty cloak
ok weird bug
"INFO:cutlery.UnslothTrainer:ERROR:main:Error loading adapter: '<=' not supported between instances of 'str' and 'int'
INFO:cutlery.UnslothTrainer:ERROR:main:Failed to load adapter: '<=' not supported between instances of 'str' and 'int'"
By the way on merging - https://github.com/unslothai/studio/blob/main/unsloth_studio/saving_utils.py might be more useful
fierce ether
im almost finished with the update for unsloth-cli-2.py 🙂 i will drop it here when its working and then maybe you guys can guys can roast me on how bad i did lol here is the feature set that i am using for Agent Chef and that im using rn to train a model for the lablab Ai Agents Hackathon 🙂 will be mentioning made with unsloth i can get it running in time deadline is tomorrow at 7pm xD
tulip hemlock
I need to try merging as soon as possible. ✅
tulip hemlock
I need to try merging as soon as possible. ✅
just quick test:
python unsloth-cli-2.py merge --base_model_path "WizardLMTeam/WizardMath-7B-V1.1" --adapter_path "/home/mario/archive-2/lora-wizardmath-hpc-accounting-epoch3/" --output_path "/home/mario/archive-3/merged-accounting-bot" --dequantize f16
and I'm getting:
ERROR:main:Error merging adapter: We need an offload_dir to dispatch this model according to this device_map, the following submodules need to be offloaded: base_model.model.model.layers.4, base_model.model.model.layers.5, base_model.model.model.layers.6, base_model.model.model.layers.7,
...
hasty cloak
Oh very cool! Great work again!
fierce ether
