Is there a good way to manage iterations of runs, what's the best way to keep track of my work.
Currently, I delete the .cache and any output between each run which is daft as I've lost some interesting stuff in my experiments.
#Uncertain of best practice for how to keep track of run versions
winter tinsel
slender nexus
have you considered changing "seed" in the llm_config? Each seed has its own cache.
winter tinsel
I have, but it writes to the same work_dir I believe? But I guess I can just rerun with the old seed to get the output back?
Then I can just keep versions of the files with a different seed value in it.
Will go test that thanks!
slender nexus
You can use a work_dir specialized for each seed, like work_dir_{seed}. If you have removed the .cache, rerunning the old seed doesn't guarantee the same output if you use OpenAI models.