CodeLlama-34B is repeating itself

[hasty pollen]

I asked CodeLlama-34B to fix its bug more than five times, each time it's just repeating its own code, get the same bug, and say the same words. There might be something wrong in prompt chaining?

[visual hull]

Yes, there's an issue in the system message txt found where interpreter is installed in site-packages. You need to edit the first line

change it to: "You are Open Interpreter, a world-class programmer that can complete any goal. You know how to write and execute code."

[hasty pollen]

that doesn't seem too different with the current prompt ''You are Open Interpreter, a world-class programmer that can complete any goal by executing code.'', but I'll give it a try

it's still repeating itself

probably related to this issue:
https://github.com/KillianLucas/open-interpreter/issues/221

[prime walrus]

I've had the same issue, it didn't seem to happen as much on smaller models for some reason, but it happens alot for me on 30B~ models

[hasty pollen]

that's quite strange, I thought the 34B model shouldn't have such a problem

[opal herald]

I tested with 2 Local models, codellama-13b-instruct.Q5_K_M get stuck in a loop instantly after startup, without user input. but if I start with --debug , it works, maybe give it a try!

[hasty pollen]

still repeating itself with --debug

[opal herald]

yea, with --debug i only got rid of the Loop at startup, but i too have the issue that you only have 1 prompt, and than the local model is repeating itself.

[short ore]

Unfortunately, it's a known issue with llama2 models. Some are worse than others. https://www.reddit.com/r/LocalLLaMA/comments/155vy0k/llama_2_too_repetitive/

[opal herald]

I don’t know if it is the same Issue.
Its more like it only really react to the first Prompt, and everything you do after that doesn’t reach it.

[short ore]

Where's the install location?

[visual hull]

site-packages in your venv or python install location in Lib/site-packages/interpreter

[glacial obsidian]

Same here. Doesn't matter what you give as input. The answer will repeat until you reboot interpreter.

[prime walrus]

This could be an issue with code llama I'm not sure but normal llama chat models have not gotten stuck in a loop for me. I have only used them on exllama though.

[opal herald]

i tested it with llama-2-13b-chat, and it is not stuck in a loop. so it seems to be something with code llama

does OI use different system parameters on instruct and chat models? where are they located?