I'm having troubles where gpt-3.5-turbo-1106 is cutting off in the middle of tool call arguments and then just adding a ton of \t, \r, spaces, or \n? It wasn't happening in the previous 3.5, but it's happening a lot now. I just noticed it sometimes also puts them in the middle as well. It often will just fill up the context window with them.
#gpt-3.5-turbo-1106 adding many tabs, spaces, or newlines in the middle of tool parameters
ashen void
Example:
[previous correct formatting/JSON] \"likelihood\": 4 }\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\n
scarlet torrent
sacred canopy
Try playing with your temperature settings as well, try temperature 0 @ashen void
ashen void
I think temperature was 0. I did try setting the presence penalty and I think that helped, but @scarlet torrent I think you're right that this is a bug with the model and more specifically json mode which I assume they use for tool calls.
hard forum
Don't use temperature=0. If you need determinism I'd recommend setting a seed. So many problems arise from using super low temperature / top_p
sacred canopy
not for me ever personally
sounds like you may need to improve your prompting, temp 0 is indeed determinismn
also shouldnt be using both temp and topp
hard forum
A lot of people get success with their same prompt just by using 1 for both 🤷 I highly suggest to OP to try with default values and see if their problem still persists
sacred canopy
Sure but you said that if you need determinism you should set a seed, that is false, temperature is still in effect even if you have a seed and will cause random variance, using a temperature of 1 for tasks like extraction, classification, will also lead to inaccurate results as it leaves things up for interpretation
also, you can't really do prompt experimentation with temperature >0 because you don't know if the behaviour is from randomness or from your prompt