#Suddenly really bad responses

1 messages · Page 1 of 1 (latest)

old basin
#

Hi 😄
im quite new to oobabooga. (I use sillytavern frontend) I suffer from the Problem seen below, suddenly really bad respones. It worked before (i always make new chats), but today, openlynn_Llama-3-Soliloquy-8B-GPTQ gives out really bad responses, as seen below (completely new chat). What i also notices that the ui seemed to stop for a while, and put out the error code seen in the second picture. I also attached my settings. Does anybody have a solution for the problem? Thanks for the help in regard 😄

image.png image.png
#

It always puts this nonesens at the and, and generates things that dont make sense...

image.png
median marsh
# old basin

Try changing preset to Llama 3 Instruct too.
As for "couldn't find character none" error, it's that WebUI didn't load a character. Probably shouldn't affect SillyTavern but if you'd use WebUI itself for chatting, you'd need to select a character in Parameters tab.

old basin
#

I already did that, and enabled instruct mode

#

it seems to be the same problem with oder llms too, i tired some other ones also

#

(astronomer_Llama-3-8B-Instruct-GPTQ-4-Bit)

median marsh
#

I mean, you have Llama 3 Intsruct + Alpaca.
It should be Llama 3 Intstruct + Llama 3 Instruct.

old basin
#

Ooh, let me try it, i will get back to you if thats okay 🙂

median marsh
#

ok

old basin
#

Thanks, it works! Do you have a tip on how to find out what template is used for what model? Or should i just stick to the Instruct one, or should i choose the one that the base model is made on?

median marsh
#

really depends.
model creators may mention supported templates, for example they could write "Template style: Alpaca" or write that it's ChatML.
sometimes they may mention that it supports simple chat format, then you need to uncheck instruct mode.
some merged models can support several templates, for example both Alpaca and ChatML and work with both. Sometimes better with one than another, sometimes equally good.

but some model creators don't write anything.
sometimes WebUI may write what template is used with the model, sometimes it's only possible to check it in model files, but it can be tricky to guess

#

and some models don't have a template at all and might even not support the chat format.
for example several storywriting and RP models from KoboldAI are like this

old basin
#

Really interesting, what ive noticed is that most of the new models ive tried dont really following the right format, i. E. Dont use ** or "", basically just write white

#

But thank you so so much for the amount of time you took for me again : D

median marsh
#

you could try writing example dialogues for characters or make "*some action* greeting message" in the character's first message.
or write your messages with some action in the very beginning of the dialogue.
it usually helped with my models, though I don't check new models recently since I'm fine with Solar models which tend to be more perceptive to context.

old basin
#

Ill just going to try the solar models too, the little more advanced stuff can follow later over time : D
Thanks : )

median marsh
#

This model is pretty smart, but it's a general-purpose mode, it might be not very expressive. It still can write pretty well if style is properly defined though.
If you find out that you need a more expressive option, you could try Lotus models, SnowLotus, KuroLotus, etc. Though I didn't extensively test them.

old basin
#

Thanks, ill try them if my PC is fast enough ^^