#LLM is speaking for me under a different name.

1 messages · Page 1 of 1 (latest)

lyric fable
#

I just switched from kobold to exllamav2 via tabbyapi. I loaded a model with tabbyAPI-gradio-loader with 32768 context. It works, and fast now, but it's speaking for me. How can I fix this? It's referring to me as "Graw" and other strange token combinations. Example response:

Normal response with description

graw yes i see graw

normal response to yes i see

graw and now what graw

etc. It'll also sometimes not include the second "graw", or start hallucinating details on {{user}} that aren't in the user description.

#

Alright, now it's doing this crap

ated, but not self-centered. She is self-sufficient, but not self-serving. She is self-actualized, but not self-absorbed. She is self-fulfilled, but not self-indulgent. She is self-aware, but not self-absorbed. She is self-possessed, but not self-obsessed. She is self-motivated, but not self-centered. She is self-sufficient, but not self-serving. She is self-actualized, but not self-absorbed. She is self-fulfilled, but not self-indulgent. She is self-aware, but not self-

lyric fable
#

looks like it was a problem with mistral nemo base, Using bullerwins/Big-Tiger-Gemma-27B-v1-exl2_4.0bpw and it's perfect

inland quiver
#

I haven't seen hallucinations with Mistral-Nemo-Base but I'd say it's a quite meh model.
Instruct version is simply bad with poor writing style, repetitions and poor instruction following.
Base model did work better but it's a hassle to setup writing style for it and it's still fairly repetitive.

I used Mistral-Nemo-Base in ST with Gemma2 template enabled and it surprisingly didn't care at all that there's some instructions template is used, probably properly figuring out it was a turn-based chat.
Never had this "graw" thing tho.

And yeah, imho Gemma is way better than either Mistral-Nemo versions.

lyric fable
vocal shore
#

I also observed degraded Mistral Nemo inference with exl2, no idea what causes this, though this can be worked around using other instruct formats or inserting an endline at the end of the prompt. Speaking of Mistral Nemo, I've read conflicting reports about the model's writing quality

lyric fable
#

I'm not using celeste 1.9 finetune of nemo, which I find perfectly fixes the slop issue. Great responces, actually surprising.

#

Can't wait until that new sampler from the guy who made DRY