Strange Gibberish Output

[uncut coral]

I installed Mixtrel 8X7B on my local desktop and run it using LM Studio. I entered a few very simple prompts and am getting crazy, gibberish output. I'm wondering if this is normal? I'm attaching some screen shots as examples. Help please, thank you!

[rancid tulip]

Definitely not normal. But you are using 2-bit quantization, which will affect model performance. I recommend using the Mistral API.

[uncut coral]

Thanks Sophia. The Mistrel API costs money, correct? Is it possible to use something better than 2-bit quantization?

@rancid tulip do you know which is better, LM Studio or Oollama?

[rancid tulip]

I don't really have recommendations. Might be a good question for the community

[orchid lily]

2 bit is a bit low for local inference, but 3 to 4 bit quants are good. and also most fiction tasks are better at lower quants (4 to 5 bit range)

[uncut coral]

Thanks Sophia