#"Best" recent models

18 messages · Page 1 of 1 (latest)

rocky salmon
#

Is there a good list of the best models for the latest "Oobabooga" Text Generation Web UI.

I recently upgraded from an older version and found models that used to work no longer load and give other errors.
Basically the same issues as this reddit thread.
https://www.reddit.com/r/Oobabooga/comments/1756vgt/didnt_update_for_a_bit_and_suddenly_all_my_old/
For example this model
https://huggingface.co/4bit/WizardLM-13B-Uncensored-4bit-128g
used to load and run fine, but with the latest Text Generation Web UI it gives
OSError: Error no file named pytorch_model.bin, tf_model.h5, model.ckpt.index or flax_model.msgpack found in directory models\WizardLM-13B-Uncensored-4bit-128g.

Assuming a 24 GB GPU, is there a list of the latest and best models compatible with the latest version or Text Generation Web UI? Best general chat model? Best uncensored model?

This Llama-2 model seems to still work.
https://huggingface.co/TheBloke/Llama-2-13B-chat-GPTQ
Is that considered "good" still, or is there a better preferred Llama-2 model out there now?
For uncensored, this one still seems compatible with Text Generation Web UI
https://huggingface.co/TheBloke/Luna-AI-Llama2-Uncensored-GPTQ
Is there a "best" uncensored these days?
Is there a good general programming/coding model?

If I was to have a list of say the best 5 models compatible with a 24 GB GPU with Text Generation Web UI, what would you suggest?

main rock
#

Pretty much all "best" models right now are Llama2 based, except "Mistral" I guess but that's only 7b. As for "uncensored", I wouldn't put too much effort into looking for that, because every model I've tried lately all gladly spew out eyebrow-raising content. You just need to nudge it into the right direction with your context and/or first message.

#

Model notable models have been quantized by TheBloke in several formats (GPTQ, GGUF, AWQ)

#

This is a list of "recent and notable"

#

The first one is very good actually (U-Amethyst 20B). But it behaves quite differently from most finetunes, at least in chat.

rocky salmon
#

Thanks. To be more clear, this is for inclusion with Visions of Chaos. I include Text Generation Web UI and would like a list of good models for first time users to try. So no strange models, just a few good general chat models, Uncensored too so users do not have to look into how to get censored models to stop censoring. So a user can see a short list of great models, pick any of them and get a decent chat result.
The new Text Generation Web UI seems to have killed support for 90% of models that used to work, so even a list of 3 good chat and 3 good uncensored models would do. I missed any official response on this. Is this a known issue being fixed? Should I roll back to the previous version that supported so many more models and leave it there?

main rock
#

It probably supports 3x more models nowadays. The ones you tried are probably old, and while they could be re-quantized, it's not even worth it because again, they're old, slow and not as "smart".

#

You could try WizardLM uncensored should work however, and it's one of the last uncensored models afaik, but came out before Llama2.
Luna AI is good, but not really uncensored.

#

I believe base Llama2 is also preferred over Llama2-chat

#

(even for chat)

#

Oh and there's also a leaderboard on HF, but I don't really use that because just chatting with a character gives me a much better idea of how well it performs (and conforms to what I want)

#

My personal favorite is still StableBeluga2, though 70B. 13B isn't that great iirc, but it's not a bad pick to include it due to it's versatility.

rocky salmon
#

Llama-2-13B-chat-GPTQ works OK here still. What is the non-chat Llama2 model?
Luna-AI-Llama2-Uncensored-GPTQ still works too.
I will try Amethyst and StableBeluga.

main rock
#

Oh and other praised ones are Athena, Mythomax and any variation of *Boros or *Chronos. But I personally don't find them very noteworthy.

#

Maybe they're just no fun for chat but really good at instruct catshrug

rocky salmon
#

OK, thanks.