Hi everyone, I've been trying different models for days and coming up with error after error, or at best, getting zero-token responses with RuntimeErrors. I'm running Windows 11 on a Intel(R) Core(TM) i7-10875H CPU @ 2.30GHz, 2304 Mhz, 8 Core(s), 16 Logical Processor(s), 64GB RAM, and an NVIDIA GeForce GTX 1650 Ti (which I think is a 4GB GPU). I'd really like to be able to run the smallest LLaMa 2 but I'm just running out of patience and really need someone to point me in the right direction (or walk me through exactly how) to get Text Generation WebUI working with any model at all. Thank you so much in advance.
#Can't get any models to work in Text Generation WebUI on Windows 11 laptop
fossil ice
somber surge
Your best bet is using a GGML model. Download this one and put it in your models folder:
https://huggingface.co/TheBloke/Llama-2-13B-GGML/resolve/main/llama-2-13b.ggmlv3.q4_K_S.bin
Other versions requiring more or less memory can be found here:
https://huggingface.co/TheBloke/Llama-2-13B-GGML/tree/main
All GGML versions of LLaMa 2 listed here:
https://huggingface.co/models?sort=trending&search=thebloke+llama-2+GGML