#No responses and Cuda error with "TheBloke_CodeLlama-7B-Instruct-GPTQ"

2 messages · Page 1 of 1 (latest)

willow osprey
#

Hi, I wanted to try the model CodeLlama 7b with AutoGPTQ Loader. It load without error, but when I send a message I have this big error :
https://pastebin.com/AksQyWGx
I'm sure it's not normal, I have a NVidia 1660 with 6gb of VRAM and 32GB of DDR5 Ram. I think the problem is linked with Cuda. Also if you know a good and light model it would be nice. Thanks 😄

Pastebin
1|start_linux | 2023-10-22 10:41:45 INFO:Loading TheBloke_CodeLlam...

Pastebin.com is the number one paste tool since 2002. Pastebin is a website where you can store text online for a set period of time.

#

I'm also gonna try to load in 8bit, and use autodevices