#ModuleNotFoundError: No module named 'gptq_for_llama'

5 messages · Page 1 of 1 (latest)

severe otter
#

Hello, any idea why is this program returning error:

 when I try to load a model with GPTQ-for-LLaMa?```

OS: ubunt server 22.04
GPU: tesla M40(24GB)
reef girder
#

What CPU does that server use? gptq_for_llama should have been installed with the requirements.txt, unless you are using an ARM CPU. I'm still looking into building ARM CUDA packages. It may be a while before I can feasibly do this.

severe otter
#

requirements.texxt returns:

#

Ignoring bitsandbytes: markers 'platform_system == "Windows"' don't match your environment
Ignoring auto-gptq: markers 'platform_system == "Windows"' don't match your environment
ERROR: auto_gptq-0.4.2+cu117-cp310-cp310-linux_x86_64.whl is not a supported wheel on this platform.

#

CPU: virtual part of(PROXMOX[QEMU]) Core i7-5820K