#Can't get cuBLAS working

4 messages · Page 1 of 1 (latest)

vivid plover
#

hey, I tried installing cublas like this:

pip uninstall -y llama-cpp-python
set CMAKE_ARGS="-DLLAMA_CUBLAS=on"
set FORCE_CMAKE=1
pip install git+https://github.com/abetlen/llama-cpp-python.git --no-cache-dir

But everytime I load a gguf model it says BLAS=0

llama_new_context_with_model: n_ctx      = 32768
llama_new_context_with_model: freq_base  = 10000.0
llama_new_context_with_model: freq_scale = 1
llama_new_context_with_model: kv self size  = 4096.00 MB
llama_new_context_with_model: compute buffer total size = 2141.88 MB
AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | SSSE3 = 0 | VSX = 0 |
2023-09-29 23:42:37 INFO:Loaded the model in 1.16 seconds.
#

This is my output when installing:

Found existing installation: llama_cpp_python 0.2.7
Uninstalling llama_cpp_python-0.2.7:
  Successfully uninstalled llama_cpp_python-0.2.7
Collecting git+https://github.com/abetlen/llama-cpp-python.git
  Cloning https://github.com/abetlen/llama-cpp-python.git to c:\users\nekora\desktop\llamacpp\installer_files\pip-req-build-n_4obs72
  Running command git clone --filter=blob:none --quiet https://github.com/abetlen/llama-cpp-python.git 'C:\Users\nekora\Desktop\llamacpp\installer_files\pip-req-build-n_4obs72'
  Resolved https://github.com/abetlen/llama-cpp-python.git to commit a945404b4a40637a50edcada3c37e77dce248ff1
  Running command git submodule update --init --recursive -q
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: typing-extensions>=4.5.0 in c:\users\nekora\desktop\llamacpp\installer_files\env\lib\site-packages (from llama_cpp_python==0.2.7) (4.8.0)
Requirement already satisfied: numpy>=1.20.0 in c:\users\nekora\desktop\llamacpp\installer_files\env\lib\site-packages (from llama_cpp_python==0.2.7) (1.24.0)
Requirement already satisfied: diskcache>=5.6.1 in c:\users\nekora\desktop\llamacpp\installer_files\env\lib\site-packages (from llama_cpp_python==0.2.7) (5.6.3)
Building wheels for collected packages: llama_cpp_python
  Building wheel for llama_cpp_python (pyproject.toml) ... done
  Created wheel for llama_cpp_python: filename=llama_cpp_python-0.2.7-cp310-cp310-win_amd64.whl size=1082619 sha256=4d55b6d31645a8d8f69a10dc41331f2a23a73802b1d6dac71341c7db89c668ab
  Stored in directory: C:\Users\nekora\Desktop\llamacpp\installer_files\pip-ephem-wheel-cache-uaw0lblt\wheels\f3\25\31\cbc4d3d5c467efdfb1081961fd5f4caa37df78de3a73a8a8cc
Successfully built llama_cpp_python
Installing collected packages: llama_cpp_python
Successfully installed llama_cpp_python-0.2.7
#

Can anyone help me?

#

I also tried:

set LLAMA_CUBLAST=1

and

pip install llama-cpp-python --no-cache-dir