I can't use my GPU | Nomic AI | Page 1

Hi @gilded briar. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:

I'm sorry to hear that you're having trouble using your RTX 3060 Ti GPU with GPT4All. There could be several reasons for this issue.

One possibility is that the models you're using are not unloading from VRAM when switching, which can cause the GPU to fill up and the system to default to using the CPU. This issue was discussed in a GitHub issue (https://github.com/nomic-ai/gpt4all/issues/1688) where a user reported a similar problem with an RTX 3060 12GB GPU.

Another user in the same thread mentioned that custom models seem to not use the GPU, and this was confirmed by a member who stated that the Vulkan backend only supports Q4_0 and Q4_1 quantizations currently, and Q4_1 is not recommended for LLaMA-2 based models.

If you're using a virtual machine, it's also worth noting that GPT4All might not be able to see your GPU unless you're doing PCIe passthrough, as mentioned in another GitHub issue (https://github.com/nomic-ai/gpt4all/issues/2204).

If none of these scenarios apply to you, I would recommend searching the Github issues (https://github.com/nomic-ai/gpt4all/issues) or the documentation FAQ (https://docs.gpt4all.io/gpt4all_faq.html) for more information. If you can't find a solution, consider opening a new issue on the GitHub page detailing your problem.

#I can't use my GPU

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon: