Hello, in Arch Linux if I have a Nvidia 3060 GPU and add another 4060 Ti (2 GPUs in total) when using artificial intelligence would add me the VRAM of both GPUs?
3060 = 12 GB VRAM
4060 Ti = 16 GB VRAM
3060+4060 Ti = 28GB VRAM?
#3060+4060 = add the VRAM?
hasty nest
wispy wolf
I don't think it quite works that neatly, but I could be mistaken
blissful basin
It's not perfect but it works for using in llms.
Some examples I have on my home systems.
It's almost like adding them up, but usually take off 1~1.5 GB's each for it to work without any issues. The one on the left is GGUF, and the one on the right is an example of ExLlama. Both running Dolphin 2.7 mixtral 4Bit models.