where it runs depends on the model you use. ggml- models run on CPU, anything else on GPU
#Ooobabooga refusing to run on3060
balmy spoke
viscid kernel
where it runs depends on the model you use. `ggml-` models run on CPU, anything ...
Will do furtger tests