#general performance-review

9 messages · Page 1 of 1 (latest)

plain steppe
#

Hi guys,

just a quick question:

Here is my hardware:

  • 3070, 8Gb, latest driver
  • AMD Ryzen 5 3600
  • 32Gb DDR4
  • OS is Win11, x64

I'm now using ExLlama_HF and the mode 'TheBloke_Nous-Hermes-13B-GPTQ' and get the results as shown on the screenshot.

Does this make sense to you? The token/s seem a little low to me.

Greetings, have a good Sunday.

Screenshot_2023-07-23_044215.png
tardy wagon
#

if you're doing some CPU offloading, looks about as expected

#

7B on pure GPU will be much faster

plain steppe
#

The GPU goes up to 99% in TaskManager. Can I set it up anywhere to use the GPU only?

#

Thanks btw 🙂

tardy wagon
#

it's up to your settings in the GUI

#

try 7B 4bit

#

with exllama