I have a nice dedicated system for this 20 cores, 32G ram, 2TB storage, GeForce RTX 5060 GPU. My understanding is that I can install ollama and whisper (faster_whisper) on that machine and have it do the stt and llm work. Is that correct?
simple nebula
I have a nice dedicated system for this 20 cores, 32G ram, 2TB storage, GeForce RTX 5060 GPU. My understanding is that I can install ollama and whisper (faster_whisper) on that machine and have it do the stt and llm work. Is that correct?
rose fulcrum
Yes
leaden oasis
And Piper for TTS too if you want
calm flower
What’s the latency going to be like?
timid sonnet
I'm not sure the addons support using the GPU though.