#Fast Local Voice Assistant with AI – Voice PE + Ollama + Mac Studio

1 messages · Page 1 of 1 (latest)

gentle tusk
#

Hey everyone! I'm new to Home Assistant voice features and I'm trying to get the best setup for fast and smart responses using Nabu.

Right now I have:

The Voice PE device already set up and working

Home Assistant running on an HP Elite desktop

A Mac Studio M1 Ultra that I can use for extra processing (LLMs, TTS, etc.). Will this good enough?

I'm looking for the best combo of:

Speech-to-Text (currently using Whisper)

AI brain (like using Ollama on my Mac for smart answers)

Text-to-Speech (currently using Piper, but open to faster options like Kokoro)

Ideally, I want it all to stay local, respond quickly, and handle more complex questions with AI.

Does anyone have recommendations or a working setup similar to this? I’d love a clean, optimized flow that balances speed and intelligence. Appreciate any advice!

shell jetty
#

I run faster-whisper with the large-v3 model on a rtx 4060ti 16GB gpu and get replies in around 200ms. Piper I have running on CPU and my replies are pretty much always instant, on the order of miliseconds. For LLM you may want to look into performance numbers for the Mac Studio. In general a GPU is always gonna be faster, but it might still offer reasonable performance.

#

As far as local models go, if you want something that handle complex tasks and answer questions, it seems the bare minimum these days is Qwen2.5-14b-instruct, though it may not be the best for answering general questions due to it's data restrictions/censorship. If you want something more reliable it seems you'd want at least a 24-30b model, maybe something like Mistral Small. But every LLM is different and each has it's pros/cons, so have to experiment a bit here to find what works best for you. 🙂

gentle tusk
#

I found this

https://www.reddit.com/r/MacStudio/comments/1j33hu5/mac_studio_server_guide_run_ollama_with_optimized/

https://github.com/anurmatov/mac-studio-server

Reddit
From the MacStudio community on Reddit: Mac Studio Server Guide: Ru...

Posted by _ggsa - 8 votes and 1 comment

GitHub
GitHub - anurmatov/mac-studio-server: Optimized Ollama LLM server c...

Optimized Ollama LLM server configuration for Mac Studio and other Apple Silicon Macs. Headless setup with automatic startup, resource optimization, and remote management via SSH. - anurmatov/mac-s...

gentle tusk