AI voiceassistant homeassistant

[next dagger]

Hi, I have a partially prototype voice assistant. What's the best language model to use so it can respond to more complex questions and commands, and perform more complex actions? For example, I could tell it to close the gate when no one's home.

[viral condor]

it depends on your available hardware but qwen3 (of various sizes) works pretty nicely

[next dagger]

Local? Maybe cloud?

[viral condor]

i run ollama locally with a gpu on one of my servers. I dont know much about online models. maybe someone else with more knowledge of that side can jump in
there is some benchmarks HERE but it may not have the latest models yet.

[long bloom]

I've built an AI powered voice assistant with wake, VTT, TTV, image rec'n, all on an RPi5 8GB with RPi AI HAT and RPi Camera Module 2. I've used Vosk and Piper to give it a voice.

I used Gemma 3n for local LLM, and Gemini APIs for online LLM

It's far From complete but works well.

[next dagger]

What about something not local?

[long bloom]

Not sure the OP is asking for local, but you can have both. The truth told, and I’ve done this, you can have online as default LLM, but will fall back to local if internet is down, BUT I have mine go to canned responses before either LLM, so for the OPs stated use case (closing the gate) you don’t need an LLM at all.

[neon flint]

I have been using qwen3:4b without major issues although the response time is pretty slow on my hardware even though I have an i7 and a 3060 ti.

[odd heart]

@neon flint I have a 14gb 3060 and I get response times usually below 3s. Try qwen3-instruct-2507 in Q4_K_M.
Check if it's faster. Then make sure you enable flash attention in ollama.

[neon flint]

Thanks for the heads up I’ll give it a try.

[odd heart]

I want to make a fine-tune of qwen3 for home assistant that is very good at calling system prompts