Best local Home Assistant conversation agent for Raspberry Pi 5 with 4 GB RAM

[hearty cradle]

I want create own local voice assistant on Pi 5 with 4 GB RAM, i have Vosk STT and Piper TTS but i can't find best local conversation agent. The internal HA Agent don't understanding me, ollama shows error: Unexpected error during intent recognition and llama process terminated

[tawny ravine]

Use llama server but I have no idea what kinda model you can run on that pi

I run my llm on my workstation

[hearty cradle]

I don't have a normal computer to run llama at all. All my computers are old and the Pi 5 is the only one that can still run it. (maybe)

[tawny ravine]

Try to find a 1b model that can do function calls, but honestly at that point it'd be better to just get used to home assistant and add custom aliases to certain actions

[hearty cradle]

llama3.2 normal?

[tawny ravine]

Maybe

[round cloud]

Tinyllama worked for me on a nuc but still with ~10s latency, that was the smallest model readily available I could find on llama. I ended up using openai integration in the end for better latency

[hearty cradle]

Yes, but you have to pay money for it, which I don't really want to do.