Local AI

[dawn dirge]

Llama 3.2

in Ollama keeps turning off wrong lights or thinks it did when it didn't

Hi everyone,

I've built several ESP32-based voice assistants for my Home Assistant setup, currently running with Ollama and Llama 3.2 as the LLM backend. While it generally works well, I'm facing an annoying issue that's driving me crazy.

When I say something like "it's quite bright in here," the assistant responds with "I'm turning off the living room lights for you" - which would be great, except:

1. Sometimes it selects the wrong room (turning off living room lights when I'm in the bedroom)
2. Even worse, sometimes it doesn't actually turn anything off but insists it did, leading to an AI argument where it swears it completed the action!

All my devices are correctly assigned to their respective rooms, and the voice assistants themselves are properly placed in Home Assistant. The system usually recognizes which room I'm in, but these glitches happen often enough to be frustrating.

Is this just the "Llama-being-Llama" problem we all know and love? 😂 Or are there specific tweaks I could make to improve reliability? Would another local LLM perform better for this specific use case?

Thanks for any suggestions!

[crisp ore]

Not the exact same thing, but this is what I did for mine: https://github.com/ollama/ollama/issues/6127#issuecomment-2264291170 - it basically tells the tool part of llama to verify if the tool is needed before running it. You aren't going to beat hallucinations sadly, as that's AI for you, but you might be to help. Also, ensure your context includes the tool call response, or you'll end up in situations where llama chat thinks it did it based on context and then lie to you.

[lost summit]

Honestly I'd recommend experimenting with other models.

The only thing I really liked about Llama was that it's responses were very natural.

But it seemed to not be able to understand the data from HA and always use a tool. I'd ask it "What's the battery level of the Vacuum?" And it'd tell me there was no vacuum in the home, but then I'd hear the vacuum starting to clean lol

Gemma27b seems really great. It doesn't natively support tool calling but it follows instructions so well you can just put instructions for tool calling in the model template and it just does it. Only thing is Ollama has had some issues with memory allocation for Gemma3

[crisp ore]

I've seen that as well. I've had issues with llama3.1 where it feels like there is a disconnect between the tool model and the answering model. Like, the command handler handles the command but then llama3.1 reads the history and goes 'Guess I did something' or simply goes 'I don't know how to deal with that', but the task might have been done.