ollama (llama3.2:1b) is really slow in HA?

[orchid wind]

so when i use HA to call ollama, it says the /api/chat calls take 3-5 seconds, but when i call it directly via docker exec, its super fast?

its a GTX 1650, i am confused

[orchid wind]

it always hangs for like 2 seconds on something like

time=2026-02-19T00:34:15.938Z level=DEBUG source=cache.go:104 msg="loading cache slot" id=0 cache=2589 prompt=2344 used=1559 remaining=785```

can i change the massive block of text its sending to ollama i think its causing the issue

it sends this "static context" that has so much info and i want to Change it

WHY is it hardcoded in the fucking python
https://github.com/home-assistant/core/blob/37f0f1869f5a564813da98150f7e9d71ac1c87d2/homeassistant/helpers/llm.py#L537-L550

[orchid wind]

seems the llama models just sug and qwen is a bit better

intriguing

[orchid wind]

i still would really like to be able to edit this but it seems tobe a yaml dump encoded in python

[tawny inlet]

The prompt is there to tell the LLM how to interact with HA in order to be able to call tools etc...
It is needed for things to function.

You may be able to improve speed by reducing the number of entities exposed to the LLM in the voice assistant settings but the instructions prompt should not be changed.

Qwen3 is generally the recommended option and is now the default with the Ollama integration.

[orchid wind]

right i get that the prompt needs to be there, but i swear just looking at it, this is not an efficient way to send things to an llm

this stupid thing is what i want to edit

so at the very least its encoded a bit more efficiently

but if i unexpose the entities then i dont think it can even modify them

like why isnt it a template

[tawny inlet]

There may be better solutions for some LLM's but I suspect this is designed to be pretty generic and provide a working setup for most setups.

[tawny inlet]

probably multiple reasons but one is probably because people will mess with it and then cry that it doesn't work. or cry that something else doesn't work but neglect to say that they messed with it and waste everyones time 😛

[orchid wind]

-# i mean isnt the entire philosiphy of HA that you can do that if you really want

[tawny inlet]

You can always fork and if you can find a better optimised way of doing it then submit a PR 🙂

[orchid wind]

that is what i am going to look into

just trying to figure out how this code is even structured to begin with

[tawny inlet]

Cool, Good luck with it. 🙂