lost hinge
I've been unhappy with the performance of Ollama when trying to use HA and OpenwebUi at the same time. The model loading and unloading takes too long when I simply need something done in my house.
What is the best way to connect HA to vLLM? Do I just use the ollama integration?
knotty oasis
All depends on your hardware your LLM is running on literally that's everything
lost hinge
While I don't disagree its partly.my hardware, vLLM always for concurrent requests. Ollama doesnt and uploads the model
cedar barn
I've been unhappy with the performance of Ollama when trying to use HA and Openw...
if its the same model and you adjust your settings so that HA and OWUI are calling the same model with the same settings then ollama doesn't have to reload the model. I remember solving this a while ago to prevent the constant reloading. or are you using different models?
lost hinge
Its the same model but I have tweaked the OWUI. Home Assistant has no way to set temperature or chunk size so it uploads it
@cedar barn
cedar barn
Its the same model but I have tweaked the OWUI. Home Assistant has no way to set...
to be fair, I think I switched my OWUI settings to match ha to do this on my setup.
lost hinge
Which would make sense for most models but GPT-OSS:20b needs a but of tweaking to get it where I want it for other users
Bit
cedar barn
Which would make sense for most models but GPT-OSS:20b needs a but of tweaking t...
yeah that's fair enough, my setup was with qwen3
lost hinge
I need to look back into qwens. I just couldn't get all 280 entities to it. It would never find sunset and stuff like that. So I've even using 20b
cedar barn
I need to look back into qwens. I just couldn't get all 280 entities to it. It w...
yeah 280 is quite a lot of entities
lost hinge
I don't really know how to limit it too much more. I havent made any scripts or anything yet, but I have roughly 60 smart switches alone in my house lol
cedar barn
I don't really know how to limit it too much more. I havent made any scripts or ...
are all of them required to have independent control? could you group some?
or maybe if the switch interaction is basic on/off you could potentially make a script for "switch control" with on/off variable and a select variable. then see if the ai can call the switch control script with the variables that you specify in command.
lost hinge
That last idea seems like the best option actually. I'm sure there are some switches I could get rid of, just havent figure out which ones yet.
So the script would be the best of both worlds I think
cedar barn
So the script would be the best of both worlds I think
yeah, it might require some tinkering to get it to work but in concept I think it should be fine.
lost hinge
Awesome thank you so much for your help
cedar barn
Awesome thank you so much for your help
no worries, good luck with it 🙂
knotty oasis
Theres a Tinyllama refined for HA only 1.0b
cloud nova
Nice