Ollama is dumb with HA

[sonic rune]

Need advise here

[weak eagle]

seeing the same thing. I have ollama running on my macbook and it pulled down the most recent version too.

[royal seal]

Same here

[sonic rune]

@prime geyser should we open issue for integration, or it is known problem?
I saw Madalena's demo, and it went well - but everyone was surprised - so IDK if it's known behaviour, or just a fluke.

[prime geyser]

The main contributors are working on extending the context window. I've been busy working on our voice hardware, so I haven't had time to try things out with Ollama lately 🙂

[sonic rune]

Don't think it's the matter of context length. It just lies to me in both read-only and Assist modes... Doesn't look like lack of context. Especially when it is able to continue conversation - so it holds previous context flawlessly.

[upper flicker]

same here, same model used. To be honest, based on the speed, thez quality of their TTS, I switched back to chatGPT (gpt4o-mini and GPT TTS)

it's instant, works quite well...and cost pennies

[fading pagoda]

Getting comparable results with models that are as small as the ones available in ollama is a challenge. The context window size is not the only variable here. Language and model size are also important. Especially if you compare it with the cloud-hosted models which potentially are vastly larger in size than lamma3.1:8b

I can confirm your experience with llama3.1:8b though, it did the same for me 😄

[sonic rune]

Yeah for me privacy is the key. I use GPT now, but only manually, though script actions for refining already parsed info into more human-like form.

[upper flicker]

yeah, well, I turn on my lights, I start playing music, ... this is not really what I'm calling privacy issues tbh

but I obviously do have LLMS/Ollama running since a long time

too bad I don't have a powerhorse to run these LLMS

[sonic rune]

I used Llama Conversation integration with weaker models than llama3.1, and it was working better.
Llama3.1 is really good, at least chatting with it is comparable to gpt3 turbo. So it should be able to do what it's intended to do here. I think the issue is somewhere in system message building. Would be cool to debug it and see raw API data.

[upper flicker]

you could/dhould join us on the ollama discord server

another place where you'll meet knowledgeable people about that is the openwebui server

I only can confirm the results are not as good (for now) using different LLMs that gpt4o-mini

[sonic rune]

All entities you have exposed are sent to it, no matter which request you have.
And if you restrict Assist to music and lights exclusively, then you cut the functionality. I want it to be functional, or not to be at all. Anyways, for now I use pure Assist, as I have other concerns about LLM. We'll see how it goes.

[upper flicker]

as a matter of fact, I exposed approx 80 entities to it for now

but I guess a lot of people simply check them all, and voilà

[sonic rune]

Whhooosh of course, comparing 7B model to GPT-4o is meaningless :))

[upper flicker]

well, sure you could run crazy models, but you also know you'll have to dedicate a mac studio with 64G of unified ram to get this working 😛

[sonic rune]

Exactly. And then it starts hallucinating and opens your door for stranger.

[sonic rune]

I don't have good hardware to run it, to be honest. Well, I do actually, but that PC is not intended to be turned on constantly. So I'm just playing now with it.
Curious if we'll get some cheap dedicated device like Tesla card, but cheaper, to be able running LLM on some ThinkCentre.

[sonic rune]

Uhh, I'm holding myself from diving into another rabbit hole so far. It's really tempting, but time is crucial... 🙂

[tender drum]

maybe try airllm i have not tried yet