Hello! I’m filling around with things, just trying to get a more reliable setup overall, and snappier if possible. I’ve got Ollama running on my desktop, which has a Ryzen 3900x CPU, an RTX 2070, and a GTX 1050ti. Home assistant is on a laptop with an i5 3340M. I have the fully local pipeline set up. Is there a way to have things first run through the default assistant, then try checking with the LLM before saying “I didn’t understand that”? I’m also just looking at other general setup and optimization tips for it, and model suggestions. Thanks!
#General Setup Tips & Ollama Suggestions
hollow current
cold lodge
Unfortunately, afaik the thing with several chained conversation agents isn't there (yet, I guess).
I build custom sentences so far, and use conversation.process action to refine answers via LLM.
hollow current
Unfortunately, afaik the thing with several chained conversation agents isn't th...
Using an LLM to refine answers definitely sounds good. How do you go about setting that up, and linking those things together?
cold lodge
Using an LLM to refine answers definitely sounds good. How do you go about setti...
I create a script, that gathers information for me and puts this to some short sentence.
For example, weather: I make the sentence "temperature is 25C, humidity is 50%, feels like 24C, conditions: cloudy". Then I use "conversation.process" on LLM with prompt "rephrase following sentence and add some joke about the weather condition: " plus my sentence. And use response from LLM as return in custom intent, to speak it.
hollow current
Is it possible to have multiple assist profiles set up with different wake words that are each independently triggered but can be side by side?
potent oar
Consider using https://github.com/m50/ha-fallback-conversation to route to HA and then ai.
hollow current
Is there a way to make a “processing” sound while it is thinking?
cold lodge
Is there a way to make a “processing” sound while it is thinking?
why not. Use on_stt_vad_end and play some beep there.
hollow current
why not. Use `on_stt_vad_end` and play some beep there.
Well.. I guess I didn’t know it existed. But the STT is not the part that’s slow. It’s the processing on Ollama
At least if the model is no longer in memory and needs reloaded
cold lodge
Well.. I guess I didn’t know it existed. But the STT is not the part that’s slow...
You asked about sound, not about processing time...
cold lodge
At least if the model is no longer in memory and needs reloaded
What do you mean?
hollow current
What do you mean?
With ollama, there’s Keep Alive time. If it’s been more than that, it takes a bit more time to process because my computer needs to load the model from the SSD to VRAM / RAM to process the request
cold lodge
Yeah, but that is set to -1 by default, so it doesn't unload the model after first time.
hollow current
I don’t want it to permanently be there for the sake of power savings and load on my desktop (I use it for other things), so I have it set to 300 at the moment. But if it’s a few hours later it will be freshly processing things
cold lodge
Ahh... Well, that's the inconvenience.
To be fair, it doesn't use that much of power or CPU while sitting there - it jsut takes VRAM..
hollow current
I’ve noticed sometimes it makes the sound like it detected wake word, but then never ends up doing anything with it (generally from false triggers, but sometime intentional ones too)
hollow current
To be fair, it doesn't use that much of power or CPU while sitting there - it js...
Yeah. And I’ve got plenty of CPU and GPU power. But I do lots of other things for school and such on my desktop. I often access it remotely via parsec.
hollow current
I’ve noticed sometimes it makes the sound like it detected wake word, but then n...
This is part of why I was wanting a sound so I can know it’s actually doing something and not just “gave up”
cold lodge
So you don't need it to ding when thinking starts, you need that ding for the whole time it's thinking?
hollow current
Yeah
I’m thinking similar to how Siri has a bit of a low level ding while it is thinking
tight cloak
I create a script, that gathers information for me and puts this to some short s...
Do you have a full example of this here in discord somewhere? or anywhere else?
hollow current
Consider using https://github.com/m50/ha-fallback-conversation to route to HA a...
Is it actually possible to make that an additional repo? I tried but it fails
cold lodge
Is it actually possible to make that an additional repo? I tried but it fails
Works for me.
hollow current
Works for me.
What do you put in?
cold lodge
Do you have a full example of this here in discord somewhere? or anywhere else?
Here's example for forecast that i wrote:
alias: Get weather forecast
sequence:
- data:
type: daily
target:
entity_id: weather.forecast_home
response_variable: w
action: weather.get_forecasts
- variables:
result:
conditions: >-
{% set next = w['weather.forecast_home'].forecast[f_days] %} {% if
(next.precipitation) < 2 %} {% set rain_chance = ', with no rain
expected.'%} {% elif (next.precipitation) < 10 %} {% set rain_chance =
', with tiny chance of rain.'%} {% elif (next.precipitation) < 50 %}
{% set rain_chance = ', with a good chance of rain.'%} {% else %} {%
set rain_chance = '. Most probably it will be raining. Don\'t forget
your umbrella!'%} {% endif %} {% if f_days == 0 %} {% set day_words =
'Today you can expect' %} {% elif f_days == 1 %} {%
set day_words = 'Tomorrow you can expect' %} {% else
%} {% set day_words = 'In ' + f_days|string + ' days
you can expect' %} {% endif %} {{ day_words }} {{ next.condition }}{{
iif(next.wind_speed |int > 33, ' and windy', '') }} weather with high
of {{ next.temperature | int }} degrees and a low of {{ next.templow |
int }} degrees {{ rain_chance }}
- data:
text: >-
Rephrase following text to sound like JARVIS from Iron Man movie. Don't
use word "sir". Use word "rain" instead of "precipitation". Here's the
input: {{ result.conditions }}
agent_id: conversation.google_generative_ai
response_variable: rephrased
action: conversation.process
- variables:
result:
conditions: "{{ rephrased.response.speech.plain.speech }}"
- stop: Result
response_variable: result
mode: single
icon: mdi:weather-cloudy-clock
fields:
f_days:
selector:
number:
min: 0
max: 5
step: 1
name: Days from today
default: 1
required: true
Where?
You need to add that repo to HACS, then install integration in HACS, then restart HA, then go to Devices and Services and install it there. I put Assist first, and OpenAI second.
hollow current
I went to add on store -> 3 dot -> repositories and put in the link. But it says it is not a valid repository
hollow current
Where?
You need to add that repo to HACS, then install integration in HACS, then...
Is ⬆️not how you do it?
cold lodge
Is ⬆️not how you do it?
Yup, 3 dots, put https://github.com/m50/ha-fallback-conversation, choose "Integration" in drop-down, hit Submit.
hollow current
Yup, 3 dots, put https://github.com/m50/ha-fallback-conversation, choose "Integr...
hollow current
Where else do you go to add repos?
hollow current
Is that an add on?
hollow current
I tagged with “new to HA” for a reason 😂
tight cloak
it is a custom integration which provides you a "store" in order to easily browse and download other custom integrations
like an unofficial integration store. It isn't required in order to use custom stuff, but it makes it easier
But...and this will be confusing...there is an addon called "get Hacs" which will make it easier to install.
So where you are right now in that screenshot, paste this repo instead: https://github.com/hacs/addons
Then in your add-ons you will see the get HACs addon. Install that. Start it. Then check that add-on's log page for further instructions
hollow current
Gotten here so far
hollow current
not an add-on repository
Sounds like WINE
tight cloak
before HACs existed, you would hear about something and then go find it on github, copy it and put it in the custom_components directory in your /config directory.
hollow current
Ahhh
tight cloak
And you still can, but this takes care of a lot of that now, as well as the discovery of what's out there. These are supported by the developers of each custom integration, or custom cards for the frontend, etc, and are not officially supported by Home Assistant itself. So they don't appear at https://www.home-assistant.io/integrations
hollow current
Gotcha
One thing I have struggled with quite a bit is getting assistant stuff to either trigger a script or shell command
Welp. Fallback seemed good when assist could handle it.. but now voice assistant is unresponsive after asking something the LLM needed to do
Actually now it seems fine
I guess the LLM’s got confused when I said “turn off computer lights” (a script name) but assist does fine with it
Y’all got any tips for more reliable TTS / less common false triggers? I’ve got a Blue Snowball hooked up as the mic, but it seems pretty unreliable and gets triggered from things that don’t even sound like “hey Jarvis”
Particularly from things on TV or my phone playing
And sometimes it struggles to be triggered too
bronze coral
Don't experience that with the voice satellites I have, but that may be due to the processing they are doing internally, as well as using MicroWakeWord with the latest trained model.
hollow current
Im using openwakeword and the local assist microphone
hollow current
Is it possible to disable HACS in the side panel and keep things I installed with it running?
hollow current
Also, can you make a voice assistant speak through airplay-able speakers rather than having to use the speaker connected to the computer running it?
cold lodge
Collin, with all respect: Google first, GPT second, docs third, real people after. All those questions can be addressed with first two 🙂