#Use Google Mini to control Home Assistant with the use of LLM

1 messages · Page 1 of 1 (latest)

trail briar
#

I have Google mini's in my home. They work fine giving commands to HA. They are connected with Nabu Casa and all the entities that I like to be exposed are correctly exposed.
Now I have installed OpenAI as LLM on HA. This works. I have used it as the provider for the Assist and now I can use normal spoken sentences to control my home.
I would like to be able to use my Google mini's to give those commands. I'm at a loss on how to get this working, if at all possible.
I tried the help of chatgpt, but i have the feeling that he just makes scripts up. It made me create a script in HA. Expose it to Google. Use Google with something like 'start Scriptname'. The triggering of a script is not the issue, I guess the sript just doesn't work.

alias: Chatgesprek
sequence:
  - data:
      text: Hoe kan ik je helpen?
    action: conversation.process
  - delay: "00:00:02"
  - target:
      entity_id: script.chatgesprek
    action: script.turn_on
    data: {}
  - entity_id: media_player.google_mini
    data:
      message: Ik luister, wat wil je weten?
    action: tts.google_translate_say
mode: single

This is the script that it proposed.

tight nebula
trail briar
#

I can indeed use google home on my phone for example but also use the mini's to give commands to HA. That works fine. What I would like to add now is the option to speak/chat to the mini's and control my home, make requests about things i'd like to know etc.

#

So I'm looking for a way to start the conversation I can have with Assist, but then through the google mini's.

zealous coral
#

Understand one thing.
With current setup you're using Google assistant with devices in Google home system. You just happened to connect your devices there through Home Assistant.
Google assistant knows nothing about your LLM. HA Assist cannot be exposed to Google.

So no, you can't use your Assist pipeline from Google. It's completely different voice systems.
There's Onju project that is replacing Google Mini guts to make HA satellite from it.

#

Google doesn't share text, received to their speakers. So you can't even make that artificially.

trail briar
#

So that explains why i cannot find anything about this online. It's just not an option?

zealous coral
#

Yes, it's incompatible systems. It's like asking Google speaker work with Alexa 🙂

trail briar
#

Bummer. What solutions do you guys use in home to talk to your house? Preferably to have a conversation like you can have with your chatgpt app on the phone (or gemini or any other) and which you can use to control your home at the same time?

teal trout
#

Voice PE, Koala Satellite, future proof homes Satellite 1 🙂

zealous coral
zealous coral
trail briar
#

Cool thanks for the help sofar

teal trout
tight nebula
zealous coral
#

If it's intended to be used in quiet room, close to speaking person - that will be enough. Even Atom Echo will do. Or any ESP32-S3 with INMP144 and DAC like MAX98357 will work. But for experience close to Alexa/Google, you will need something with noise reduction and acoustic echo cancellation, like those i mentioned.

zealous coral
tight nebula
zealous coral
tight nebula
zealous coral
# tight nebula Well not really 😅 Should I use micro wake word on the device itself? I thought ...

It depends. If you have one satellite, it's fine (unless you're uncomfortable with constantly streaming your unencrypted audio through UDP in the network). But as the number of satellites will grow, it will introduce increasing load on your STT server. This solution isn't scalable, and it's recommended to offload wake word to satellite. Additionally, it's faster to pick wake word on device. 🙂 Might be negligible, but still.

#

But you won't be able to do it with one microphone.