#Voice PE - send tts.speak and wait for yes/no to run an action?

1 messages · Page 1 of 1 (latest)

crisp latch
#

I have an automation that runs tts.speak on my Voice PE (using piper) that asks "Would you like to run the vacuum now?". I can then trigger the Voice PE to listen remotely (adopted device in esphome and added a custom button that the automation presses):

  - platform: template
    name: "Trigger Listening (Custom)"
    on_press:
      - script.execute:
          id: play_sound
          priority: true
          sound_file: !lambda return id(wake_word_triggered_sound);
      - voice_assistant.start:```
Now, I'm wondering if it's possible to have the automation tell whether yes or no was detected by whisper or Voice PE, and run an action accordingly (in this case, run my vacuum).
icy basalt
#

No, there's no "actionable notifications" yet on HA voice. I'm waiting for this too, it's the only thing I miss...

ruby thorn
#

Exactly this. I am also specifically browsing Here to find a solution for this 😉

thorn mantle
#

thanks to that snippet i was able to pull off this scenario 🎉

following the announce then listening you can do :

wait_for_trigger:
  - trigger: conversation
    command: "yes"
continue_on_timeout: false
timeout:
  seconds: 10

OR
you can enable another automation that triggers on a "yes" sentence, then disable itself

urban shell
#

or, of course, if you say "ok nabu... yes"

urban shell
thorn mantle
#

OP gives the snippet that allows to trigger from HA.

Yes trigger still being registered when no wait for trigger or an automation being disabled with that trigger, should probably be considered a bug?

urban shell
#

not really. the wait for sentence trigger was not meant for such a usecase, so it's more of a workaround on your part. you can't call failure to hammer some nails with your smartphone "a bug" 😅

#

however, we could create a feature request for the trigger to support this usecase

thorn mantle
#

I do not understand in which case a wait for sentence trigger is not intended to be ephemeral ?

urban shell
#

it was specifically designed to start automations. not to be used within them. i can't find the discussion where i mentioned your usecase (more than 1.5years ago) and it was dismissed as "we'll see about that" 😅

thorn mantle
#

OK understood. For sure this is while waiting for an official implementation. There is so much approaches to that issue 😁

urban shell
#

Don't get me wrong, it's a great solution to emulate a conversation. But it's subject to change as it is not designed to be used as such and it does have its shortcomings. E.g. if your sentence trigger is a wildcard, you've screwed up all your sentences.

VA: "what item would you like to add to your shopping list?"
Sentence trigger: {item}

Boom! There go all your sentences over which this takes precedence 😅

thorn mantle
#

Just did some tests to understand how bad is it : Having a sentence trigger defined makes it trigger-able. Even if the automation is disabled, even if the step is disabled, even if the automation has never been started.

thorn mantle
urban shell
icy basalt
#

@urban shell thanks for inspiration. Made it for yes/no on Respeaker satellite. Works great. Of course, would be nice to have inbuilt service like assist_satellite.listen, but even with this i was able to make generic script, that, given question like "do you want to turn PC off?" and area to ask in, returns result yes/no/unanswered. Involves intent script and two text helpers, but works good. 🙂

icy basalt
urban shell
thorn mantle
urban shell
#

i don't think the voice_assistant component exposes the used pipeline so as to use it in internal ESPHome automations 🤔