#I d like to revive a conversation about

1 messages · Page 1 of 1 (latest)

violet pawn
#

@zealous peak @fresh mirage your input is more than welcome on this

zealous peak
#

So I think that all those need to be covered by HassGetState

#

So a single intent in HA

violet pawn
#

agreed. but what i'm saying is that I have cover.bedroom_door with state open and binary_sensor.bathroom_door with state on, and then ask "how many doors are open?", I should get 2. and I'm not sure that's how it works in HA at the moment

zealous peak
#

We should be able to solve that in HA I think

#

We need to add some more logic there

#

We need to hardcode links between binary sensor device classes and domains

violet pawn
#

Wouldn't it be more future-proof to allow intent sentences to query more than one domain? there may be more links in the future, i'm guessing

wet furnace
violet pawn
zealous peak
#

If HA matches two intents , it doesn’t know how to merge that

#

So it needs to always end up in one handler

wet furnace
#

So it should be one intent which checks for multiple domains, with possibly different states on and open for example

#

But maybe I'm thinking too simple 🙂

violet pawn
#

that's the gist of it, as far as I understand as well

zealous peak
#

Yep that’s correct

fresh mirage
#

HassGetState already supports multiple domains and states 🙂

language: "en"
intents:
  HassGetState:
    data:
      - sentences:
          - count open doors
        response: how_many
        slots:
          domain:
            - cover
            - binary_sensor
          device_class: door
          state:
            - "open"
            - "on"
violet pawn
#

In the example above, the same device_class is used in both domains, door. But for locks, this doesn't work.

It's good we can do it for doors and windows, I'll give that a try.

zealous peak
#

Yeah that’s something that needs to be fixed

#

Maybe slots can be a list with matches

violet pawn
#

Like a list of what they are now, right?

fresh mirage
#

I'd imagine something like:

slots:
  - domain: cover
    device_class: door
    state: open
  - domain: binary_sensor
    device_class: door
    state: "on"
violet pawn
#

Exactly! That sounds great!

wet furnace
fresh mirage
#

It's used in the intents repo for validation, but not by HassIL itself.

zealous peak
#

We just validate the file name to enforce certain organization

violet pawn
# fresh mirage I'd imagine something like: ```yaml slots: - domain: cover device_class: ...

what if I need requires_context or excludes_context to vary from domain to domain as well? for example:

language: "en"
intents:
  HassGetState:
    data:
      - sentences:
          - "how many locks are locked [in <area>]"
        response: how_many
        match_contexts:
          - requires_context:
              domain: binary_sensor
              device_class: lock
            slots:
              state: "on"
          - requires_context:
              domain: lock
            slots:
              state: "locked"
#

also, in the example above, I couldn't write a sentence for how many locks are {lock_states:state} [in <area>], as lock_states would vary between the lock domain (locked, unlocked) and the binary_sensor domain (on, off)

#

what would be needed is domain-dependent variation of the out value of lists. something like:

language: "en"
lists:
  lock_states:
    values:
      - in: locked
        dict_out:
          lock: locked
          binary_sensor: on
#

and then

intents:
  HassGetState:
    data:
      - sentences:
          - "how many locks are {lock_states(domain):state} [in <area>]"
        response: how_many
        match_contexts:
          - requires_context:
              domain: binary_sensor
              device_class: lock
            slots:
              domain: binary_sensor
              state: "on"
          - requires_context:
              domain: lock
            slots:
              domain: lock
              state: "locked"
#

where we could use any of the slots as a "parameter" for the list name. What do you think?

fresh mirage
#

We can discuss here for now.
If you need this, it would seem like you could do it just by splitting into two sentences, no?

intents:
  HassGetState:
    data:
      - sentences:
          - "how many locks are {binary_lock_states} [in <area>]"
        response: how_many
        requires_context:
          domain: binary_sensor
          device_class: lock
      - sentences:
          - "how many locks are {lock_states} [in <area>]"
        response: how_many
        requires_context:
          domain: lock

I realize there's some duplication, but the complexity has to go somewhere and I think we should try to keep the YAML language simple when possible.

violet pawn
#

That's not the issue. The issue is that "how many windows are open in the living room?" in common speech should not differentiate between covers and binary sensors. Not to mention you can't properly match an intent if you have both in the same area.

#

Even your example fails if there are both locks and binary_sensors in the same area (albeit less common)

fresh mirage
#

Ah, good point.

fresh mirage
#

I'd suggest constraints instead of match_context; any thoughts?

violet pawn
#

Much better! I had major doubts about the naming, but i just wanted to convey the idea

zealous peak
#

Maybe filter? easier word

zealous peak
#

Or matches

violet pawn
# zealous peak Or matches

Given that excludes_context would be a possible key, i don't think matches is appropriate. I think filters is ok, though

zealous peak
#

So what would be needed right now to move forward with this

violet pawn
#

I've talked to @fresh mirage and decided to have a crack at this. I'm working on hassIL at the moment. My plan is to have the intents backwards compatible, but the HA change will have to use the new version on HassIL

#

I'm not the most proficient Python dev, though, as you can tell from this and my other PRs (e.g. from core) 😋

zealous peak
#

Cool!

wet furnace
#

Certainly!

violet pawn
fresh mirage
#

I'm out sick today, but I'll take a look when I get back 👍

violet pawn
violet pawn
#

Any news?

fresh mirage
#

Looks good, I just need to make the necessary changes to the intents repo scripts.

wet furnace
#

Are the changes to the repo scripts already made?

fresh mirage
#

Not yet, sorry :/

wet furnace
#

No worries, just checking 🙂

zealous peak
#

@fresh mirage can we review this first thing after chapter 3 presentation

fresh mirage
violet pawn
#

@fresh mirage please don't merge the PR just yet even if by some miracle everything looks OK. I'd like to add a bit of documentation to the readme in the same PR if there are no major changes required

fresh mirage
wet furnace
#

Any news on this? 😇

violet pawn
#

Nope, the wildcard overhaul intefered with all my work and i didn't have the time to pick it up again

fresh mirage
#

I'm off on wakeword stuff at the moment, so it'll be a while before I can get back to this.

wet furnace
#

I can undertand that has priority now 🙂

fresh mirage
#

I've been creating a list of accomplishments for the Year of Voice so far, and it's pretty large! But they're spread out over so many areas that anyone interested in one particular thing is probably wondering why progress has stalled 😄

violet pawn
#

We all appreciate your hard work, Mr. H. And i am really curious about that list

fresh mirage
#

Thanks! Here's what I have so far, excluding more details about the in-progress wake word stuff:

  • Assist
    • Pipeline architecture
      • Wake (in progress)
      • VAD (webrtc)
      • STT (cloud/whisper)
      • intent (HassIL/OpenAI)
      • TTS (cloud/piper)
    • HassIL intent recognizer
      • Template-based recognition
      • Lists + ranges
      • Context
      • Wildcards
    • Community sentences
      • 50 languages, 176 contributors
      • 5 intents
      • Responses
      • Tests
    • Custom sentences (YAML)
    • Sentence triggers
    • Debugging tools
      • View pipeline runs
      • Listen to saved audio
  • Hardware
    • M5 ATOM Echo
      • ESPHome extension
      • Push-to-talk and continuous mode
    • VoIP (Grandstream)
      • Custom SIP/RTP stack
    • Android App
      • Native integration as Android assistant
      • Phones and tablets
    • Android Watch
      • Native integration as Wear assistant
      • Single button launch
  • Piper
    • 100+ voices trained across 34 languages
    • C++ native client
    • PyTorch training rewrite
    • NVDA plugin for blind users
    • Arabic diacritization
    • Academic collaborations with 2 universities
    • Community contributed voices
      • 16 contributors, 11 languages
      • Website and user management
  • Rhasspy
    • Wyoming protocol
      • Glue for all voice services
    • Rhasspy 3 + services
      • 5 wake
      • 6 STT
      • 6 TTS
      • 1 intent
      • 2 audio I/O
      • 3 audio frontend
violet pawn
#

Not too shabby

#

😁

fresh mirage
#

(need to fix formatting)

violet pawn
#

Really, congrats! Help us help you, though, and review some of those PRs so we add functionality on top of the great things you've already done

#

*please 😅

fresh mirage
#

This is definitely a problem I face in my projects. I get spread so thin that context switching starts to become a barrier :/

zealous peak
#

Depending on your Python skills, we also need someone to look at passing the area as context

#

So when someone says "turn on the lights" it turns on the lights in the area the listening device is located

#

(we already have device ID passed in )

#

The wake word stuff is sadly a bit tougher than we antitipcated and Mike is fighting with ESP-ADF

violet pawn
violet pawn
#

And i am trying, but there are some PRs i made for EN where i could really use another set of eyes

zealous peak
#

ha sorry yes, this was in response to the "help us help you"

#

Maybe @wet furnace can help with that

#

we also know timers will have to come sometime, so I wonder if we can make a proposal for how the intents will look to start one, and we can start collecting sentences for that

violet pawn
violet pawn
zealous peak
violet pawn
violet pawn
zealous peak
#

you wouldn't need to add support for the intent itself, but you can come up with a proposal how the intent will look like facing the sentences

#

HassStartTimer with slots <duration> and optional slot <name> for example

violet pawn
#

Roger that

zealous peak
#

if we can agree on that, we could start gathering sentences

zealous peak
#

@fresh mirage what do you think, duration required and name optional for a timer ? or should we just start with a single timer and make duration the only requirement, no other slots ? Just to get people to start making sentences

#

I guess the challenge is that duration needs to be translated into a machine readable format

#

we can't just have "5 minutes" be passed on

violet pawn
#

I can already think of a remind me to {timer_name} in {duration} sentence, but maybe simpler is better at first

fresh mirage
wet furnace
wet furnace
zealous peak
#

We’re reverting that