#Random (?) hangs with Wyoming stack

1 messages · Page 1 of 1 (latest)

south elk
#

When using the wyoming-openwakeword/whisper/piper/satellite combo, I'm experiencing seemingly random hangs that require a full restart of the whisper container each time.

It seems any response that works (i.e., "Marvin, please turn on the bedroom ceiling light") will not trigger the hang.

But when it misunderstands something ("Marvin, please fumble with the didgereedoo bronze wight"), then whisper will hang and need to be restarted.

Is anyone else experiencing this? I can't remember having had that before dockerizing everything. The containers I'm using are all the official rhasspy ones, though, except for the satellite, where I'm using the one by sker65.

#

I should add that I don't mean the wyoming-satellite hangs that occur while also using the 2mic-hat's LED service. That is a known problem with an Issue for it on GitHub, and I'm not using the LEDs for that reason.

cedar idol
#

would want to check logs and see where it might be hanging (check whisper logs to see if it even received the request, check voice assist pipeline debug to see if it is there or not and where it may have stopped, check Piper TTS logs to see if something broke there)

south elk
# cedar idol would want to check logs and see where it might be hanging (check whisper logs t...

Hi Nick! I always have logs open on the side when debugging such things. This is a bit of a difficult one, though, as neither the satellite log nor the whisper log show any activity. But restarting whisper is sufficient, satellite doesn't need to be restarted. Hence why my suspicions weigh a little heavier on whisper. Piper has no log output at all (even when things are working fine - it's not in verbose mode), but also keeps working fine once whisper has been restarted.

#

This is to say, I do think whisper is where it hangs. Unfortunately, no clue how to proceed from there.

cedar idol
#

might need to turn on debug logging

#

and you are checking the addon log right?

#

and I'd take a look at the voice pipeline trace, that gives a high level overview of what it tried to do

south elk
#

(All based around Docker ... in this screenshot, there's 3 different machines involved that I'm ssh'd into)

#

Aaaand I've found a solution, for the time being: instead of rhasspy's wyoming-whisper docker container, I'm using slackr31337's wyoming-whisper-gpu container now, which I've modified to take a DEVICE=cpu argument. There's a few changes, and possibly fixes, that slackr31337 made to his version of the code. No more hang times now! Thank you, Nick, for getting me started and widening my horizon the for the problem a little.

#

Interestingly, even with the 2mic LED service running and hanging, wyoming-satellite is now unaffected by that.

#

There's a lot of unhandled exceptions in a lot of the Python code in the HASS universe. Perhaps at some point I'll get annoyed enough by that to start adding some exception handlers with proper error messages.

cedar idol
#

huh, any reason you wouldn't use GPU to process things?

#

I run Piper and Whisper in k8s containers and haven't seen these issues you have 😮

south elk
#

Yes, my GPU RAM is not enough to hold both the ollama model and the whisper model 😦

cedar idol
#

with gpu acceleration

#

ah.

south elk
#

It's a GeForce GTX960. But at the moment I just don't have the money to buy something better.

#

Which issues do you mean? The hanging? Or something else?

cedar idol
#

Hanging, I've never experienced that

#

usually my tts/stt come back in aroound 200-300ms, but I am using GPU 😅

south elk
#

Ah, okay. But so if you do whisper via GPU, I'm guessing you're also using slackr31337's container?

cedar idol
#

that and the piper container from them as well

#

these are kept pretty up to date

south elk
#

I'll definitely try these out, thank you for the link!

cedar idol
#

Also if you want an easier time investigating your logs and GPU usage, you may want to take a look at Loki/Prometheus/Grafana 🙂

south elk
south elk
#

Do you have something that causes any music that's playing out of some speaker to be muted once the wakeword is spoken, Nick?

cedar idol
#

so they have that built into the Voice PE, it ducks audio that is playing on it

#

otherwise there's been a few people throughout the discord who have made scripts to achieve that on an area level and such

south elk
#

Ah, alright then. I need to put together another support question. My multiroom audio solution is based on Snapcast, and I'm not getting that integrated into HomeAssistant yet. But I don't think I understand HomeAssistant well enough yet to even phrase the question properly.

cedar idol
#

Probably want to look into Music Assistant 🙂

south elk
#

Did. Would prefer to be able to make do without that. Interferes a little with the existing Snapcast setup unfortunately.

#

Wait, looking at your photo now, aren't you the guy who builds the ceiling assistant devices based on ESP32? I absolutely loved your YouTube video on that. Awesome setup, and the device looks awesome as well!

cedar idol
#

Ah nope, that's the FutureProofHomes guy 😄

south elk
#

Oic ... brother from another mother then 😄

cedar idol
#

I did some work with the Willow team a while back, but unfortunately the lead dev for that project died so there hasn't been any more work there, so I bounce around the HA and MA discords helping people out 😄

south elk
#

😢

#

It's real cool you do that, though!

#

I have to head off to prep dinner. Talk to you on another thread at another time! Take care and thanks again, much appreciated! 💚

cedar idol
#

No problem, take care!

raven bay
#

BTW i'm --that guy-- the one of that guys who is ducking media players in area, when satellite is working 🙂

#

I have the code somewhere here on Discord.

south elk
#

"Satellite1", YES! That's the one I meant, just didn't remember the name.

south elk
raven bay
# south elk If you ever find it again, do let me know 🙂

Oh it's easier to copy the automation here. I'm sure we need some repo for snippets like this. And for solutions in general - like, everyone right now tries to use Wyoming Satellite, but don't know about XMOS-based alternatives to PE like Satellite1 or Koala.

south elk
raven bay
# 
alias: Adjust area players volume while assist in progress
triggers:
  - trigger: state
    entity_id:
      - assist_satellite.koala_satellite_c85894_assist_satellite
      - assist_satellite.respeaker_satellite_4_assist_satellite
      - assist_satellite.koala_satellite_c121c8_assist_satellite
      - assist_satellite.koala_satellite_c857a0_assist_satellite
      - assist_satellite.koala_satellite_c8bcc0_assist_satellite
    to: listening
actions:
  - variables:
      v_players: >-
        {{ states.media_player|selectattr('state', '==', 'playing')
        |selectattr('attributes.volume_level', 'defined')
        |selectattr('entity_id', 'in',
        area_entities(area_id(trigger.entity_id)))
        |rejectattr('entity_id', 'in',
        device_entities(device_id(trigger.entity_id)))
        |rejectattr('attributes.active_queue', 'in',
        device_entities(device_id(trigger.entity_id)))
        |map(attribute='entity_id')
        |list }}
      v_volumes: |-
        {% set vol = namespace(umes = []) %}
        {% for i in v_players %}
          {% set vol.umes = vol.umes + [{'id':i, 'volume_diff':state_attr(i, 'volume_level') - 0.2 }] %}
        {% endfor %}
        {{ vol.umes }}
  - action: media_player.volume_set
    data:
      volume_level: 0.2
    target:
      entity_id: "{{ v_players }}"
  - wait_for_trigger:
      - trigger: template
        value_template: "{{ is_state(trigger.entity_id, 'idle') }}"
    timeout:
      minutes: 5
  - repeat:
      sequence:
        - action: media_player.volume_set
          data:
            volume_level: >-
              {{ state_attr(repeat.item.id, 'volume_level') + repeat.item.volume_diff }}
          target:
            entity_id: "{{ repeat.item.id }}"
      for_each: "{{ v_volumes }}"
mode: parallel
max: 10
south elk
#

Thank you kindly sir! Looks like there's a lot to learn from that!

raven bay
#

Whoosh edited to look better

#

Ask if something is unclear

south elk
#

Will do ... won't be able to make use of it until I haven't figured out how to properly marry my Snapcast setup with Homeassistant. Still fiddling with that.

raven bay
raven bay
south elk
#

Yes, there's an integration, and it finds my snapserver and all my snapclients. But it doesn't properly add them as media_player entities, unfortunately.

#

Alas ... nothing came out of the kitchen speakers.

image.png
raven bay
#

Well, i'd still use Music Assistant for those (it works great with HA, so even if you're playing music from different source, it's still beneficial)...

south elk
#

Hmm, Nick mentioned MA for this, too. I shall give it a try now.

raven bay
south elk
# 
alias: Output a static TTS sentence on the bedroom speakers
sequence:
  - action: tts.speak
    target:
      entity_id: tts.piper
    data:
      message: May the force be with you.
      media_player_entity_id: media_player.kitchen_speakers_snapcast_client_2
description: (Test script)

YAML of the above screenshot. Does that look correct? Should it work?

raven bay
south elk
#

Interesting.

#

Just asked Marvin what the temperature in the kitchen is, and he answered faithfully. That, too, goes through Piper.

raven bay
#

Yeah it works for me