Home Assistant Voice Preview Edition - No Audible Responses | Home Assistant | Page 1

daring onyx Apr 8, 2025, 9:48 PM

#

I am trying to get the Home Assistant Voice hardware setup. I have it integrated and I am able to wake the device vis Hey Nabu and give it a command. The lights on the front flash like its thinking but it never responds. I can ask it to start a 1 minute timer, the LED ring starts to count down and after 1 minute, the device does sound an audible alarm. But no voice ever says anything.

I then ran a test action, the device actually did audible speak those words.

action: tts.speak
metadata: {}
data:
  cache: true
  language: en_US
  media_player_entity_id: media_player.home_assistant_voice_09e6a4_media_player
  message: Test Announcement
target:
  entity_id: tts.piper

I am struggling to find the issue, any help would be greatly appreciated.

vernal bronze Apr 8, 2025, 9:51 PM

#

daring onyx I am trying to get the Home Assistant Voice hardware setup. I have it integrated...

what hardware are you running on?
what does your voice pipeline look like?

daring onyx Apr 8, 2025, 9:51 PM

#

Oh, also, when I set it up, when I tried to hit this button, I got the error below

#

My setup is in a container, I have whisper and piper running on other containers with their proper ports exposed

#

Whisper logs show it understands my question

INFO:faster_whisper:Processing audio with duration 00:04.560                                                           INFO:wyoming_faster_whisper.handler: What's the current time?                                                       INFO:faster_whisper:Processing audio with duration 00:05.820                                                     INFO:wyoming_faster_whisper.handler: What time is it?                                                         INFO:faster_whisper:Processing audio with duration 00:02.670                                               INFO:wyoming_faster_whisper.handler: What time is it?

#

Piper shows it downloading models

INFO:wyoming_piper.download:Downloaded /data/en_US-kathleen-low.onnx.json (https://huggingface.co/rhasspy/piper-voices/resolve/v1.0.0/en/en_US/kathleen/low/en_US-kathleen-low.onnx.json)                                                INFO:wyoming_piper.download:Downloaded /data/en_US-kathleen-low.onnx (https://huggingface.co/rhasspy/piper-voices/resolve/v1.0.0/en/en_US/kathleen/low/en_US-kathleen-low.onnx)                                                     INFO:wyoming_piper.download:Downloaded /data/en_US-kusal-medium.onnx.json (https://huggingface.co/rhasspy/piper-voices/resolve/v1.0.0/en/en_US/kusal/medium/en_US-kusal-medium.onnx.json)                                                INFO:wyoming_piper.download:Downloaded /data/en_US-kusal-medium.onnx (https://huggingface.co/rhasspy/piper-voices/resolve/v1.0.0/en/en_US/kusal/medium/en_US-kusal-medium.onnx)

#

The voice assistant settings look like the following

vernal bronze Apr 8, 2025, 9:55 PM

#

on the assist page hit the 3 dots on your pipeline and press debug. this will open the trace for the voice call

#

daring onyx Apr 8, 2025, 9:56 PM

#

vernal bronze Apr 8, 2025, 9:56 PM

#

okay so the pipeline is working

daring onyx Apr 8, 2025, 9:57 PM

#

I also found some github discussions about this setting

vernal bronze Apr 8, 2025, 9:58 PM

#

yeah that can be a problem sometimes

daring onyx Apr 8, 2025, 9:58 PM

#

Ive tried https://home.elcarpenter.com as well

#

Neither made any difference

vernal bronze Apr 8, 2025, 9:58 PM

#

you mentioned your test using tts.speak worked

#

could you try a test in dev tools using assist_satellite.announce

daring onyx Apr 8, 2025, 9:58 PM

#

Sure!

#

action: assist_satellite.announce
target:
  device_id: 319ea7cc3414f2bb54d0aaf468ec1fed
data:
  message: Test Announcement
  preannounce: true

Played the pre-announcement jingle, lights flashed, no voice

vernal bronze Apr 8, 2025, 10:00 PM

#

can you show screenshot of the voicepe device page?

daring onyx Apr 8, 2025, 10:00 PM

#

The hardware's own web ui?

vernal bronze Apr 8, 2025, 10:01 PM

#

no
devices - esphome - voice pe

#

this one

daring onyx Apr 8, 2025, 10:02 PM

#

vernal bronze Apr 8, 2025, 10:04 PM

#

ok this is odd

#

we could look at the device log

#

can you plug the vpe into your pc?

daring onyx Apr 8, 2025, 10:06 PM

#

Already is!

vernal bronze Apr 8, 2025, 10:06 PM

#

go here - https://web.esphome.io/

#

connect to the device and select to view the log

daring onyx Apr 8, 2025, 10:08 PM

#

From power-up to me asking it what today's date it 2x

📎 logs.txt

#

Ahh!

#

The URL for the .flac response didnt work
Replaced it with my FQDN and in my web browser it worked

#

Let me change that in HA

vernal bronze Apr 8, 2025, 10:10 PM

#

progress!... hopefully

daring onyx Apr 8, 2025, 10:12 PM

#

So the logs show it changed the URL, but no audio on the VPE

#

I can visit that URL from chrome and it plays the correct audio

#

here is the relavant log section

[15:12:09][D][voice_assistant:723]: Response: "April 8th, 2025"
[15:12:09][D][light:036]: 'voice_assistant_leds' Setting:
[15:12:09][D][light:051]:   Brightness: 66%
[15:12:09][D][light:109]:   Effect: 'Replying'
[15:12:09][D][voice_assistant:665]: Event Type: 8
[15:12:09][D][voice_assistant:745]: Response URL: "https://home.elcarpenter.com/api/tts_proxy/_rDpWL5-pSoB6EtktGnfUw.flac"
[15:12:09][D][voice_assistant:528]: State changed from AWAITING_RESPONSE to STREAMING_RESPONSE
[15:12:09][D][voice_assistant:535]: Desired state set to STREAMING_RESPONSE
[15:12:09][D][media_player:074]: 'Media Player' - Setting
[15:12:09][D][media_player:081]:   Media URL: https://home.elcarpenter.com/api/tts_proxy/_rDpWL5-pSoB6EtktGnfUw.flac
[15:12:09][D][media_player:087]:  Announcement: yes
[15:12:09][D][speaker_media_player:426]: State changed to ANNOUNCING
[15:12:09][D][voice_assistant:665]: Event Type: 2
[15:12:09][D][voice_assistant:764]: Assist Pipeline ended
[15:12:09][D][esp-idf:000][ann_read]: I (77832) esp-x509-crt-bundle: Certificate validated
[15:12:09]
[15:12:09][D][ring_buffer:034][ann_read]: Created ring buffer with size 1000000
[15:12:09][D][speaker_media_player.pipeline:114]: Reading MP3 file type
[15:12:10][D][speaker_media_player:426]: State changed to IDLE
[15:12:10][D][voice_assistant:401]: Announcement finished playing
[15:12:10][D][voice_assistant:528]: State changed from STREAMING_RESPONSE to RESPONSE_FINISHED
[15:12:10][D][voice_assistant:535]: Desired state set to RESPONSE_FINISHED
[15:12:10][D][voice_assistant:528]: State changed from RESPONSE_FINISHED to IDLE
[15:12:10][D][voice_assistant:535]: Desired state set to IDLE
[15:12:10][D][light:036]: 'voice_assistant_leds' Setting:
[15:12:10][D][light:047]:   State: OFF
[15:12:10][D][light:109]:   Effect: 'None'
[15:12:20][D][power_supply:048]: Disabling power supply.

vernal bronze Apr 8, 2025, 10:16 PM

#

can you try the assist_satellite.announce test again now that the url is fixed

daring onyx Apr 8, 2025, 10:16 PM

#

Yep

vernal bronze Apr 8, 2025, 10:17 PM

#

can try both with and without preannounce

daring onyx Apr 8, 2025, 10:17 PM

#

With pre-announce, jingle but no audio

#

No pre-announce, no audio at all but the lights do flash

vernal bronze Apr 8, 2025, 10:18 PM

#

and does tts.speak work still?

daring onyx Apr 8, 2025, 10:18 PM

#

yes it does

#

The media URL is different between the two

vernal bronze Apr 8, 2025, 10:19 PM

#

in what way?

daring onyx Apr 8, 2025, 10:20 PM

#

tts.speak
https://home.elcarpenter.com/api/esphome/ffmpeg_proxy/319ea7cc3414f2bb54d0aaf468ec1fed/nu9KoYKdTP1bEtcNBaV9vw.flac

assist_satellite.announce
https://home.elcarpenter.com/api/tts_proxy/ZJiML_3iw8U54zkHQycFpg.flac

#

Not sure if thats expected or not

vernal bronze Apr 8, 2025, 10:20 PM

#

yeah the begining bit is the same which is the important thing

#

okay on the voice assistant trace on the TTS section you can hit "play audio"

#

that plays the TTS message it generated on the browser

#

this one

#

does that play tts on your browser?

daring onyx Apr 8, 2025, 10:22 PM

#

Yep, plays in my browser

#

I did notice in the ESPHome logs, the audo that actually plays has a entry about decoding the audio, while the one that doesnt play, has no such line

#

tts.speak

[15:18:46][D][speaker_media_player.pipeline:114]: Reading FLAC file type
[15:18:46][D][speaker_media_player.pipeline:124]: Decoded audio has 1 channels, 48000 Hz sample rate, and 16 bits per sample

vernal bronze Apr 8, 2025, 10:24 PM

#

yeah, its super strange

daring onyx Apr 8, 2025, 10:24 PM

#

assist_satellite.announce

[15:17:39][D][ring_buffer:034][ann_read]: Created ring buffer with size 1000000
[15:17:39][D][speaker_media_player.pipeline:114]: Reading MP3 file type

#

It also detects it as MP3 and not FLAC

vernal bronze Apr 8, 2025, 10:25 PM

#

are you running the latest HA version?

daring onyx Apr 8, 2025, 10:25 PM

#

2025.4.1

vernal bronze Apr 8, 2025, 10:25 PM

#

cool, was just making sure. gotta check the easy

#

does the vpe play sound effects when you flip the mic mute switch on and off?

daring onyx Apr 8, 2025, 10:27 PM

#

Yea, like a sweeping noise

vernal bronze Apr 8, 2025, 10:28 PM

#

at this point all i can think of suggesting is a manual reinstall of firmware to rule out any corruption there

daring onyx Apr 8, 2025, 10:29 PM

#

Already did that. Earlier I had tried having ESPHome own it and after that made no difference, I reverted it via https://esphome.github.io/home-assistant-voice-pe/

Home Assistant Voice PE

Install firmware on Home Assistant Voice PE.

vernal bronze Apr 8, 2025, 10:30 PM

#

ok

marsh sparrow Apr 8, 2025, 10:30 PM

#

If you look at the debug voice assistant's raw part, does it list mime_type for the mp3 as something else than mp3? I don't know why that could be an issue but just a thought.

vernal bronze Apr 8, 2025, 10:31 PM

#

on the dev tools try running the action
tts.clear_cache
then
restart both home assistant and the voice pe

daring onyx Apr 8, 2025, 10:31 PM

#

marsh sparrow If you look at the debug voice assistant's raw part, does it list mime_type for ...

You mean this

stage: done
run:
  pipeline: 01jr9et3b4xg0dgc29zmwempdn
  language: en
  conversation_id: 01JRBRD8M77647E8QA8WMDYZJX
  tts_output:
    token: g8QzKpak1fviiRsbQ5G95g.flac
    url: /api/tts_proxy/g8QzKpak1fviiRsbQ5G95g.flac
    mime_type: audio/mpeg

marsh sparrow Apr 8, 2025, 10:31 PM

#

Ye

daring onyx Apr 8, 2025, 10:32 PM

#

vernal bronze on the dev tools try running the action `tts.clear_cache` then restart both home...

I ran tts.clear_cache first and the VPE said "Test"

#

Restarting both now

marsh sparrow Apr 8, 2025, 10:33 PM

#

daring onyx You mean this ``` stage: done run: pipeline: 01jr9et3b4xg0dgc29zmwempdn lang...

To me that mime_type seems a bit strange, but does that happen with the mp3 too?
However not sure if PE even cares about it.

daring onyx Apr 8, 2025, 10:34 PM

#

vernal bronze on the dev tools try running the action `tts.clear_cache` then restart both home...

No change, ESPHome log looks similar

#

Full yaml of the voice pipeline
https://gist.github.com/dcplaya/d3c5c48c6fcf1dcca6f0900c6563a86e

Gist

gist:d3c5c48c6fcf1dcca6f0900c6563a86e

GitHub Gist: instantly share code, notes, and snippets.

vernal bronze Apr 8, 2025, 10:45 PM

#

i am running out of suggestions at this point (if you couldnt tell 😛 )

#

on the esphome integration page delete the vpe from HA and then add device and manually re add it using the add device button on the integration page. (you will need its ip)

#

daring onyx Apr 8, 2025, 10:50 PM

#

I have progress!

#

This works

action: assist_satellite.announce
target:
  entity_id: assist_satellite.home_assistant_voice_09e6a4_assist_satellite
data:
  preannounce: true
  media_id: https://www.soundhelix.com/examples/mp3/SoundHelix-Song-1.mp3

#

ESPHome log
https://gist.github.com/dcplaya/00cb2aa6ca45b0e83795ab3334f7d85d
Seems like the ring bufffer size is much larger, could just be due to the file be bigger though

Gist

gist:00cb2aa6ca45b0e83795ab3334f7d85d

GitHub Gist: instantly share code, notes, and snippets.

vernal bronze Apr 8, 2025, 10:52 PM

#

yeah the file is bigger so its a bigger buffer

#

can copy the url that has the TTS in it, comfirm it works then try playing that as a media file like you did with the song

daring onyx Apr 8, 2025, 10:54 PM

#

The pre announce jingle?

vernal bronze Apr 8, 2025, 10:54 PM

#

no, when you do a announce tts it sends 2 urls

#

1 is the jingle and the other is the tts

#

the jingle plays right? but the tts seems to fail. i am wondering what happens if you try and tell it to play the tts that it previously generated

#

if that makes any sense?

daring onyx Apr 8, 2025, 10:57 PM

#

So lets just turn the pre-announce jingle off.

You are saying re-run that action above (prenounce= false), grab the URL from ESP logs, then use it again to do another announce?

#

If so, that fails

vernal bronze Apr 8, 2025, 10:59 PM

#

what does the esphome log look like for that attempt with the url in the media id?

#

does it fail in the same way?

daring onyx Apr 8, 2025, 11:01 PM

#

https://gist.github.com/dcplaya/d9fd402ae6c2609bd24f931f55b09f02

First one is the original, working sound
2nd one is the 1st one's URL as the media_id

Gist

gist:d9fd402ae6c2609bd24f931f55b09f02

GitHub Gist: instantly share code, notes, and snippets.

#

It seems to get it's IDLE state triggered early

vernal bronze Apr 8, 2025, 11:03 PM

#

yeah it seems its failing to play silently and continueing

daring onyx Apr 8, 2025, 11:03 PM

#

IDLE gets set, then further down it actually requests IDLE

vernal bronze Apr 8, 2025, 11:05 PM

#

maybe piper is doing something strange with the output file that your browser can deal with but the vpe cant

#

whats your config of piper look like?

#

cos it did that strange thing with the mp3 decode earlier

daring onyx Apr 8, 2025, 11:07 PM

#

I have the container run with the following arguments

image:
  repository: rhasspy/wyoming-piper
  tag: 1.5.0
  pullPolicy: IfNotPresent
args:
  - --voice
  - en-us-lessac-low

vernal bronze Apr 8, 2025, 11:12 PM

#

you have a /data volume mounted?

daring onyx Apr 8, 2025, 11:13 PM

#

Yea, its full of the models

vernal bronze Apr 8, 2025, 11:13 PM

#

the default recomended config looks like this sort of thing

    --voice en_US-lessac-medium

daring onyx Apr 8, 2025, 11:13 PM

#

Yea, thats what I am running, its just in a helm chart format

vernal bronze Apr 8, 2025, 11:13 PM

#

you have 1.5.0 tagged directly but i am not sure that matters

daring onyx Apr 8, 2025, 11:13 PM

#

vernal bronze Apr 8, 2025, 11:14 PM

#

that is latest i think

daring onyx Apr 8, 2025, 11:14 PM

#

I can use whatever version, takes me 30 seconds to change it

vernal bronze Apr 8, 2025, 11:14 PM

#

can switch to "latest"

daring onyx Apr 8, 2025, 11:14 PM

#

1.5.2 seems to be the latest

#

I can switch it

vernal bronze Apr 8, 2025, 11:15 PM

#

i am not sure it will make any difference

#

its more that we have checked pretty much everything else

#

maybe it will be the kick it needs. you will want to run the clear tts cache action again too though

daring onyx Apr 8, 2025, 11:19 PM

#

Doesnt seem to be any different

vernal bronze Apr 8, 2025, 11:23 PM

#

you could try spinning up another home assistant instance in a different container deleting it from your main one and add it to the fresh one

#

its starting to look like a hardware fault if i am honest

#

a fresh HA instance would rule out any random config issues on the HA side

daring onyx Apr 8, 2025, 11:25 PM

#

But then wouldn't no audio work?

#

Yea, let me get another instance spun up

vernal bronze Apr 8, 2025, 11:25 PM

#

probably gunna have to change the ports on the container to avoid clash

#

or use another box i guess

daring onyx Apr 8, 2025, 11:26 PM

#

Its k8s, it takes care of all of that

vernal bronze Apr 8, 2025, 11:28 PM

#

the only other thing i can suggest at this point is trying to use cloud tts. you could sign up for home assistant cloud (you get 31 day trial) and use that as your providrer instead of piper

daring onyx Apr 8, 2025, 11:28 PM

#

Oh, true! It would let me rule out a hardware issue at the least

vernal sundial Apr 8, 2025, 11:29 PM

#

I have it working of it helps

vernal bronze Apr 8, 2025, 11:30 PM

#

by trying a fresh spin of HA and a cloud speech stuff you have ruled out everything except the vpe itself

vernal sundial Apr 8, 2025, 11:30 PM

#

https://github.com/uhthomas/automata/blob/645d46f684b10b39140251b1c428bebcaa7d7470/k8s/magiclove/home_assistant/piper/stateful_set_list.cue

GitHub

automata/k8s/magiclove/home_assistant/piper/stateful_set_list.cue a...

Monorepo for Starjunk and subsidiaries. Contribute to uhthomas/automata development by creating an account on GitHub.

daring onyx Apr 8, 2025, 11:30 PM

#

Thomas!

#

Now to just decode your weird cluster code 🤣

vernal sundial Apr 8, 2025, 11:30 PM

#

haha

#

it's a very simple deployment

#

it needs a ridiculous amount of cpu fwiw

#

the 1 core I have given it is not enough and I will change it to a limit of like 8 maybe

#

ideally I could get it working with cuda, but it didn't like it

vernal bronze Apr 8, 2025, 11:31 PM

#

piper doesnt use that much cpu, whisper uses loads mind

daring onyx Apr 8, 2025, 11:32 PM

#

The only difference is the arguments passed

vernal sundial Apr 8, 2025, 11:32 PM

#

daring onyx Apr 8, 2025, 11:34 PM

#

Spinning up a new instance now

vernal bronze Apr 8, 2025, 11:34 PM

#

best way to run HA is HAOS 😛

daring onyx Apr 8, 2025, 11:35 PM

#

HAOS isnt high availability! 😜

vernal bronze Apr 8, 2025, 11:35 PM

#

it could be if you host it on proxmox nodes...

daring onyx Apr 8, 2025, 11:35 PM

#

But then I have to use a mouse...yuck

vernal bronze Apr 8, 2025, 11:36 PM

#

mines on proxmox but its not HA to be fair

barren wolf Apr 8, 2025, 11:36 PM

#

vernal bronze it could be if you host it on proxmox nodes...

in my opinion waste of resources just having them sit there waiting to take over

vernal bronze Apr 8, 2025, 11:37 PM

#

yeah thats fair

daring onyx Apr 8, 2025, 11:37 PM

#

New arguments for Piper made no difference

vernal sundial Apr 8, 2025, 11:37 PM

#

what is the error sorry

#

and how are you testing it

#

if you go to /config/voice-assistants/assistants, you can test the TTS service directly

#

vernal bronze Apr 8, 2025, 11:39 PM

#

it does seem to be generating the TTS file but for some reason when it tries to play it on the VPE its not playing correctly. looks like a failure to decode on the device

#

was worried that piper was producing a strange format

#

for no particular reason but everything else has been ruled out

vernal sundial Apr 8, 2025, 11:40 PM

#

gotcha, so piper is probably okay, but the home assistant voice device is not?

#

guess if it works in the web ui then that 100% confirms it right

vernal bronze Apr 8, 2025, 11:40 PM

#

i am leaning towards a hardware issue

#

trying a fresh HA instance and trying HA cloud for voice pretty much completly rules out everything except the device. maybe one of the internals of the chip is failing and causing it to fail to decode the file type it needs to

#

we are in to long shots here now to be fair

daring onyx Apr 8, 2025, 11:43 PM

#

I guess doing cloud right now should rule out everything, right? Hell, I dont even care if I pay for a year to test. It goes to more development anyway

vernal bronze Apr 8, 2025, 11:44 PM

#

i do encourage you to do that but you can use the trial initally for testing anyway

#

if still no joy with cloud and fresh spin up then looks like returning it for a replacement will be the next step

#

it could be that its struggling with https but it said the cert was verified in the log and other stuff works so i dont think thats it

#

a fresh spin would rule that too as i assume your not gunna https that one just to test

daring onyx Apr 8, 2025, 11:47 PM

#

It gets https automagically

#

But I can give it a IP and bypass it

vernal bronze Apr 8, 2025, 11:49 PM

#

yeah, as i said. i am well into long shots and ruling out every possible thing i can think of except the physical hardware

#

although if its a hardware failure it could also be on the xmos chip which is why theres no errors

daring onyx Apr 8, 2025, 11:52 PM

#

Ok, instance spun up
I need to remove it from my original HA instance, right? And then I can add piper/whisper

vernal bronze Apr 8, 2025, 11:52 PM

#

you need to remove the device from the esphome integration

#

so you dont have 2 HA instances trying to talk to it

#

you could just use cloud trial on the freshly spun up one too. you can move it later

#

can only be used on 1 instance but its 1 at a time

#

if it works with fresh and cloud then you could add whisper/piper back in and see if it falls over there

#

if not then its the inital HA instance

#

thats my chain of thought

daring onyx Apr 8, 2025, 11:55 PM

#

I can use it on the new test instance

vernal bronze Apr 8, 2025, 11:55 PM

#

it works as expected?

#

or you mean you can use the cloud? but not there yet?

daring onyx Apr 8, 2025, 11:58 PM

#

New instance using local, same result

#

Adding the cloud to the new instancenow

#

With cloud...same result

vernal bronze Apr 9, 2025, 12:04 AM

#

then i think its gotta be hardware.

#

sad to say it but i think returning for a replacement is the best next move

#

i feel that your rma request "could" be comprehensive but you might want to take it easy 😛

daring onyx Apr 9, 2025, 12:09 AM

#

I contacted Ameridroid for an RMA

vernal bronze Apr 9, 2025, 12:12 AM

#

dam, i was really hoping we would get this one solved

#

i guess if it ends up being hardware fault we did solve it

daring onyx Apr 9, 2025, 12:14 AM

#

Would be one of the craziest new device hardware faults ive personally experienced

vernal bronze Apr 9, 2025, 12:15 AM

#

yeah its super strange to be fair

daring onyx Apr 9, 2025, 12:16 AM

#

Email sent off, ill keep this updated when I get a new device

vernal bronze Apr 9, 2025, 12:16 AM

#

unless it really just doesnt like k8 containers? (mostly joking)

daring onyx Apr 9, 2025, 12:17 AM

#

Thomas having his working proves it doesnt care about k8s

vernal bronze Apr 9, 2025, 12:17 AM

#

thats a fair point

#

anyways, hopefully you get it replaced in a few days and new one works out the box

daring onyx Apr 9, 2025, 12:17 AM

#

Thanks for the help!

#

Also sent an email to hello@nabucasa.com, found that info in the pamphlet that came inside the box

#Home Assistant Voice Preview Edition - No Audible Responses