#Help: ReSpeaker Satellite No Wake Word Response & Wrong Audio Output (Laptop vs. Sonos)

1 messages · Page 1 of 1 (latest)

merry palm
#

Dear HA Community,

I need some final assistance with my local Voice Assistant setup.

My ReSpeaker XVF3800/XIAO ESP32S3 was added via HA and is connected to my Wi-Fi. I used the ESPHome Device Builder add-on to take control but have made no manual YAML edits. The pipeline (Whisper/Piper/German) is working for processing.

The Two Core Issues
Incorrect Audio Output (TTS): Piper's voice responses are playing through the USB audio output of the ReSpeaker (i.e., my laptop headphones), even though my Sonos Five is integrated into HA. The option to assign the media_player is missing from the HA UI.

No Wake Word Recognition: The ReSpeaker does not react to the wake word (Hey Assist).

My Question
Since both the UI speaker assignment and the mic activation are failing:

Do I definitively need to manually edit the YAML configuration within the ESPHome add-on to add the required blocks (voice_assistant: with microphone: and media_player:) to solve both problems simultaneously?

I am looking for the simplest, most reliable fix for these two common issues.

Thanks for any input!

civic temple
#

Hi, for the audio output other than device-output itself you have to specify sonos mediaplayer in yaml(and disable internal. For Wakeword also, but not sure though..

native kernel
#

Please describe your setup and goals better. Why you refer to headphones, if XVF is connected via ESPHome? What is "hey Assist"? Custom wake word? For which system, and how is it integrated to the device and pipeline?
So far it looks all over the place. Please clarify if you want to use it as USB mic/speaker, or as separate satellite via ESPHome?

native kernel
#

Also running pipeline response on separate speaker isn't something that is easily doable on the pipeline itself. You may use some tricks like automation, that redirects response to other speaker - but those are clunky still - no official support, hacks only.

merry palm
#

Sorry, I'm still very new to HA.

I've been completely paralyzed since a car accident and have relied on Alexa for years to control my smart home via voice commands. I'm currently in the process of switching to HA and want to control my smart home via voice using HA going forward.

This is my current setup:

Host: Geekom Air12 mini PC N150 16 GB RAM + 512 GB SSD

Microphone: ReSpeaker XVF3800 with XIAO ESP32S3

Speaker: Sonos

I intend to use my microphone as a satellite. I've successfully connected my ReSpeaker to my Wi-Fi network via HA and used the ESPHome Device Builder to take control of it.

Unfortunately, I'm not very experienced with this and simply want my XVF to react to my wake word as a satellite so I can use HA with voice control. I'd also love to integrate this with an LLM eventually. That would be fantastic.

Please let me know if you need any further information.

merry palm
#

I would be very grateful for any help.

native kernel
#

Sorry for your situation.

IDK if it's worth switching to HA Assistant right now. It's definitely far from mature, and in the situation when your life can depend on it, I'd rather use something well-established, even if it's proprietary shit... 🙂 Especially talking LLMs with their hallucinations.

Regarding the XVF: it's not microphone, it's dev board with mic array, DAC and audio processor XMOS. It's basically ready-made satellite, you just need speaker for it.
However, it's very early product, and the ESPHome integration for it is still in development (as well as the XMOS chip firmware). I worked on Respeaker Lite integration, and work on this one - and although it's pretty promising, it's far from ideal. For stable solution at the moment I'd rather take Respeaker Lite.

Check my repos for these devices, I work with Seeed directly to make sure I use them right. Also I aim to have feature parity with Voice PE (and a bit more).

Now to the setup. Having separate speaker, as I said, is very, very tricky. Much easier to put some 5W speaker into Respeaker board directly and use it for voice responses - and use Sonos for music etc. It's all changing every day, so the setup that you may be able to do now, will break tomorrow most probably. Stick to the most stable solution.

merry palm
#

Thank you so much for your kind reply and the incredibly helpful information!

I truly understand your concerns about stability. I want to assure you that my life doesn't depend on this specific system—I am never home alone. My main goal is simply to bring my smart home, TV, and music control into Home Assistant, and eventually explore conversational features with an LLM.

I'm aware the maturity level isn't quite at Alexa or Google's standard yet, but I'm eager to try and make this work, especially since I saved up for this specific hardware. If it proves unsuitable, I can always revert to Alexa!

Your advice brought up a few points I'm hoping you can clarify for a beginner like me:

ReSpeaker Lite vs. Mine: You suggested the ReSpeaker Lite for stability. Does that mean the XVF3800 that I purchased is significantly less suitable or much harder to integrate right now?

"Repos": Could you kindly explain what you mean by "repos"? I'm not familiar with the terminology.

Satellite Placement: My understanding was that an Assist Satellite is designed to work independently, sending audio over Wi-Fi to a network speaker (like my Sonos). Does the current state of the integration mean I must physically wire the satellite to the speaker for a stable setup?

Finally, given my situation and lack of technical experience, I would be incredibly grateful for any hands-on help in setting up the system. If you were willing to assist, I would be happy to discuss providing temporary, secure access to my Home Assistant instance to help fix the configuration directly.

Thank you again for all your hard work on these integrations. Your assistance means a lot!

native kernel
#

Thanks for your kind words, and for explaining your situation in detail.

Let's go one by one:

XVF vs Lite: yes, Lite is a bit older device, with the firmware that works for at least a year without major trouble. XVF just released, and while it's very promising device, so far the software part is far from ideal. I write it, and I wrote one for Lite (well it's pretty bold statements, I'm practically just building them from pieces that others made) - and I'm not ready to replace my Respeaker Lite satellites with XVF yet, although main functionality is sudo there.

Repos: sorry for bringing confusion here. It means GitHub repositories.
Respeaker Lite: https://github.com/formatBCE/Respeaker-Lite-ESPHome-integration
XVF3800: https://github.com/formatBCE/Respeaker-XVF3800-ESPHome-integration

Placement: the satellite is made to be single device, like Alexa Echo or Google Home Mini. It supports JST speaker up to 5W on 4 Ohm impedance. Alternatively it can output over 3.5mm line jack. There's a lot of speech-based services integrated into assist_satellite entity in Home Assistant, and they assume that mic and speaker are single device.
There is a hacky way to direct the answer audio to other media player, but it will degrade your experience (e.g. there's no way to play "ding" preannouncing sound etc.)

Setting up the satellite requires more than HA access, and likely would involve physical presence. 🙂 So unless you're in Vancouver area, I can help informationally only.

GitHub
GitHub - formatBCE/Respeaker-Lite-ESPHome-integration

Contribute to formatBCE/Respeaker-Lite-ESPHome-integration development by creating an account on GitHub.

GitHub
GitHub - formatBCE/Respeaker-XVF3800-ESPHome-integration

Contribute to formatBCE/Respeaker-XVF3800-ESPHome-integration development by creating an account on GitHub.

merry palm
#

Thank you for your comprehensive response.

Unfortunately, I really can't make heads or tails of it; I don't even know how to use GitHub.

Since I've already purchased the XVF, I need to try to get the system running somehow to see if all the further work will be worth it.

The good news is that my Sonos has a 3.5 mm input, so I could physically connect my ReSpeaker to the speaker with a 3.5 mm cable. That would help, right?

My only concern is how to get the assistant running after that, specifically how to enable wake-word activation. Could you possibly list the steps I need to complete? Then I will try to follow them somehow.

What do you think?

civic temple
#

Hi, about the 3.5 audio input on sonos that would work indeed if source on sonos is selected to line input (3.5 jack) , for your other question about wakeword, you have to wait for FormatBCE again. Sorry about your situation too!

merry palm
#

Okay thanks

#

Just one quick question, how can I reset the XVF to factory settings?

native kernel
#

There's no factory settings on this device. 🙂

#

Do you have version with soldered ESP32-S3?

merry palm
#

yea

#

I could really cry. I've spent so much money and I can't do anything with it. I just tried implementing your repo with GPT, but it just doesn't work.

native kernel
#

Do you have ESPHome add-on installed?

merry palm
#

yea

native kernel
#

Just make sure you use ESP32 USB port.

merry palm
#

Where should I insert it? In the YAML file of my XVF?

#

I tried it, but I get an error message saying my OTA password isn't set. Do I need to set it to install the code?

native kernel
#

Yes, in the XVF

#

That things are i guess because you're trying to install it over the air?

#

Install it via USB

#

Case is that you probably had OTA password on your old config - so you either need the same here, or install directly via USB connection.

merry palm
#

When I try to download the file, I get this error message.

image.png
#

When I try to upload it via USB with the connected computer, it says it can't because I don't have HTTPS.

native kernel
native kernel
merry palm
#

I simply removed the line with the OTA password as you suggested. Now the installation seems to have been successful via the ESP Web Tool. I also sent you a screenvideo of my assist configuration. What do I need to do now to try it out?

image.png
native kernel
#

Your pipeline looks okay. I guess you've checked it for work.

Now you need to add the device to HA, if you didn't do it before. Go to Settings - Devices and Services, plug the XVF in and check - it should appear on top. If it's not, add it from ESPHome integration page.

merry palm
#

Sorry I haven't given any feedback yet. I have a cold and will try it out in the next few days and get back to you. Okay?

civic temple
#

Get well soon!

merry palm
#

Hey, hope you're doing well. I've been off and pretty sick for the past few days. But thank God, everything's back to normal. I've also connected the XVF to the Sonos via a 3.5mm cable and plugged it in. As you can see, ESPHome now shows me many configuration options and tells me that my XVF is online.

Can you explain to me what I need to do next to finally test the system? I would be incredibly grateful and think that your help has helped me a whole lot.

image.png
civic temple
#

Hi, good you're feeling better now! Have you tested if your microphone and wakeword is working? Just say "okay nabu" and see if device reacts.

native kernel
#

Yeah test it for wake word, then try to send announcement to it from Developer tools.
Also mute/unmute button on device should trigger sound.

If you feel extra fancy, you can connect it to Music Assistant and play music there.

merry palm
#

Hey, when I say "okay nabu," nothing happens on the XVF. A red LED just stays on all the time.

Even when I click mute, I can't hear any sound. Why is that?

Can you explain this in more detail with the developer tools? I'm not very familiar with HA.

How can I connect it to Music Assistant?

merry palm
#

Update: I turned off the XVF and restarted it and now when I say the activation word, the LEDs on the XVF change, but I don't hear anything.

civic temple
#

Hi, is your sonos powered on and is using the line input?

#

what happens if you disconnect the audio plug from the XVF and then again say the wakeword?

#

If you then hear sound, you would have to check the line in input on the sonos (or sonos app)

#

If you hear sound, you can use (wakeword) wie spaet ist es. it should answer with the current time.

#

you can do the same in developer tools using the action tab. select announce (assist_satellite.announce) then select entity (your XVF) and enter a question in the message field. The answer would then be played on the XVF.

merry palm
#

I chatted a bit with ChatGPT and then changed the source level from 4 to 10 in the Sonos app. I think that made a difference. Because now when I click mute, I hear a sound. Now when I call the wake word, my XVF responds and I hear a wake tone, but when I ask what time it is, it doesn't give me a response. In the developer tools, I was able to have a message spoken in the announcement entity, which then plays on my Sonos. But when I type what time it is in the message field, my Sonos just speaks the message and doesn't give me the time.

So what else am I doing wrong? Or is this actually good news?

image.png
civic temple
civic temple
native kernel
#

Yes, the announcement is just what it is - it's not for the voice communication, it's for informing you that something's up.
For voice assistant to work you need to make sure your voice pipeline is working.

#

Check Settings - Devices & services - ESPHome - your XVF. You should see wake word drop-downs and pipeline drop-downs. So far adjust just the first wake word and first pipeline. Choose what you want there. Then go to Settings - Voice assistant, and adjust the pipeline, set conversation agent, STT and TTS there.
After that everything should be working.

civic temple
#

my brain is having a bad day today lol 😉

civic temple
#

and check box in ESPHome > XVF to allow xvf to control HA

image.png
#

zahnrad / erlauben..

#

@native kernel pipelines looked ok you said above in post 3-10, would still be ok?

civic temple
native kernel
native kernel
civic temple
#

didnt know that.. gonna try, so here a restart would be enough after double-checking?

native kernel
#

Guess so.at least it works for me.
That flag is needed if you call HA services in YAML, which we dont

merry palm
#

Hi guys, thank you so much for your input, I really appreciate it. I've restarted Piper and Whisper, and I think my XVF settings should be fine. See my video with the settings. Is everything correct? How can I test if it's working?

image.png image.png
civic temple
#

Hi, looks ok to me, just say the wakeword and ask for the time

#

it should respond through your sonos if connected and volume is high enough

#

Best thing to do is a reboot first of ha

merry palm
#

Okay, so as I said, after I say the wake word, the microphone responds, and I hear an activation tone. If I then ask for the time, it doesn't respond. If I try again, it responds after about 10-15 seconds, sometimes with the time, sometimes with an error message saying it can't understand me.

I've restarted all relevant add-ons and HA, but the problem still persists.

native kernel
#

If you go to Settings - Voice Assistants, and on Local Assistant row click three dots on the right - there's Debug option. Can you go there and show us that info?

merry palm
native kernel
#

Ok, so several things.

  1. Your Whisper is pretty slow. Almost 8 seconds for speech-to-text. You may want to choose smaller model there, move Whisper to different, better machine, or use NabuCasa cloud STT engine instead.
  2. You have OpenAI conversation agent. In pipeline settings to that put the checkbox "Prefer handling commands locally". It will save you tokens and time on simple commands.
civic temple
#

can confirm that speech is working flawless with nabu casa cloud. (and you have online backups)

merry palm
#

Okay, I've activated nabu casa cloud, and things are speeding up now. Some responses come very quickly, but others take quite a while. How can I speed things up? Is there anything other than OpenAI alt AI and would it make sense to use HA with Google or Alexa?

image.png image.png image.png image.png
civic temple
#

google/alexa... no, would not do that, but instead of openai you can try google gemini, for me it's working fine. Have you checked the box "Prefer handling commands locally"

#

this will force commands that HA can handle itself to be handled locally, otherwise redirects to ai. Saves tokens indeed.

#

less google/alexa=better..

merry palm
#

Yes, I checked the box. How can I use Google Gemini instead?

civic temple
#

go to integrations > add integration. Look for Google and select Google Gemini. You have to get an api key, wich is explained during setup if i remember correctly

merry palm
#

Okay thanks, I'll try it.

civic temple
#

after installation, set your conversation agent to google ai and reboot, just to be sure.

merry palm
#

Okay. But STT and TTS will remain HA, or will they also be changed to Google AI?

civic temple