OpenAI answer not played in M5 Atom Echo | Home Assistant | Page 1

fiery moon May 23, 2025, 7:05 AM

#

Hi guys,

I have Voice Assistant running locally on my thin client with Whisper and Piper Add-Ons. I’ve linked a M5 Atom Echo as local satellite.

Voice commands work fairly okay to control my home and it responds verbally without issue.

Now I’ve integrated OpenAI via API and it works perfectly if I send a prompt via the HA App console (setting local execution preferred on). However, when I ask a question via the M5, such as “explain what big bang is” it doesn’t output the answer acoustically on the M5.

What am I doing wrong?

split river May 23, 2025, 7:08 AM

#

fiery moon Hi guys, I have Voice Assistant running locally on my thin client with Whisper ...

have you set up an assist pipeline that uses your whisper/piper with the openai as a conversation agent?

have you set the device to use that assist pipeline?

fiery moon May 23, 2025, 10:00 AM

#

Sorry I’m a noob still. I have integrated Whisper and Piper as ad ons and Whyoming Protocol. I have then connected OpenAI via the Voice assistant menu. What am I missing?

split river May 23, 2025, 10:11 AM

#

fiery moon Sorry I’m a noob still. I have integrated Whisper and Piper as ad ons and Whyomi...

if you have the voice assistant pipeline set up. you may just need to select it on the device page

#

fiery moon May 23, 2025, 10:14 AM

#

Oh I have done so. I’ve tried both entries: „Voice Assistant“ and „Preferred“. Both don’t make a change. However to clarify: when I simply ask „Ok nabu, turn on kitchen lights“ it does so and also responds verbally through the M5. Only GPT requests don’t end up in a verbal response.

split river May 23, 2025, 10:16 AM

#

gotcha, on the voice assistant settings page you can hit the 3 dots and go to debug

#

#

you get something like this:

#

is the processing part of it working?

fiery moon May 23, 2025, 1:28 PM

#

This is what is outputted with the same text prompt of yours. Typed into the interface in the app:

split river May 23, 2025, 1:56 PM

#

fiery moon This is what is outputted with the same text prompt of yours. Typed into the int...

but if you call the command from a voice assistant where it doesnt work do you see what goes wrong there?

fiery moon May 23, 2025, 6:21 PM

#

Okay this is my result. I asked Nabu where humanity is coming from. It didn’t output any spoken response. Below please find the debug protocol.

#

However, when I ask Nabu to turn off the living room lights, it does so and also output a spoken response. Here are the (to me completely similar looking logs).

fiery moon May 23, 2025, 10:23 PM

#

Just installed the latest HA Update which also bumped the ESP software. And now also ChatGPT is talking to me. At least every now and then… I am open for ideas to obtain a consistent experience of always obtaining a spoken answer 🙂 The „Wetter“ prompt worked, the „Donald Trump“ prompt however didn’t output a spoken answer…

manic briar May 23, 2025, 10:44 PM

#

the reason is that Piper is slow to generate a response. If more than 5 seconds pass, the satellite resets the task.

fiery moon May 24, 2025, 6:22 AM

#

Oh okay this makes sense! Any ways to improve speed or is there a better TTS solution? I run HA in Proxmox as Virtual Machine. Piper currently as HA Add On.

manic briar May 24, 2025, 1:01 PM

#

fiery moon Oh okay this makes sense! Any ways to improve speed or is there a better TTS sol...

Unfortunately, Piper is already as fast as it gets in its class, but the ONNXRuntime library cannot provide instant generation on any modern CPU. The only question is at what character limit the time restriction will occur.
Using accelerators like cuda significantly improves Piper's performance, but this solution isn't suitable for everyone. There is also onnxruntime acceleration for other platforms, but it is a quest to build a working build because of different dependency versions.
However, it's not all bad—developers are working on streaming response generation, which might appear in the next few releases. This will address two issues at once: the long generation time for lengthy texts and the delay caused by LLM response generation. For now, I can suggest looking for cloud integrations, or if offline use is required, use a simpler tts generator (maybe RHVoice?).

coarse pelican May 24, 2025, 8:49 PM

#

I have a voice assistant I made myself. I use Cloud and ChatGPT. I have the same problem. It worked perfectly fine for a while, then after an ESPHome update it broke. It doesn't say long answers. But for example, it doesn't always say my morning wake-up routine. That would be a longer text too. I'll grab a log entry: [17:14:56][W][esp-idf:000][ann_read]: W (1998086) HTTP_CLIENT: Connection timed out before data was ready!
[17:14:56][E][speaker_media_player.pipeline:112]: Media reader encountered an error: ESP_FAIL

#OpenAI answer not played in M5 Atom Echo