#Sound effect at wake word detected not playing
1 messages ยท Page 1 of 1 (latest)
this is the log for when i'm not commanding anything
[13:48:22][D][media_player:059]: 'ESPHome I2S Media Player' - Setting [13:48:22][D][media_player:066]: Media URL: http://192.168.1.239:8123/media/local/audio/activatedSound.mp3?authSig=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiI3ZWI1OGIzZTQxZWM0M2YzYTM1ZDhkOGMxZGM1ODMyMiIsInBhdGgiOiIvbWVkaWEvbG9jYWwvYXVkaW8vYWN0aXZhdGVkU291bmQubXAzIiwicGFyYW1zIjpbXSwiaWF0IjoxNzA4NjA2MTAyLCJleHAiOjE3MDg2OTI1MDJ9.J6pJSNA5L0bZrot6JpiNDpL3556Id5El2dhLItpKHXc [13:48:37][D][voice_assistant:521]: Event Type: 12 [13:48:37][D][voice_assistant:679]: STT by VAD end [13:48:37][D][voice_assistant:414]: State changed from STREAMING_MICROPHONE to STOP_MICROPHONE [13:48:37][D][voice_assistant:420]: Desired state set to AWAITING_RESPONSE [13:48:37][D][voice_assistant:414]: State changed from STOP_MICROPHONE to STOPPING_MICROPHONE [13:48:37][D][voice_assistant:521]: Event Type: 0 [13:48:37][E][voice_assistant:651]: Error: stt-no-text-recognized - No text recognized [13:48:37][D][voice_assistant:514]: Signaling stop... [13:48:37][D][voice_assistant:414]: State changed from STOPPING_MICROPHONE to STOP_MICROPHONE [13:48:37][D][voice_assistant:420]: Desired state set to IDLE [13:48:37][D][voice_assistant:414]: State changed from STOP_MICROPHONE to IDLE [13:48:37][D][main:226]: Playback started! [13:48:37][W][component:214]: Component i2s_audio.media_player took a long time for an operation (0.56 s). [13:48:37][W][component:215]: Components should block for at most 20-30ms. [13:48:37][D][switch:016]: 'Use wake word' Turning OFF. [13:48:37][D][switch:055]: 'Use wake word': Sending state OFF [13:48:37][D][switch:012]: 'Use wake word' Turning ON. [13:48:37][D][switch:055]: 'Use wake word': Sending state ON [13:48:37][D][voice_assistant:414]: State changed from IDLE to START_PIPELINE
can you post your config
`media_player:
-
platform: i2s_audio
name: ESPHome I2S Media Player
dac_type: external
i2s_dout_pin:
number: GPIO27 #DIN
allow_other_uses: true
mode: monoon_play:
- delay: 700ms
- logger.log: "Playback started!"
- voice_assistant.stop:
on_idle:
- logger.log: "Playback finished!"
- voice_assistant.start_continuous:
output:
- platform: gpio
pin:
number: GPIO27
allow_other_uses: true
id: set_low_speaker
voice_assistant:
microphone: mic_i2s
id: va
noise_suppression_level: 4
auto_gain: 31dBFS
volume_multiplier: 8.0
use_wake_word: false
speaker: my_speaker
on_wake_word_detected:
- homeassistant.service:
service: media_player.play_media
data:
entity_id: media_player.jarvis_esphome_i2s_media_player
media_content_id: >-
http://192.168.1.239:8123/media/local/audio/activatedSound.mp3
media_content_type: music
- switch.turn_on:
id: led_light
on_listening:
- switch.turn_on:
id: led_light
on_error:
- if:
condition:
switch.is_on: use_wake_word
then:
- switch.turn_off: use_wake_word
- switch.turn_on: use_wake_word
on_client_connected:
- if:
condition:
switch.is_on: use_wake_word
then:
- voice_assistant.start_continuous:
on_client_disconnected:
- if:
condition:
switch.is_on: use_wake_word
then:
- voice_assistant.stop:
on_end:
- switch.turn_off:
id: led_light`
Do you need the full config?
that should be enough , i'll have a look through it in a short while ๐
what is the output for on gpio27? id: set_low_speaker ?
set_low_speaker yes
This is for fixing a buzz that happened on the speakers when powering on the device
try formatting the media url like below , if the file is in your media folder, and also move the block to 'on_listening' rather than 'on_wake_word_detected' and see what that does.
- homeassistant.service:
service: media_player.play_media
data:
media_content_id: media-source://media_source/local/vad.mp3
media_content_type: music
entity_id: media_player.tree_media
thanks for the tip. Gonna try that tonight
I haven't physically set one up for a while to test as I only output to other media players and don't use one on the devices personally. however there are a few people trying to do the same, so I will put a test rig together later, makes it easier to advise , i've got the same setup ๐
This setup does not work ๐ฆ
`voice_assistant:
microphone: mic_i2s
id: va
noise_suppression_level: 4
auto_gain: 31dBFS
volume_multiplier: 8.0
use_wake_word: false
speaker: my_speaker
on_wake_word_detected:
- switch.turn_on:
id: led_light
on_listening:
- switch.turn_on:
id: led_light
- homeassistant.service:
service: media_player.play_media
data:
entity_id: media_player.jarvis_esphome_i2s_media_player
media_content_id: >-
media-source://media_source/local/audio/activatedSound.mp3
media_content_type: music`
and for some reason I can't see the logs via de wireless option
Error connecting to [AddrInfo(family=<AddressFamily.AF_INET: 2>, type=<SocketKind.SOCK_STREAM: 1>, proto=6, sockaddr=IPv4Sockaddr(address='192.168.1.140', port=6053))]: [Errno 113]
However I have set up the manual_ip option:
manual_ip: static_ip: 192.168.1.140 subnet: 255.255.255.0 gateway: 192.168.1.254
and the logs are filled with "Speaker buffer full, trying again next loop" error messages.
Do you have any suggestions on how I could fix this? ๐
hi, I have been having a look into this earlier this morning and got side tracked, I can get it to play the sound on wakeword, no problem and then it wont pickup the command. I know previously I have had it working but I can't find the config... so i'm having to figure it out again. Are you unable to get any sound at all from the device? even responses?
It does recognize the wake word, led's turn on but still no sound, in the log it doesn't show that it is executing the service call anymore. previously I could see that in the log. And the audio response when the action is finished is very stutterly
It also doesn't work when I turn use wake word off to turn the pipeline off, and then do the service call via HA itself. No sound at all. But now it does show something in the logs
[D][media_player:059]: 'ESPHome I2S Media Player' - Setting [D][media_player:066]: Media URL: http://192.168.1.239:8123/media/local/audio/activatedSound.mp3?authSig=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJlZDhhMjdhY2ZhOGQ0MzM1OTM3ZTkwNGMxNmJhZTg1ZSIsInBhdGgiOiIvbWVkaWEvbG9jYWwvYXVkaW8vYWN0aXZhdGVkU291bmQubXAzIiwicGFyYW1zIjpbXSwiaWF0IjoxNzA4Njg4ODYwLCJleHAiOjE3MDg3NzUyNjB9.RhCvOVZLBURYIGpFpjGA7DOKe940xCJLjicPAj-pon4
well I need to leave soon, so it has no priority because I can't test anything.
I think you need to strip your config back to basics first , in order to rectify the overall sound issue, then start to build back up once the sound plays normally as it should. Once you have the basic config with audio working. then can look at adding the service calls etc.
Iยดm uploading a new version without the service call to see if that fixes the sound issues
by basic config , just have wifi: ota: api: logger: 12s_audio: microphone : media_player : and a basic VA with no automations
ok ๐ , what board, dac/amp is it ?
The sound is fixed again, so it does work better without the service call at the on_listening block. But the logs are still filled with 'Speaker buffer full, trying again next loop' error messages.
I'm using a esp32dev board with MAX98357A dac and INMP441 mic
Alright, so when I switch of the Use wake word switch, then I can play the audio file via the UI but not via a service call ๐ค
a way I have also done it in the past is to use an automation in HA. Let me see if i can find that
That's what I thought of as well. But would that be fast enough?
i think it worked ok iirc - so trigger = assist in progress then actions, switch ww off , wait for time = 200ms, then media play, then switch ww back on. worth a try and can adjust on the fly ... without having to keep re-compile the device. not ideal .. but a workaround without upsetting audio on the device
ah right yes. Gonna try that now ๐
good luck
I am still looking into getting it to trigger from the device side, I just have a couple of other projects on at the moment that are taking up a bit of time, so i start something then whilst waiting for something to upload or whatever .. i then start looking at another thing and then get deep into that so apologise if i'm a little slow at times getting back with ideas and info ๐
haha thats no problem! I get it, I'm doing like 5 things at same time as well lol
I'm just glad that someone is willing to figure this out with me.
it is something that is asked for a lot, so i don't know why i haven't got a working answer lol.
But we are getting somewhere! with the new automation it is starting to work! ๐ When I trigger the wake word, the led turn on, then after i think 200ms they turn off and then the audio file plays. But it wont listen to the command
can you have the esphome logs open for the device to watch what happens, you should be able to see everything as it happens
Yes I can see that.
Event Type: 10 [21:17:43][D][voice_assistant:530]: Wake word detected [21:17:43][D][switch:012]: 'Eyes' Turning ON. [21:17:43][D][switch:055]: 'Eyes': Sending state ON [21:17:43][D][voice_assistant:521]: Event Type: 3 [21:17:43][D][voice_assistant:535]: STT started [21:17:43][D][switch:012]: 'Eyes' Turning ON. [21:17:43][D][switch:016]: 'Use wake word' Turning OFF. [21:17:43][D][switch:055]: 'Use wake word': Sending state OFF [21:17:43][D][voice_assistant:514]: Signaling stop... [21:17:43][D][voice_assistant:414]: State changed from STREAMING_MICROPHONE to STOP_MICROPHONE [21:17:43][D][voice_assistant:420]: Desired state set to IDLE [21:17:43][D][voice_assistant:414]: State changed from STOP_MICROPHONE to STOPPING_MICROPHONE [21:17:43][D][voice_assistant:414]: State changed from STOPPING_MICROPHONE to IDLE [21:17:44][D][voice_assistant:521]: Event Type: 0 [21:17:44][E][voice_assistant:651]: Error: stt-no-text-recognized - No text recognized [21:17:44][D][voice_assistant:521]: Event Type: 2 [21:17:44][D][voice_assistant:611]: Assist Pipeline ended [21:17:44][D][switch:016]: 'Eyes' Turning OFF. [21:17:44][D][switch:055]: 'Eyes': Sending state OFF [21:17:44][D][media_player:059]: 'ESPHome I2S Media Player' - Setting [21:17:44][D][media_player:066]: Media URL: http://192.168.1.239:8123/media/local/audio/activatedSound.mp3? [21:17:44][W][component:214]: Component i2s_audio.media_player took a long time for an operation (0.54 s). [21:17:44][W][component:215]: Components should block for at most 20-30ms. [21:17:44][D][switch:012]: 'Use wake word' Turning ON. [21:17:44][D][switch:055]: 'Use wake word': Sending state ON
This is what happening in short
- wake word said
- wake word detected
- LED turn on for like 200ms
- LED turn off
- audio file starts playing
can you post the config for your switch: component
`switch:
-
platform: template
name: Use wake word
id: use_wake_word
optimistic: true
restore_mode: RESTORE_DEFAULT_ON
entity_category: config
on_turn_on:- lambda: id(va).set_use_wake_word(true);
- if:
condition:
not:
- voice_assistant.is_running
then:
- voice_assistant.start_continuous
on_turn_off:
- voice_assistant.stop
- lambda: id(va).set_use_wake_word(false);`
I have come a little further now. I right now I have the same behaviour but only with the config from the yaml. No automations. The sound does play finally. But only after it ended to listen for the command and the led turn off
i think something must have recently changes in how VA handles being stopped and started, because this [21:17:44][E][voice_assistant:651]: Error: stt-no-text-recognized - No text recognized i'm sure didn't used to happen, and the stream just started as soon as voice_assistant: was running, without the need for a wakeword. Well I know that it does work , as I have just out a push to talk in one of my builds... if i use wakeword it works. but if i just press the 'button' i can say a command without ww. which really is what's happening here ... so it's a little odd that it's giving that error
yes, so at the point the mic is off and the media can play. and all we want to do is turn mic off to let audio play then turn back on to give the command.! sounds simple
Yes that is what we want. But it doesn't do that ๐ฆ
if only things worked as well in practice as they do in theory. and the stupid thing is, i've done it before. but because I don't actually do it that way myself... it's been lost in configs and my mind lol
has this error 'Component i2s_audio.media_player took a long time for an operation (0.54 s).' something to do with that as well or no?
You do it via automations? or only push to talk?
no thats fine, thats just a warning , that only shows because they changed the logging level in ESPHome a few months ago, previously the errors were still there but supressed
ah right
and the annoying thing is that other people have figured it out and got it working perfectly like this
https://community.home-assistant.io/t/r2d2-satellite/690701/8
so it is possible, and I wont stop trying until if have it working xD
I don't use any devices with mic and speaker. i have them on separate devices, i so i can have mics in different areas and speakers just in central areas, if that makes sense ( sounds like i live in a castle lol ) i don't need audio feedback everywhere
ah yes... on the wyoming satellite it just works! i've got one of those and it's great lol
Oh really. I wanted to start that project once I have this one working. This will be the main "Jarvis" in the living room
3d printed helmet and all
and I was thinking of making some little r2d2 devices as satellites
have you got room in it to put another esp ? as that is a solution and works ! and at the cost of an esp doesnt brake the bank. I've built a lamp that I finished yesterday and I put 2 in that 1 , (well 3 in total lol)
Oh wow nice! also with touch trigger. Thats a nice addition!
yes. i was quite suprised how well the touch works, and it also works through the painted outer. and all that uses is a piece of wire on a gpio . using the esp32_touch: component
oh thats a very clever idea yes!
but with that. I have the mic on 1 esp32 and the media on another. and that way it's no problem calling the media to play whilst the va is running
i did it that way to share the load , with using onboard wakeword, an oled, a mmwave sensor and temp sensor etc
oh right, the only tutorial that I have watched uses a raspberry pi zero 2 w and a mic hat? that is what seperates them?
the pizerow uses the mic hat which is basically a soundcard in it's own right. so it can handle the audio side of things much better
and it isn't limited by the esp voice_assistant limitations of only being able to do either mic or speaker at a time and not both together
ohhh thats why that is working so flawelessly! I am for sure gonna make that after this project
btw it just good to know that I have changed some thing with the media play service call.
I came to the conclusion that I was trying to play on the wrong entity_id xD
and I changed media_content_id: to -> media-source://media_source/local/audio/{audio_file}
and I changed the media_content_type: from music -> audio/mpeg
but it should be doable what we are trying to do right?
unless something has changed in the VA code that's stopping it. right , I think ive just caught up with everything. so i can plug my test device in and have another play.
ah nice ๐
unfortunatley the only one with media player to hand is an atom echo! so I can't hear it very well anyway ๐
you mean the tts? Whats important that it works right xD optimalization comes later
totally! just going to add a switch to the config and then try from HA
what are you trying to make right now? also a voice assistant with esp?
i haven't got any projects on the go at the moment, just need to finish the final bits on that lamp. but i'm just testing stuff now and trying to figure out how to get stuff to work ๐
ah oke cool
ooops totally broke the config lol , need to did another out to fix it
lol happens a lot sometimes, but once it starts working it all fine
just compiling
hmm same error Error: stt-no-text-recognized - No text recognized
yea it's so annoying, im trying to make it work as well. But still the same results as before
that didn't used to come up i'm sure hmm
I think whats going wrong as well is that once I do the voice_assistant.start_continuous it cancels the listening phase or something? but the weird thing is that the audio file only plays after the listening phase is done
I need to take a break for while, i'm getting brain fog ... been at it since 3am i think i need some food to refuel.
good idea. I need to get some sleep as well in a bit. its 11:45 pm where i am
yeah 22.49 here now
i'll pick it up again tomorrow with a fresh set of eyes ๐
me too, have a good night
cheers , you too catcha later
well some people have been sleeping ๐คช i might have sorted it!!!
really?! is it working for you?
so far so good!
the occaisional error, but it carries on working and doesn't cause it to fail
How did you manage to fix it? ๐
if you transfer 10,000000 into my bitcoin account i will reveal the secrets ๐คฃ
ok it was really simple in the end!
you mean you fixed the wake word detected sound right? ๐
yes, you say hey wakeword - and it plays a sound and you say .turn on the light, and it does it!
yessss
Please show me, im very curious right now xD
voice_assistant:
id: va
microphone: echo_microphone
media_player: echo_speaker
noise_suppression_level: 2
auto_gain: 31dBFS
volume_multiplier: 4.0
use_wake_word: true
on_wake_word_detected:
- voice_assistant.stop
- lambda: id(va).set_use_wake_word(false);
- delay: 100ms
- media_player.play_media: http://192.168.1.8:8123/local/vad.mp3
- delay: 100ms
- voice_assistant.start_continuous
on_tts_end:
then:
- homeassistant.service:
service: media_player.play_media
data:
entity_id: media_player.m5media
media_content_id: !lambda 'return x;'
media_content_type: music
announce: "true"
- delay: 50ms
- switch.turn_on: use_wake_word
is it the announce true??
its just the order that the actions are listed under on_wake_word_detected:
oh no thats the normal response from the action
ohh right. But I kinda had that already why wouldn't that work xD im gonna test this right now !
and also after on_tts_end add the delay and the switch.turn on
thes last 2 entries in the code
on_tts_end is not the same as on_end?
no
and at the on wake word detected block you platy the sound via the url, but no entity specified?
correct
it uses the media player configured on the device, which is how we wanted it originally , but tried with service calls ... but this 100% works for me
compiling the code now ๐
๐ค
I hope it works!
omg it took so long that I had a timeout: Error receiving acknowledge version: timed out
uploading directly to the esp right now instead of wirelessly
btw do you know if there is a way to rename the COM port devices in device manager? I have multiple connected so sometimes it difficult to find the right device
its not working yet ๐ฆ
the logs are spammed with
[11:29:32][E][voice_assistant:651]: Error: no_wake_word - No wake word detected [11:29:32][D][voice_assistant:514]: Signaling stop... [11:29:32][D][voice_assistant:414]: State changed from STARTING_PIPELINE to STOP_MICROPHONE [11:29:32][D][voice_assistant:420]: Desired state set to IDLE [11:29:32][D][voice_assistant:414]: State changed from STOP_MICROPHONE to IDLE [11:29:32][D][voice_assistant:521]: Event Type: 2 [11:29:32][D][voice_assistant:611]: Assist Pipeline ended [11:29:32][D][voice_assistant:414]: State changed from IDLE to START_PIPELINE [11:29:32][D][voice_assistant:420]: Desired state set to START_MICROPHONE [11:29:32][D][voice_assistant:514]: Signaling stop... [11:29:32][D][voice_assistant:118]: microphone not running [11:29:32][D][voice_assistant:202]: Requesting start... [11:29:32][D][voice_assistant:414]: State changed from START_PIPELINE to STARTING_PIPELINE [11:29:32][D][voice_assistant:521]: Event Type: 1 [11:29:32][D][voice_assistant:524]: Assist Pipeline running [11:29:32][D][voice_assistant:118]: microphone not running [11:29:32][D][voice_assistant:521]: Event Type: 9 [11:29:32][D][voice_assistant:118]: microphone not running [11:29:32][D][voice_assistant:521]: Event Type: 0
Over and over again
unplug and replug
it's gone very quiet over there ... is that a good or a bad thing ?
kinda bad thing ๐ฆ it's not really working anymore. Even sound is not playing ๐ค
this is the config right now
`voice_assistant:
microphone: mic_i2s
id: va
noise_suppression_level: 4
auto_gain: 31dBFS
volume_multiplier: 8.0
use_wake_word: false
speaker: my_speaker
on_wake_word_detected:
- switch.turn_on:
id: led_light
- voice_assistant.stop
- lambda: id(va).set_use_wake_word(false);
- delay: 100ms
- media_player.play_media: media-source://media_source/local/audio/activatedSound.mp3
- delay: 100ms
- voice_assistant.start_continuous
on_tts_end:
then:
- homeassistant.service:
service: media_player.play_media
data:
entity_id: media_player.ultimate_jarvis_esphome_i2s_media_player
media_content_id: !lambda 'return x;'
media_content_type: music
announce: "true"
- delay: 50ms
- switch.turn_on: use_wake_word
on_listening:
- switch.turn_on:
id: led_light
on_error:
- if:
condition:
switch.is_on: use_wake_word
then:- switch.turn_off: use_wake_word
- switch.turn_on: use_wake_word
on_client_connected:
- if:
condition:
switch.is_on: use_wake_word
then:
- voice_assistant.start_continuous:
on_client_disconnected:
- if:
condition:
switch.is_on: use_wake_word
then:
- voice_assistant.stop:
on_end:
- switch.turn_off:
id: led_light`
I can try to play the sound via ip url
and now I have this issue again...
Can't connect to ESPHome API for ultimate-jarvis @ 192.168.1.140: Error connecting to [AddrInfo(family=<AddressFamily.AF_INET: 2>, type=<SocketKind.SOCK_STREAM: 1>, proto=6, sockaddr=IPv4Sockaddr(address='192.168.1.140', port=6053))]: [Errno 113] Connect call failed ('192.168.1.140', 6053) (SocketAPIError)
ill be back soon! just gotta do something with family
ok no worries, i'll be about and we WILL get it working!
ok boss
first thing i notices is that you have use_wake_word: false in the voice_assistant section
needs to be true
but well need to get you connected to the device again , can you paste the top part of your config, before the voice stuff you posted earler
I replugged the device in, seems to be working again now.
ok cool
I changed the use_wake_word to true
nothing else stands out from what i can see, so upload and try again, after flashing unplugit, give it a few seconds and plug it back in again
still no working sound.. also when I give it the command I get a notification in HA
Login attempt or request with invalid authentication from 192.168.1.140 (192.168.1.140). See the log for details.
ok , so it looks like it needs reconfiguring in HA - is there an api key in your config ?
yes there is
I do also have this in the config
manual_ip: static_ip: 192.168.1.140 subnet: 255.255.255.0 gateway: 192.168.1.254
but that shouldn't give problems right?
ok you will need that api - what you need to do is delete the device from the integration but not the addon. then from the addon view the logs. this should then trigger HA to discover the device again. Having the static in there shouldn't be an issue. do you have a static reserved for it in the router?
No i don't have it set as static ip in my router. I will add that as well. And rediscover the device in HA.
yes, if you don't have it reserved in the router, then it can give that address to another device, then you run into problems.
once you have added it back to HA if you look down the list of devices in the ESPHome integration, there will be a configure button next to each device, click that and then tick the box to allow service calls
still the same login attempt error message ๐ค and what I have noticed is that the device in HA only has the sensor 'Assist in progress' and nothing else. No media player no eyes switch
oh wait now there in the device in HA
it will take a moment for the device to update the entities whne you first add it... how you getting on
this is the voice assistant config block right now
`voice_assistant:
microphone: mic_i2s
id: va
noise_suppression_level: 4
auto_gain: 31dBFS
volume_multiplier: 8.0
use_wake_word: true
speaker: my_speaker
on_wake_word_detected:
- switch.turn_on:
id: led_light
- voice_assistant.stop
- lambda: id(va).set_use_wake_word(false);
- delay: 100ms
- media_player.play_media: http://192.168.1.239:8123/media/local/audio/activatedSound.mp3
- delay: 100ms
- voice_assistant.start_continuous
on_tts_end:
then:
- homeassistant.service:
service: media_player.play_media
data:
entity_id: media_player.ultimate_jarvis_esphome_i2s_media_player
media_content_id: !lambda 'return x;'
media_content_type: music
announce: "true"
- delay: 50ms
- switch.turn_on: use_wake_word
- switch.turn_off:
id: led_light
on_error:
- if:
condition:
switch.is_on: use_wake_word
then:- switch.turn_off: use_wake_word
- switch.turn_on: use_wake_word
on_client_connected:
- if:
condition:
switch.is_on: use_wake_word
then:
- voice_assistant.start_continuous:
on_client_disconnected:
- if:
condition:
switch.is_on: use_wake_word
then:
- voice_assistant.stop:
on_end:
- switch.turn_off:
id: led_light`
What i have noticed is that when I turn off the wake word switch manually and then trigger the sound via the service call like we did before, then it works
So I think I need to try the service call again?
you can try it , but it seems strange
the sound does work again. But still only after the led eyes turned off and the pipeline ended
just for a test I will turn off all the things related to the LED eyes
yes it's worth trying as minimal as possible
esphome:
name: m5stack-atom-echo-889258
friendly_name: M5Stack Atom Echo 889258
api:
on_client_connected:
- delay: 500ms
- switch.turn_on: use_wake_word
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
esp32:
board: m5stack-atom
framework:
type: arduino
logger:
ota:
button:
- platform: restart
id: restart_but
name: Reboot Device
i2s_audio:
i2s_lrclk_pin: GPIO33
i2s_bclk_pin: GPIO19
microphone:
- platform: i2s_audio
id: echo_microphone
i2s_din_pin: GPIO23
adc_type: external
pdm: true
media_player:
- platform: i2s_audio
name: m5media
id: echo_speaker
i2s_dout_pin: 22
dac_type: external
mode: mono
voice_assistant:
id: va
microphone: echo_microphone
media_player: echo_speaker
noise_suppression_level: 2
auto_gain: 31dBFS
volume_multiplier: 4.0
use_wake_word: true
on_wake_word_detected:
- voice_assistant.stop
- lambda: id(va).set_use_wake_word(false);
- delay: 100ms
- media_player.play_media: http://192.168.1.8:8123/local/vad.mp3
- delay: 100ms
- voice_assistant.start_continuous
on_tts_end:
then:
- homeassistant.service:
service: media_player.play_media
data:
entity_id: media_player.m5media
media_content_id: !lambda 'return x;'
media_content_type: music
announce: "true"
- delay: 50ms
- switch.turn_on: use_wake_word
switch:
- platform: template
name: Use wake word
id: use_wake_word
optimistic: true
entity_category: config
on_turn_on:
- lambda: id(va).set_use_wake_word(true);
- delay: 200ms
- voice_assistant.start_continuous
on_turn_off:
- voice_assistant.stop
- lambda: id(va).set_use_wake_word(false);
that is my complete yaml for the test device i used
alright thanks, I have changed some things based on your config. Maybe it will work now.
heres hoping ๐ฌ
you dont use LED in your assistant right?
thats just a test rig, I have only just got round to setting them up for myself to use. I just try different things to help people out
ahh right
i've got quite a few setups for testing with different boards etc .. it's much easier to help if you can replicate the setup
Im still getting these 'Error: stt-no-text-recognized - No text recognized' error messages..
this is my config now
`voice_assistant:
microphone: mic_i2s
id: va
noise_suppression_level: 4
auto_gain: 31dBFS
volume_multiplier: 8.0
use_wake_word: true
speaker: my_speaker
media_player: jarvis_speaker
on_wake_word_detected:
# - switch.turn_on:
# id: led_light
- voice_assistant.stop
- lambda: id(va).set_use_wake_word(false);
- delay: 100ms
- homeassistant.service:
service: media_player.play_media
data:
entity_id: media_player.ultimate_jarvis_esphome_i2s_media_player
media_content_id: !lambda 'return "http://192.168.1.239:8123/media/local/audio/activatedSound.mp3";'
media_content_type: audio/mpeg
- delay: 100ms
- voice_assistant.start_continuous
on_tts_end:
then:
- homeassistant.service:
service: media_player.play_media
data:
entity_id: media_player.ultimate_jarvis_esphome_i2s_media_player
media_content_id: !lambda 'return x;'
media_content_type: music
announce: "true"
- delay: 50ms
- switch.turn_on: use_wake_word
# - switch.turn_off:
# id: led_light`
how long is the sound
as in time duration
try increasing the 2 x 100ms delays to 200ms
and is the sound a sound or a voice?
Its a sound, windows player says its 0:00 secs๐
thats fast ๐คฃ
or should i say short... i mean it plays and stops fast lol
you could to test with just copy and paste my whole config and just change the device name, network and pin numbers and name of mp3 to see if that works.
what are you using for tts and stt >
?
I tried that, but that doesn't work. The audio file will only play with the service call
home assistant cloud
I just changed the delay from 100ms to 200ms, lets see if that is better
same a sme , so that shouldnt cause an issue. it makes no sense that it will play from a service call , as it's calling what your'e doing internally on the device. where abouts in HA is the file? is it in media or just in a folder in www
if you put the url in a broswer does it play the file
im getting a 401: unauthorized error
and the audio file is in media -> my media -> audio -> activatedSound.mp3
its giving me the same error message when I try to acces it via the external nabu casa url
so https
ive just put mine in the www folder so it accessible from /local
hmm will try that for a change xD
hey! that works. It does play the sound when I acces that file via url
a step closer to success
lets hope for some progress
trying it now with your config line
it wont recognize the wake word now ๐ฆ and the logs are constantly filled with these logs
'[15:22:18][D][voice_assistant:521]: Event Type: 0
[15:22:18][E][voice_assistant:651]: Error: wake-stream-failed - Unexpected error during wake-word-detection
[15:22:18][D][voice_assistant:514]: Signaling stop...
[15:22:18][D][voice_assistant:414]: State changed from STREAMING_MICROPHONE to STOP_MICROPHONE
[15:22:18][D][voice_assistant:420]: Desired state set to IDLE
[15:22:18][D][voice_assistant:414]: State changed from STOP_MICROPHONE to STOPPING_MICROPHONE
[15:22:18][D][voice_assistant:521]: Event Type: 2
[15:22:18][D][voice_assistant:611]: Assist Pipeline ended
[15:22:18][D][voice_assistant:414]: State changed from STOPPING_MICROPHONE to IDLE
[15:22:18][D][voice_assistant:514]: Signaling stop...
[15:22:18][D][voice_assistant:414]: State changed from IDLE to START_PIPELINE
[15:22:18][D][voice_assistant:420]: Desired state set to START_MICROPHONE
[15:22:18][D][voice_assistant:521]: Event Type: 1
[15:22:18][D][voice_assistant:524]: Assist Pipeline running
[15:22:18][D][voice_assistant:118]: microphone not running
[15:22:18][D][voice_assistant:202]: Requesting start...
[15:22:18][D][voice_assistant:414]: State changed from START_PIPELINE to STARTING_PIPELINE
[15:22:18][D][voice_assistant:521]: Event Type: 9'
adding the on_error block back now to see if that fixes the spam\
did you unplug after flashing
no?
just unplug and replug if you get errors after flashing , best to have a full reboot
ah right, thanks for the tip
i'm just pluging my test one back in to make sure i wasnt imagining it worked ๐
something is going wrong when this error occurs:
Error: stt-no-text-recognized - No text recognized
because then the pipeline gets reset or something, and it wont listen for the command anymore
this error happens everytime I trigger the wake word ๐ค but not when its just waiting for the wake word to be detected
lol
its because its expecting a sound, mines not working now ๐คฃ thats ridiculous, it keeps just leaving the mic open so i dont need to use wakeword
which is quite good really hahahah
just say turn off kitchen and it does without wake word ๐คฌ
it was perfect this morning
ohh that sucks! It's so annoying that something will work one time and then it doesn't anymore
right.... i need a break from it , and then it will work! it will not beat me! man verses machine!
hahaha we are not giving up indeed!
yoooo it worked for a moment! but then the whole device crashed xD
Guru Meditation Error: Core 1 panic'ed (LoadProhibited). Exception was unhandled.
wow and it worked for a second time as well. but still with the crash lol
So I guess this needs to be fixed by nabu casa
This error means the CPU was trying to load from an illegal address in memory.
This is what I read online
i'll have a look again shortly... need some food and sort a few things
Thats okay, take your time. No worries
it works withoout any messing using an ESP32-S3 and micro_wake_word ๐คฆโโ๏ธ
i was just testing with a different setup. using local wake word (micro_wake_word) it works fine, but you need an esp32-s3 with psram it doesnt work on a normal esp32
Yea im thinking of ordering a board as well soon. Just ordered a raspberry pi zero 2 w with mic HAT to make a satellite.
Does it work with the wake sound as well?
yes, the pi zero does ๐
everything is moving quite quickly with esp's now and a lot of new projects like voice, cameras, media etc need something with psram and the S3n16r8 fits the bill well
Yes it does. very exciting weeks to see all the progress everyone has made with these projects
also esp-idf supported media player might arrive quite soon... but will have to wait and see what happens
I reset my config back to when it was working with only LED on wake word detected. For now at least it works and I'll have a look at it soon again. Maybe someone else will figure it out and I can use that config for the sound issue
Oh really? that could be quite interesting
but I don't know if this as enough pins
yes, with an s3 local wake word and esp-idf media , it is pretty much a match made in heaven... everything should play nicely together
I hope so. Lets hope it will come out soon and we can play with it ๐
i'm already testing it ๐
Next project for me will be setting up the satellites
So it does already work with a normal s3 board then?
those boards i'll have a look at the specs, i've got loads of those c3 ones and they are great for mic only .and sensors... wled and relays etc. .. very low power too ..work really well. but no media support for the c3 . just need to make sure that it is an s3 with PSRAM it needs that additional ram
It does have s3 version as well on the page. Its the second option
just looking and it doesn't give any specs .. hmmm it needs to have psram else it's no good
in the description when you expand it. You can see the specs of the s3 zero. There it does say it has 2MB of PSRAM
https://www.aliexpress.com/item/1005004637451669.html these are the full size boards i use a lot
oh in that case i will order some
These are a better choice I think? How many pins do you need for a working setup? I don't have it at hand at,
atm*
for a voice setup will use 3 for mic 3 for dac and power
just ordered some of the minis i like the size of those boards
haha nice
and you'll need more for speakers as well
i shouldn't really as i already have loads and already another 15 of the big boards on the way ๐คฃ but i can't resist
6 in total + led
Yes I just counted for wires going into the board with the setup I'm using right now
they do have an onboard addressable led wchis is quite bright, so depending on project that can act as the wake , progress light etc
I use those https://nl.aliexpress.com/item/4001313545899.html?spm=a2g0o.order_list.order_list_main.32.7be079d2pAc4lU&gatewayAdapt=glo2nld for led to act as wake light ๐
just a simple positive and negative
Jarvis will take over the house soon xD
oh thats an easy but effective solution ๐
it does work pretty well yes ๐
only thing that is left is this freaking wake word sound.. but maybe it can be done with the new s3 boards
100% works with S3 and local wakeword ... because the pipeline doesnt contstantly run and the wake word is completley seperate, so after wake word detected, you can run whatever you want befor opening the pipeline for voice
ohhhh yes thats true! haven't thought about that. Yes then it should work ๐ Im gonna order some boards now
do you recommend the bigger ones or the smaller?
I need to 4 gpio pins, v5, gnd and 3v3
they will both run it... personally if you have the space in your project i would go for the larger it has 16mb flash and 8mb ram... so if you want to add more complex stuff in the future then you have scope to expand
I have the full iron man helmet to put stuff in, so space enough. I will order some larger boards then
oh cool ... lol , my daughter would love that!
https://www.thingiverse.com/thing:4629346 I printed and painted this one
Will send I picture of it in a bit
boards are ordered ๐
how long did that take to print ?
it does come in pieces because my build volume is just a little too small. So it took around 2,5 - 3 days of printing.
The sanding and painting took over 2 weeks to do lol. That was no fun after 3 days
a labour of love... i don't think i would have the patience .... i only really ever print parts for stuff or square boxes to put things in ๐
This has been my only big project so far as well. But it's so awesome to see it come to life. So already thinking about next projects xD
welcome to the rabbit hole ๐
im thinking of printing the arc reactor and making those into voice satellites. That would be awesome
happy to be in it lol
yeah , it's reall good fun and interesting and great to learn new stuff
what do you think will be shown in the next HA livestream about Grace?