dark lion Dec 1, 2023, 10:38 AM

#

esp32:
  board: esp32dev
  framework:
    type: arduino

logger:
api:
ota:

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

i2s_audio:
  i2s_lrclk_pin: GPIO33
  i2s_bclk_pin: GPIO19

microphone:
  - platform: i2s_audio
    id: mic
    i2s_din_pin: GPIO23
    adc_type: external
    pdm: true

media_player:
  - platform: i2s_audio
    id: media_out
    name: None
    dac_type: external
    i2s_dout_pin: GPIO22
    mode: mono

voice_assistant:
  microphone: mic
  media_player: media_out
  use_wake_word: true
  on_tts_end:
    - media_player.play_media: !lambda return x;
  on_end:
    - delay: 100ms
    - wait_until:
        not:
          media_player.is_playing: media_out
  on_client_connected:
        then:
          - voice_assistant.start_continuous:
  on_client_disconnected:
        then:
          - voice_assistant.stop:

gritty bane Dec 1, 2023, 4:08 PM

#

Gotta turn the wake word on/off and hold the button for a little while for some reason or another, but it works continously after that

dark lion Dec 1, 2023, 8:08 PM

#

strange i've removed the button from the config and it works fine.

#

and the wakeword switch

civic garden Dec 1, 2023, 8:48 PM

#

Just been trying to play with this, but so far can't get it to work. If I use the following code, I can get the mic and speaker to work no problem, but of course only as a satellite (so no media player).

#

esphome:
name: noahs-sensor
friendly_name: noahs-sensor
on_boot:
- priority: -100
then:
- wait_until: api.connected
- delay: 1s
- if:
condition:
switch.is_on: use_wake_word
then:
- voice_assistant.start_continuous:

esp32:
board: esp32dev
framework:
type: arduino
version: recommended

#

VOICE ASSISTANT

i2s_audio:
i2s_lrclk_pin: GPIO27
i2s_bclk_pin: GPIO26

microphone:

platform: i2s_audio
id: mic
adc_type: external
i2s_din_pin: GPIO13
pdm: false

speaker:

platform: i2s_audio
id: big_speaker
dac_type: external
i2s_dout_pin: GPIO25
mode: mono

voice_assistant:
microphone: mic
use_wake_word: false
noise_suppression_level: 2
auto_gain: 31dBFS
volume_multiplier: 5.0
speaker: big_speaker
id: assist
on_listening:
- light.turn_on:
id: led
on_end:
- light.turn_off:
id: led
switch:

platform: template
name: Use wake word
id: use_wake_word
optimistic: true
restore_mode: RESTORE_DEFAULT_ON
entity_category: config
on_turn_on:
- lambda: id(assist).set_use_wake_word(true);
- if:
  condition:
  not:
  - voice_assistant.is_running
  then:
  - voice_assistant.start_continuous
  on_turn_off:
- voice_assistant.stop
- lambda: id(assist).set_use_wake_word(false);
platform: restart
name: "Noah's mic Restart"

interval:

interval: 1s
then:
- if:
  condition:
  api.connected:
  then:
  - if:
  condition:
  and:
  - switch.is_on: use_wake_word
  - not:
  - voice_assistant.is_running
  then:
  - voice_assistant.start_continuous:

Light

light:

platform: binary
id: led
name: "Noah's sensor LED"
output: light_output

output:

id: light_output
platform: ledc
pin: GPIO23

#

If I try and change it to the following, I can't seem to get any response from Assist, or via the media player entity (tried a TTS cloud service call, with no response). Anything obvious you can see, that I'm doing wrong?

#

esp32:
board: esp32dev
framework:
type: arduino

i2s_audio:
i2s_lrclk_pin: GPIO27
i2s_bclk_pin: GPIO26

microphone:

platform: i2s_audio
id: mic
i2s_din_pin: GPIO13
adc_type: external
pdm: true

media_player:

platform: i2s_audio
id: media_out
name: None
dac_type: external
i2s_dout_pin: GPIO25
mode: mono

voice_assistant:
microphone: mic
media_player: media_out
use_wake_word: true
on_listening:
- light.turn_on:
id: led
on_tts_end:
- media_player.play_media: !lambda return x;
on_end:
- light.turn_off:
id: led
- delay: 100ms
- wait_until:
not:
media_player.is_playing: media_out
on_client_connected:
then:
- voice_assistant.start_continuous:
on_client_disconnected:
then:
- voice_assistant.stop:switch:

platform: template
name: Use wake word
id: use_wake_word
optimistic: true
restore_mode: RESTORE_DEFAULT_ON
entity_category: config
on_turn_on:
- lambda: id(assist).set_use_wake_word(true);
- if:
  condition:
  not:
  - voice_assistant.is_running
  then:
  - voice_assistant.start_continuous
  on_turn_off:
- voice_assistant.stop
- lambda: id(assist).set_use_wake_word(false);
platform: restart
name: "Noah's mic Restart"

interval:

interval: 1s
then:
- if:
  condition:
  api.connected:
  then:
  - if:
  condition:
  and:
  - switch.is_on: use_wake_word
  - not:
  - voice_assistant.is_running
  then:
  - voice_assistant.start_continuous:

Light

light:

platform: binary
id: led
name: "Noah's sensor LED"
output: light_output

output:

id: light_output
platform: ledc
pin: GPIO23

civic garden Dec 1, 2023, 9:10 PM

#

I should also add that this is on an ESP32 device 🙂

gritty bane Dec 1, 2023, 9:28 PM

#

dark lion strange i've removed the button from the config and it works fine.

So are you able to reference it as a media player and use a wake word?

gritty bane Dec 1, 2023, 9:30 PM

#

civic garden I should also add that this is on an ESP32 device 🙂

Do the lights change as if it is trying to speak? Piper doesn't work on the media_player component , but HA Cloud does

dark lion Dec 1, 2023, 9:31 PM

#

piper works with 2023.12 of HA

gritty bane Dec 1, 2023, 9:32 PM

#

dark lion piper works with 2023.12 of HA

This is true. Sorry, I'm not on the beta versions yet. I would be if it were just the voice assistant/assist side

civic garden Dec 1, 2023, 9:32 PM

#

gritty bane Do the lights change as if it is trying to speak? Piper doesn't work on the **me...

So far, I've not had any response from it, when trying to use media_player. For now, I've just set it up without, so I can use the satellite, but of course would love to be able to control the volume, and use the satellite for TTS cloud service calls

gritty bane Dec 1, 2023, 9:34 PM

#

What esp board are you using?

dark lion Dec 1, 2023, 9:34 PM

#

i'll be about in a few mins :can go through it

gritty bane Dec 1, 2023, 9:35 PM

#

civic garden So far, I've not had any response from it, when trying to use media_player. For ...

Mine looks like this https://imgur.com/a/ouy5GOk

civic garden Dec 1, 2023, 9:36 PM

#

Using a cheap ESP32 from the German Amazon (AZ delivery vroom ESP32)

civic garden Dec 1, 2023, 9:37 PM

#

gritty bane Mine looks like this https://imgur.com/a/ouy5GOk

Yeah I get basically the same on my device, but no response when trying to talk to assist, or when trying to make a TTS service call

gritty bane Dec 1, 2023, 9:39 PM

#

Hmm

#

I'll have to mess with it again later

dark lion Dec 1, 2023, 9:39 PM

#

@celondb what hardware do you have? esp32 + max98357? and what mic?

gritty bane Dec 1, 2023, 9:40 PM

#

Can you try https://github.com/Gtt1229/firmware/blob/dev8/media-player/m5stack-atom-echo-va.yaml instead

#

I tried changing things on dev9

#

but I may have broke it

dark lion Dec 1, 2023, 9:41 PM

#

gritty bane Can you try https://github.com/Gtt1229/firmware/blob/dev8/media-player/m5stack-a...

that doesnt use wakeword though

gritty bane Dec 1, 2023, 9:41 PM

#

It do

#

  - platform: template
    name: Use wake word
    id: use_wake_word
    optimistic: true
    restore_mode: RESTORE_DEFAULT_ON
    entity_category: config
    on_turn_on:
      - lambda: id(va).set_use_wake_word(true);
      - if:
          condition:
            not:
              - voice_assistant.is_running
          then:
            - voice_assistant.start_continuous
      - script.execute: reset_led
    on_turn_off:
      - voice_assistant.stop
      - lambda: id(va).set_use_wake_word(false);
      - script.execute: reset_led
  - platform: template
    name: Use Listen Light
    id: use_listen_light
    optimistic: true
    restore_mode: RESTORE_DEFAULT_ON
    entity_category: config
    on_turn_on:
      - script.execute: reset_led
    on_turn_off:
      - script.execute: reset_led

#

Dev branch 8

#

I have a bunch of dev branches because of the way ESP home looks up

dark lion Dec 1, 2023, 9:41 PM

#

gritty bane It do

sorry , ignore me 🙂 i was thinking of different link

gritty bane Dec 1, 2023, 9:42 PM

#

Yeaaa, too many links!

dark lion Dec 1, 2023, 9:42 PM

#

isn't there just and half of them don't work.

gritty bane Dec 1, 2023, 9:43 PM

#

I take that personally!

dark lion Dec 1, 2023, 9:43 PM

#

you shouldn't as it wasn't aimed at you

civic garden Dec 2, 2023, 5:40 AM

#

dark lion @celondb what hardware do you have? esp32 + max98357? and what mic?

Hi, yes using ESP32 + max98357 + INMP441

dark lion Dec 2, 2023, 5:41 AM

#

and it's working ok as just a speaker?

dark lion Dec 2, 2023, 6:08 AM

#

if that is the case then that says A. wiring is ok B. pipeline is setup ok 👍 so I can't see any reason why the media player config shouldn't work. It's just a case of starting with the basic working config and building it up from there. So forgetting switches, lights, buttons etc for now. can you try the following.

Make a note of the device name: and friendly_name: if you have 1 set.
from esphome dashboard , locate the card for your device and click on the 3 dot menu and then 'download yaml' and save that file. This will give us your config safe so if need be can swap back.
open the device card and select all (ctrl + a ) then delete. so that you have an empty config. and paste the code that I post at the end.
change the name and friendly_name to your names. don't make any other changes, I have already added the pins that you are using. click save but DON't install.
go to ESPHome integration in HA devices & services and delete the device from the integration.
go back to ESPHome dashboard, open the card and click 'install' and then wirelessly (assuming the device is powered on and connected to wifi 🙂 )
HA should automatically detect the device and ask you to configure, click configure and re-add it to the integration.
unplug the device and plug it back in. Go back to ESPHome dashboard open the device card and click 'logs' and 'wirelessly' watch the logs and you should see a block of entries repeat every 5 seconds. Now test! say your wakeword and a command. You should see the logs change and hopefully it will work 🤞
let me know how it goes. Apologise if steps above are 'stating the obvious' but i thought it just better to be clear so hopefully it gets it working 👍 let me know how it goes. CODE BELOW

#

esphome:
  name: voice_assistant
  friendly_name: Voice Assistant
  
esp32:
  board: esp32dev
  framework:
    type: arduino

logger:
api:
ota:

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

i2s_audio:
  i2s_lrclk_pin: GPIO27
  i2s_bclk_pin: GPIO26

microphone:
  - platform: i2s_audio
    id: mic
    i2s_din_pin: GPIO13
    adc_type: external
    pdm: true

media_player:
  - platform: i2s_audio
    id: media_out
    name: None
    dac_type: external
    i2s_dout_pin: GPIO25
    mode: mono

voice_assistant:
  microphone: mic
  media_player: media_out
  use_wake_word: true
  on_tts_end:
    - media_player.play_media: !lambda return x;
  on_end:
    - delay: 100ms
    - wait_until:
        not:
          media_player.is_playing: media_out
  on_client_connected:
        then:
          - voice_assistant.start_continuous:
  on_client_disconnected:
        then:
          - voice_assistant.stop:

dark lion Dec 2, 2023, 6:42 AM

#

i have just tested now with an esp32 wroom32 and MAX98357 and inmp441 and all is working great

dark lion Dec 2, 2023, 7:15 AM

#

and proof of concept https://youtube.com/shorts/yjE_eCndbHI?feature=share

civic garden Dec 2, 2023, 11:19 AM

#

dark lion and it's working ok as just a speaker?

I haven't tried it as just a speaker yet, but it works fine as a satellite wired as follows: https://imgur.com/B4oqgWU

civic garden Dec 2, 2023, 11:20 AM

#

dark lion if that is the case then that says A. wiring is ok B. pipeline is setup ok 👍 s...

Thanks very much for the guide, I'll go through it asap 🙂

dark lion Dec 2, 2023, 11:20 AM

#

civic garden I haven't tried it as just a speaker yet, but it works fine as a satellite wired...

yes, sorry i meant that it can hear you through the mic and outputs responses through the speaker

civic garden Dec 2, 2023, 11:20 AM

#

dark lion yes, sorry i meant that it can hear you through the mic and outputs responses th...

Yes, works no problem (I have another running just like this in the office, and it works great). Just need to be able to control the volume, and use it as a media player, so this one I'm playing with now is just a test set up

dark lion Dec 2, 2023, 11:22 AM

#

thats good then at least the wiring and pin numbers etc are all correct. 👍 just drop a message in here if you get stuck with anything going through the setup. i'm usually around at all hours 😄

civic garden Dec 2, 2023, 11:34 AM

#

And just to double check, is it OK for me to use the same pins (BCLK/SCK & LRCLK/WS) for the amp and mic, as shown in the picture I linked to? I assume it is judging by your code 🙂

dark lion Dec 2, 2023, 11:50 AM

#

yes thats how it needs to be and just a seperate data pin 👍

#

the only thing i can see that may need to be added is a wire between gnd and LR on the mic as this defines whether the mic outputs to the left or right cannel as the i2s component expects a left or right. however if it is working now i would leave it as it is (guessing this is from the everythingeverywhere tutorial ) the normal would be to have it like below . but as it works I would leave it.

#

oh it wont let me post a pic

#

but basically if a wire is between LR and gnd on the mic it is in left channel mode , and if wire is from LR to VCC then it is in right channel mode. but i wouldnt worry for now

civic garden Dec 2, 2023, 2:52 PM

#

dark lion yes thats how it needs to be and just a seperate data pin 👍

Hi again, so I've followed your instructions to the letter, but so far I still can't get it to work. I used the exact code provided above, where the only changes were the name at the top, plus I added the ota password. I can see the pipeline entities, media player entity etc in the newly added device, but the wakeword doesn't respond. In the ESPhome logs, I can see a block entries repeat. I'll post my exact code below first, followed by a section from the logs 🙂

#

esphome:
  name: noahs-sensor
  friendly_name: noahs-sensor
  
esp32:
  board: esp32dev
  framework:
    type: arduino

logger:
api:
ota:
  password: "xxx"

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

i2s_audio:
  i2s_lrclk_pin: GPIO27
  i2s_bclk_pin: GPIO26

microphone:
  - platform: i2s_audio
    id: mic
    i2s_din_pin: GPIO13
    adc_type: external
    pdm: true

media_player:
  - platform: i2s_audio
    id: media_out
    name: None
    dac_type: external
    i2s_dout_pin: GPIO25
    mode: mono

voice_assistant:
  microphone: mic
  media_player: media_out
  use_wake_word: true
  on_tts_end:
    - media_player.play_media: !lambda return x;
  on_end:
    - delay: 100ms
    - wait_until:
        not:
          media_player.is_playing: media_out
  on_client_connected:
        then:
          - voice_assistant.start_continuous:
  on_client_disconnected:
        then:
          - voice_assistant.stop:

dark lion Dec 2, 2023, 2:56 PM

#

can you try adding
channel: left in place of pdm: true in the microphone: section

#

if that makes no change then change it from left to right i've had it where different espboard have required the opposite of what it should be ... just to make it even more confusing 🤦‍♂️

civic garden Dec 2, 2023, 3:04 PM

#

Sorry, just had to swap over to my phone. Making that change marks the mic text in red...

gritty bane Dec 4, 2023, 5:31 PM

#

@civic garden Your code doesn't have a wakeword switch, so the voice_assistant script doesn't run.

#

Something like:

  - id: turn_on_wake_word
    then:
      - if:
          condition:
            and:
              - switch.is_on: use_wake_word
          then:
            - lambda: id(va).set_use_wake_word(true);
            - if:
                condition:
                  not:
                    - voice_assistant.is_running
                then:
                  - voice_assistant.start_continuous
            - script.execute: reset_led
  - id: turn_off_wake_word
    then:
      - voice_assistant.stop
      - lambda: id(va).set_use_wake_word(false);
      - script.execute: reset_led
      
  - id: reset_led
    then:
      - if:
          condition:
            - switch.is_on: use_wake_word
            - switch.is_on: use_listen_light
          then:
            - light.turn_on:
                id: led
                red: 100%
                green: 89%
                blue: 71%
                brightness: 60%
                effect: none
          else:
            - light.turn_off: led
      
switch:
  - platform: template
    name: Use Wake Word
    id: use_wake_word
    optimistic: true
    restore_mode: RESTORE_DEFAULT_ON
    on_turn_on:
      - script.execute: turn_on_wake_word
    on_turn_off:
      - script.execute: turn_off_wake_word
      
  - platform: template
    name: Use Listen Light
    id: use_listen_light
    optimistic: true
    restore_mode: RESTORE_DEFAULT_ON
    entity_category: config
    on_turn_on:
      - script.execute: reset_led
    on_turn_off:
      - script.execute: reset_led```

dark lion Dec 5, 2023, 4:47 AM

#

gritty bane Something like: ```script: - id: turn_on_wake_word then: - if: ...

it is using

 on_client_connected:
        then:
          - voice_assistant.start_continuous:
  on_client_disconnected:
        then:
          - voice_assistant.stop:

so a switch isn't required

#Wakeword - media player and esp32 config

VOICE ASSISTANT

Light

Light