#esp-idf media player and micro_wake_word on the seeed studio esp32 s3 xiao, 8MB PSRAM.
496 messages ยท Page 1 of 1 (latest)
no i didnt create the component ๐ that was the fantastic work of @somber field
How did you get it to react so fast?
Oh sorry I didn't know ๐
use a good power supply and be close to your router
And what config are you runing on HA and what Hardvare?
Though its not really that fast
Same speed as my google home that i want to repleace ๐
I'm running it on proxmox as a virtual machine and for the hardware its an old i5 2500k with 2 cores and 8GB of ram
Oh so alexa is faster than google home
C3 has one single core and one single i2s bus so its better to get the s3
also need PSRAM ๐
But for microphone and repro you ar using only one I2s
Shout by speed diferent for C3 vs S3? With voice?
No two
the xiao s3 on the docs they say that its got one single i2s but but its working with two now
I think esphome uses only one anyway
From the datasheet: ESP32-S3 includes two standard I2S interfaces.
esphome uses whatever you tell it to. ๐
no I mean in the seeed studio docs of this specific board
Can I pulse the one color on board led?
Ok, but that just means the docs are wrong.
Yes
the core esp32-s3 chip has 2 x i2s i would assume that the board would have this chip
But maybe there are gpio pins for the second i2s bus that they are not using because the board is very tiny
You can use almost any pins for it.
most s3's have multiuse pins
All, not most. ๐ Whatever the module is, it's still the same S3 core.
true that ... i just didn't want to commit ๐คฃ
When this custom component will be natively in esphome?
still a work in progress and the answer at the moment is your guess is as good as mine ๐ there are updates and discussion here, so worth keeping an eye on it https://discord.com/channels/429907082951524364/1214417455133237319
Hmmm what is this magic. So Looking for pinou esp32 s3 xiao
And i see that only 4,5 are i2C pins...
On that mini board.
why not
i2c as well can use most pins.
Ok.
i2s_audio:
- id: i2s_in
i2s_lrclk_pin: 9 #WS
i2s_bclk_pin: 8 #SCK - id: i2s_out
i2s_lrclk_pin: 2 #LRC
i2s_bclk_pin: 3 #BCLK
I i transfer this to mi C3 it will be working?
I don't think so
No because the C3 has only one
I've already tested
Why do you say that?
i2c is pin specific?
It's not.
Its not the same as i2s
uart, i2c, i2s, ledc, etc. can use most pins.
But on all the pinouts they specify the i2c pins but not the i2s ones
Things like the DAC are fixed.
They do. They might be the default pins, but it doesn't matter.
On the original ESP32, if you use the default pins, you can get higher speeds, but for normal use, it doesn't make any difference.
Oh
e.g. for the big displays, you probably want to use the default SPI pins, because you can get the maximum speed. (assuming your wiring can handle it)
But that's like 40 (80?) MHz.
Yes
But if you're just using a gpio expander, you can use whatever you want for SPI.
c3 doesn't support media_player: either. it works ok as just a mic using wakeword via the pipeline . but you cant run microwakeword or media player
What can I use as led platform other than the binary so I can control the brightness?
Ok so i must buy new S3 from somwhere to get this working right.
Whole voice assistant is new for mee. Get in working with C3 but with poor performance.
does it have an addressable led?
ledc
I don't know if it supports pwm
If it's an LED, it does. ๐
yep sorry ๐
single color is unlikely to be addressable.
what's the set_frequency
Whatever you want. I think the default is 1000, which is more than enough.
So I can add pulse effect to the ledc?
You need to use a monochromatic light in front of it, then you can add effects.
that's what I've done
try clean build
Its the pin I think its not analog
It will take forever
just click the 3 dot menu on the device card and click 'clean build files' only takes a few seconds
No the compiling will take forever ๐๐
You need a better system to compile on. ๐
Its just the tensorflow things its too big
The speaker doesn't work on battery
The amp needs 5v
I didn't know this even exists
it's what i used in my build
No you used the zero but this is the super mini
it got more gpios and a bettery recharge module
there is no rgb led
Only single color I think linked to gpio 48
battery charging sounds like a bonus ๐
Yes also the xiao have one but voice assistant doesn't seems to work on battery anyways
Or it works but with a smaller speaker
what battery have you tried?
3.7v rechargable
you can see it on the video
ah yes i see , have you tried the DAC on 3v3 ?
Yes it works when plugged only
have you got a link to that supermini... i need some ๐
Its a ali-express link
cool, i just found them ๐ they must be newly released as couldn't find any a couple of weeks ago
Yes also I didn't found any docs for them
interesting little board ๐ i'll order a couple to play with
can't see any mention os PSRAM ๐ค
Ill search
These are the smallest ESP32-S3 development boards with an amazing 34 IO pins available and LiPo battery management built in.
Only 23.5mm x 18mm
A dual-core XTensa LX7 MCU
capable of running at 240MHz
512 KB of internal SRAM, it also comes with
integrated 2.4 GHz, 802.11 b/g/n Wi-Fi and
Bluetooth 5 (LE)
Type-C USB connector
Shipping list:
1x ESP32-S3 SuperMini
2x male pins
here is the description of the same board on ebay
It doesn't seems to have psram but it has 512kB of ram
i've just messaged the seller to ask ๐
good
i'll let you know if they reply
The RAM is the same for all S3 modules.
you don't need google... you just need a ssieb ๐ better than any Ai
Here is the same on ebay
All the ESP chips of the same type have the same amount of SRAM. It's part of the core chip.
Auto replay integrated in discord
But why with the xiao s3 they don't even talk about sram
On-chip 8M PSRAM & 8MB Flash
Only this
just look at the espressif docs for the core s3 , manufactures arent the greatest at producing specs/docs they either miss stuff out or make it up ๐
Because it's a constant, so it doesn't need to be specified.
When someone want to buy it he will not look into the espressif doc he even didn't know that its an s3 if its a beginner
Then that's his problem. It says S3 and if he doesn't know what he's looking at, he should be doing a bit more research first...
Only because I've just learned this about the XIAO ESP32-C3: it might have battery over-charge protection, but possibly no over-discharge protection
Not certain if there is a schematic to refer to so as to infer support aside from try it and see.
they replied saying 'yes' and that is all they said, no details ... i ordered some, so we will see when it arrives ๐
Testing is the best way to know
I didn't tried the battery charging but batteries are always bad ๐
I was potentially mistaken with regards to the XIAO ESP32-C3 battery module not supporting overdischarge protection. It depends on the battery and while it seems a higher voltage like 3v seems like a common recommended minimum for longevity, I've not yet observed discharge below 2.1v or so.
They recommand a 3.7v lithium battery so I'm using that
arrived
Nice can't wait to know if it has psram ๐
To use the extra pins we need these I think and a custom pcb
i'll just solder onto them ๐
Its so tiny how would you solder
I need this I think because my robot project need more pins
If it works with esphome as a voice assistant it will defenitly work with my project
that's enough I think
I found these they could help you
The only pinout
Octal ?
quad
Its better?
bit slower spi transfer , but probably not noticeable ... i will try it with VA later
Go ahead and make the first doc page for this board ๐
Technically, it can store 1min of audio file
that would be great to see... we can store image files so there is hope
Yes
You also have 512kb sram
Charging current selection jumper: Connect this jumper "BOOST", the charging current increases from 100ma to 300ma. Only do this if your battery is larger than 500mAh to avoid explosion.
The majority of 3.7v batteries are more than 500mAh
In the pinout picture its written in chinese
I'm not english but I speak well
Also the rgb led is adressable
It has one pin
48
๐
Tell me if it works well with va so I buy one
will do , will be this evening... mrs bobba has me doing chores ๐ ๏ธ
School is time wasting
they've already made a breakout for this board
not on this build, i did on the last one i did. but i just get this other occasional warning.
i'll have a look at the logs on the other, i've not checked recently
yes still getting those
plz do .. I'm using your https://github.com/bigbobbas/esphome_audio adf + i2s audio external componenets + the nice cue tone in mp3 and my feeling is that there is something there that load the cpu so it throws audio sample and impact detection rate
try changing to https://github.com/gnumpi/esphome_audio as that will be more up to date and mine is only a fork I made when i first tested but nothing has been altered in there ... so better to use the original source
i'm just updating the one with the same errors as it's not on the latest release , i'll see how that goes
changed url and now I'm getting the same warning as you
but no samples drop and better detection ...
ive just updated to latest release on that device and it is now without those errors.
what is this speaker? how do you control it?
i would like to make alarm.. with time and to play a song/radio
possible?๐ค
Niceee is it working better than the zero ?
#865158094514290698 message
They will introduce the C6
The xiao will be much smaller
๐ง
What i am dooing wrong?
it's relating to the esp32-voice-spania.yaml i don't think that is valid for an external component. it'sa yaml file not a component
external_components:
- source: my_components
- source:
type: git
url: https://github.com/gnumpi/esphome_audio
ref: main
components: [ adf_pipeline, i2s_audio ]
refresh: 0s
I just copiing pastebin from this post ๐
you just need this part
external_components:
- source:
type: git
url: https://github.com/gnumpi/esphome_audio
ref: main
components: [ adf_pipeline, i2s_audio ]
Yeeh great more problems ๐
well it's failing on another external component and the one I linked doesn't have that component. but hey ...was just offering an option. @fickle bough will know what that component is
Wich one
- my_components i think it was
external_components:
- source: my_components
- source:
type: git
url: https://github.com/gnumpi/esphome_audio
ref: main
components: [ adf_pipeline, i2s_audio ]
refresh: 0s
I used your code
oh ok ... ๐
Yes its the adf pipeline
@covert fiber can you paste your actual config, rather than a screen shot. use the format i'll paste below
if it complains it's too many lines, click upload as a file.
To format your text as code, enter three backticks on the first line, press Enter for a new line, paste your code, press Enter again for another new line, and lastly three more backticks.
```yaml
example: here
```
Don't forget you can edit your post rather than repeatedly posting the same thing.
remove that line - source: my_components
Then i get this...
Unable to find action with the name 'micro_wake_word.start'.
And i just deleted: - source: my_components
so you need to change this part
my_micro_wake_word:
model: okay_nabu
probability_cutoff: 70%
on_wake_word_detected:
then:
- logger.log:
format: "I heard : %s"
args: ['wake_word.c_str()']
level: INFO
- voice_assistant.start:
to
micro_wake_word:
model: okay_nabu
probability_cutoff: 70%
on_wake_word_detected:
then:
- logger.log:
format: "I heard : %s"
args: ['wake_word.c_str()']
level: INFO
- voice_assistant.start:
OK i get it. Compiling ๐
Great ๐
my_,micro_wake_word
Becouse of my_component.
That stupid ๐ But i get it now.
So if i get it now in whole. I shoud not need to have instaled open wake word on HA side?
This runs it localy?
correct
it's pretty cool for sure ๐
[15:10:19][D][light:036]: 'LED' Setting:
[15:10:19][D][light:047]: State: ON
[15:10:20][D][light:036]: 'LED' Setting:
[15:10:20][D][light:047]: State: OFF
[15:10:20][W][micro_wake_word:157]: Wake word is already running
[15:10:21][D][light:036]: 'LED' Setting:
Perfect. I just need to get in on some temporary board. ๐ To test.
Thank you @tall merlin
your welcome, any issues ... make a thread in #voice-assistant ๐
One more thing ๐ Has anybody make like a Oled expansion for this? To doo some animation or show the output of sensor or something on display?
i did a little .96" display on my lamp build here https://discord.com/channels/429907082951524364/1210369185612562512 which could easily be expanded to do more, i have a lvgl display board ready too ... which i plan to turn into something voice related
the one on the lamp, just shows a smiley whne wake word heard and a thumbs up when command complete
Ohhh OK Perfect that exactly what i Want ๐
Som visual representation ๐
I will look into this ๐ Thanks ๐
adf_pipeline:
-
platform: i2s_audio
type: sink
id: adf_i2s_out
i2s_audio_id: i2s_out
i2s_dout_pin: GPIO4 -
platform: i2s_audio
type: source
id: adf_i2s_in
i2s_audio_id: i2s_in
i2s_din_pin: GPIO7
channel: left
sample_rate: 16000
bits_per_sample: 16bit
Is thare like a conection to where?
Like here?
i2s_audio:
- id: i2s_in
i2s_lrclk_pin: 9 #WS
i2s_bclk_pin: 8 #SCK - id: i2s_out
i2s_lrclk_pin: 2 #LRC
i2s_bclk_pin: 3 #BCLK
nope .. just as it is in the original code
Hmm i going to chceck mi wires becouse its shits on my big time ๐
Red light goes out. Ant than i ask anything and esp ist just "dead"
No respond anything ๐
yoou need to watch the esphome logs to see whats happening - what esp board is it ?
SeedStudio
XIAO-ESP32=S3
I dont get anything in logs.
Thats interesting.
That is all ๐
And after some time:
ARNING esp32-voice-spalna @ 192.168.201.36: Connection error occurred: [Errno 104] Connection reset by peer
INFO Processing unexpected disconnect from ESPHome API for esp32-voice-spalna @ 192.168.201.36
WARNING Disconnected from API
INFO Successfully connected to esp32-voice-spalna @ 192.168.201.36 in 0.005s
INFO Successful handshake with esp32-voice-spalna @ 192.168.201.36 in 0.079s
it should change when you say the wake word, if not then something is not right with your wiring or pin numbers probs, what mic and dac do you have ?
do you have LR connected to gnd on the mic?
Yeah
9 - WS
8 - SCK
7 - SD
2 - LRC
3 - BCLK
4 - I dont know is to connect do DIN od SD?
I have conected to SD.
yes so i2s_din_pin: goes to sd on mic
Yup wiering is OK.
[16:47:51][D][api:102]: Accepted 192.168.201.2
[16:47:51][D][api.connection:1159]: Home Assistant 2024.3.0 (192.168.201.2): Connected successfully
And after that i get nothing
That strange.
I try new microphone. And its same.
yes, it will just sit and do nothing until it hears a wakeword
just to double check , the mic WS do youo have that to the lrclk pin?
Yep pin 9
Continuity OK.
Code same
- id: i2s_in
i2s_lrclk_pin: 9 #WS
i2s_bclk_pin: 8 #SCK-
id: i2s_out
i2s_lrclk_pin: 2 #LRC
i2s_bclk_pin: 3 #BCLKi2s_dout_pin: GPIO4 # SD
i2s_din_pin: GPIO7 # SD
-
That i habe pluged in ๐
LR to GND
Mic is
9,8,7
WS, SCK, SD
ok the i2s_dout is to the din pin on the max98357 ... sorry i thought you meant the mic previously. but i don't think that would effect the mic
check the top of mic pins on gnd and vcc/vdd that you have 3v
3.296
yup thats perfect
try changing model to hey_jarvis ... and see if 'hey jarvis' works as ww
can you upload config again with the formatting
To format your text as code, enter three backticks on the first line, press Enter for a new line, paste your code, press Enter again for another new line, and lastly three more backticks.
```yaml
example: here
```
Don't forget you can edit your post rather than repeatedly posting the same thing.
are you using the D numbers or GPIOnumbers on the board ?
Oh noooo
I am stupid.
Who the F... Move GPIO apart from D pins...
Soooos stupid.
yes , it is somewhat silly! if they need to label them ..at least make the numbers the same
Ok its finaly can recognize mi voice ๐ Thats stupid...
yayyy well, thats another thing you have learned today... if things just worked you wouldn't find it interesting ๐
Yeeh no problem with that ๐
But i use C3 normaly and my Pinout is same on board and in ESPHome ๐ So i just good dumfonuded...
๐
Thanks for you time ๐
you're welcome ๐ glad youve got it going
@fickle bough dou you have any problems running this code?
I experiencing problem after reboot.
Wich code
Yours ๐
It doesn't work on c3
Yeh i know ๐
I have Seed Studio XIAO-ESP32-S3
And it runs this code no problem. Until i reboot it.
And it the same code expect that "My custom component"
So something is sketchy @fickle bough
What happend when you reboot it?
It is online. I can ping it. But it has high latenci 100+ms
And its not recognized by homeassistant
When i do reflash it will come online. It works fine. Until i reboot it. Or disconect power and conect back.
I can see it online in ESPhome. But it does not work. I can read logs and everithing. But no response for wake word. No aditional output to log noting....
Its like a zombee @fickle bough
๐
this didn't happend with me, did you plug the antenna?
Yeeh ๐ I have ๐
Before reboot everithing works. ๐ With clean install.
After reboot nothing works ๐
What is that - source: my_components all about? @fickle bough
Can this be a problem when i deleted it? As @bigbobba sugested?
Wait how to go to the top to see the code url ๐
I remember I've copied bigbobba code I didn't wrote anything myself except the my_micro_wake_word
So @tall merlin we are missing something ๐
@fickle bough do you have that my_components anywher where i can download it?
i'm at a loss what it could be.... i'm thinking perhaps something with the router not liking something but ๐คทโโ๏ธ
I got 7 other devices that are working perfect.
Mi network is bulet proff ๐
On the other S3 i runn mi rack controler with PDI fan PWM Leds and temp monitoring just fine.
This Voice assistant has been giving me hard pass for 2 mounths...
No because the first time it works
And thats true.
can you show me the peice of code where is my components written
external_components:
- source: my_components
- source:
type: git
url: https://github.com/gnumpi/esphome_audio
ref: main
components: [ adf_pipeline, i2s_audio ]
refresh: 0s
And than you have this asociated with it:
my_micro_wake_word:
model: okay_nabu
probability_cutoff: 70%
on_wake_word_detected:
then:
- logger.log:
format: "I heard : %s"
args: ['wake_word.c_str()']
level: INFO
- voice_assistant.start:
Wait
This is directly from your: https://pastebin.com/raw/scxA4L6Z
You should remove the
- source: my_components
line
And modify the
my_micro_wake_word
to
micro_wake_word
Yeah. That what i done.
And removed my_ from my_micro_wake_word:
An that my curent state.
But it is not working after reboot.
But now they merged it into the original source
that's weird
I do install (With ota or direct) and it works. Until i reboot it.
Eeeee ๐
You are running on usb?
Yeh
plugged into your pc?
I try conecting to PC and Powerbank.
Same
What about power plug
Also mine if I reboot it smth weird happend like I can't lower the speaker volume until I make the first conversation
Nope
The yellow
Nope.
And if you flash it it blinks?
No I mean when it boots
you are giving it a static ip?
And when you flash it does it blinks just after the flash is finished?
I dont know.
But what i just discover.
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
ap:
ssid: "Esp32-Voice-Spalna"
password: "qwnRl8la9Qxf"
power_save_mode: none
output_power: 8.5db
manual_ip:
static_ip: 192.168.201.35
gateway: 192.168.201.1
subnet: 255.255.255.0
dns1: 192.168.201.1
I have statick IP.
After flash it listen to it.
But after reboot it gets IP from DHCP.
Sor totaly ignoring this config.
It could be a problem from your network configuration
Why?
When i give it statick IP in config?
Something that BigBobba sugested in proces of testing.
Oh so it changed nothing
yesp
Try remove the static ip
From the config and from your ruoter
If the led doesn't blink in the beginning it means the the mww didn't started and the wifi never connected
and if you click the reset button the led never comes on?
Nope.
Oh
that's could be a hardware problem
because if it connected the led should blink
Even on_boot it should blink
do you have another board?
Yes I see but the led don't blink
But wifi is conected.
the led on_boot is commented out, did you uncomment after testing ? @covert fiber
Yeh i habe it uncommented.
on_boot and on_connect
esphome:
name: esp32-voice-spalna
friendly_name: esp32-voice-spalna
platformio_options:
board_build.flash_mode: dio
on_boot:
- priority: -100
then:
- wait_until: api.connected
- light.turn_on: ledd
- delay: 250ms
- light.turn_off: ledd
- delay: 1s
- light.turn_on: ledd
- delay: 2s
- light.turn_off: ledd
esp32:
board: esp32-s3-devkitc-1
variant: esp32s3
flash_size: 8MB
framework:
type: esp-idf
version: recommended
sdkconfig_options:
CONFIG_ESP32_S3_BOX_BOARD: "y"
psram:
mode: octal
speed: 80MHz
# Enable logging
logger:
# Enable Home Assistant API
api:
encryption:
key: "bS3h+cUR8IzTDoIEnN8ncnQYdfPjJAf/ghEG7xTmuTA="
# on_client_connected:
# then:
# - delay: 50ms
# - light.turn_on: ledd
# - delay: 2s
# - light.turn_off: ledd
# - micro_wake_word.start:
#on_client_disconnected:
# then:
# - voice_assistant.stop:
ota:
password: "5d59a5486ef286df5fa459232389122b"
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
ap:
ssid: "Esp32-Voice-Spalna"
password: "qwnRl8la9Qxf"
power_save_mode: none
#on_connect:
# then:
# - micro_wake_word.start:
# - light.turn_on: ledd
# - delay: 250ms
# - light.turn_off: ledd
# - delay: 500ms
# - light.turn_on: ledd
# - delay: 250ms
# - light.turn_off: ledd
daaam sorry
but like this?
on_boot:
- light.turn_on: ledd
- delay: 1s
- light.turn_off: ledd
- delay: 1s
- light.turn_on: ledd
- delay: 1s
- light.turn_off: ledd
Replace the circles with -
Bruh I hate discord
To format your text as code, enter three backticks on the first line, press Enter for a new line, paste your code, press Enter again for another new line, and lastly three more backticks.
```yaml
example: here
```
Don't forget you can edit your post rather than repeatedly posting the same thing.
No led after reboot
Oh
you reboot it by clicking the reset button or unplug and replug?
This is the first boot after the flash
yes
Yes
SW reboot with button in console.
Try click the button
Nothing. No led.
that's a hardware problem
Try flashing a simple code in arduino ide
With wifi
and see if it do the same thing
I have to go sleep
Shut the door.
I hate this stuf...
Its working.
After flashing it with that default blinking led...
I just cant...
@fickle bough @tall merlin
I flash it with that blinking stuff...
And back to your config. And it is working now...
I just hate this ghost broken stuff.
Maybe clean build files in the future when you find ghosts?
Ah usually needs 11 times ๐
Nice
@tall merlin I'm working on the gpio expander
Working on putting everything inside the box
nice! ๐
What to do in this case
use flux and wick it away with braid and then re-tin
Hope it still works ๐
add solder to the wire first and trim so only short exposed wire is showing. less risk of getting any shorts.
Is it normal that the red led on the board turns off after certain time ?
Good sign
Compiling micro wake word ๐๐
It works perfectly the first time but when I say the wake word the second time it detect it but never do the speech to text and blocks
It changes to "UNINITIALIZED" state and then never respond
Can you share your config? Depending on the components you are using it might be related to the issue I am trying to resolve right nowโฆ
I will check when I am back home, but what you can check for now is using the dev-next branch instead of main:
ref: dev-next
Ok ill try that
Oh its working now thankyou
now I just need to order a new amp because I broke it somehow
I ordered 5 amps just in case ๐
Hi there all... and sorry if I reopen thisn thread.
I just want to make this VA using the ESP32 S3 and the source from: https://github.com/gnumpi/esphome_audio/blob/main/examples/esp32-s3-N16R8-adf.yaml
But I can't understand the diagram in order to link the pins.
I have used the pin of the project here: https://smarthomecircle.com/How-I-created-my-voice-assistant-with-on-device-wake-word-using-home-assistant and it's work.
Now I just want to refresh the source with the last news (I think) ...
So is there any picture or maybe a youtube video in order to understand the pin mapping please ?
Thanks in advance
@ivory fjord the pin numbers on the s3 are labelled 1, 2,3 etc... the pins are......
MIC:
SD >> 4
WS >> 5
SCK >> 6
L/R >>gnd
VDD >> 3v
GND >>gnd
DAC:
LRC>>46
SCLK>>9
DIN>>10
GAIN>> NOT CONNECTED
SD>>NOT CONNECTED
GND>>GND
Vin>> 5V
Thanks a lot for the quick replay... I just turn at home in order to try it ๐
It' work !!! ๐
Thanks a lot again
@tall merlin can you create integrations?