Use Amazon Echo or Google Nest as an Voice Assistant for HA

[gleaming needle]

Hey, is it possible to flash an Amazon Echo or Google Nest with custom firmware to use it as an IO device for HA.
I want to use an NVIDIA Jetson Nano to run an local LLM. Now I need some sort of IO device for voice input and output.
Are there any recommendations out there to use these devices with HA? And I know that there is probably already an easy way using there API from Amazon or Google but I want to use it entirely local.

If anyone already tried that, please give me a hint. Thanks!

[feral pawn]

No, I recommend you get a Voice PE instead

https://www.home-assistant.io/voice-pe/

[gleaming needle]

Is it able to use a local LLM?

Ahh found the specs

Looks interesting

[feral pawn]

yes, you set up a pipeline inside of home assistant using whichever STT, TTS and assist provider you want

[gleaming needle]

How is the audio quality on it?

[shut stratus]

it can eb connected to external speakers via 3.5mm audio jack, so audio quality is as good as you want 🙂

[feral pawn]

for TTS its fine. if you want music you will probably want to add a speaker to it. but it has a 3.5mm line out so thats easy

[shut stratus]

I have mine hooked up to a 2000watt amp 😄

[gleaming needle]

I mean I have external speakers but I rather would like to use it standalone. I only want to use it for voice not for music playback or else

[feral pawn]

i just have a little Anker Soundcore mini and that works nicely

[gleaming needle]

Okay.

[shut stratus]

for voice the internal speaker is decent, tts is clear

I have a few around the house standalone and I have no trouble hearing them clearly from across the room

no distortion or anything like that

[gleaming needle]

👍🏻

[feral pawn]

yeah for just voice its absolutely fine

[shut stratus]

think I have mine at 70% volume and they are pretty loud

[gleaming needle]

Thanks for the recommendation. I think I will buy one and give it a shot

[shut stratus]

could go higher and I think it'd still be fine, but 70 is plenty loud. In some rooms that ar esmaller I have it around 50% because it's too loud then 😄

[gleaming needle]

How about the local LLM running on the Jetson?

[shut stratus]

I haven't tried that, I've tried running Qwen2.57b on my RTX 4050ti and it sort of works for basic combo commands, but anythin more complex I have found it to be unreliable.

[gleaming needle]

I looked up some tutorials on how to set it up. I also want to use the Jetson for other LLMs not just Voice assistant

[feral pawn]

i dont have much experience with that but i assume ollama? then just attach with the ollama integration and you should be fine

[shut stratus]

Yeah ollama is the easiest way to get it working with HA

[gleaming needle]

Want something low power 😅 that why I would use the Jetson

It just draws 8W maximum on load

[shut stratus]

gpu itself is low power draw, but yeah I get ya 😄

I run a k8s cluster with a bunch of things going on anyways, so wasn't really an issue for me haha

[gleaming needle]

👍🏻

[shut stratus]

but GPU power draw itself is pretty low, most times GPU is idle unless it is inferrencing

and even then it's a spike for like 1-2 seconds and then it goes right back down to idle level

With my 4060ti I get around 30 t/s on Qwen2.5-7b. Not sure how the Jetson does

[gleaming needle]

But you don’t get an 4050ti for 250€

[shut stratus]

No, it's a matter of use and performance

the memory bandwidth on the 4060ti is probably higher than the jetson, so it likely gets more t/s. But that's a matter of use case and how long you are willing to wait for responses 🙂

[gleaming needle]

https://nvdam.widen.net/s/zkfqjmtds2/jetson-orin-datasheet-nano-developer-kit-3575392-r2

[shut stratus]

aso GPU allows other fun stuff. For instance I can stream my steam library games to any screen in the house via SUnshine/Moonlight 🙂

[gleaming needle]

I have my Gaming Projector already hooked up to 4K HDMI so yea 😅

[feral pawn]

i kind of want to mess with llms but i cant decide which way to go on hardware

[gleaming needle]

Yea. That why I thing for starters the Jetson is the way to go

[feral pawn]

8gb is a bit limited on the memory side

[gleaming needle]

Depends on what you want to do

[steady verge]

There are, however, esp32 projects that gut an echo and replace it's internals so it still has the same looks.

[gleaming needle]

Yea I have seen this and I thinks it’s also a very interesting project

Don’t know how reliable it is

[steady verge]

But if you want to run a local LLM, you'll definitely need something with more power.

[feral pawn]

the voice-pe has the xmos chip in it which helps alot with audio clarity. so would stick with that. you could build something custom with that kind of thing but its harder

[gleaming needle]

That’s a tomorrow problem 😅 currently I am just searching for the voice communication hardware

[shut stratus]

Yeah 8gb for HA will be rough

for my setup for instance I need a minimum context window of 16k

that with a q8 7b model already goes to like 10GB vram

[feral pawn]

at the risk of sounding like a shill, the voice PE is probably the best performence you are going to get for the money

[steady verge]

For me, the little 13.00 atom Echo works great as a satellite, the speaker sucks, but its a basic esp32 device, so you could, in theory, just rewire the pins to go to an amp and external speaker.

[shut stratus]

it's worth noting, if you just want voice control strictly LLM is not a requirement

[feral pawn]

the atom echo is great for development but not for production

[shut stratus]

you can do it all with intent processing which doesn't require a LLM

[steady verge]

And, in fact, there is a way to actually have your google mini speaker do the responding. I have a video saved in my watch later list of a guy who did just that

[shut stratus]

yeah the Voice PE benefits greatly from the XMOS chip

[gleaming needle]

Yea. I already saw that but I really like the hardware esthetics of the Echo and Nest

[shut stratus]

mine can hear me across the room, with music blasting

atom echo isn't gonna fare too well in that scenario 😅

[gleaming needle]

But not entirely local?

[feral pawn]

the voice PE design is open. you could 3d print a case to make it look however you want 🙂

[steady verge]

I'm sure, but I'm just dipping my toes into that part of HA, so a cheap little mic/speaker box that works is perfectly fine for me ATM. 😁

[gleaming needle]

Yea true

[shut stratus]

also checkout the FutureProofHomes Sattelite1

That one may be more to your liking, though it is still under development and all you can get now are dev units, and they are already sold out I think lol

[steady verge]

Plus, I'm an electronics analyst by trade, so little bitty gadgets that I can easily "make mine" are much more fun for me.

[shut stratus]

could also check out Koala satellite

that one is a DIY variant of the Voice PE more or less

[gleaming needle]

Same

[steady verge]

I just might.

[shut stratus]

https://github.com/formatBCE/Koala-Satellite/tree/main

[steady verge]

Currently sitting at work, waiting for either our UPS contractor to show up to finish yearly inspection, or for something to break. 🤣

[shut stratus]

https://futureproofhomes.net/ if you are interested in the Sat1

these two and Voice PE are basically SOTA for open local voice assist satellites at this point I think

[steady verge]

works for the county, at the local jail I, along with the rest of the team, typically keep the fire alarm, CCTV, and locking controls working.

[gleaming needle]

Thanks for all your suggestions and the support I will check all of these projects to see which one is the best for me

[shut stratus]

Also to figure out VRAM requirements for LLMs this is a handy tool: https://huggingface.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator

[gleaming needle]

THX

Great community!

[pine island]

there's not ,much point to the echo when you can use assist from a phone, watch and other devices. Deconstructing the VPE and changing the form factor is a better project. As others have pointed out its the mic array that makes the difference. I have the echo. In addition to reskining the VPE to not white, there are a few 3D designs around that add a better speaker in a large case . Unfortuenetly all VPE case mods are a long way from the design elegance of a homepod mini and similar. I will probably print black cases for the VPE as that change will hide it a bit. I do think the LEDs are well done.

[neon geyser]

Instead of reusing VPE guts, you could take Respeaker Lite Voice Kit. Same S3 (well, it's N8R8 instead of N16R8, but it's completely fine), same XU316 module, 2 mics, and even same DAC. 🙂 Firmware is ready, and it will be like 3 times cheaper.

[pine island]

Respeaker may be a better project base. Does it use the voice firmare?. But I have five Thinksmart Views that I need to get working with voice. The Thinksmarts are upscale products and still cost less than the VPE. But they are a significant project to get up and running. At least the first one. I think everyone beyond HA beginner level who is looking for a project would find the TSV as good as it gets. Anyone who has voice working on TSV and also own a VPE want to give a comparitive review?

[neon geyser]

Sorry, I have no experience with TSV.
On your question about Respeaker - it is using VPE firmware with slight changes.

[bleak iron]

Respeaker lite is an awesome platform
Can recommend

[gleaming needle]

How about using VPE and install it in an Google Nest or Echo Dot?

You can get an Echo Dot on EBay for around 10€

Then I would ripp out the original pcb and replace it with VPE

Probably some additional hardware is required (like amp and/or mic)

Esthetic wise I don’t think the VPE is ugly but I would prefer the design of the Echo Dot or Google Nest over the VPE.

[steady verge]

There's a project for that somewhere.

[steady verge]

https://discord.com/channels/429907082951524364/1325602879133712415

[gleaming needle]

I can not access the link