#Translation/aliases for areas

1 messages · Page 1 of 1 (latest)

versed lily
#

My areas are named in Swedish but as the language support for sv is very lacking, I run assist in english atm. Is there no way to translate areas, or give them aliases?

Otherwise I would need to rename all areas to english which would have other implications.

mellow anvil
#

You can add aliases to areas, no problem. Settings > Areas, labels & Zones > Areas (default tab) > edit an area > Aliases section

versed lily
#

Oh, nice. Thanks. Confusing that they are not exposed like entities.

woeful monolith
mellow anvil
mellow anvil
woeful monolith
#

Ah sorry. They will work for default stuff, the problem I had is that Hassil isn't returning area in slot, but exact alias that was said - and there's no way to use that in intent_script because there's no "get area by alias" method.

mellow anvil
#

that's a different story and more of a tinkerer's issue than a common use case

woeful monolith
versed lily
# mellow anvil Areas are not entities. I'm not sure what you would like to see

Overall the assist config is a bit all over the place. I guess that is what it comes down to. Setup is far from straight forward for getting it to be useful, imo, after having the PE for two days.

I expected it to be lacking in Swedish, but I haven't got it to understand a single command yet, even simple things like turning on lights. So I'm forced to use english at the moment which requires me to set aliases for almost every device.

So far the experience has been quite a let down tbh. But I hope it will get better in the future.

mellow anvil
versed lily
mellow anvil
#

is it any good? does it properly transcribe what you are saying? my best guess is no, which is why your experience is hindered

versed lily
#

I don't know? But I guess not. What should I use instead?

image.png
mellow anvil
woeful monolith
mellow anvil
#

you can use a larger Whisper model with larger beam size, but that's going to need a GPU

#

well... it won't need a GPU, but your patience probably will 😅

versed lily
woeful monolith
#

You can add alias "case that's office" 🙂

versed lily
#

My wife has a name that is quite hard to pronunce correctly in english. Are there any... workarounds for that?

#

Besides a nickname

mellow anvil
mellow anvil
versed lily
#

Is the tiny-int8 = wisper_faster?

mellow anvil
#

ouch, that's small! it's bound to suck, unfortunately

#

set up a 1mo trial for NC cloud and see if you get much better results with the cloud STT

versed lily
#

Just an assumption, since it states it here... and whisper_faster isnt listed in the dropdown

image.png
#

I could give my vm more cpu, but I would need to know the starting point to beef it up

#

How do I know which one is used?

mellow anvil
#

you select the engine in the pipeline definition and the model in the engine settings. I understand you're running it as an addon (which is ok), so you select the model in the addon settings

versed lily
#

Ok, thanks! But can I know which model is currently used when it is set to auto?

mellow anvil
mellow anvil
#

i have no reason to believe anything else is used

fervent burrow
#

you could probably check the addon logs and see if it says what model it loaded or is using

versed lily
#

I assumed it would adapt to the hardware, from the definition of auto

fervent burrow
#

but as tetele said, if you are running larger models on just CPU, expect to wait several seconds before it even convers your speech to text

versed lily
fervent burrow
#

if the response times are presently fast, I'd think it is running tiny or small

#

and yeah, you'd need to run wyoming faster-whisper in a separate VM, in Docker, with GPU exposed for best speeds/performance

#

or k8s if that's your thing 😁

versed lily
#

Nice. So you're pretty much bound to use a gpu or nb cloud to get anything useful out of it. Doesn't exactly say on the box.

mellow anvil
#

not just NB cloud. there are other cloud providers with (custom) integrations

versed lily
#

As in OpenAI and such?

mellow anvil
mellow anvil
versed lily
#

Not very local though 😦

mellow anvil
mellow anvil
#

https://community.home-assistant.io/t/use-google-cloud-speech-to-text-in-home-assistant/570113

Home Assistant Community
Use Google Cloud Speech-to-Text in Home Assistant

The recently announced voice stuff in 2023.5 are pretty neat, and of course both the local (whisper) and cloud speech-to-text are awesome. But the more choices we have the better, so I made an integration that allows to use Google Cloud Speech-to-Text in HA. It’s pretty fast, supports a ton of languages and can be included in an assist pipeline...

#

STT is resource-intensive. I've said this many times before: you need to pay for it one way or the other. Your options are:

  • with time (i.e. slow transcription)
  • with hardware costs (i.e. get a GPU and run Whisper)
  • with subscription/usage fees (i.e. use someone else's hardware and pay a fee, but also lose out on privacy)
versed lily
#

thanks

#

I assume this would be enough? Characters = characters in a stt sentence?

image.png
#

Not sure if all this is worth it to replace the google homes though 😄

mellow anvil
mellow anvil
versed lily
#

I mean, going through all this setup and still sending the data to google. I dont see the point really, when their hardware is also better. I'd probably be better off contributing to the sv translations then.

#

What are you looking for a in a gpu for running whisper? vram?

mellow anvil
#

if you're gonna use it just for Whisper, make sure it has CUDA cores (i.e. is nvidia). you'll never cap out the VRAM with Whisper alone

mellow anvil
#

NC Cloud, for example, uses Azure as the STT provider, but all users go to the same Azure account (for NC). it's anonymity by numbers in that case 😄

#

however, if you're not keen on supporting HA development, it's much more expensive than a simple STT service subscription

fervent burrow
#

for TTS/STT you don't need crazy hardware. a decent nvidia GPU with 8GB of vram would be plenty

#

Think people run it on used GTX 1070 GPUs and get responses in under a second

#

I am using a 4060ti and my stt takes about 250ms

#

and that's running large-v3-turbo in my case

#

pretty sure you can run the large model on a 1070 and get responses in under a second tbh

mellow anvil
#

I've tried that on a Tesla P4 and i could, so yeah, it works on a 1070 as well

versed lily
#

I might actually buy one, I've been considering it for video transcoding for a while anyway. Need something that fits a optiplex sff though.

#

Is there documentation for running whisper in a separate vm for HA?

mellow anvil
versed lily
#

Anything will do, I just want to have a grasp of how complicated it would be before diving in

#

Or could I pass through a gpu to HA and benefit from that?

mellow anvil
#

how are you running things?

versed lily
#

proxmox, HAOS in a VM

#

and the wisper addon, as you stated

mellow anvil
#

i have an LXC running docker on my proxmox host (yeah, i know, but whatever works). I have passed the GPU to that LXC so docker can use it. it's then a matter of installing drivers and finding a docker image capable of using CUDA cores

#

installing the Linux drivers for nvidia was marginally complicated, but required on any system you choose. they had to be installed on both the host and the LXC. keeping the drivers in sync between the 2 is a bit of a bitch

versed lily
#

don't think I'll have the time for the maintenance required tbh 😦

mellow anvil
#

well... it's not mandatory

versed lily
#

so, there's no way to utilize a gpu from within HA?

#

as in, passing it through HA and simply selecting a larger model. Or something like that.

fervent burrow
#

no, HA doesn't utilize GPU hardware in addons or anything like that

#

an alternate approach you can do is just spin up another VM in proxmox and pass the GPU to that. For instance I have a k8s cluster, and I just have a VM that is my worker node with the gpu passed to it

#

then any containers I run on that worker have GPU access

#

so you can run multiple docker containers, like whisper, piper, a transcode server like Plex

#

I have all those running and sharing the GPU

#

I even have a sunshine container that runs steam so I can stream games to my TV/Phone 😁

versed lily
#

Yeah, but that would still require the driver maintenance etc I suppose?

mellow anvil
mellow anvil
fervent burrow
#

Yeah LXC has the caveat you mentioned, where you have to install the driver at the host and LXC level

versed lily
#

I just care for it to work 😂

fervent burrow
#

VM you just maintain it on that one VM, so a little bit easier maintenance 😄

#

but LXC has the benefit that multiple LXCs could use the same GPU

versed lily
#

How do you tell HA to use the external wisper/piper vms? Can you direct it to an ip in the integration or something?

#

Or do you need to tinker with the esp code perhaps

mellow anvil
#

yes. you add an instance of the Wyoming integration and it asks for host and port

fervent burrow
#

yup

versed lily
#

Ah, great

fervent burrow
#

that's how mine is set up

versed lily
#

Well I might start to hunt for a gpu then. And probably find out that I need a new host from that. It will be an expensive voice assistant 😛

fervent burrow
mellow anvil
fervent burrow
#

yeah but if you are serious about privacy, getting the good hardware pays off 🙂

#

I got a powerhouse system, and everything is local, even the LLM. And performance in terms of speed is on-par with Google/Alexa IMO.

versed lily
#

I'm serious about the relation between caring about privacy and not be forced to sleep on the couch for the next month

#

😉

fervent burrow
#

with the LLM you can issue more complex commands and it understands quite well, at least in my experience 🙂

versed lily
#

Will a 1070 be enough to run a decent llm as welll? Or do you need beefier stuff

mellow anvil
mellow anvil
versed lily
#

3060? I'm not sure what to look for really, except I know that vram is good

#

But thats probably another discussion isnt it 😛

#

Anyway, thanks for the help. It's certiantly not plug and play yet, thats my conclusion, for my language anyway

fervent burrow
#

with LLM I'd personally try to shoot for at least 16GB vram

mellow anvil
fervent burrow
#

if you are planning to do LLM+STT+TTS

#

and expect decent LLM performance 😅

versed lily
#

I'll probably leave llms to another year lol

fervent burrow
#

fair enough. Good to consider though if you are gonna make the investment, in terms of future-proofing 🙂

mellow anvil
#

yeah, a 1070 is dirt cheap and will get you 80% there in terms of STT/TTS

fervent burrow
#

Definitely, if you only want TTS/STT a 1070 will give great performance for the value

versed lily
#

Im pretty sure it wont fit in my case. Do you know about the rx6400? Or do you need nvidia for these things.

fervent burrow
#

3060 of course will be even better

mellow anvil
fervent burrow
#

need CUDA, so has to be nvidia

versed lily
#

A new host it is...

mellow anvil
#

there are low profile nvidia cards, though

#

also PCIe accessories which allow you to mount GPUs outside enclosures or at an angle

versed lily
#

Not sure height is so much an issue as overall space tbh, I'm gonna open it up and have a look

mellow anvil
#

just don't be an idiot like i was and buy a server GPU (i.e. the Tesla P4) and then realize it has passive cooling

fervent burrow
#

ouch yeah 😄

#

though if you have the space in your case, people have made printable shrouds for those to inject cooling

mellow anvil
#

that's what i started doing

#

...after buying another case 😅

#

which was the limiting factor for the size in the first place

versed lily
#

Do you recon a gtx 1650 would be enough?

#

for sff + tts

mellow anvil
#

most likely

#

get one with as much VRAM as you can

versed lily
#

4gb for the one I'm looking at

mellow anvil
#

but it will be worse than the 1070, though

fervent burrow
#

yeah 4gb won't be enough for decent models

mellow anvil
#

6-8GB should do

fervent burrow
#

at least I don't *think

#

yeah, think the large model for whisper uses something like 4.5-5gb

versed lily
#

Nice

fervent burrow
#

but that runs a full k8s cluster. I do alot of research stuff for work 😅

versed lily
#

I seem to at least be able to run base-int8 instead of tiny without problems

#

is small-int8 more capable than base? Or what's the heirarchy

fervent burrow
#

so I checked on my server,

#

evidently the large-v3-turbo model is only using 1.866GB of VRAM

versed lily
#

that sounds promising, maybe ill buy a 1650 and give it a go, its only 70 euros used

mellow anvil
#

i wish i could provide more insight, but i haven't played with the GPU for a while (mostly because i couldn't be bothered to print a cooling shroud)

versed lily
#

This is the small model running

INFO:faster_whisper:Processing audio with duration 00:03.750
INFO:wyoming_faster_whisper.handler: What's the weather like?
INFO:faster_whisper:Processing audio with duration 00:03.590
INFO:wyoming_faster_whisper.handler: Turn off the lights
#

Decent I think?

mellow anvil
#

decency is subjective in this case. it is what it is. if it feels slow to you, then it's slow. otherwise it's decent

#

it's by no means fast, i can tell you that 😅

fervent burrow
#

this is the performance I have with large-v3-turbo:

image.png
#

that's before the model is primed, so first time talking to it

mellow anvil
fervent burrow
#

465ms

#

the audio duration is how long the speech audio is

#

so 3 seconds of audio converted to text in 465ms

versed lily
#

Now my PE flashed red when trying to wake it, what does that mean?

fervent burrow
#

the primed request comes in at 219ms

mellow anvil
mellow anvil
#

you'll need to check logs (prefferably ESPHome logs) to understand what's wrong

versed lily
#

Now Whisper is stuck in a reboot loop with [16:46:53] INFO: Service exited with code 256 (by signal 9) 🙄

fervent burrow
#

might want to turn on debug logging to see why it is crashing

#

but if you are using a larger model size, it uses more ram, is the system running out of memory maybe?

versed lily
#

But I'll try a full ha reboot

# 
Trying to resume download...```
#

Things are not going great 😄

versed lily
#

Up and running again, the cpu spiked from trying a too big model

#

Could one use a TPU instead of a gpu for local stt?

fervent burrow
#

nope, TPU doesn't have the capacity to handle those kinds of models :/