JARVIS assistant

[native laurel]

hey guys I'm working on a real life JARVIS and I've heard about home assistant integration but do not know how to use it

also im a bit of an ironman nerd

and if you want to kow more about my jarvis ask me here

[west junco]

I can ask my Karin about a ambulance passing by where its going.

[native laurel]

huh?

[haughty anchor]

hallo o/ im also working on a similar voice assistant to help me manage my day to day activities, what tts are you using for ur va?

[native laurel]

whisper

[haughty anchor]

bcus i cant find a good one

[haughty anchor]

i see

is it free?

[native laurel]

faster-whisper is free and offline

wanna see a snippet of my code?

[haughty anchor]

sure!

also how would u rate the speech generated?

[west junco]

home assistant and friends even has a hey jarvice wake words pre installed.

[haughty anchor]

i have everything set in my own code, i just want a nice tts so i can finish it 😅

im using openwakework for the wake word managment

[west junco]

Here is a short shippet of my assistant

[native laurel]

disclaimer about faster-whisper it takes a i bit longer than some stt plugins only a little bit longer though

also your snippet is coming

[haughty anchor]

oh tht sounds pretty good

[haughty anchor]

np take your time :D

[native laurel]

ef record_audio(duration=5, samplerate=16000):
print("🎤 Recording...")
recording = sd.rec(int(duration * samplerate), samplerate=samplerate, channels=1, dtype='int16')
sd.wait()
path = tempfile.NamedTemporaryFile(suffix=".wav", delete=False).name
write(path, samplerate, recording)
return path

def offline_speech_to_text(duration=5):
try:
path = record_audio(duration)
print("🤖 Transcribing...")
model = WhisperModel("base", compute_type="int8") # Use "tiny" for faster performance
segments, _ = model.transcribe(path)
return " ".join(segment.text for segment in segments).strip()
except Exception as e:
return f"[Offline STT Error] {e}"

i thinks that's it

[haughty anchor]

:0

il check it out! tysm, im new to all this so this is very exciting :))

[west junco]

if you pick home assistant as the basis for your jarvis the possiblities are almost endless

but you have to make a lot of scripts your self for now

[native laurel]

i have a full version of jarvis designed to work on a pc and a version designed to go on a raspberry pi

[native laurel]

also same

ive just gotten into it the past few wekkss and picking up a lot from freinds

[haughty anchor]

samee thts my end goal as well

[west junco]

whisper on a pi isnt that fast

[native laurel]

guys wann see my rasberry pi version of jarvis in action becasue my full version has a lot of my personal info on it

[native laurel]

yeah sadly

[west junco]

But you can run whisper on a remote system.

[haughty anchor]

also i think this stt? i was asking abt text-to-speech 😅

[haughty anchor]

i wld love to

[west junco]

I have a script that makes a local dynamic dns to pick the fastest whisper available.

[native laurel]

ohhh than idk i just use pyxtss or what ever it's called

[native laurel]

ok coming in hot

sending soon

[haughty anchor]

ah i was also using that, but for me recently it kept stopping in the middle of the sentence and i have no idea how to fix it, so im looking for something more natural and free x)

[haughty anchor]

ayy, excited

[native laurel]

ahh dang it it's too long of a video i might be able to go into a vc? i will be defeaned tho

and show you what he can do

[native laurel]

idk man

well ok i will send a video but it will be a bit short ok

[haughty anchor]

np np

also for ur hardware setup, are u running just a pi (5/4), or are do you have anything else connected to it?

[native laurel]

i dont have a pi yet

im getting one soon and plus i cant seend you a video becasue its still too long

[west junco]

cant you compress it?

[native laurel]

true

i try that then

[west junco]

if you recorded it on a phone there are good free working apps

or else ask a AI for the linux compress command

[native laurel]

ummm i will have to give it too you in snippets but it will be only like 3

wanna see the code for your self?

@haughty anchor

@west junco

[haughty anchor]

np

[haughty anchor]

sure

also i wont be able to respond for a while cus im gonna go study for my exams xD

also

i found a tts called coqui tts

imma try it out and let uk

[native laurel]

good for you

[native laurel]

yo @haughty anchor

what should i add to jarvis now?

[dire chasm]

gonna need an explanation on this one lol

[haughty anchor]

Yeah i had this too

Also i might train a voice model for Jarvis with coqui tts

Il start doing that once my exams get over

[haughty anchor]

Have u implemented a music player yet?

[west junco]

https://github.com/Bram-diederik/home-assistant-LLM-scripts

[fierce hamlet]

Hi, "hey Jarvis" doesn't work on my devices (Voice PE) only Ok Nabu works, I didn't finde a solution to get this to work, does anyone have an idea what to do?

[west junco]

Its in the openwakeword setup open wakeword and use that

[autumn glade]

Hey Jarvis is built into Microwakeword. Also using PE with Openwakeword requires a lot of changes to YAML, and results in losing some functionality (like STOP word).

[fierce hamlet]

I thought it's not possible to change the wakewords on voice PE you just have "ok Nabu" "hey microft" and "hey jarvis" which are fix, am I wrong?

[autumn glade]

So far there's three official words for MWW, yes.
Well, in theory you can train your own wake word for Microwakeword (also for Openwakeword too, and it's easier). There's notebook available for training. But for good results, you'd have to get real voice samples, and big positive/negative set. For example, Ok Nabu is the only wake word for Microwakeword, that was trained on big synthetic dataset plus ~20000 real samples, donated by community.
Same goes with Openwakeword, but training and tuning models for it is a bit easier. However, official PE firmware doesn't work with Openwakeword (which is on HA side), and uses Microwakeword (which is on-device ESPHome algorithm).

"Hey Jarvis" works for some people better than "Ok Nabu". It heavily depends on voice height, stress, accent and pronunciation. Also because of synthetic dataset, it doesn't work good with children/women voices..

[fierce hamlet]

Ok but the I need to change the configurations files of the Voice PE right, I know how to train wakewords, I already did this but I‘m not sure if I really want to this with Voice PE, as I often realized problems when an ESP update arrived.

[karmic thorn]

i'm working on an iteration where i'll re-introduce OpenWakeWord (or anything compatible) wake word detection, but that's for experimenting only. I think MicroWakeWord is superior.

[autumn glade]

Well, MWW works only for ESP32. OWW is universal, although server based... Each has advantages and disadvantages.

[autumn glade]

Yes, you'd need to take control and make changes to the YAML.