wintry wing
I know this has been asked before (even by me, I think!) but what is the current way to determine which satellite is active? I want to be able to select the appropriate media player (etc) based upon the satellite that was triggered.
I read that it should be part of the "homeassistant_satellite_event" but mine doesn't seem to fire with Assist PE hardware. I'm using standard firmware on the PE, with the openai extended conversation integration.
Any pointers would be very much appreciated!
balmy smelt
what makes a satellite "active" in your opinion? because technically speaking, they're all active 😅
more to the point, can you better describe what you want to achieve?
wintry wing
Yep - good point - I wasn't clear. I want to know which satellite heard the wake word and is being used to process the response. This would then allow me to determine the room, and which devices in that room could be targeted for actions.
balmy smelt
that's already possible. how are you handling the intents? custom intents or automations?
for custom intents, there's a slot called area you can use
for sentence triggers, use trigger.user_input.device_id and then get the area of that device_id
sweet flower
You can do this to be exact {{ area_name(trigger.entity_id) }}
wintry wing
I'm using the openai extended conversation integration, with funtions (as this seems the only way to get the level of performance I need). So I need the "active" satellite device_id to be accessible for the functions in that integration. I'm not hugely familiar with custom intents so is "area" and "trigger.user_input.device_id" only available to the custom intent?
balmy smelt
oh, in that case it should/might work out of the box, although you can't guarantee any results, as the LLM decides what it needs to do. just to make sure, your satellites have areas assigned to them, right?
wintry wing
Hmmm. I don't think it does. I have a test area with an assist PE and a single light. When I say "turn off the light" it asks me which one.... but, interestingly, it does work sometimes. Not consistently though.
misty otter
Hmmm. I don't think it does. I have a test area with an assist PE and a single...
But did you add your satellite to the area?
spiral tree
how is the function defined?
usually the description needs to instruct the LLM how to use the tool, including how to determine area
for instance something like "Use the area as specified by the user, or if the user does not explicitly state an area, then use the area the request came from"
wintry wing
But did you add your satellite to the area?
yes, that's the assist PE device.
wintry wing
for instance something like "Use the area as specified by the user, or if the us...
Ah, right. I'll give that a go now. Thanks
spiral tree
Hopefully it helps, but a bit hard to figure out what you need without the context of what you are trying to make. If it doesn't work, may need to post the yaml of what you are trying to do so we can understand where the hiccup is 🙂
wintry wing
So it turned off the light ok. When I asked to turn it back on it replied with "theres no light to turn on". I tried a second time and it worked.
Do you want to see the extended conversation code I'm using? It's largely a cut and paste of other examples but would welcome any comments.
spiral tree
hmm odd, the light turn off/on by area is built into HA's conversation agents, both OpenAI and Ollama
We could look, I don't personally use extended conversation so I imagine it may have stuff not typical to the core integrations 😅
spiral tree
Could be the model doing something funky
wintry wing
I'm using gpt-4o, not sure how you debug that or if it's even possible?
spiral tree
so I tried this locally using the stock OpenAI core integration, and it understood the area I am in without adding anything special to my prompt
wintry wing
Without going off topic - I'm using extended conversation because I went to openai mainly because I wanted better selecting of tracks in music assistant. I then found music assistant didn't support UK BBC radio stations, so I used extended conversation to allow these to be handled with functions. Maybe I'm going down a rabbit hole here but it's part of the fun learning curve....!
spiral tree
granted, I said to turn on the lights, not light
Oh, we added an entire voice-support repo with LLMs for MA, should go give that a look
but
the core integrations also allow using tool/function calling 🙂
wintry wing
TBH, adding your suggestion didn't seem to make it any better! It was already a bit hit and miss without that instruction.
spiral tree
you can expose scripts to assist, and the LLM will use them as tools
https://github.com/music-assistant/voice-support for more info if you like
wintry wing
ok - interesting. This is where I feel that I'm still quite a novice and probably exploring the wrong paths to get the job done.
I'll look at the blueprints - thanks!
Is it better to use MA from the repo or is the built in ok for voice now? My understanding was the built-in was initially lacking?
spiral tree
are you referring to HACS?
the above is designed to work with the core integration, the HACS one has actually been discontinued and removed from HACS
wide sky
I have a terrible solution to this that I use in my fork of extended_openai_conversation, written in Jinja:
{%- for area in areas() %}
{%- for device in area_devices(area) %}
{%- if not device_attr(device, "disabled_by") and not device_attr(device, "entry_type") and device_attr(device, "name") %}
{%- for entity in device_entities(device) %}
{%- set entity_domain = entity.split('.')[0] %}
{%- if not is_state(entity,'unavailable') and not is_state(entity,'unknown') and not is_state(entity,"None") and not is_hidden_entity(entity) %}
{%- if entity_domain == "assist_satellite" and is_state(entity, 'processing') %}
* The user is in the {{area}} area. If an action is required but there is no information on the entity, area, or floor, use the {{area}} area ID.
{%- endif %}
{%- endif %}
{%- endfor %}
{%- endif %}
{%- endfor %}
{%- endfor %}
it is of course nightmare fuel, but it does work...
it has the side effect of having multiple areas defined that way if two satellites are active at once. i don't know of any way to fix it. if you do, please let me know!
i do think most people should just switch to the official integration, but sometimes it's not feasible
wintry wing
the above is designed to work with the core integration, the HACS one has actual...
yes, the HACS one. Sounds like I should "upgrade"?
wintry wing
This is a new one for me. The blueprint (option 3) failed with the following error:
I'm trying to find an error in the logs. Any ideas?
spiral tree
Never seen that before, would check HA logs and see what they say.
civic drift
I tried the Option 3 blueprint but somehow its Not starting to Play on my ReSpeaker Lite. The Script itself says it wasnt Run once as well. I can Play music on my ReSpeaker If i select IT via Music Assist UI.
wintry wing
The script issue is possibly a more generic issue with HA, if this link is correct: https://github.com/home-assistant/core/issues/134967
limpid aspen
I've seen that error as well, but every time I got it, the script/automation was added and worked
wintry wing
I've not used blueprints or scripts before. It is expected that the blueprint adds its code to scripts.yaml?
ok - I "fixed" it by deleting my scripts.yaml and adding:
script: !include scripts.yaml
to my configuration.yaml. Not sure which one (or both?) changes fixed it but my script is now created. OpenAI correctly responded but my track didn't play on the mediaplayer - that's the next thing to debug....
wintry wing
Any idea why this is failing? It seems like it can't determine a valid player? Initially I left the default player empty but, even when I added a default, it seems unable to find a target?
wintry wing
ok- getting closer! I realised that I needed to add a room to the media player created by MA.
@limpid aspen , I think you're the author of the script so maybe you can assist with the error. I tried "play songs by Queen" and a list of tracks looked like they were selected but the response I get is "I am unable to play songs by Queen at the moment, please try other songs..." and the error in the trace is "recommendations not found".
limpid aspen
This error comes from Music Assistant.
action: music_assistant.play_media
target:
area_id
- music
data:
media_id:
- Bohemian Rhapsody
- Don't stop me now
- We will Rock you
- Somebody to love
media_type: track
artist: Queen
radio_mode: true
This is what the script tried to perform, you'll probably get the same error if you try that in developer tools > actions
wintry wing
Yep - correct - it fails there too 😦
Looks like setting radio mode to "Always" causes the problem. I've changed to "use player settings" and this seems to be working...
limpid aspen
Looks like setting radio mode to "Always" causes the problem. I've changed to "u...
In the last version that's also the default setting
limpid aspen
This is a new one for me. The blueprint (option 3) failed with the following er...
Are you using local Tuya? I just read a post somewhere that this was causing this error
wintry wing
I am using local tuya. That's a strange side effect?