Looking for any information on how this can be done.
This video makes it seem possible...
https://www.youtube.com/watch?v=XvbVePuP7NY
#Triggering Automations/Scripts with ollama conversation agents
pastel rose
fluid nova
Why automations? Automations are automated, not launched by request. What you want it scripts.
That you can do with LLM. Expose script to Assist, and give it good name and description, like "this tool is used to get my to-do list". LLM will be able to use that tool, and gather information from it.
pastel rose
I'm open to ideas for sure. Just saw it done with automations.
So, lets say I have a script: "play 90's music"
I want it to pull in the media_player entity from the voice command and send that into the script, I can then expose the script, and say:
"play 90's music in the living room" and it should hopefully play on the media_player.living_room?
Trying it... can't get it to run the script ...
If you have any more specific information about how to achieve what you suggested... I'm all ears.
I have created a script with a name: "play music in den"
I exposed it to assist
I gave it a description: "This tool should be used to play music in the den"
I added to the llm prompt: "When asked to play music, execute the appropriate script to do so."
When I ask it to "play music in the den" it says "I'll play music using the available scripts" and then does nothing. or... "there seems to be an issue".
Yes... the script runs if I manually execute it. 🤷♂️
fluid nova
Probably not enough context.
tough thistle
I'd ask the model what tool it tried to use and what arguments it gave it. Use the llm this debug itself 😁
Odds are it's having trouble figuring out how to call the tool, or is using a tool call format that isn't quite jiving with home Assistant
Each model can have it's own way of calling tools. I think HA initially tested using llama 3.1 on ollama, so models that follow that same methodology would probably fare best.
tough thistle
Hmm checked the leaderboard as I haven't looked in a while, guess qwen2.5 is scoring above llama3.2 at the moment on synthetic benchmarks. So I'd ask it what it did, might need some more tweaking to the description or prompt.
tough thistle
I just tried qwen2.5, even put it on the 14b model, and when I asked for the weather it replied in what I think is sanscrit 😄
Ah, google says it is Thai
pseudo current
I just tried qwen2.5, even put it on the 14b model, and when I asked for the wea...
Yes, you need to provide a prompt for it, such as: please reply in English or we can communicate in the same language. But it cannot be ruled out that language hallucinations may occur. It can reduce language hallucinations.
tough thistle
Ah makes sense
pastel rose
Triggering Automations/Scripts with ollama conversation agents
strong ingot
I created this script:
Anyone knows why my LLM can't caall it?
I see this in the ollama logs:
on, saloon, salon, Saloon\n attributes:\n unit_of_measurement: °C\n device_class: temperature\n- names: sensor salon temperature Humidity\n domain: sensor\n state: '22.02'\n areas: Salon, saloon, salon, Saloon\n attributes:\n unit_of_measurement: '%'\n device_class: humidity\n- names: plug hallway tablet\n domain: switch\n state: 'off'\n areas: Hallway\n- names: plug closet fan\n domain: switch\n state: 'off'\n areas: Closet\n- names: cover salon blinds, Salon Window, Salon blinds, Salon windows, Salon blind,\n Salon Covers, Salon Cover\n domain: update\n state: 'off'\n areas: Salon, saloon, salon, Saloon\n attributes:\n device_class: firmware\n- names: Kitchen Google hub, Kitchen Tv, Kitchen Display, Kitchen Chromecast, Kitchen\n hub\n domain: media_player\n state: idle\n areas: Kitchen\n attributes:\n volume_level: 0.0\n- names: Hotbox Top TV\n domain: media_player\n state: 'off'\n areas: Hotbox\n- names: Bedroom Chromecast, Bedroom TV\n domain: media_player\n state: 'off'\n areas: Bedroom\n- names: maxi is listening\n domain: binary_sensor\n state: unavailable\n- names: Office Thermostat\n domain: climate\n state: unavailable\n- names: Bedroom Thermostat\n domain: climate\n state: unavailable\n- names: maxi desktop\n domain: binary_sensor\n state: unavailable\n attributes:\n device_class: motion\n- names: Plex (Plex Cast - Chromecast)\n domain: media_player\n state: unavailable\n- names: Plex (Plex Cast - Chromecast)\n domain: media_player\n state: unavailable\n- names: home slimproto\n domain: media_player\n state: unavailable\n attributes:\n device_class: speaker\n- names: all home speakers\n domain: media_player\n state: unavailable\n attributes:\n device_class: speaker\n- names: Plex (Plex Cast - Chromecast)\n domain: media_player\n state: unavailable\n- names: group mass home airplay speakers old, All speakers, Home Speakers, Home Group\n domain: media_player\n state: unavailable\n attributes:\n device_class: speaker[/SYSTEM_PROMPT][INST]turn all on lights to random colors[/INST][TOOL_CALLS][{\"name\": \"changes_all_on_lights_to_random_colors\", \"arguments\": {}}]</s>[TOOL_RESULTS]{\"content\": {\"error\": \"ServiceNotFound\", \"error_text\": \"Action script.changes_all_on_lights_to_random_colors not found\"}}[/TOOL_RESULTS]I'm sorry, Maxi. It seems that the action to change all lights to random colors is not available. Would you like me to try something else?</s>[INST]isn't there a script for that?[/INST][TOOL_CALLS][{\"name\": \"changes_all_on_lights_to_random_colors\", \"arguments\": {}}]</s>[TOOL_RESULTS]{\"content\": {\"error\": \"ServiceNotFound\", \"error_text\": \"Action script.changes_all_on_lights_to_random_colors not found\"}}[/TOOL_RESULTS]I'm sorry, Maxi. It seems that the action to change all lights to random colors is not available. Would you like me to try something else?</s>[INST]Do you see the script?[/INST]Yes, I do. It is called \"changes_all_on_lights_to_random_colors\".</s>[INST]why can't you use it?[/INST][TOOL_CALLS][{\"name\": \"changes_all_on_lights_to_random_colors\", \"arguments\": {}}]</s>[TOOL_RESULTS]{\"content\": {\"error\": \"ServiceNotFound\", \"error_text\": \"Action script.changes_all_on_lights_to_random_colors not found\"}}[/TOOL_RESULTS]"