It is my desire to have HA voice (currently via Home Assistant Voice PE) able to interpret plain-language queries to manage HA entities when a matching entity is found, and then be generally conversational when none is found. I have a Claude-based AI assistant set up in Home Assistant, and my PE is set up with Claude as the first assistant, and Home Assistant as the second. I have attempted to mess with different settings (local preferred, etc) but I can't quite get the experience to match my desired outcome. If I hop into an Assist pipeline debug session with the Claude engine, I can ask it "Turn on the bedroom fan" and have it do so successfully, as well as query "what is bigger, a blue whale or a sperm whale?" (a real-life example from earlier today) and have it provide me a general AI-based answer. However, if I attempt the same via the PE, with Claude set as the first assistant and the default/preferred "Home Assistant" as the second, entity-commands work well, while general, non-entity-related queries fail. If I ask my whale question via the voice PE, I get "I am not aware of any device named what is bigger, a blue whale or a sperm whale" as a voice response. Is it possible to expect queries that don't match HA-specific entities or actions be answered by the AI-based conversation agent via the PE satellite?
#Getting AI assistants to be generally conversational when no matching entity found
bleak karma
somber bolt
In the Claude agent settings, enable local command priority.
On VPE, select Cloude. Now everything will work as you expect.
Satellites do not have a primary and backup agent—these are two equivalent agents, each of which can be configured with its own wake word.
bleak karma
OH. 😬
i guess because "Assistant/Wake Word" and "Assistant2/Wake Word 2" are grouped as "Assistant/Assistant2" and "Wake Word/Wake Word 2" I assumed a primary and backup assistant, and a primary and secondary wake word. I had no idea there were two distinct agents accessible from two distinct wake words.
is this so you can make a choice at any given time between an AI and local assistant? man, i've been thinking of things backwards the entire time.