Default Assist Pipeline | Home Assistant | Page 1

supple meadow Dec 6, 2024, 9:32 PM

#

I think I got the right channel sorry if I got it wrong. I am looking for some help/insight.

I have been playing with connecting Assist to Open.ai. I have the connection working great and when I use the 'chat with assist' internally or from one of our devices with the homeassistant app installed it works perfectly!

My next phase where the issue is happening is when trying to interact with the Assist through API. When posting to /api/conversation/process it seems to keep defaulting to some other Assist option. I have the one I want as default set to 'prority' through the Voice Assistant settings page but it still does not his that specific Assist.

I was unable to find a way to specify what Assist or AssistPipeline I wanted to use.

supple meadow Dec 8, 2024, 1:37 PM

#

The documentation page does not indicate a way to choose which Assist to use.

https://developers.home-assistant.io/docs/intent_conversation_api

Conversation API | Home Assistant Developer Docs

Intents can be recognized from text and fired using the conversation integration.

narrow prism Dec 8, 2024, 5:20 PM

#

Set your got pipeline as preferred (star).

supple meadow Dec 8, 2024, 8:14 PM

#

Hi formatBCE. In the Voice Assistant page I have a single Assist and it is set to Preferred currently.

I have tried removing and adding new Assist and changing which is set to preferred but it seems to use a different default when interacting with the API.

narrow prism Dec 8, 2024, 9:49 PM

#

Hmmm...

gloomy gust Dec 9, 2024, 12:08 PM

#

What leads you to think it's using a different default?

supple meadow Dec 9, 2024, 3:39 PM

#

When I interact directly with the Assist I get an expected response. When I do it through the API with the same prompt I get a different response.

When I view the logs they appear to show a difference as well.

The Assist I am trying to use which I only have one setup and it is preferred is connected to OpenAI so the responses are very different vs the standard assist responses

gloomy gust Dec 9, 2024, 4:15 PM

#

That's what's weird. If you only have one pipeline set up, there is no "other" pipeline it could possibly be using..

#

might need to see some examples to understand what is going on here

narrow prism Dec 9, 2024, 4:19 PM

#

Worth try to interact with other ways: through satellite with assigned area, through satellite without assigned area, through text input...

supple meadow Dec 9, 2024, 5:55 PM

#

It will take me a day or 2 but I’ll collect the logs up and post some examples.

supple meadow Dec 10, 2024, 8:47 PM

#

This is what my logs are showing when I interact directly with Assist Conversation vs the API.

#

My 'voice assists' is only 1 and it is set to preferred as indicated here.

#

The 'voice assist' is set to use 'GPT' as its chat agent

#

The response I get when using API is always
"Sorry, I couldn't understand that"

When using the Assist chat directly I get various responses that would be expected likst "how can I help" or some other various customer service style response.

gloomy gust Dec 10, 2024, 8:53 PM

#

think I see what is going on

#

so when you use conversation process, that API talks to HASSIL, the intent handler in HA

#

it doesn't have anything to do with the assist pipeline

#

I think wht you are looking for is this: https://developers.home-assistant.io/docs/voice/pipelines/

Assist pipelines | Home Assistant Developer Docs

The Assist pipeline integration runs the common steps of a voice assistant:

#

Which I think you use that in tandem with this https://developers.home-assistant.io/docs/api/websocket

WebSocket API | Home Assistant Developer Docs

Home Assistant hosts a WebSocket API at /api/websocket. This API can be used to stream information from a Home Assistant instance to any client that implements WebSockets. We maintain a JavaScript library which we use in our frontend.

supple meadow Dec 10, 2024, 9:00 PM

#

I was trying to use this the other day. I do think it would work but for my overall goal I am trying to use Apple ShortCuts to push the API's so I can make a connection to siri and my other apple devices.

#

When attempting this method it seems it required a continue authentication (that was my understanding) which apple shortcuts does not support as it only passes the auth with each request.

#

Because of this I was attempting to go the other API route. If i am understanding you though the Restful API does not interact with the 'Assist' feature but some other similair feature?

gloomy gust Dec 10, 2024, 9:04 PM

#

The API you were trying to use goes to the intent processor

#

Basically it is not the assist pipeline, it is going straight to the command intent processor. So doesn't talk to LLM or anything like that

#

At least that's what the docs seem to indicate.

supple meadow Dec 10, 2024, 9:06 PM

#

What I find interesting is in the logs they both start in the same 'homeassistant.components.conversation.agent_manager'

#

This made me think that the 'agent_manager' was directing to a location I was not expecting.

gloomy gust Dec 10, 2024, 9:07 PM

#

think it's just part of the architecture of the APIs

#

even in the docs, that conversation API is referred to as the "intent_conversation_api"

#

so in HA, HASSIL is technically a conversation agent

#

but it's the one that handles intents, and is also the default agent if no other is defined

#

so makes sense that the logs would show the default_agent kicking in on the conversation api

#

since HASSIL is the agent for intents

#

Also not sure if you saw https://www.home-assistant.io/voice_control/apple/

Home Assistant

Assist on Apple devices

Open source home automation that puts local control and privacy first. Powered by a worldwide community of tinkerers and DIY enthusiasts. Perfect to run on a Raspberry Pi or a local server.

supple meadow Dec 10, 2024, 9:13 PM

#

gloomy gust Also not sure if you saw https://www.home-assistant.io/voice_control/apple/

I did see this! I quickly realized a downside to this. It forces open the 'home assistant app' on the phone and you have to repeat what you are doing and physically interact with the device. It also does not auto turn the mic on after response from Assist so continuous conversations do not work. As a middle ground its got some good features.

narrow prism Dec 10, 2024, 11:51 PM

#

I guess you want too much from HA. It's exposing API for internal functions, but it's too much to expect it exposing LLMs. Ollama has its own API - use that. 🙂

gloomy gust Dec 11, 2024, 1:49 PM

#

Yeah, even HA's own satellites don't support continuous conversation yet, though I suppose some people have modified the firmwares to make that possible.

#Default Assist Pipeline