Bypassing intents helper | Home Assistant | Page 1

steady flint Feb 9, 2025, 12:14 AM

#

Is there any way to bypass the intents helper in Assist when using an LLM service?

I'd really like to use it to pass what I'm saying directly to the LLM and play back the output. Basically to use the voice preview module (and the assist on my watch and phone) as an interface to my local LLM. I have set that up with tools too so it could also trigger stuff, not just in home assistant but also send messages etc.

The problem is, when I ask general questions it says 'unexpected error during intent recognition'. I think that's the intent helper that tries to find the names of entities in my prompt or something. It would be really nice if I could bypass that.

#

PS: The reason I want to use it that way is that the voice preview and assistant interface would be a really handy way to access the LLM. Handsfree and with voice. I can use voice in the OpenWebUI interface too but not handsfree with hardware like the voice preview module. Or as an assistant on my phone/watch.

jaunty geyser Feb 9, 2025, 1:05 AM

#

Well you need to put your LLM as conversation agent to the pipeline. If you want LLM only, you disable "local first" toggle there. That's it.

#

"unexpected error" is happening when you're using scripts and they fail. Or maybe your LLM connection is broken. The local Assist says 'sorry, I couldn't understand that', not the unexpected error.

steady flint Feb 9, 2025, 5:19 AM

#

Ahh thanks! It was the intent thing that threw me off. I'll check that out. It must be another issue then.

versed drum Feb 9, 2025, 8:16 AM

#

"Unexpected error" means there will be a more explicit error message in your core logs. Paste it here for more info

steady flint Feb 9, 2025, 8:18 AM

#

Ah thanks I didn't know where to find troubleshooting data

#

Hmmm it's a weird error I'm getting

Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/components/assist_pipeline/pipeline.py", line 1117, in recognize_intent conversation_result = await conversation.async_converse( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<8 lines>... ) ^ File "/usr/src/homeassistant/homeassistant/components/conversation/agent_manager.py", line 114, in async_converse result = await method(conversation_input) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/extended_openai_conversation/__init__.py", line 199, in async_process query_response = await self.query(user_input, messages, exposed_entities, 0) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/extended_openai_conversation/__init__.py", line 376, in query if response.usage.total_tokens > context_threshold: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: '>' not supported between instances of 'NoneType' and 'int'

#

So it looks like response.usage.total_tokens is None

#

But why... Hm

versed drum Feb 9, 2025, 8:20 AM

#

Ah, extended openai. Any reason you are not using the core integration?

steady flint Feb 9, 2025, 8:20 AM

#

Yes it doesn't allow a custom API base URL 😦 The developers keep shooting down any requests for that

#

I use it with my own server

#

That's the only reason I need to use that one

#

I do need the OpenAI interface, I can't go directly to ollama for this because OpenWebUI provides the tools

versed drum Feb 9, 2025, 8:22 AM

#

Got it. I don't have much experience with it, but at least you now know what the issue is. You can seek further help using the integration docs/community or raise an issue in case it's not known

#Bypassing intents helper