I'm playing around with the Voice PE using Google Gemini as the assist agent. I was a little surprised that using a cloud based AI that it can't answer basic questions like "when is the NBA trade deadline?" Does anyone know if there is a way to enable the assist to answer questions like this? I kind of assumed since you can ask Gemini these questions you would be able to do the same with HA Assist. Does the GPT Assistant can answer these sorts of questions or is this more of a HA thing that just isn't supported yet across the board?
#Google AI - local questions like "business hours" or "NBA Schedule"
weak igloo
spare quest
Might be that gemini on the phones talks to a different "path", where they have an instance of gemini that has access to tools that let it pull data from google/the web
you could write a script to get the data, and expose that to assist so the LLM knows to use that tool to get the data relevant to your question
weak igloo
you could write a script to get the data, and expose that to assist so the LLM k...
Interesting. You mean, write a script to pull 'google search' data? I wonder if GPT does this already. Just surprised an LLM wouldn't do this by default i guess.
spare quest
depends on the implementation and engine running the LLM
by default any LLM only has it's internal knowledge, not necessarily real-time knowledge
to get real-time data they use tools to retrieve it
that's why some of these big LLMs post a "Knowledge Cutoff Date", that basically lets you know the LLM only contains knowledge up to a certain point in time
weak igloo
that makes a lot more sense. Thanks for the explanation. Sounds like I need to research how to get goodle search data into HA or something similar.
tranquil fox
Giving the LLM access to a script, or tool, is basically how Gemini does pull that kind of data. That is called Retrieval Augmented Generation (RAG)
spare quest
This also works in ha 🙂