#Voice Agents - past, present, and future
plain anvil
(2s finishing an ice cream, it's super hot where i'm at right now ahah)
i'm back xD
Okay so I'm not 100% sure how it should be done really, but I remember the Assistant page about things to do / integrations being really clunky
and also being quite.. hidden?
I think that one thing I would really like to see is kind of a proper "storefront" page for extensions
Maybe even a category & category page on the play store?
- a label or badge
Similar to how it's done for Wear OS compatible apps, or apps that have watch faces
--
Sidenote:
The biggest thing I keep thinking about though, and that i'm not sure how they could do, is how they would present to users the difference between:
• Apps that have "actions" that can be triggered by Gemini
• Gemini extensions that work on all platforms, kind of like the workspace extension or the Google Flights one
brazen bronze
well, one issue is that Assistant didn't need to install apps before hand. So you could just ask for them or ask it to do something.
But there were issues with that, not least of which discovery.
And there was an "app store".
But making it part of the Play store was unlikely for two reasons:
- Actions didn't have anything at all to do with Android
- It was the web team that ran Assistant at the time.
Once the Android team took over Assistant, they shut that half of the world down and left us with App Actions. Which are... just terrible.
how they would present to users the difference between:
Right... that's the problem that they just boxed themselves into again.
plain anvil
well, one issue is that Assistant didn't need to install apps before hand. So yo...
oooh didn't know about that at all (the web/android team thing)
brazen bronze
oooh didn't know about that at all (the web/android team thing)
It wasn't well known, but those in the industry watching saw it happen. Around the same time that the Nest team became part of Android.
plain anvil
I feel that maybe they should just increase what's possible to declare when making app "slices" / shortcuts
(on Android)
Kind of regardless of Gemini (would help a ton for accessibility tools, but also for the builtin search and so on)
But would also maybe let assistant apps exploit that
brazen bronze
But would also maybe let assistant apps exploit that
Eh... the real issue is that App Actions are an invocation thing, rather than something truly interactive.
It is certainly possible they might make Gemini/AA more interactive. But that is a LOT of work, and I don't see them doing that very quickly.
Thef unny part is... the technology to do that is a LOT better than it was back when they discontinued Conversational Actions.
plain anvil
Eh... the real issue is that App Actions are an invocation thing, rather than so...
True yeah, but I feel like just having a way to at least trigger all the "dumb actions" on most apps would already be a huge step forward
Maybe by making it so instead of just declaring a name / icon and so on, to also declare a few "prompts" that could be used to trigger it?
(same style as the routine triggers?)
(i fear for the possible abuse of that though)
--
Now that I think about it 👀
If they added a proper "actions" thing on Android, maybe even in the android settings page of said apps
brazen bronze
Maybe by making it so instead of just declaring a name / icon and so on, to also...
Maybe by making it so instead of just declaring a name / icon and so on, to also declare a few "prompts" that could be used to trigger it?
But why not let the LLM do that!
Instead, you should be describing what your sub-action can do and return.
And let Gemini piece them together in the most reasonable way possible.
(See what Bixby did 5 years ago.)
plain anvil
It would be quite awesome to be able to individually turn on/off declared actions from apps
but also would maybe help integrate them into the android-pixel "rules/automation' thingy that's... quite empty atm
brazen bronze
(i fear for the possible abuse of that though)
This is the other problem. But we ahve established ways of selecting what we want to use.
plain anvil
> Maybe by making it so instead of just declaring a name / icon and so on, to al...
That could be really awesome 👀
brazen bronze
That could be really awesome 👀
But this is the problem. The Android folks don't understand things beyond a screen. They just don't think that way.
So they don't get units smaller than a widget. They barely understand a widget.
plain anvil
Hmmm, you don't think a screen kinda like this one:
but under an app
would be sufficient UX-wise for the end users to understand its purpose?
brazen bronze
would be sufficient UX-wise for the end users to understand its purpose?
Do you think that would scale for every app on your phone?
plain anvil
Hmm what do you mean by that?
(as in developer adoption of this feature to be high enough?)
brazen bronze
as in UX usability
plain anvil
I thiiink it would be okay?
You could have it just be a column on phones
and for each card not to take too much room vertically you could hide the suggestion chips under a collapsible thing maybe?
lemme quickly google-draw something xD
plain anvil
I think so yeah, because I doubt most apps would have a lot of them and it's something you would rarely end up interacting with 🤔
plain anvil
how many apps do you have on your phone?
great question 🤣
(including the default & system ones 192 apparently 👀)
brazen bronze
Can you imagine that screen with 192 of those cards?
