Interacting with Mealie Webpage Cards via HA Voice

[dusk shore]

Hello, new to HA, currently have HA green and HA voice preview set up and running.

Currently working towards setting up a kitchen display for displaying recipes. Running through setting up dashboards and checking what things work/what I can do before investing in buying hardware.

I want to be able to have mealie open on a dashboard (have server set up and working, currently showing on dashboard through a webpage card) and interact with it with voice assistant commands rather than having to utilize a touch screen. Main things I wanted to be able to do:
-Look up a recipe through the search menu, and then select it to open up recipe page
-Scroll recipe page up and down to see steps

I'm not really sure how the best way to go about this would be, hard coding automation commands to click/scroll the webpage would be silly. Not sure if this needs to be set up as a custom card or something rather than a webpage card, or if it would just be much more efficient/reasonable to get a touch screen display rather than trying to handle everything with voice command.

Any assistance/guidance would be appreciated.

[wheat tusk]

I would assume that changing anything within the mealie webpage itself is not feasible. Usually the "parent" page does not have access to the content of the iframe (aside from changing the iframe url).

Regarding touch or no touch, my main decision criteria was display size and cost. The 27" monitor was already there and would allow seeing it from wverywhere in the kitchen, a 27" touch device is expensive and a smaller tablet cannot be seen from everywhere. So, I went for a combination of raspberry with browser to display the dashboard(s).

What should work (I plan to set it up but did not find the time):

• Set up the mealie integration
• Create a custom dashboard to display a mealie recipe. See the instructions here: https://community.home-assistant.io/t/mealie-in-ha/603560/130
• Instead of some REST calls used there, mealie integration actions could be used. But, not for all.
• Instead of getting a fixed mealie reciepe based on the mealplan, use a template select to provide all available recipes: https://community.home-assistant.io/t/template-select-entities/715177
  (Respective, I plan to have a toggle to switch between custom selected recipe (e.g. to bake a cake) and the one from the mealplan.

Then you can change the select value from anywhere (mobile phone or automation triggered by voice) to update the shown recipe.

You could use browser mod to navigate to a different dashboard via voice which then allows the search (or simply use a mobile phone for it).

To not always have the monitor runnning, this script will be used: https://blog.cavelab.dev/2021/10/control-rpi-hdmi-with-mqtt/