Devices and actions when using `conversation.process` | Home Assistant | Page 1

Hey hey,
i'm playing around with OpenAI a bit. I have a configured agent for OpenAI where i changed the prompt/instructions to this

You are part of a Home Assistent smart home.
The user will provide you with a prompt and you have to come up with a Home Assistant script executes the correct services in order.
Output the script and only the script. Do not give any explanations. Use the correct actions/services for the devices in the script. Just create the script.

When i am running this via the conversation.process it roughly does what i'm expecting it to do:
"turn off the lights at my desk for 10 seconds "

response:
  speech:
    plain:
      speech: |-
        ---yaml
        sequence:
          - service: HassTurnOff
            data:
              name: Schreibtisch
              domain: light
          - delay: 
              seconds: 10
          - service: HassTurnOn
            data:
              name: Schreibtisch
              domain: light
        ---
      extra_data: null
  card: {}
  language: en
  response_type: action_done
  data:
    targets: []
    success: []
    failed: []

(i replaced the markdown code tags with --- for discord)

But as you can see from the "service: HassTurnOff" it looks like it doesnt know which devices and actions exist in my system.
I think in a regualr voice conversation context, HASS adds additional context to my message, between the system prompt and the users input.
Is this not added when running an agent via conversation.process? Can i i MAKE it add that?

To give some context: The idea here is to make OpenAI generate a script, save and run that script, then delete it when its finished.
Think of it as "ephemeral scripts". Imagine telling it "Mow the lawn, and when thats done, run the sprinklers for 20 minutes" and it would run the robo mower, then add a "wait for trigger: mower parked" and then run the sprinklers.
I dont think you can run complex processes like this right now using normal assist.

I'm pretty sure its a bad idea, but i'm just curious about the possibilities and want to play around with it

#Devices and actions when using `conversation.process`