Is an LLM Vision setup with WebRTC google nest cameras possible? | Home Assistant | Page 1

I've been trying to set up an LLM vision timeline feed for events triggered from my nest doorbell camera. However, I haven't been able to get it to view the feed.

I'm trying to use the llmvision.stream_analyzer action like so:

sequence:
  - action: llmvision.stream_analyzer
    metadata: {}
    data:
      remember: true
      use_memory: true
      duration: 10
      max_frames: 5
      include_filename: true
      target_width: 1280
      max_tokens: 100
      temperature: 0.5
      generate_title: true
      expose_images: true
      provider: 01JPSK68NBR1M15EC8HDD21CXV
      image_entity:
        - camera.indoor_living_room
      model: gpt-4o-mini
    response_variable: response
alias: Analyse cameras
description: ""

It only sees the black screen with a white/grey blob (see pic 1). The doorbell (and indoor wired camera that I've been testing with) use WebRTC, which doesn't support snapshots per https://www.home-assistant.io/integrations/nest/#camera. I have a nest hub max camera, which uses RTSP Recording, and does support snapshots. I was able to successfully get the script to analyse that (see pic 2).

So from the above, I'm assuming that llmvision.stream_analyzer relies on snapshots to be able to perform it's analysis, and it isn't able to actually capture any stream feed.

Is this simply not possible with WebRTC google nest camera because of the lack of snapshot support, or is there another method available that doesn't rely on snapshots?

Home Assistant

Google Nest

Instructions on how to integrate Nest into Home Assistant.

#Is an LLM Vision setup with WebRTC google nest cameras possible?