Frigate + HA + Google Home (latency) | Home Assistant | Page 1

grave river Oct 2, 2024, 1:30 PM

#

Hey folks have my Frigate instance working great (stand alone Docker install) with my HA (docker install) and camera entities exposed to Google Home (we have a bunch of google home screen devices like the nest hubs) and I can pull them up on the GH screens, and I can choose a camera to play on a screen as part of an automation etc. Works a treat. Go2RTC is running as I expect and seems to be working.

I'm now focused on reducing latency from the camera to the GH screen as much as possible. Anyone have any initial tips or results? I'm seeing 5-10+ seconds of latency depending on the camera and the instance that it's started. It creates an awkward use case to throw the doorbell camera on the screens when frigate detects a person approaching if the person has already rang the doorbell and is in the house by the time they appear on the screen.

Sub-Question for bonus, how much latency does Birdseye add? I'd like to display a birdseye stream when a person is detected, using it to combine a doorbell (up close) and a standard PTZ (wider shot, person tracking active). I know I can't have multiple birdseye or composite shots but when I do use birdseye piped into HA as a "Generic Camera" it's an even longer delay than a frigate integration camera.

#

I have birdseye configured as such in my config:
birdseye:
enabled: true
width: 1920
height: 1080
restream: true

#

And in HA I have the generic camera entity set to preload camera stream

hybrid sapphire Oct 2, 2024, 1:59 PM

#

I don't think there is much that can be done about the latency to GHA as I believe only HLS is supported which has this latency.

#

The latency that birdseye adds depends on your hwaccel decoder and how much latency it introduces

#

you could try enabling the use webrtc setting in the frigate integration configure menu and see if that helps but if I had to guess that will break the GHA camera streams

grave river Oct 2, 2024, 2:02 PM

#

Thanks for the response, @hybrid sapphire I appreciate it. I'll fool around with that flag and see. I'm using an Nvidia card for decode, so hopefully that's better than CPU latency wise? I have enough CPU overhead to use that if it would lower it.

#

Side note/Enhancement thought: "Dyanmic Composite Views" using the birdseye functionality to assemble a single stream with multiple cameras based on request, so I could have HA say "oh this camera in this HA Area picked up a person, I want these 3 cameras to give me a full view of what is going on for my realtime alert" You've basically got that now with the new multi-cam time scrubber which is AWESOME

hybrid sapphire Oct 2, 2024, 2:05 PM

#

nvidia is the fasted decode that I am aware of

grave river Oct 2, 2024, 2:05 PM

#

Oh, I do have "Enable WebRTC for camera steams" on in HA Frigate Integration. GH integration still works inside the house. The home app looses the ability to display cameras not on the home network

hybrid sapphire Oct 2, 2024, 2:06 PM

#

yeah I am not sure, we have talked about multiple birdseye cameras but that would require a major refactor of a lot of parts so it is probably a long way out

grave river Oct 2, 2024, 2:06 PM

#

makes sense, not conceived as a reusable component

#

we're still in the weird limbo of "are cameras smart home tech or security system/NVR devices" and Frigate + HA is the only thing blending them as well as it does, so trying to get to a scenario of "Hey someone is walking up to the house and you are home so I will alert you so you have the most amount of time possible to see/react to them" is big on my list of things. 2 way doorbell communication without any other app is also one of those solid use cases for me.

hybrid sapphire Oct 2, 2024, 2:14 PM

#

2 way doorbell communication without any other app is also one of those solid use cases for me.

yeah, this has been working in Frigate for a while now

#

I don't really use HA for anything frigate related besides automations at this point

#

Using Frigate as a PWA it behaves as its own app, especially in 0.15 with native notifications

grave river Oct 2, 2024, 2:15 PM

#

I have had two way doorbell working with my reolink doorbells, but I was using the frigate card in HA and latency was too much to be usable for my wife, I'll have to see if the native interface solves that issue

#Frigate + HA + Google Home (latency)