Perhaps you could avoid rendering on

[glossy knoll]

I will look what URP or eventually HDRP can do, but it would be great to make it stop processing things when the user is not doing anything