real hazel
I'm looking to make the best/most-efficient use of my server hardware in order to accomplish my goals in HA. They are:
- record security camera footage and use motion detection w/Frigate
- process Frigate events (object ID) through an AI (CodeProject.AI?)
- process voice requests using HA Voice Nabu Casa devices (using an LLM)
- other basic automations for lights/thermostat/TV/media players/etc
The system I am using has both a Coral Edge TPU (USB) and an Nvidia RTX 4000 GPU. Right now, I have Frigate doing all object ID using CodeProject.AI in a separate Docker container using the GPU (using iGPU for decoding w/FFMpeg). The Coral Edge TPU is currently unused.
Would it be wise for me to replace CodeProject w/something like Ollama that would integrate better w/HA? I am also not real certain of how to go about designing the proper 'pipeline' of processing the object recognition making the most-efficient use of this hardware.
Should I maybe leverage the Coral TPU in Frigate, but use HA (and the GPU) for the more in-depth AI processing (such as the LLM)? Is it possible to map the /dev/<GPU> devices in a docker-compose to more than one container (thus sharing it) or will it conflict/error-out?
I have tried researching things and there's not really a clear direction, just many many ways to accomplish what I'm looking to do (which isn't a bad thing, I am just not wanting to get too entrenched in which I could be limiting myself later on)
torn hinge