#WIP: Port immich_ml for Jetson NVIDIA

1 messages · Page 1 of 1 (latest)

outer tundra
#

I'm currently trying to build the immich_ml container for NVIDAS Jetson hardware (edge computing). The challenge here, is that the Jetsons running on an ARM, and that most of the CUDA stuff is albeit working, specifically tailored for Jetson. So, the normal cuda container would not work. I managed to get already half there. I used a tailored Jetson container, which contains onnxruntime and python based on an ubuntu already. I had to add a jetson cpu option into poetries config, to make sure, it tries not to install onnxruntime again. Now I have to change the Dockerfile of immich_ml and extend it with a builder and prod image for the jetson. Here I struggle to understand the following lines:

...
RUN pip install --upgrade pip && pip install poetry
RUN poetry config installer.max-workers 10 && \
    poetry config virtualenvs.create false
RUN python3 -m venv /opt/venv

COPY poetry.lock pyproject.toml ./
RUN  poetry lock
RUN poetry install --sync --no-interaction --no-ansi --no-root .... goes on

line four creates an venv, but that never gets activated. The last line installs all dependencies for immich_ml but it is set in line 3 to not create an venv.
So for what is that line 4 for?
Where does all those dependencies get installed?
For the prod-image I have to copy it over, so I am a bit lost from where I have to copy now.

tardy island
#

cc @odd marsh

#

previous discussion in #10647

prisma jettyBOT
odd marsh
#

There's a PR for this (#12456) but I got kinda confused by the whole JetPack 4 vs 5 vs 6 thing and them all needing different CUDA versions. Which Jetson do you have?

prisma jettyBOT
outer tundra
#

Ha I am not alone 🙂

#

Honestly, this NVIDIA stuff reminds me a lot to FPGAs and other "big company" stuff. The typical "Just download this 16GB tar ball and execute the script inside as root on your host machine, to flash the board"-vibes. If you get it running you face a trillian of different OS-versions vs. different JetPack versions vs. different CUDA versions... why are all the cooperate docs are so confusing and feel so much like dirty hacks....

#

@odd marsh I would try to get it working on my particular machine and share what I have. I agree that it might become a nightmare to support all those different hardware models, with different software stacks, but once started with a working proof-of-concept, it might be easier to continue from there

odd marsh
#

Sounds good! Just a heads up though that if you're using a board that needs JetPack 4, it needs CUDA 10.2 and that's kind of impossible for us to support

outer tundra
#

I have a recent board Jetson Orion 16GB NX, running Jetpack 6

odd marsh
#

Ah, perfect. That should work fine

#

"theoretically"

outer tundra
#

Well as son as I managed to understand what the venv stuff and poetry is doing and where in the build-image to find the files for the prod-image.... btw. do you know if there is a way to check the file system of the build stage? I checked some ressources, like adding lables, etc. but I could not make it, that I can run the build-image look into it.

outer tundra
odd marsh
odd marsh
prisma jettyBOT
outer tundra
#

yes, I think I am almost there... I just was confused about the poetry no-venv and the creating of a venv which never gets activated