#How can I verify that machine learning is using NVIDIA CUDA

1 messages · Page 1 of 1 (latest)

solemn forge
#

I setup NVIDIA, I actually got an error that NVIDIA was not available when starting the docker container, which I then fixed by installing the nvidia container toolkit, and now the ML docker started fine.

However, I would like to verify that it is being used, how can I do that?

plucky pagodaBOT
#

:wave: Hey @solemn forge,

Thanks for reaching out to us. Please follow the recommended actions below; this will help us be more effective in our support effort and leave more time for building Immich immich.

References

Checklist

  1. :ballot_box_with_check: I have verified I'm on the latest release(note that mobile app releases may take some time).
  2. :ballot_box_with_check: I have read applicable release notes.
  3. :ballot_box_with_check: I have reviewed the FAQs for known issues.
  4. :ballot_box_with_check: I have reviewed Github for known issues.
  5. :ballot_box_with_check: I have tried accessing Immich via local ip (without a custom reverse proxy).
  6. :ballot_box_with_check: I have uploaded the relevant logs, docker compose, and .env files, making sure to use code formatting.
  7. :blue_square: I have tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable

(an item can be marked as "complete" by reacting with the appropriate number)

If this ticket can be closed you can use the /close command, and re-open it later if needed.

plucky pagodaBOT
feral current
#

...point 6

solemn forge
# 
root@docker:/storage/appdata/immich-compose# docker logs immich_machine_learning
[09/24/24 11:31:56] INFO     Starting gunicorn 23.0.0                           
[09/24/24 11:31:56] INFO     Listening at: http://[::]:3003 (9)                 
[09/24/24 11:31:56] INFO     Using worker: app.config.CustomUvicornWorker       
[09/24/24 11:31:56] INFO     Booting worker with pid: 10                        
[09/24/24 11:32:00] INFO     Started server process [10]                        
[09/24/24 11:32:00] INFO     Waiting for application startup.                   
[09/24/24 11:32:00] INFO     Created in-memory cache with unloading after 300s  
                             of inactivity.                                     
[09/24/24 11:32:00] INFO     Initialized request thread pool with 8 threads.    
[09/24/24 11:32:00] INFO     Application startup complete.                      
[09/24/24 11:34:16] INFO     Loading visual model 'ViT-B-32__openai' to memory  
[09/24/24 11:34:16] INFO     Setting execution providers to                     
                             ['CPUExecutionProvider'], in descending order of   
                             preference                                         
[09/24/24 11:34:35] INFO     Loading detection model 'antelopev2' to memory     
[09/24/24 11:34:35] INFO     Setting execution providers to                     
                             ['CPUExecutionProvider'], in descending order of   
                             preference                                         
[09/24/24 11:34:35] INFO     Loading recognition model 'antelopev2' to memory   
[09/24/24 11:34:35] INFO     Setting execution providers to                     
                             ['CPUExecutionProvider'], in descending order of   
                             preference
# 
  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
      file: hwaccel.ml.yml
      service: cuda # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: unless-stopped
feral current
# 
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}-cuda
    extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
      file: hwaccel.ml.yml
      service: cuda # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: unless-stopped```
#

I believe that should be your env file

#

image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}-cuda

solemn forge
#

ya i just saw that. once i restart is there something i can check in logs to see 'using cuda'?

feral current
#

also, for the little curiosity in me, how did you get your code formatted with color?

feral current
solemn forge
#

three backticks then the word yaml, e.g.; '''yaml

#

but backticks

#

woot!

[09/24/24 22:39:15] INFO     Initialized request thread pool with 8 threads.    
[09/24/24 22:39:15] INFO     Application startup complete.                      
[09/24/24 22:39:46] INFO     Loading detection model 'antelopev2' to memory     
[09/24/24 22:39:46] INFO     Setting execution providers to                     
                             ['CUDAExecutionProvider', 'CPUExecutionProvider'], 
                             in descending order of preference                  
[09/24/24 22:39:48] INFO     Loading recognition model 'antelopev2' to memory   
[09/24/24 22:39:48] INFO     Setting execution providers to                     
                             ['CUDAExecutionProvider', 'CPUExecutionProvider'], 
                             in descending order of preference
#

seeing gpu util with nvidia-smi now and face recognition going a lot faster

feral current
#

Haha nice!!