#Help with ever expanding library jobs!!

1 messages · Page 1 of 1 (latest)

upper barn
#

Hey! I recently had to reinstall Immich after having it running perfectly for months. I'm now having an issue where my library jobs are expanding constantly - I have about 95k assets, but the library went up to about 8m last night before my computer just gave up 😛

I have had to stop the job for now and can't work out what I've configured incorrectly... The format of the docker-compose.yaml has changed since the last time I did it. Hoping someone might be able to point me in the right direction!

Running Zorin OS 17 (Ubuntu derivative), Immich installed in Docker following the usual instructions. Remote access through Tailscale.

Photos are stored on an external library - my setup is that I use Synology Photos for backup to my NAS, then I have a job that clones the photos to a drive connected to my Linux PC and that folder is set as external library in Immich. I do not back up photos directly to Immich. Thought that might be important to share.

My docker-compose.yml and .env are attached.

I thought I might have a duplicated volume entry in the YML but I can't see anything.... I've exhausted my ideas as to what might be wrong now! Could it be an issue with the Redis setup? I've not changed anything there though!

Any help you can offer is appreciated!

Thanks in advance 🙏

📎 env 📎 docker-compose.yml
umbral hemlockBOT
#

:wave: Hey @upper barn,

Thanks for reaching out to us. Please follow the recommended actions below; this will help us be more effective in our support effort and leave more time for building Immich immich.

References

Checklist

  1. :ballot_box_with_check: I have verified I'm on the latest release(note that mobile app releases may take some time).
  2. :ballot_box_with_check: I have read applicable release notes.
  3. :ballot_box_with_check: I have reviewed the FAQs for known issues.
  4. :ballot_box_with_check: I have reviewed Github for known issues.
  5. :ballot_box_with_check: I have tried accessing Immich via local ip (without a custom reverse proxy).
  6. :ballot_box_with_check: I have uploaded the relevant logs, docker compose, and .env files, making sure to use code formatting.
  7. :ballot_box_with_check: I have tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable

(an item can be marked as "complete" by reacting with the appropriate number)

If this ticket can be closed you can use the /close command, and re-open it later if needed.

umbral hemlockBOT
copper briar
#

I'm on mobile, can you paste here your UPLOAD_LOCATION line from the .env?

upper barn
#

Thanks @copper briar

The location where your uploaded files are stored

UPLOAD_LOCATION=./library

#

# The location where your uploaded files are stored UPLOAD_LOCATION=./library

copper briar
#

and what path is that on your host? the full path

#

like, what folder did you run docker compose up -d in

upper barn
#

immich-app

copper briar
#

the full path please

vague grove
#

That's not a full path

upper barn
#

ahh sorry

copper briar
#

i'm guessing it's /usb/files/Photos/immich-app lol

upper barn
#

/home/clevercode/immich-app

stored on my main device drive - not on the USB

copper briar
#

are you 100% sure of this

#

what is the external library path that you added in Immich?

upper barn
#

Just to prove it! this is the /photos folder on USB

image.png
#

import path is /usr/src/app/external

knotty parrot
#

Right so

#

You set ./library to /home/clevercode/immich-app/library

#

And then you add /usb/files/Photos as external path

#

And the usb is not mounted aaaaaaanywhere else perchance

upper barn
#

As far as I can see it's not mounted anywhere else! I can't say I'm a linux pro - I moved over from Mac 6 months or so ago!

image.png
knotty parrot
#

bizarre bizarre

#

Are you running the container now? I assume not

upper barn
#

I am, but with the job paused

#

what would you like to know (thanks btw)

knotty parrot
#

You could check inside the container with docker exec -it immich_server /bin/bash

#

That should put you in /usr/src/upload

#

Could you check if the folders like /usr/src/upload/external and /usr/src/upload/upload contain the right files/folders?

upper barn
#

root@ee3574ae380e:/usr/src/app# ls /usr/src/upload/external ls /usr/src/upload/upload ls: cannot access '/usr/src/upload/external': No such file or directory ls: cannot access '/usr/src/upload/upload': No such file or directory

What am I doing wrong? 🙃

vague grove
#

/usr/src/app/upload

upper barn
#

root@ee3574ae380e:/usr/src/app# ls /usr/src/app/upload encoded-video library profile thumbs upload

knotty parrot
#

Yeah my bad 👀

#

That seems to be okay so far

#

Usually when the jobs keeps increasing it means your external library somehow containes the upload folder or vice versa

#

And by usually I mean that has always been the case

upper barn
#

I am as confused as anyone! I could understand if I had a duplicate and it was stuck looping - but that doesn't seem to be the case.

Are there any logs I can share that might help?

knotty parrot
#

You could check the latest logs for immich_server, see which asset was last created (thumnailed) and go check out what picture that is

#

That might give a clue

upper barn
knotty parrot
#

It seems to queue your library about 7 times in a single minute 🤔

placid rock
#

Oh, that’s not great

#

So if you run docker compose down and docker compose up, it should clear all the jobs in redis. If you then start a scan, does the infinite queueing persist after that?

upper barn
#

It does sadly. It climbs up to about 700k then goes down a little then starts climbing again and doesn't seem to stop.

placid rock
#

cc @warped eagle @wet kestrel

wet kestrel
#

Hey @upper barn , would you be able to run find . -type l within the root of your external library please? 🙂

upper barn
#

Hey - thanks for looking into this. Sadly (or happily?) no results

image.png
wet kestrel
#

So no recursive loops then, that's good. Hmmm

upper barn
#

Any ideas or should I give up and try again from fresh? 🥴

placid rock
#

There’s something in the code that’s repeatedly queueing these scans based on the logs. It doesn’t seem like a folder traversal issue. I’m just not sure what’s causing that to happen

#

I’d say you can make a separate instance for now and see if the issue occurs there as well. But keep the current one because it’ll be helpful for figuring this out

upper barn
#

ok cool - I'll see if I have time to do a fresh install and hopefully you will work it out 😛

wet kestrel
#

I'm personally off the entire next week, so I won't be able to take a look for a while

upper barn
#

I tried to install a separate instance, but it wouldn't load for some reason, so I'm just gonna wait it out until someone has time to take a look at this. If it helps - I plan on paying for Immich soon! Just need enough cash (and to have it working :P)

upper barn
#

@wet kestrel hope you had a good week offline 🙂 Aware you probably have a million things to deal with, but if you get a chance to take a look at this, I'd appreciate it. I've kept the install for testing purposes, but currently can't use Immich due to this, so would love to resolve or destroy and start again soon if possible 🙂 Thanks in advance!

wet kestrel
upper barn
#

yeah - that seems to be the case, but not sure why or how to reset it... Thanks for the speedy response!

#

could it be that I have a sync task with Synology on my nas set up? I had it running without issue previously with this config..

wet kestrel
#

If you restart the redis container, it should lose anything in the queue

upper barn
#

did that, and it comes back each time...

wet kestrel
#

Have you set something up externally to sync the library through the API?

#

It's almost every 30 seconds it invokes the refresh

upper barn
#

no so the queue is

  1. Photos get backed up to my Synology NAS through Synology photos on respective devices (iPhone, Android etc)
  2. Synology Drive client app on Linux syncs that folder on my NAS to my desktop PC where Immich is installed when there are any changes.
  3. Immich checks that folder regularly for updates

I think I'll play around with the task to see if I can decrease the frequency of the checks and also will make sure my cron job in Immich isn't set to too frequently and that might fix it...

wet kestrel
#

What is your cron expression set to now in immich?

#

Given the files are on the device (not mounted via network share) you would probably be better to change the periodic scan to run nightly or something, but then turn on library watching

#

That should automatically detect changes and only queue the single file that got added/deleted/modified

upper barn
#

The cron was set to every minute, AND I had watch for changes on... that might be the issue... I'll try the watch feature and set nightly scanning on the cron and see if that fixes it. If not I can turn off nightly and decrease the cron to every 10 minutes or something

#

Thanks - I think you've found the cause!!

upper barn
#

Can confirm - that fixed it! All library tasks complete 🎉 Thanks again!