Perpetually creating thumbnails | Immich | Page 1

livid flame Dec 15, 2023, 4:07 PM

#

I had this issue a couple of versions ago but it's happening right now so reaching out to see if we can live diagnose.

I just processed a library import via immich-go, and now Immich is creating thumbnails.... that never ends.

The counter increases by about 25 every few seconds and exceeds the number of photos in the library. Even knowing that each photo might need 2-3 thumbnails, the numbers shown below are after about 10 hours of processing and still increasing.

The last time this happened it went for 24+ hours, all the while the counter kept increasing and a seemingly infinite number of thumbnail images were created on disk.

How can I help figure out what is happening?

oblique capeBOT Dec 15, 2023, 4:07 PM

#

:wave: Hey @livid flame,

Thanks for reaching out to us. Please follow the recommended actions below; this will help us be more effective in our support effort and leave more time for building Immich immich .

References

Container Logs: docker compose logs docs
Container Status: docker compose ps docs
Reverse Proxy: https://immich.app/docs/administration/reverse-proxy
Typesense: You can't fix this. Just wait until we get rid of it. Sorry.

Checklist

:ballot_box_with_check: I have verified I'm on the latest release(note that mobile app releases may take some time).
:ballot_box_with_check: I have read applicable release notes.
:blue_square: I have reviewed the FAQs for known issues.
:blue_square: I have reviewed Github for known issues.
:blue_square: I have tried accessing Immich via local ip (without a custom reverse proxy).
:blue_square: I have uploaded the relevant logs, docker compose, and .env files using the buttons below or the /upload command.
:ballot_box_with_check: I have tried an incognito window, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable

(an item can be marked as "complete" by reacting with the appropriate number)

If this ticket can be closed you can use the /close command, and re-open it later if needed.

crisp igloo Dec 15, 2023, 4:09 PM

#

Images can have a lot of thumbnails. These mostly come from the people in them, as for the facial recognition to work, we have to run face detection, then generate an image of that face to store so that we can use that further to recognise who is who

#

This means that while the facial recognition tasks are ongoing, these thumbnail jobs can keep growing significantly

#

You'll just have to wait it out while it completes, but this should only ever be a problem on your initial big import

#

(or if you re-run the facial recognition jobs I believe)

livid flame Dec 15, 2023, 4:15 PM

#

Ok let me keep monitoring. I should mention that I've disabled reverse geo AND ML

#

So I wasn't expecting it to do anything too involved.

#

(last time it was enabled so could well have been that)

crisp igloo Dec 15, 2023, 4:17 PM

#

Side note: if you had reverse-geo disabled for a long time, we recently re-worked the implementation and it now requires very little extra memory, so might be worth looking at enabling again if you want it 🙂

livid flame Dec 15, 2023, 4:18 PM

#

Noted in the release note 🙂 Mostly this time I had it disabled because I'm aware that development is very active 😜 and so may well want to start over again a few more times.

crisp igloo Dec 15, 2023, 4:18 PM

#

Regardless of ML, we always generate more thumbnails than the amount of images uploaded to the server, I'm not sure exactly how many

#

But this is certainly expected behaviour

#

On lower-end hardware it'll definitely not be able to keep up with the rate you can ingest the files

livid flame Dec 15, 2023, 4:19 PM

#

Ok will keep you updated. The first time I did an update in a low 0.8x release the whole thing did actually complete fairly quickly

crisp igloo Dec 15, 2023, 4:19 PM

#

There have been a lot of changes since then 😛

livid flame Dec 15, 2023, 10:26 PM

#

So not to be a conspiracy theorist or anything, but I hit 'Pause' on the task for a few hours while doing some other disk-intensive tasks. Just unpaused it now and it's immediately churning through the thumbnails - fast - and the counter is decreasing whereas right before the pause it was increasing.

livid flame Dec 15, 2023, 11:06 PM

#

Seems unlikely to be a complete coincidence that I paused right as it got on top of them, but *shrug there we go haha

distant wadi Dec 15, 2023, 11:11 PM

#

The thumbnail jobs aren't equal in how long they take. The large JPEG "thumbnails" (preview images) get processed first while the smaller and much faster thumbnails get processed after. And the smaller thumbnails for an asset get queued only after the JPEG thumbnail completes.

So at first the number of queued jobs will increase since it's both processing the slower jobs and queueing several jobs for each one. But after some time there will be fewer or no thumbnails being queued and the jobs will start being processed much more quickly since it's doing the small thumbnails.

#

The way we present it is a bit confusing since it makes it look like there's no progress as more and more jobs get queued at a faster rate than the images get processed

livid flame Dec 15, 2023, 11:25 PM

#

Interesting - especially since it's not doing image-by-image but size-by-size batching. Thanks for the info!

crisp igloo Dec 15, 2023, 11:29 PM

#

Also, due to some importing i am doing myself atm

#

The jobs operate in a certain order

#Perpetually creating thumbnails

References

Checklist