Immich randomly broke upon updating. | Immich | Page 1

rugged tide Jul 4, 2024, 1:48 AM

#

R730 running unRAID, updated Immich in docker and it just broke it. It now gets stuck on ```text error warn system array login

[mod-init] Running Docker Modification Logic
[mod-init] Adding imagegenius/mods:universal-redis to container
[mod-init] Downloading imagegenius/mods:universal-redis from registry-1.docker.io
[mod-init] Installing imagegenius/mods:universal-redis
[mod-init] imagegenius/mods:universal-redis applied to container
[migrations] started
[migrations] no migrations found

╔═══════════════════════════════╗
__ _____ _____ __
/ / |_ / | / /
/ / | || | __ / /
/ / | || | | | / /
/ / | || |__| | / /
// |______| /_/

Baseimage from linuxserver.io
╠═══════════════════════════════╣
To support this applications developer(s) visit:
Immich: https://immich.app/docs/overview/support-the-project
╠═══════════════════════════════╣
User/Group ID:
User UID: 99
User GID: 100
╚═══════════════════════════════╝

for over 45 minutes before starting. I am running the Cuda version--runtime=nvidia``` and passing through the cards, ids, etc. Everything worked before updating. Any idea where to start? I already uninstalled it 6 seperate times using new folders for everything, using different versions.

brave mistBOT Jul 4, 2024, 1:48 AM

#

:wave: Hey @rugged tide,

Thanks for reaching out to us. Please follow the recommended actions below; this will help us be more effective in our support effort and leave more time for building Immich immich .

References

Container Logs: docker compose logs docs
Container Status: docker compose ps docs
Reverse Proxy: https://immich.app/docs/administration/reverse-proxy

Checklist

:ballot_box_with_check: I have verified I'm on the latest release(note that mobile app releases may take some time).
:ballot_box_with_check: I have read applicable release notes.
:ballot_box_with_check: I have reviewed the FAQs for known issues.
:ballot_box_with_check: I have reviewed Github for known issues.
:ballot_box_with_check: I have tried accessing Immich via local ip (without a custom reverse proxy).
:ballot_box_with_check: I have uploaded the relevant logs, docker compose, and .env files using the buttons below or the /upload command.
:ballot_box_with_check: I have tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable

(an item can be marked as "complete" by reacting with the appropriate number)

If this ticket can be closed you can use the /close command, and re-open it later if needed.

brave mistBOT Jul 4, 2024, 1:48 AM

#

brave mist :wave: Hey <@372172031397068803>, Thanks for reaching out to us. Please follow ...

Successfully submitted, a tag has been added to inform contributors. :white_check_mark:

rugged tide Jul 4, 2024, 2:48 AM

#

It still loads but it's taking over 45 minutes instead of the 8 it was

steep pollen Jul 4, 2024, 3:18 AM

#

🤔

#

Any more info that you can provide?

rugged tide Jul 4, 2024, 5:02 AM

#

steep pollen Any more info that you can provide?

I've got no real ideas on what to provide, that's the problem. It's not showing any errors, just halting now for a long time before resuming. It's a random amount of time, I even updated unRAID as my last resort (to the newest beta). Is there anything I could grab for you? The config is the same as it's been for well over a month, default using the internal redis container. Redis starts up first and appears to be all good so I'm lost.

#

It's not overall a huge problem as I've got it mostly dialed in where I want it. Though it is a bit annoying to have to take a whole nap between setting changes 🤣. Works perfectly fine after it starts up. Even removed all the Nvidia, Cuda, etc, from it just running it barebones.

#

Using a separate docker container for redis didn't have an effect other than being a bit of a hassle so I just switched it back to using the internal one.

#

I've been over forums for "similar" issues but they didn't really apply/were different. Tried their fixes anyways no difference. (I'd be more specific on that but I honestly don't remember at this point it's been a hectic day getting family in)

heady gyro Jul 4, 2024, 8:13 AM

#

Does this happen every time the container restarts?

rugged tide Jul 4, 2024, 12:47 PM

#

heady gyro Does this happen every time the container restarts?

Every time now

#

Even did a whole new docker image, installed Immich first and had the same thing.

heady gyro Jul 4, 2024, 1:02 PM

#

Can you try to change the ownership for your folders mounted to /config and /photos to nobody:users ?

rugged tide Jul 4, 2024, 1:10 PM

#

#

Already been set that way

#

Both folders* not just the config

heady gyro Jul 4, 2024, 1:51 PM

#

Can you ssh in the immich container and tell me what’s the output for these commands:
time find /app/immich -path "*/node_modules" -prune -o -exec chown abc:abc {} +

#

Can you re-execute that command ?

rugged tide Jul 4, 2024, 2:35 PM

#

heady gyro Can you ssh in the immich container and tell me what’s the output for these comm...

heady gyro Jul 4, 2024, 2:48 PM

#

Are you using cuda for the gpu acceleration ?

rugged tide Jul 4, 2024, 2:52 PM

#

Yes

#

GTX 1060

#

Even fiddled with different Nvidia drivers

#

No change

#

It's working with Plex and Frigate for transcoding/detection so I'm not really sure if that has anything to do with it.

heady gyro Jul 4, 2024, 2:58 PM

#

GPU acceleration just changed for immich

#

Can you replace the latest docker tag by cuda?

rugged tide Jul 4, 2024, 2:59 PM

#

I can, though I tried that the other day too

heady gyro Jul 4, 2024, 2:59 PM

#

Same issue ?

rugged tide Jul 4, 2024, 2:59 PM

#

I rolled back 3 versions with the Cuda tag

#

And 3 just regular ones

rugged tide Jul 4, 2024, 2:59 PM

#

heady gyro Same issue ?

Yeah

#

It's really strange

#

Any other tests I can do for you?

#

I'm heading out to work in 2 hours

heady gyro Jul 4, 2024, 3:01 PM

#

Yep please

#

Can you execute that command

#

lsiown -R abc:abc
/config

#

And

#

lsiown -R abc:abc
/usr/src/resources

rugged tide Jul 4, 2024, 3:02 PM

#

also

heady gyro Jul 4, 2024, 3:02 PM

#

One of the directory causes the issue

rugged tide Jul 4, 2024, 3:02 PM

#

running those now

#

restart after?

heady gyro Jul 4, 2024, 3:03 PM

#

Nope, not needed

rugged tide Jul 4, 2024, 3:07 PM

#

Just ran both of them, now what?

heady gyro Jul 4, 2024, 3:08 PM

#

Damn sorry I forgot the time before each command

rugged tide Jul 4, 2024, 3:08 PM

#

darn

#

lol

heady gyro Jul 4, 2024, 3:08 PM

#

😅😅

rugged tide Jul 4, 2024, 3:08 PM

#

well

#

uh

#

the first command went from 10:03 to now

#

so about uh 5 minutes

#

ig

#

the second one finished instantly

#

sooo

#

idk if that helps

#

Anything else?

heady gyro Jul 4, 2024, 3:12 PM

#

Can you ssh in the container and run find /config -type f | wc -l

rugged tide Jul 4, 2024, 3:12 PM

#

running now

#

done

heady gyro Jul 4, 2024, 3:14 PM

#

With /config right ?

rugged tide Jul 4, 2024, 3:14 PM

#

yep

#

oh

#

yeah

#

my /photos folder is in there

#

always has been

#

running off my cache drive

#

makes my disks spin up less

heady gyro Jul 4, 2024, 3:14 PM

#

Yep but the issue is on /config

#

Sorry I’m not at home so I can’t check quickly. Is there a cuda folder in /config

#

?

rugged tide Jul 4, 2024, 3:16 PM

#

let me check

#

#

that's all thats in there

#

anything else?

heady gyro Jul 4, 2024, 3:19 PM

#

In machine-learning/

rugged tide Jul 4, 2024, 3:19 PM

#

oh

#

yep

#

whoops, sorry

heady gyro Jul 4, 2024, 3:30 PM

#

You can delete the cuda folder

#

Can you restart after that?

rugged tide Jul 4, 2024, 3:33 PM

#

Checking

#

Restarting now

#

It appears to be starting but it'll be a while before it actually launches

#

Previously I would start it and it would take a minute

#

then it would start installing all the extra packages for Nvidia/Cuda I believe

#

now it's just waiting on that one screen for half an hour before doing that

#

I'll update you after it starts.

heady gyro Jul 4, 2024, 3:39 PM

#

Can you update the docker image ?

rugged tide Jul 4, 2024, 3:39 PM

#

holy crap

#

it just started

#

5 minutes is the fast its been in a while

#

Will it still use my gpu even without the cuda folder or hows that working?

#

aw crap

#

the web gui just crashed

#

heady gyro Jul 4, 2024, 3:42 PM

#

rugged tide Will it still use my gpu even without the cuda folder or hows that working?

If you’re using the cuda tag, yes

heady gyro Jul 4, 2024, 3:42 PM

#

rugged tide

Can you share your docker logs ?

rugged tide Jul 4, 2024, 3:43 PM

#

heady gyro Jul 4, 2024, 3:43 PM

#

Nothing wrong here 🤔🤔

rugged tide Jul 4, 2024, 3:44 PM

#

Absolutely crazy

#

Now it just started working again???

#

My cpu usage didn't even change the whole time

#

not like it bogged out

#

Oh well I got to start getting ready

#

leave anything else I should do in here

heady gyro Jul 4, 2024, 3:47 PM

#

Niceeeeee

rugged tide Jul 4, 2024, 3:48 PM

#

It's not like it was wifi

#

Was still able to access other containers and Unraid

#

🤷‍♂️

#

It just hates me 🤣

rugged tide Jul 4, 2024, 4:09 PM

#

...

#

#

it

#

is

#

doing

#

it

#

again

#

fml

heady gyro Jul 4, 2024, 4:35 PM

#

Can you confirm you updated the docker image and you use the cuda tag ?

rugged tide Jul 4, 2024, 4:37 PM

#

I did that about 25 minutes ago

#

Still hasn't started yet

#

Changed :latest to :cuda

heady gyro Jul 4, 2024, 4:39 PM

#

What is the folder with the most files?

rugged tide Jul 4, 2024, 4:51 PM

#

Overall? In config it would be /photos where I keep everything

heady gyro Jul 4, 2024, 6:00 PM

#

Nope in /config

rugged tide Jul 4, 2024, 6:38 PM

#

#

Other than that it would be machine learning models

heady gyro Jul 4, 2024, 6:59 PM

#

Just to be sure, has the cuda folder be re-installed or not ?

rugged tide Jul 4, 2024, 7:13 PM

#

#

It has not

heady gyro Jul 4, 2024, 7:15 PM

#

Nice

rugged tide Jul 4, 2024, 7:27 PM

#

well its back to starting slow

#

so ground zero but even less of an idea

heady gyro Jul 4, 2024, 7:30 PM

#

You can to try to see which folder causes the issue with the commands I sent before

rugged tide Jul 5, 2024, 2:27 AM

#

Aight

rugged tide Jul 5, 2024, 4:17 AM

#

rugged tide Jul 5, 2024, 4:18 AM

#

rugged tide

#

rugged tide Jul 5, 2024, 4:31 AM

#

rugged tide

42 minutes and it started to start

heady gyro Jul 5, 2024, 6:05 AM

#

You can try to delete /config entirely

#

It just contains the machine-learning models which are re-downloaded if not find

rugged tide Jul 5, 2024, 6:19 AM

#

Done, also moved everything out of that folder too just incase it was somehow scanning it and going through all the thumbs (incase that could somehow slow it down) but not change

#

honestly the most interesting bug I've seen with my limited experiences with docker

rugged tide Jul 5, 2024, 6:20 AM

#

rugged tide Done, also moved everything out of that folder too just incase it was somehow sc...

eh

#

minor change I guess

#

normally it would hold that for like 30 seconds

#

then that would pop up after

#

#

now both are popping up quicker

#

but still not starting quickly

heady gyro Jul 5, 2024, 6:24 AM

#

What takes the most time?

rugged tide Jul 5, 2024, 6:25 AM

#

at this point I'm not sure

#

both of those pop up like one after the other now

#

pretty darn quick

#

then it just holds on that screen for over 30 minutes

#

before proceding to actually start immich

heady gyro Jul 5, 2024, 6:40 AM

#

Is this command helping? time find /config -maxdepth 0 ( ! -user abc -o ! -group abc ) -exec lsiown -R abc:abc {} ;

rugged tide Jul 5, 2024, 6:42 AM

#

I am too braindead to figure out where it goes right now lmao

#

I haven't slept since 2 days ago whoops

heady gyro Jul 5, 2024, 6:44 AM

#

rugged tide

Inside your immich container ?

rugged tide Jul 5, 2024, 6:44 AM

#

yeah when I tried running it

heady gyro Jul 5, 2024, 6:44 AM

#

rugged tide I haven't slept since 2 days ago whoops

Whooops, go to sleep

rugged tide Jul 5, 2024, 6:44 AM

#

nah

#

I got things to do

#

and sleeping isn't one of them

#

hectic schedules + family is

#

well

#

chaos

heady gyro Jul 5, 2024, 6:45 AM

#

rugged tide yeah when I tried running it

It’s working fine in my immich container 🤔🤔🤔

rugged tide Jul 5, 2024, 6:45 AM

#

time find /config -maxdepth 0 this part works

#

uh

#

past that it starts not working

heady gyro Jul 5, 2024, 6:46 AM

#

Does /config exist ?

rugged tide Jul 5, 2024, 6:46 AM

#

it does

#

with only the first half of that command

#

heady gyro Jul 5, 2024, 6:48 AM

#

How are you ssh in the immich container ?

#

With docker exec -it immich bash ?

rugged tide Jul 5, 2024, 6:48 AM

#

yep

heady gyro Jul 5, 2024, 6:49 AM

#

Man wtf, docker exec -it Immich bash
root@8d22a22c78fd:/# time find /config -maxdepth 0 ( ! -user abc -o ! -group abc ) -exec lsiown -R abc:abc {} ;

real 0m0.001s
user 0m0.000s
sys 0m0.001s
root@8d22a22c78fd:/#

#

It’s working just fine for me

rugged tide Jul 5, 2024, 6:49 AM

#

hell let me run it again

#

wtf

#

it just ran

heady gyro Jul 5, 2024, 6:50 AM

#

Is your /config on a ssd or hdd?

rugged tide Jul 5, 2024, 6:50 AM

#

what the hell

heady gyro Jul 5, 2024, 6:50 AM

#

rugged tide it just ran

Niceee

rugged tide Jul 5, 2024, 6:50 AM

#

wtf

#

softwares more bi-polar than my MIL

heady gyro Jul 5, 2024, 6:51 AM

#

rugged tide

That’s not what I expected

#

😅

rugged tide Jul 5, 2024, 6:51 AM

#

NOW ITS NOT WORKING AGAIN

#

WTF

#

what the hell

#

Software is weird

#

Mechanical parts are weirder

#

oddly enough both can sometimes be fixed with a hammer

heady gyro Jul 5, 2024, 7:26 AM

#

I'm not sure this will work the way you want 😂😂

rugged tide Jul 5, 2024, 7:44 AM

#

I MEAN NOTHING ELSE IS WORKING

#

HAMMERS FIX MOST PROBLEMS

rugged tide Jul 6, 2024, 3:04 PM

#

It's back to working normally

#

I

#

Give

#

Up

#Immich randomly broke upon updating.

References

Checklist