#Immich crashed for no reason

1 messages · Page 1 of 1 (latest)

drowsy shore
#

immich crashed for no reason.
I tried to add an external library and the server didnt start in 800s (normaly it starts in 500) and i reversed the change in docker.yml and the server came live after i dont know like 1200s and it shows running without problems, but the web and nothing works
How can i explain that in help desk

rare swallowBOT
#

:wave: Hey @drowsy shore,

Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich immich.

References

#

Checklist

I have...

  1. :blue_square: verified I'm on the latest release(note that mobile app releases may take some time).
  2. :blue_square: read applicable release notes.
  3. :blue_square: reviewed the FAQs for known issues.
  4. :blue_square: reviewed Github for known issues.
  5. :blue_square: tried accessing Immich via local ip (without a custom reverse proxy).
  6. :blue_square: uploaded the relevant information (see below).
  7. :blue_square: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable

(an item can be marked as "complete" by reacting with the appropriate number)

Information

In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider:

  • Your docker-compose.yml and .env files.
  • Logs from all the containers and their status (see above).
  • All the troubleshooting steps you've tried so far.
  • Any recent changes you've made to Immich or your system.
  • Details about your system (both software/OS and hardware).
  • Details about your storage (filesystems, type of disks, output of commands like fdisk -l and df -h).
  • The version of the Immich server, mobile app, and other relevant pieces.
  • Any other information that you think might be relevant.

Please paste files and logs with proper code formatting, and especially avoid blurry screenshots.
Without the right information we can't work out what the problem is. Help us help you ;)

If this ticket can be closed you can use the /close command, and re-open it later if needed.

potent matrix
#

Raspberry pi?

drowsy shore
#

Noup.

potent matrix
#

Then keep posting all the things the bot asked for, sorry :)

drowsy shore
#

Mac mini 2014, intel i5, 16gb ram, 128gb ssd, 1tb hd, running natively ubuntu

#

I tried everything from the list

#

there are no logs in docker

#

just shows as running

#

Also, i was running the 1.140 version

potent matrix
#

You haven't shared your compose file for instance 👀

#

You haven't shared exactly what you changed

drowsy shore
#

Oh, sorry

potent matrix
#

There must be some startup logs

drowsy shore
#

I forgot about that

#

I will do that

#

Under this:

  • ${UPLOAD_LOCATION}:/data
    - /etc/localtime:/etc/localtime:ro
#

I added:

#

something like home/home/DATA_HDD/2004-2023

potent matrix
#

Is your database running?

drowsy shore
#

ea8052d84362 ghcr.io/immich-app/immich-server:release "tini -- /bin/bash -…" 2 days ago Up 2 days (healthy) 0.0.0.0:2283->2283/tcp immich_server
24a094188a05 ghcr.io/immich-app/postgres:14-vectorchord0.4.3-pgvectors0.2.0 "/usr/local/bin/immi…" 2 days ago Up 2 days (unhealthy) 5432/tcp immich_postgres
2007c5451afb ghcr.io/immich-app/immich-machine-learning:release "tini -- python -m i…" 2 days ago Up 2 days (healthy) immich_machine_learning
5bb78f731e51 valkey/valkey:8-bookworm "docker-entrypoint.s…" 2 days ago Up 2 days (healthy) 6379/tcp immich_redis

#

Seems like it is

#

This is on web

potent matrix
#

Can you share the logs?

drowsy shore
restive arch
#

Your database might have corruption. Please follow the instructions in the FAQ

rare swallowBOT
restive arch
#

It’s one of the last ones

drowsy shore
# restive arch Your database might have corruption. Please follow the instructions in the FAQ

emm, i think this isnt ok:
home@home-Macmini:~/Documents/stacks/immich-app$ docker exec -it immich_postgres psql --dbname=postgres --username=postgres --command="SELECT datname, checksum_failures, checksum_last_failure FROM pg_stat_database WHERE datname IS NOT NULL"
psql: error: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: Resource temporarily unavailable
Is the server running locally and accepting connections on that socket?
home@home-Macmini:~/Documents/stacks/immich-app$

potent matrix
#

Maybe try restarting the entire stack

drowsy shore
#

home@home-Macmini:~/Documents/stacks/immich-app$ docker rm -f immich_postgres
Error response from daemon: cannot remove container "/immich_postgres": could not kill: tried to kill container, but did not receive an exit event
home@home-Macmini:~/Documents/stacks/immich-app$

#

I cant. I tried even restarting docker

restive arch
#

You can reboot the host but my guess is the DB is cooked.

drowsy shore
#

So complete restore?

#

But what caused it

restive arch
#

If it’s corrupt, then yes you’d have to restore from backup

#

Usually bad storage device

potent matrix
#

Failing drive?

restive arch
#

Databases are far less fault tolerant to small corruption than most other uses of a drive are, so it’s a common early failure sign

drowsy shore
#

I mean i dont know, but i was transfering like 300gb of images from old and bad hd (external) to immich via immich go and it froze like 3 times completely

restive arch
#

So your drive is failing under load, that sounds exactly right 😅

drowsy shore
#

So that means probably the disk is cooked

#

Shit

#

Even if i had the db configuration on ssd?

restive arch
#

Well the SSD might be what’s failing here

#

It’s whatever PG is stored on

#

But it’s hard to say without more logs or data

drowsy shore
#

Is it possible that this happened because i was transfering data from (probably 1/2 dead) hdd

potent matrix
#

Yes, it could've been broken before

#

Well, did you restore from a database dump or did you copy over the postgres folder?

drowsy shore
#

I will restore it now from database dump probably as this worked for me the last time

potent matrix
#

I meant when you were transferring data over from the dying hdd

restive arch
#

Importing images from a failing disk wouldn’t corrupt a different Postgres disk, no

drowsy shore
#

No. I have hdd internal in the computer with the immich images. Then i went and took an external disk and transfered the images to the internal hdd using immich go.

potent matrix
#

Then what Zeus says

#

The SSD postgres is on, is that an especially cheap one or something?

drowsy shore
#

No, its apple quality.

#

Is it maybe that i tried to add external library to docker compose and it broke the postgres?

potent matrix
#

Well tbf that mac mini is 11 years old though 😅

drowsy shore
#

Cause that was right after that

potent matrix
#

At least most likely

drowsy shore
#

But im to cheap to replace it

#

But i didnt know it is corrupt

#

Ill try and restore the database dumb and ill post more details then

potent matrix
#

There's a chance it's not. As Zeus said it's impossible to tell from here

drowsy shore
#

Will the immich freak out if the database has in it that some user has some pictures that arent in the upload folder

potent matrix
#

It will start up, the assets will simply show up as missing in the UI

drowsy shore
#

How can i fix that then.

#

I mean i added one user for the images i wanted to upload and then deleted all of them as i wanted to add it as an external library

potent matrix
#

How would you end up with a database that lacks some assets?

potent matrix
drowsy shore
#

from immich

potent matrix
drowsy shore
#

Cause i uploaded it one day, then there was the db dump at 2am and then i changed the docker.yml and removed the assets.

#

I will restore the version before the change just to be sure

potent matrix
#

Ah got it

#

Yeah in that case you'll just gonna have to delete the assets from Immich again

drowsy shore
#

from web?

potent matrix
#

You should definitely take backups in general though

potent matrix
drowsy shore
potent matrix
#

Yes

rare swallowBOT
drowsy shore
#

But isnt the automatic database dump enough

potent matrix
#

Read the link

drowsy shore
#

Emm, i need a little help for restoring:
home@home-Macmini:~/Documents/stacks/immich-app$ docker compose down
[+] Running 4/4
✔ Container immich_machine_learning Removed 0.1s
✔ Container immich_server Removed 0.1s
✘ Container immich_postgres Error while Stopping 24.1s
✔ Container immich_redis Removed 0.1s
Error response from daemon: cannot stop container: 24a094188a05454b8a592c74dbffa6efc3521a272096b72bfde8f0909e68608a: tried to kill container, but did not receive an exit event

#

I cant remove it

potent matrix
#

Try killing the docker VM

#

Or restart the host

drowsy shore
drowsy shore
# potent matrix Or restart the host

I made the restore but now immich server is taking forever:
home@home-Macmini:~/Documents/stacks/immich-app$ docker compose up -d
[+] Running 3/4
✔ Container immich_postgres Running 0.0s
✔ Container immich_machine_learning Started 0.4s
✔ Container immich_redis Started 0.3s
⠧ Container immich_server Starting 873.7s