#Immich keeps crashing with db connection timeout but DB logs show no errors

1 messages · Page 1 of 1 (latest)

burnt venture
#

Server crashed overnight while I was doing a very large photo upload via immich-go. Restarted the server several times this morning. Immich takes forever to boot, then the web interface loads but as soon as you access it, I get the following errors in logs. Immich is runing in docker containers using Saltbox setup script.

[Nest] 203 - 08/22/2024, 10:05:45 AM ERROR [Api:ExceptionsHandler~5ipotsbq] Connection terminated due to connection timeout Error: Connection terminated due to connection timeout at Connection.<anonymous> (/app/immich/server/node_modules/pg/lib/client.js:132:73) at Object.onceWrapper (node:events:633:28) at Connection.emit (node:events:519:28) at Socket.<anonymous> (/app/immich/server/node_modules/pg/lib/connection.js:63:12) at Socket.emit (node:events:519:28) at TCP.<anonymous> (node:net:339:12) at TCP.callbackTrampoline (node:internal/async_hooks:130:17

DB logs show no errors (apart from chmod error which I dont think is a problem)

PostgreSQL Database directory appears to contain a database; Skipping initialization

`2024-08-22 09:22:23.578 PDT [1] LOG: redirecting log output to logging collector process
2024-08-22 09:22:23.578 PDT [1] HINT: Future log output will appear in directory "log".
chmod: changing permissions of '/var/run/postgresql': Operation not permitted

PostgreSQL Database directory appears to contain a database; Skipping initialization

2024-08-22 09:33:25.376 PDT [1] LOG: redirecting log output to logging collector process
2024-08-22 09:33:25.376 PDT [1] HINT: Future log output will appear in directory "log".`

upbeat wedgeBOT
#

:wave: Hey @burnt venture,

Thanks for reaching out to us. Please follow the recommended actions below; this will help us be more effective in our support effort and leave more time for building Immich immich.

References

Checklist

  1. :ballot_box_with_check: I have verified I'm on the latest release(note that mobile app releases may take some time).
  2. :ballot_box_with_check: I have read applicable release notes.
  3. :ballot_box_with_check: I have reviewed the FAQs for known issues.
  4. :ballot_box_with_check: I have reviewed Github for known issues.
  5. :ballot_box_with_check: I have tried accessing Immich via local ip (without a custom reverse proxy).
  6. :ballot_box_with_check: I have uploaded the relevant logs, docker compose, and .env files, making sure to use code formatting.
  7. :ballot_box_with_check: I have tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable

(an item can be marked as "complete" by reacting with the appropriate number)

If this ticket can be closed you can use the /close command, and re-open it later if needed.

upbeat wedgeBOT
unkempt sand
#

you are probably increasing the job concurrency, and it push the limit of your disk IO

#

If you can, slowly increasing the job concurrency or keep it as default values

burnt venture
#

ive not changed any of the defaults, and its still crashing after several restarts

#

also, server runs many other apps which are not affected

#

will run some i/o diagnostics

burnt venture
#

@unkempt sand thanks for your help, will mark this as complete as it looks like it was an IO issue, however note that everything was set to default