#500 Error - Failed to get server features

1 messages · Page 1 of 1 (latest)

white stream
#

Started to get these errors recently, have not seen it prior to 1.103.1. Restarting the stack fixes the issue for a while, but have not been able to correlate to anything triggering it however between my last stack restart and last time it crashed, only activity would have been background uploads from connected Android apps.

Failed to get server config (500)

Stacktrace
Error: Error: 500
at Object.he [as ok] (http://10.10.20.50:2283/_app/immutable/chunks/fetch-client.FL1-MUp9.js:1:2835)
at async He (http://10.10.20.50:2283/_app/immutable/nodes/31.CRSG-kf6.js:1:1094)
at async ke (http://10.10.20.50:2283/_app/immutable/chunks/entry.Kq2O8KLA.js:1:13443)

Since restarting this morning after noticing the last crash, it did not last long and I cannot view logs beyond the last line here:

[Nest] 8 - 05/12/2024, 11:50:01 AM LOG [ImmichServer] Immich Server is listening on http://[::1]:3001 [v1.103.1] [PRODUCTION]
[Nest] 8 - 05/12/2024, 11:50:18 AM LOG [EventRepository] Websocket Connect: gvMeNB5XV37xEkW4AAAB
[Nest] 8 - 05/12/2024, 11:50:30 AM LOG [EventRepository] Websocket Connect: xvCH0o0Iuo45_hRhAAAD
[Nest] 8 - 05/12/2024, 11:51:46 AM LOG [EventRepository] Websocket Disconnect: xvCH0o0Iuo45_hRhAAAD
[Nest] 8 - 05/12/2024, 11:56:50 AM LOG [EventRepository] Websocket Connect: fPhdkQdW-pGhE6JHAAAJ
#Removed a number of logs above and below due to Discord 2000 char limit, but they were all Websocket connect/disconnects.
[Nest] 8 - 05/12/2024, 6:08:26 PM LOG [EventRepository] Websocket Connect: pa8PuXvAvrJ2hUWfAAAi
[Nest] 8 - 05/12/2024, 6:09:01 PM LOG [EventRepository] Websocket Disconnect: pa8PuXvAvrJ2hUWfAAAi
[Nest] 8 - 05/12/2024, 9:03:01 PM LOG [EventRepository] Websocket Connect: Rc4n1LO1huMZy1DAAAAk
error from daemon in stream: Error grabbing logs: invalid character 'l' after object key:value pair

image.png
vocal hatchBOT
#

:wave: Hey @white stream,

Thanks for reaching out to us. Please follow the recommended actions below; this will help us be more effective in our support effort and leave more time for building Immich immich.

References

Checklist

  1. :ballot_box_with_check: I have verified I'm on the latest release(note that mobile app releases may take some time).
  2. :ballot_box_with_check: I have read applicable release notes.
  3. :ballot_box_with_check: I have reviewed the FAQs for known issues.
  4. :ballot_box_with_check: I have reviewed Github for known issues.
  5. :ballot_box_with_check: I have tried accessing Immich via local ip (without a custom reverse proxy).
  6. :blue_square: I have uploaded the relevant logs, docker compose, and .env files using the buttons below or the /upload command.
  7. :ballot_box_with_check: I have tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable

(an item can be marked as "complete" by reacting with the appropriate number)

If this ticket can be closed you can use the /close command, and re-open it later if needed.

paper storm
#

then we can capture the logs next time

#

Easiest way is probably to rm the offending container and re-up it

vocal hatchBOT
white stream
#

Interesting... looked in Portainer, the immich_postgres container was in a "running" status. However drilling in to it, the Start option was available. After clicking it started and now the UI works.

I have a feeling root cause has still not been solved and I'll run into this soon enough.

paper storm
#

Based on everything so far I think you might have an issue with portainer/docker tbh, not Immich itself

#

log corruption/issues, containers running but not running, etc

#

how good is the system you are running on and what OS/setup is it?

white stream
#

System is pretty capable. R740xd, specs in screenshot of physical host and Portainer VM. (I just dumped over 1100 photos and videos from the weekend to an external library, so CPU is high on Portainer.)

Ubuntu 22.04.4
Portainer 2.19.4

Last I ran apt-get upgrade back on 5/6. As you probe for these questions, I would probably say I did not see the 500 error prior to the 6th. 🤔

Start-Date: 2024-05-06 17:31:27
Commandline: apt-get upgrade
Requested-By: xxxxx
Upgrade: containerd.io:amd64 (1.6.28-2, 1.6.31-1), docker-compose-plugin:amd64 (2.25.0-1~ubuntu.22.04~jammy, 2.27.0-1~ubuntu.22.04~jammy), docker-ce-cli:amd64 (5:26.0.0-1~ubuntu.22.04~jammy, 5:26.1.1-1~ubuntu.22.04~jammy), openssh-client:amd64 (1:8.9p1-3ubuntu0.6, 1:8.9p1-3ubuntu0.7), qemu-guest-agent:amd64 (1:6.2+dfsg-2ubuntu6.18, 1:6.2+dfsg-2ubuntu6.19), openssh-server:amd64 (1:8.9p1-3ubuntu0.6, 1:8.9p1-3ubuntu0.7), landscape-common:amd64 (19.12-0ubuntu13, 23.02-0ubuntu1~22.04.2), docker-buildx-plugin:amd64 (0.13.1-1~ubuntu.22.04~jammy, 0.14.0-1~ubuntu.22.04~jammy), docker-ce:amd64 (5:26.0.0-1~ubuntu.22.04~jammy, 5:26.1.1-1~ubuntu.22.04~jammy), ubuntu-pro-client-l10n:amd64 (31.2~22.04, 31.2.2~22.04), cloud-init:amd64 (23.4.4-0ubuntu0~22.04.1, 24.1.3-0ubuntu1~22.04.1), docker-ce-rootless-extras:amd64 (5:26.0.0-1~ubuntu.22.04~jammy, 5:26.1.1-1~ubuntu.22.04~jammy), distro-info-data:amd64 (0.52ubuntu0.6, 0.52ubuntu0.7), openssh-sftp-server:amd64 (1:8.9p1-3ubuntu0.6, 1:8.9p1-3ubuntu0.7), ubuntu-advantage-tools:amd64 (31.2~22.04, 31.2.2~22.04), ubuntu-pro-client:amd64 (31.2~22.04, 31.2.2~22.04)
End-Date: 2024-05-06 17:32:05

image.png image.png
paper storm
#

oh yeah, you have a monster system

#

I would honestly put my money on some kind of containerd/portainer issue

#

are you opposed to switching to a plaintext docker compose deployment? will that break your workflow?

#

oh while I can pick your brain I am looking for another hypervisor system. Only used single socket so far. Have you run into many NUMA issues with a 2 socket server?