#Server never starts up after upgrading to v1.118.2

1 messages · Page 1 of 1 (latest)

trim verge
#

Hello,
I've recently updated the images to use v1.118.2, as well as the Helm chart to use 0.8.2, however this results in the server never starting up.

Logs from one of the server instances:

Initializing Immich v1.118.2
Detected CPU Cores: 4
Starting api worker
Starting microservices worker
[Nest] 7  - 10/23/2024, 4:22:51 PM     LOG [Microservices:EventRepository] Initialized websocket server
[Nest] 17  - 10/23/2024, 4:22:51 PM     LOG [Api:EventRepository] Initialized websocket server
[Nest] 7  - 10/23/2024, 4:23:14 PM     LOG [Microservices:MapRepository] Initializing metadata repository

However, even after 10 minutes of waiting, the startup probe keeps failing as the server never starts:

Startup probe failed: Get "http://10.42.2.70:2283/api/server/ping": dial tcp 10.42.2.70:2283: connect: connection refused

Both, PostgreSQL and Redis instances are working fine and there seems to be no issues in their logs.

image.png
sacred dockBOT
#

:wave: Hey @trim verge,

Thanks for reaching out to us. Please follow the recommended actions below; this will help us be more effective in our support effort and leave more time for building Immich immich.

References

Checklist

  1. :ballot_box_with_check: I have verified I'm on the latest release(note that mobile app releases may take some time).
  2. :ballot_box_with_check: I have read applicable release notes.
  3. :ballot_box_with_check: I have reviewed the FAQs for known issues.
  4. :ballot_box_with_check: I have reviewed Github for known issues.
  5. :ballot_box_with_check: I have tried accessing Immich via local ip (without a custom reverse proxy).
  6. :ballot_box_with_check: I have uploaded the relevant logs, docker compose, and .env files, making sure to use code formatting.
  7. :ballot_box_with_check: I have tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable

(an item can be marked as "complete" by reacting with the appropriate number)

If this ticket can be closed you can use the /close command, and re-open it later if needed.

sacred dockBOT
trim verge
#

I don't see how my values.yaml might influence this outcome, but in any case I'm pasting it here as well:

immich:
  env:
    # secrets...

  persistence:
    external:
      enabled: true
      type: nfs
      server: 10.0.2.219
      path: /volume1/NAS/Sync
      globalMounts:
      - path: /mnt/external

  immich:
    metrics:
      enabled: true
    persistence:
      library:
        existingClaim: immich-pvc
# Dependencies omitted for brevity
# ...
  server:
    enabled: true
    controller:
      replicas: 3
      pod:
        topologySpreadConstraints:
        - maxSkew: 1
          topologyKey: kubernetes.io/hostname
          whenUnsatisfiable: DoNotSchedule
          labelSelector:
            matchLabels:
              app.kubernetes.io/instance: immich
    image:
      repository: ghcr.io/immich-app/immich-server
      pullPolicy: IfNotPresent
      tag: v1.118.2
    ingress:
      main:
        enabled: true
        ingressClassName: nginx
        annotations:
          kubernetes.io/ingress.class: nginx
          cert-manager.io/cluster-issuer: letsencrypt-prod
          nginx.ingress.kubernetes.io/ssl-passthrough: "true"
          nginx.ingress.kubernetes.io/proxy-body-size: "0"
        hosts:
        - host: &host <MY_HOST>
          paths:
          - path: /
            pathType: Prefix
        tls:
        - secretName: immich-tls-certificate
          hosts:
          - *host

  machine-learning:
    enabled: true
    image:
      repository: ghcr.io/immich-app/immich-machine-learning
      pullPolicy: IfNotPresent
      tag: v1.118.2
    env:
    - name: TRANSFORMERS_CACHE
      value: /cache
    persistence:
      cache:
        enabled: true
        size: 10Gi
        type: pvc
        accessMode: ReadWriteMany
        storageClass: nfs-nas-client
crystal mortar
#

Hello,

I've similar problem, my app start but the status has stuck "Deploying" because the container pgbouncer is broken ...
ix-immich immich-cnpg-main-rw-6487f489b9-98gq2 0/1 CrashLoopBackOff 8 (75s ago) 17

{"level":"error","ts":"2024-10-23T13:37:52Z","msg":"Error while running manager","error":"while initializing reconciler: while reading secrets: while getting auth query secret immich-cnpg-main-pooler: secrets \"immich-cnpg-main-pooler\" not found","stacktrace":"github.com/cloudnative-pg/cloudnative-pg/pkg/management/log.(*logger).Error\n\tpkg/management/log/log.go:128\ngithub.com/cloudnative-pg/cloudnative-pg/pkg/management/log.Error\n\tpkg/management/log/log.go:166\ngithub.com/cloudnative-pg/cloudnative-pg/internal/cmd/manager/pgbouncer/run.NewCmd.func2\n\tinternal/cmd/manager/pgbouncer/run/cmd.go:68\ngithub.com/spf13/cobra.(*Command).execute\n\tpkg/mod/github.com/spf13/[email protected]/command.go:983\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\tpkg/mod/github.com/spf13/[email protected]/command.go:1115\ngithub.com/spf13/cobra.(*Command).Execute\n\tpkg/mod/github.com/spf13/[email protected]/command.go:1039\nmain.main\n\tcmd/manager/main.go:64\nruntime.main\n\t/opt/hostedtoolcache/go/1.21.5/x64/src/runtime/proc.go:267"}

i've another instance of immich (test) without problem ...

prisma crane
#

For the startup being stuck: make sure the postgres pod doesn't have ridiculously low resource limits

crystal mortar