#Corrupted migration after upgrading to 1.133.1

1 messages · Page 1 of 1 (latest)

ashen mason
#

Hi all, first of all thank you for developing this fantastic program. I've recommended this to massive of my friends.

I'm having a problem with 1.133.1. I followed the upgrade commands and these worked perfectly last time I upgraded. But this time the server is down and outputs something like corrupted migrations.

Is there any way to get it back work other than restoring... I'm so afraid of losing images and since I have 10k+ huge images and videos, it could take super long time to recreate the daataset.

If I have to recreate the db, how should I do it? I now have a backup.sql file ready, which I created after encountering this migration issue.

📎 message.txt 📎 env 📎 docker-compose.yml
midnight archBOT
#

:wave: Hey @ashen mason,

Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich immich.

References

#

Checklist

I have...

  1. :blue_square: verified I'm on the latest release(note that mobile app releases may take some time).
  2. :blue_square: read applicable release notes.
  3. :blue_square: reviewed the FAQs for known issues.
  4. :blue_square: reviewed Github for known issues.
  5. :blue_square: tried accessing Immich via local ip (without a custom reverse proxy).
  6. :blue_square: uploaded the relevant information (see below).
  7. :blue_square: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable

(an item can be marked as "complete" by reacting with the appropriate number)

Information

In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider:

  • Your docker-compose.yml and .env files.
  • Logs from all the containers and their status (see above).
  • All the troubleshooting steps you've tried so far.
  • Any recent changes you've made to Immich or your system.
  • Details about your system (both software/OS and hardware).
  • Details about your storage (filesystems, type of disks, output of commands like fdisk -l and df -h).
  • The version of the Immich server, mobile app, and other relevant pieces.
  • Any other information that you think might be relevant.

Please paste files and logs with proper code formatting, and especially avoid blurry screenshots.
Without the right information we can't work out what the problem is. Help us help you ;)

If this ticket can be closed you can use the /close command, and re-open it later if needed.

ashen mason
#

I'm running on Docker v4.41.2 on a Wins11, harddisk of 8TB. I tried running the upgrade commands again. I haven't revised anything on docker-compose.yml or .env files since a few months ago.
immich_server keeps the status "started".
✔ Container immich_redis Running 0.0s
✔ Container immich_machine_learning Running 0.0s
✔ Container immich_postgres Running 0.0s
✔ Container immich_server Started

midnight archBOT
ebon falcon
#

Please read and apply the breaking changes. You should always read the release notes.

solar sage
#

did you happen to switch to main branch at any time?

ashen mason
dull socket
#

I'm not sure how you could get into that situation without running the main branch at some point

ashen mason
#

I'm not sure about what the main branch refers to. For me I didn't do anything than:
docker compose down
docker compose pull
docker image prune
docker compose up -d

ashen mason
#

Ok after carefully following the breaking changes, I still got the migration problem:
microservices worker error: Error: corrupted migrations: expected previously executed migration 1747664684909-AddAlbumAuditTables to be at index 3 but 1745902563899-AddAssetVisibilityColumn was found in its place. New migrations must always have a name that comes alphabetically after the last executed migration., stack: Error: corrupted migrations: expected previously executed migration 1747664684909-AddAlbumAuditTables to be at index 3 but 1745902563899-AddAssetVisibilityColumn was found in its place. New migrations must always have a name that comes alphabetically after the last executed migration.

I followed the TIP when restoring dataset:
Some deployment methods make it difficult to start the database without also starting the server. In these cases, you may set the environment variable DB_SKIP_MIGRATIONS=true before starting the services. This will prevent the server from running migrations that interfere with the restore process. Be sure to remove this variable and restart the services after the database is restored.

It works well with the variable = true. However, when I set it back to false (I mean delete this line), the same migration confliction problem pops up, and prevents me again logging in the page.

📎 docker-compose.yml
ashen mason
#

Since I'm not only storing my own pictures but also my girlfriend's album, I'm really afraid of losing any of the data. Can anyone help?

ebon falcon
#

I would like to note you are running the database on NTFS

#

If you have a backup from before 1.133.1 you could set IMMICH_VERSION=v.1.132.3 , create the pgdata volume like the recommendation says, and then restore to that version before upgrading to release later

#

Also, because you are worrying about the pictures, the database has metadata only, the pictures aren't touched

ashen mason
#

Thank you! How could I know which version I was using?

#

Ops, I did store the db in NTFS

ebon falcon
#

like when was the last time you updated before that

#

It's uh... stored in the database

ashen mason
#

Here is what I recently did. Now I already upgraded to 1.133, then [System.IO.File]::WriteAllLines("C:\absolute\path\to\backup\dump.sql", (docker exec -t immich_postgres pg_dumpall --clean --if-exists --username=postgres)). Following https://immich.app/docs/administration/backup-and-restore. In this sense, it seems to me that the database is 1.133?

ashen mason
# ebon falcon Also, because you are worrying about the pictures, the database has metadata onl...

Guess I could do a clean reset right? without hurting the meta data. Then I'm wondering how to import the folder to the new created server.

  1. I have an external library, which is totally fine.
  2. For the recent months I uploaded my newly captured photos to UPLOAD_LOCATION=E:\Immich\library. But I could not find a place to import the folder in the webpage.
    my .yml file:
    services:
    immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}

    extends:

    file: hwaccel.transcoding.yml

    service: cpu # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding

    volumes:

    Do not edit the next line. If you want to change the media storage location on your system, edit the value of UPLOAD_LOCATION in the .env file

    • ${UPLOAD_LOCATION}:/usr/src/app/upload
    • /etc/localtime:/etc/localtime:ro
    • E:/Photos:/mnt/media/photos

.env file:

You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

The location where your uploaded files are stored

UPLOAD_LOCATION=E:\Immich\library

The location where your database files are stored

DB_DATA_LOCATION=E:\Immich\postgres

To set a timezone, uncomment the next line and change Etc/UTC to a TZ identifier from this list: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List

TZ=Europe/Stockholm

The Immich version to use. You can pin this to a specific version like "v1.71.0"

IMMICH_VERSION=release

Connection secret for postgres. You should change it to a random password

Please use only the characters A-Za-z0-9, without special characters or spaces

DB_PASSWORD=postgres

The values below this line do not need to be changed

###################################################################################
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

ebon falcon
#

Do you only have one database backup?

ashen mason
#

There are a few automatically saved backup. I could find a folder called backup inside library. Looks like this

image.png
#

I guess it's automatically saved cuz I don't have a memory for creating those..

ebon falcon
#

Exactly, that's the daily backup

#

-backup +dump

#

(but yeah)

#

Are you turning your server on/off a lot?

ashen mason
#

Will it work? Those are 'only' 500mb, but if I [System.IO.File]::WriteAllLines, the sql file is 1.2GB or sth.

ebon falcon
#

these are .gz files, they are compressed

ashen mason
ashen mason
ebon falcon
#

It means that it's technically not a backup, it's a dump, because semantics matter

ebon falcon
#

Anyhooo

#

From the backup restore instructions:

If your backup ends in .gz, replace cat with gunzip --stdout

#

But you still need to mount it to the DB container of course

ashen mason
#

If I'm considering resetting the whole stuff.. Perhaps it's easier for me. Then how could I import the uploaded folder?

ebon falcon
#

Did you use storage templating or not?

ashen mason
#

Yeah it looks like

image.png
ebon falcon
#

and resetting will require all of the ML to run again right

ashen mason
#

yeah, it's only 100k photos, not gonna take long🥲

ebon falcon
#

The easiest would be using immich-go and doing a recursive search on each user's directory for their account

#

Note that you'll need to switch API keys between users

ashen mason
#

I haven't used Immich for very long, so the ML and face detection are ok to dispose.

ashen mason