Help with error "[Microservices:DatabaseService] Could not run vector reindexing checks." | Immich | Page 1

waxen vortex Jan 29, 2025, 5:35 AM

#

https://github.com/immich-app/immich/discussions/15401

Hello. I do not really enjoy using discord as it's closed source trash but since it's been over a week without a response to this thread I was wondering if anyone here could help me?

slow quartzBOT Jan 29, 2025, 5:35 AM

#

:wave: Hey @waxen vortex,

Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich immich .

References

Container Logs: docker compose logs docs
Container Status: docker ps -a docs
Reverse Proxy: https://immich.app/docs/administration/reverse-proxy
Code Formatting https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline#h_01GY0DAKGXDEHE263BCAYEGFJA

#

Checklist

I have...

:ballot_box_with_check: verified I'm on the latest release(note that mobile app releases may take some time).
:ballot_box_with_check: read applicable release notes.
:ballot_box_with_check: reviewed the FAQs for known issues.
:ballot_box_with_check: reviewed Github for known issues.
:ballot_box_with_check: tried accessing Immich via local ip (without a custom reverse proxy).
:ballot_box_with_check: uploaded the relevant information (see below).
:ballot_box_with_check: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable

(an item can be marked as "complete" by reacting with the appropriate number)

Information

In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider:

Your docker-compose.yml and .env files.
Logs from all the containers and their status (see above).
All the troubleshooting steps you've tried so far.
Any recent changes you've made to Immich or your system.
Details about your system (both software/OS and hardware).
Details about your storage (filesystems, type of disks, output of commands like fdisk -l and df -h).
The version of the Immich server, mobile app, and other relevant pieces.
Any other information that you think might be relevant.

Please paste files and logs with proper code formatting, and especially avoid blurry screenshots.
Without the right information we can't work out what the problem is. Help us help you ;)

If this ticket can be closed you can use the /close command, and re-open it later if needed.

GitHub

immich-app immich · Discussions

Explore the GitHub Discussions forum for immich-app immich. Discuss code, ask questions & collaborate with the developer community.

FAQ | Immich

User

GitHub

Issues · immich-app/immich

High performance self-hosted photo and video management solution. - Issues · immich-app/immich

slow quartzBOT Jan 29, 2025, 5:36 AM

#

slow quartz ## Checklist I have... 1. :ballot_box_with_check: verified I'm on the latest rel...

Successfully submitted, a tag has been added to inform contributors. :white_check_mark:

whole coral Jan 29, 2025, 5:44 AM

#

Hmm, it seems something wrong happened with the index. You can try running these SQL commands:

DROP INDEX IF EXISTS clip_index;

SET vectors.pgvector_compatibility=on;

CREATE INDEX clip_index ON smart_search
USING hnsw (embedding vector_cosine_ops)
WITH (ef_construction = 300, m = 16);

waxen vortex Jan 29, 2025, 5:50 AM

#

whole coral Hmm, it seems something wrong happened with the index. You can try running these...

It looks like it throws a similar error, I don't think it ran the first command successfully

root@81cfb0b1ad1b:/# psql -U postgres -d immich
psql (14.10 (Debian 14.10-1.pgdg120+1))
Type "help" for help.

immich=# DROP INDEX IF EXISTS clip_index;

SET vectors.pgvector_compatibility=on;

CREATE INDEX clip_index ON smart_search
USING hnsw (embedding vector_cosine_ops)
WITH (ef_construction = 300, m = 16);
ERROR:  pgvecto.rs: IPC connection is closed unexpected.
ADVICE: The error is raisen by background worker errors. Please check the full PostgreSQL log to get more information. Please read `https://docs.pgvecto.rs/admin/configuration.html`.
SET
ERROR:  relation "clip_index" already exists

karmic pawn Jan 29, 2025, 8:47 AM

#

Did you paste it all in one go? If yes, run the commands one at a time

waxen vortex Jan 29, 2025, 10:50 AM

#

karmic pawn Did you paste it all in one go? If yes, run the commands one at a time

The same error occured after the first command. Do i continue with the rest?


root@81cfb0b1ad1b:/# psql -U postgres -d immich
psql (14.10 (Debian 14.10-1.pgdg120+1))
Type "help" for help.

immich=# DROP INDEX IF EXISTS clip_index;
ERROR:  pgvecto.rs: IPC connection is closed unexpected.
ADVICE: The error is raisen by background worker errors. Please check the full PostgreSQL log to get more information. Please read `https://docs.pgvecto.rs/admin/configuration.html`.
immich=#

crude hinge Jan 29, 2025, 10:52 AM

#

Which pgvecto.rs version are you running?

karmic pawn Jan 29, 2025, 10:53 AM

#

https://github.com/tensorchord/pgvecto.rs/issues/409#issuecomment-1978161420 this dev suggests dropping the index by deleting the files on disk 👀

waxen vortex Jan 29, 2025, 10:54 AM

#

crude hinge Which pgvecto.rs version are you running?

This is from my docker compose yaml. I am assuming this is the correct information?
image: docker.io/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0

waxen vortex Jan 29, 2025, 10:54 AM

#

karmic pawn https://github.com/tensorchord/pgvecto.rs/issues/409#issuecomment-1978161420 thi...

Like, getting rid of all my photos ?

karmic pawn Jan 29, 2025, 10:55 AM

#

No no

#

the indices are stored in a specific folder and deleting the content of the folder is the same as dropping the index

waxen vortex Jan 29, 2025, 10:55 AM

#

Oh okay, i see. Just checked the link. I'll back mv the folder and restart and see where that goes

crude hinge Jan 29, 2025, 10:56 AM

#

I think it might be worth to try updating that to 0.3.0

#

tbc, you do have backups right?

#

Before we end up destroying something 😅

waxen vortex Jan 29, 2025, 10:57 AM

#

crude hinge tbc, you do have backups right?

I have a backup but it's a couple months outdated + I have not validated it yet since it was only an rsync to an external drive that I currently have unplugged. Let me double check the data is there and looks correct and get back to you

crude hinge Jan 29, 2025, 10:58 AM

#

Yeah definitely make a backup before continuing

#

(and you should really have a regular automated backup in place tbh)

waxen vortex Jan 29, 2025, 11:00 AM

#

crude hinge (and you should really have a regular automated backup in place tbh)

I've been in the process of converting my stack to replicatable VMs so I can have a better backup solution than my previous which was a weekly systemd timer that ran an rsync to a drive plugged into the computer. My current plan is to have a 2nd PC at a friends house that it backs up to now. Waiting on my new drive to come in for that and then that problem will be solved. 😛

karmic pawn Jan 29, 2025, 11:00 AM

#

nice 🙂

#

for this effort, a database dump should suffice

waxen vortex Jan 29, 2025, 11:07 AM

#

Just did the database dump, should I try updating to 0.3.0 first or moving pgdata/pg_vectors to a temp directory and restarting?

crude hinge Jan 29, 2025, 11:08 AM

#

I think there's a good chance you may end up needing both together

#

So up to you :P

waxen vortex Jan 29, 2025, 11:09 AM

#

Got it, lol

#

I tried moving the pg_vectors and starting it up and so far it looks to be working fine? What would the impact of upgrading to 0.3.0 be, and should I still try it?

crude hinge Jan 29, 2025, 11:23 AM

#

If it ain't broken don't fix it ;)

karmic pawn Jan 29, 2025, 11:23 AM

#

Be sure to still:

SET vectors.pgvector_compatibility=on;

CREATE INDEX clip_index ON smart_search
USING hnsw (embedding vector_cosine_ops)
WITH (ef_construction = 300, m = 16);

waxen vortex Jan 29, 2025, 11:28 AM

#

karmic pawn Be sure to still: ```SQL SET vectors.pgvector_compatibility=on; CREATE INDEX cl...

Is this a problem at all? It still seems to be properly loading images on the webpage

immich-# WITH (ef_construction = 300, m = 16);
ERROR:  relation "clip_index" already exists

karmic pawn Jan 29, 2025, 11:29 AM

#

Does DROP INDEX IF EXISTS clip_index; work now?

#

CLIP is for the smart search, so it shouldn't hinder everything else

waxen vortex Jan 29, 2025, 11:30 AM

#

Yes it did

immich=# DROP INDEX IF EXISTS clip_index;
DROP INDEX

karmic pawn Jan 29, 2025, 11:30 AM

#

and can you now create it again? 😛

waxen vortex Jan 29, 2025, 11:33 AM

#

Looks like it!

immich-# WITH (ef_construction = 300, m = 16);
CREATE INDEX

karmic pawn Jan 29, 2025, 11:42 AM

#

After that is done I'm not sure smart search will work yet, feel free to try first

waxen vortex Jan 29, 2025, 11:44 AM

#

I mean, it looks like it is

karmic pawn Jan 29, 2025, 11:48 AM

#

Excellent 🙂

waxen vortex Jan 29, 2025, 11:48 AM

#

Thank you, mertalev and bo0tzz so much for your help, I thought my immich was going to be dead for a little there lol

karmic pawn Jan 29, 2025, 12:02 PM

#

You should have the automated backups, unless you disabled it

#Help with error "[Microservices:DatabaseService] Could not run vector reindexing checks."

References

Checklist

Information