#The pgvecto.rs extension version is 0.3.0-alpha.2, but Immich only supports 0.2.x.

1 messages · Page 1 of 1 (latest)

burnt hawk
#

I was accidentally running pgvector 0.3 and Immich 1.104 fine. It seems Immich 1.106 added a check for the pgvector version and now Immich fails to start. I've downgraded pgvector (SELECT * FROM pg_extension confirms this) this but Immich still complains

ripe rampartBOT
#

:wave: Hey @burnt hawk,

Thanks for reaching out to us. Please follow the recommended actions below; this will help us be more effective in our support effort and leave more time for building Immich immich.

References

Checklist

  1. :ballot_box_with_check: I have verified I'm on the latest release(note that mobile app releases may take some time).
  2. :ballot_box_with_check: I have read applicable release notes.
  3. :ballot_box_with_check: I have reviewed the FAQs for known issues.
  4. :ballot_box_with_check: I have reviewed Github for known issues.
  5. :ballot_box_with_check: I have tried accessing Immich via local ip (without a custom reverse proxy).
  6. :ballot_box_with_check: I have uploaded the relevant logs, docker compose, and .env files using the buttons below or the /upload command.
  7. :ballot_box_with_check: I have tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable

(an item can be marked as "complete" by reacting with the appropriate number)

If this ticket can be closed you can use the /close command, and re-open it later if needed.

burnt hawk
#

Doh, I got it. Somehow pgres was still running the 3.x extension in the immich db

#

Fixed 🙂

burnt hawk
#

Okay, reviving this

#

because I cannot find a solution

#

I got immich up but facial stuff is broken

#

I cannot find a clean way to downgrade from vectors 0.3 to 0.2, as any attempt to unload the extension requires a CASCADE which nukes a bunch of columns from asset_faces and smart_search

fading spear
#

You might have to restore from backup

burnt hawk
#

as in dump all the data, drop the db, then import?

#

so I've restored my working 1.104 Immich with vectors 0.3 and all is good

#

I was thinking of reimporting my databases but I think it exports the plugin extension as part of a dump

#

so even if I remove vectors 0.3 it ends up back again

fading spear
#

Yes. You should use a backup from before you upgraded

burnt hawk
#

I've done that, now trying to restore my database into a database with vectors .2 installed

#

but so many issues

#

CREATE INDEX "face_index" ON "public"."asset_faces" USING btree ("embedding");

gives the error

Error in query: ERROR: data type vectors.vector has no default operator class for access method "btree"
HINT: You must specify an operator class for the index or define a default operator class for the data type.

feral nest
#

This is a hack but you shouldn’t need to restore from an old backup if you do this:

UPDATE pg_catalog.pg_extension SET extversion = ‘0.2.1’ WHERE extname = ‘vectors’;
burnt hawk
#

I've almost managed to downgrade but I'm stuck on creating two indexes

#

when I exported the table I see: CREATE INDEX "IDX_asset_faces_assetId_personId" ON "public"."asset_faces" USING btree but it looks like people are recreating the tables using CREATE INDEX IF NOT EXISTS clip_index ON smart_search
USING hnsw

feral nest
#

Those are different indices

burnt hawk
#

sorry

#

same thing though

#

CREATE INDEX IF NOT EXISTS face_index ON asset_faces
USING hnsw (embedding vector_cosine_ops)
WITH (ef_construction = 300, m = 16);

#

vs CREATE INDEX "IDX_asset_faces_assetId_personId" ON "public"."asset_faces" USING btree

#

argh

#

sorry I keep copying the wrong things

#

its the same index though

#

CREATE INDEX "face_index" ON "public"."asset_faces" USING btree ("embedding");

vs

CREATE INDEX IF NOT EXISTS face_index ON asset_faces
USING hnsw (embedding vector_cosine_ops)
WITH (ef_construction = 300, m = 16);

#

there we go 😎

#

I dunno what hnsw is and neither does my postgres

feral nest
#

The latter is right, not sure where the btree is coming from

burnt hawk
#

I just exported my 1.104 database as an sql file

feral nest
#

If you restore that and change the extversion, it should probably work

feral nest
burnt hawk
#

@feral nest so I hacked it so that vectors appears as 0.2

#

but immich reports:

WARN [Microservices:DatabaseService] Could not run vector reindexing checks. If the extension was updated, please restart the Postgres instance.
immich_server | QueryFailedError: pgvecto.rs: The extension is upgraded so all index files are outdated.
immich_server | ADVICE: Delete all index files. Please read https://docs.pgvecto.rs/admin/upgrading.html.

#

I restarted postgres

#

I also tried dropping face_index and recreating it, after running SET vectors.pgvector_compatibility=on; but same error about hnsw

#

I tried forcing a reindex too (REINDEX INDEX face_index)

#

but I get a similar complaint from postgres

feral nest
#

Run SELECT pgvectors_upgrade(); and restart

burnt hawk
#

Error in query: ERROR: function pgvectors_upgrade() does not exist 😦

#

I deleted both indexes now

#

and immich runs fine

#

including the facial recognition job

#

but those two indexes definitely dont exist anymore hah

burnt hawk
#

there is no *hnsw *

#

wonder if there is for everyone else :\

#

hmm so I'm guessing that pgvector alhpa 3 has issues with compatibility mode

feral nest
#

Did you run the compatibility mode command right before trying to create the index?

#

It’s a little weird, datagrip doesn’t think hnsw exists either but the command does run successfully

burnt hawk
burnt hawk
#

I've logged an issue with pgvecto, maybe it's a 0.3 issue 🙂

feral nest
#

You can try their toml syntax instead without compatibility mode

CREATE INDEX face_index ON asset_faces USING vectors (embedding vector_cos_ops)
WITH (options = $$
[indexing.hnsw]
m = 16
ef_construction = 300
$$);
burnt hawk
#

operator class "vector_cos_ops" does not exist for access method "vectors"

#

hehe getting real deep here into vectors x_x

#

I guess worse case, my immich will just be slow at face stuff until it supports pgvector 3.x? :S

feral nest
#

The next release reworks the face embeddings so it should create another index. But smart search won’t create an index on its own

#

Try this

ALTER DATABASE <immichdatabasename> SET search_path TO "$user", public, vectors;
ALTER SCHEMA vectors OWNER TO <immichdbusername>;
#

Then try the index command again

burnt hawk
#

nope no joy

#

really appreciate your help though 🙂

feral nest
#

Can you try vectors.vector_cos_ops instead?

#

For the index command

#

Also the ALTER DATABASE command only applies to later sessions, so maybe restart Postgres and sprinkle some prayers

burnt hawk
#

naw same thing

feral nest
#

Alright that’s weird. You can try actually downgrading to 0.2.1

burnt hawk
#

hmm I think I did try that

#

although I've learned a lot more since

#

think postgres moaned there was no upgrade path from 3 -> 2

#

which is fair 😄

feral nest
#

Might need to set the extversion manually

#

Although you’re “already” at 0.2.1 haha

burnt hawk
#

just reverted my snapshot so back to 3 😄

#

so you're thinking fake being at 1.x?

#

and then an alter extension update?

feral nest
#

The cleanest way to downgrade would be to drop the extension and change the image to 0.2.1, then create things again. To avoid losing your current embeddings for facial recognition etc., change the embedding columns in smart_search and asset_faces to real array so it doesn’t rely on the extension

#

You can change them back to vector(512) after creating the extension again

burnt hawk
#

omg.

#

Please don't ban me lol

#

so you know how there is pgvector but also pgvecto.rs

#

😐

#

well guess what I guess Immich works with pgvector 0.3 alpha, I have no idea if pgvecto.rs 0.3 is even a thing 🤦‍♂️

#

oh no wait, scratch that

#

I don't even know where I got the alpha, was just looking at my wget command from 4 weeks ago

#

it's not even listed under their relases

#

right I'm gonna try what you suggested 😄

#

ALTER TABLE "asset_faces" ALTER "embedding" TYPE real[], ALTER "embedding" DROP DEFAULT, ALTER "embedding" SET NOT NULL;

#

operator class "vectors.vector_cos_ops" does not accept data type real[]

#

oh need to drop the index

#

I think

#

yesssss

feral nest
#

Yep

burnt hawk
#

Error in query: ERROR: access method "hnsw" does not exist

#

one thing I'm having to do is use vectors.vector rather than just vector, noticed that a few times

#

i.e. ALTER "embedding" TYPE vectors.vector(512)

#

but I'm running true 0.2.1 vectors now

#

hmm if I do SET vectors.pgvector_compatibility=on; in the same query

#

I get a different error

#

Error in query: ERROR: operator class "vector_cos_ops" does not exist for access method "vectors"

#

and I've done the old ALTER SCHEMA vectors OWNER TO immich

feral nest
#

Did you set the search path too?

# 
ALTER DATABASE immich SET search_path TO "$user", public, vectors;
SET search_path TO "$user", public, vectors;
burnt hawk
#

I did yeah

#

it's still not correct

#

aaj

#

wat 😐

#

the index was deleted

#

and vectors was fully removed befored I restarted postgres twice and checked

feral nest
#

That just means the index folder is still there. Running SELECT pgvectors_upgrade(); deletes that folder

#

Also just checking but are you connected as the postgres user?

burnt hawk
#

connected as the immich user

#

which has superuser

feral nest
#

Ayyyyy

burnt hawk
#

just needed a VACUUM FULL and a restart of postgres 😅

feral nest
#

VACUUM FULL is like Postgres’ WD-40 lol

burnt hawk
#

thanks so much for your extended help! I feel like there were 20 hoops to jump through all in the right order, and you really helped me get through them all!!

feral nest
#

No problem! I’ve gone through these hoops myself so I know it can be rough sometimes haha

#

Also run that for the asset_faces table too. Although it’d only matter until the next release

burnt hawk
#

yep done 😄