How to speed up resetting MongoDB sync flags for 1.6M documents

[marble lantern]

Hi all, running the MeiliSearch reset sync script (reset-meili-sync) on a production DocumentDB instance with ~1.6M messages. The script resets the _meiliIndex flag on all documents so they can be re-indexed into MeiliSearch.

The problem is that there is no database index on the _meiliIndex field, so every query that filters on it does a full collection scan.

We tried this approach hit latency issues at this scale:

• Batched updates (10k/batch), each batch takes ~10 minutes due to the collection scan, making the full reset 15+ hours

Has anyone dealt with a similar issue? Any other approaches the community has found effective?

Thank you in advance

[flat crescent]

This looks like a beneficial topic for 'enterprise-scale'. Referencing this post there.

[flat crescent]

This was implemented in 2025 https://github.com/danny-avila/LibreChat/issues/6740 and was supposed to help determining which messages have already been indexed.

Which version of LibreChat have you deployed?

[flat crescent]

DocumentDB is an AWS repackaging of MongoDB. And its known that its not a 100% replacement for MongoDB. Chances are that there could be an issue with how mongoMeili.js is working with DocumentDB:

api/models/plugins/mongoMeili.js

[still pendant]

Cough, replace meili with mongo imo.

[flat crescent]

I remember someone mentioning that a couple months back. I haven't tried it, but sounds like its been viable.