#Diffing two collections

7 messages · Page 1 of 1 (latest)

fringe skiff
#

I've got two identical collections: MutableData and ImmutableData. I would like to get a diff between what has changed in MutableData since the initial cloning.

I'm learning how to do this via mongoose aggregations at the moment, but I'm curious if there is some solution out there to this that someone has already encountered.

Thank you

misty jacinth
#

You're already ahead of my knowledge it seems with using aggregations for this. It is essentially just comparing JSON for differences?
My approach might be to write out the diff as the changes are written instead of trying to compare them in a read operation.

fringe skiff
#

Oh, thanks for responding @misty jacinth ! I forgot I posted this. I did end up getting an aggregate going that allowed me to tell the docs that changed between two collections.

[
    {
        $lookup: { // provides the mutable data to compare against
            from: "immutable-data-collection-name",
            localField: "_id",
            foreignField: "_id",
            as: "mutableData",
        },
    },
    {
        $addFields: { // Filters the data down to only that which has changed and puts it in a new field called "diff"
            diff: {
                $filter: {
                    input: "$mutableData",
                    cond: {
                        $or: [
                            { $ne: ["$$this.Name", "$Name"] },
                            // ...and so on for all the other relevant fields you want to check against when determining an change has happened.
                        ],
                    },
                },
            },
        },
    },
    {
        $match: { // further filters out any documents that have no changes, leaving only the ones that have changed
            $expr: { $gt: [ { $size: "$diff", }, 0, ], },
        },
    },
]
#

Then to actually use this in a payload endpoint, I ran code that looked like the following:

const getDiff = () => {
    return new Promise((resolve, reject) => {
        payload.collections['mutable-data-collection-name'].Model.aggregate(diffAggregate)
        .then((result) => {
            resolve(result)
        })
        .catch((err) => {
            reject(err)
        })
    })
}
#

diffAggregate being the initial array of objects code I pasted.

misty jacinth
#

Marking as Answered so it shows up in our site's community help. Great work @fringe skiff!