Mongoose virtual getter causes recursion

[sour seal]

I am trying to use getter on Mongoose virtual populated field, to filter out some props from populated object before actually returning it, but it enters some infinite recursion and throws an error.

userSchema
  .virtual('ownedProducts', {
    ref: COLLECTION_NAMES.Product,
    localField: '_id',
    foreignField: 'sellerOwner',
  })
  // @ts-ignore
  .get((value, virtual, doc) => {
    console.log('(virtual get) /value:', value, '/virtual:', virtual, '/doc:', doc);
    return doc;
  });

What i noticed is that if i remove doc from console.log, then error is gone (so mere fact of reading it's value causes recursion). I don't understand it, because otherwise how it's supposed to be used if reading it causes a recursion?

[mighty elk]

It's probably a matter of the cyclical structures having specified toJSONs.
If you're inspecting objects it's always recommended to use console.dir instead

console.dir(doc, { colors: true, depth: 3 })

Example limits to three attributes deep and also adds colours ;)

https://nodejs.org/api/console.html#consoledirobj-options

[sour seal]

Huh, i haven't thought about it 🙂 Actually spreading it { ...doc } solved the problem. Thanks!

(although maybe your suggestion would be better, because spreading makes that object quite difficult to read 🤔 )

(or not quite, because both ways log internal stuff, but console.dir() let's limit depth)

[mighty elk]

Also colours

[sour seal]

Sorry, but is it normal that populated stuff accessible inside getter is buried in some weird props like doc.$$populatedVirtuals or doc.$__.populated? I mean, i have a doubt that it's meant to be read by end developer, but attempt to read target field just straight from the doc ends with infinite recursion again.

Even console.dir with depth: 1 doesn't help

Nor spreading console.dir([...doc.ownedProducts], { colors: true, depth: 1 });

Though logging other virtual - doc.accountType - seems to work

[sour seal]

Hmm, despite having these options passed to Schema constructor:

    toObject: { virtuals: true, getters: true },
    toJSON: { virtuals: true, getters: true },

,my virtual field getter's return value doesn't seem to have any effect on caller.

userSchema
  .virtual('ownedProducts', {
    ref: COLLECTION_NAMES.Product,
    localField: '_id',
    foreignField: 'sellerOwner',
  })
  // @ts-ignore
  .get((value, virtual, doc) => {
    const _ = doc.$$populatedVirtuals.ownedProducts.map(
      ({ _id, name }: { _id: any; name: any }) => ({ _id, name })
    );
    console.log('(virtual get) /value:', value, '/virtual:', virtual, '/doc.ownedProducts:', _);

    return _;
  });

userSchema.methods.toJSON = function (): TUserPublic {
  const user: IUser = this.toObject();

  console.log('(toJSON) user:', user);

  if (!user.accountType?.roleName) { // doesn't matter here
    throw new Error('User role was not successfully populated!');
  }

  const userToExpose: TUserPublic = {
    login: user.login,
    email: user.email,
    observedProductsIDs: user.observedProductsIDs || [],
    accountType: user.accountType.roleName,
  };

  if (userToExpose.accountType === USER_ROLES_MAP.seller) {
    if (!user.ownedProducts) {
      throw new Error('User ownedProducts not succesfully populated for seller role!');
    }

    console.log('(toJSON) /user.ownedProducts:', user.ownedProducts);

    userToExpose.ownedProducts = user.ownedProducts;
  }

  return userToExpose;
};

Logs are on screenshot (getter correctly maps array to contain objects with only 2 props, but code inside .toJSON() doesn't seem to trigger/be affected by getters, because user.ownedProducts virtual field returns whole array instead of mapped one).

So, by unknown reason, user.ownedProducts returns array of "whole" objects, instead of mapped to objects with skipped props. (and there are no errors)

[mighty elk]

Wait

I'm just noticing that you're combining the syntax for virtuals and populated virtuals

I don't think that's how you're supposed to go about things

Populated virtuals are automagic, virtuals are manual

[sour seal]

Could you explain?
ownedProducts is virtual field, which is supposed to gather ids of products, which have sellerOwner field pointing to certain user id. Getter of ownedProducts virtual should map down it's objects to only a few fields, which are then sent to client

[mighty elk]

No, ownedProducts is a populated virtual and they are very different from virtuals

You are treating it like a virtual when it's not

It already has a getter and setter (by virtue of being a populated virtual not a virtual)

Virtuals ≠ Populate Virtuals

[sour seal]

So what is a better way to map down populated virtual output, if getter is wrong there?

[mighty elk]

Do you need to modify the object? I think the transform option would do that

If you just need select, you should be able to just use the select field

[sour seal]

I have to read about it 🤔

I don't need to modify, but just narrow down props (throw out not needed ones)

(hence i do .map() in getter)

[mighty elk]

Sounds like a select to me

userSchema
  .virtual('ownedProducts', {
    ref: COLLECTION_NAMES.Product,
    localField: '_id',
    foreignField: 'sellerOwner',
    options: {
      select: 'name'
    }
  })

Something like this oughta do it

[sour seal]

Yes, thank you 🙏 Though i couldn't find in the docs why (custom) getters are available on virtuals, but not on populated-virtuals

[mighty elk]

Because they are not the same :D

[sour seal]

They are just mentioned for virtuals, but it's not said why populated-virtuals don't support them 😦

[mighty elk]

You need to think of Populate Virtuals as preconfigured populate operations, not as virtuals

[sour seal]

And i am confused when i should use options.select and when getter

[mighty elk]

Getter when you are making a virtual

options.select when you want to limit the fields on a populated virtual

Virtuals have getters and setters, Populate Virtuals don't

(from your perspective, the library internally is of course the other way around)

[sour seal]

This should be somehow better documented

[mighty elk]

To be fair they are distinct and seperate parts of the documentation

https://mongoosejs.com/docs/populate.html#populate-virtuals

https://mongoosejs.com/docs/guide.html#virtuals

https://mongoosejs.com/docs/api/schema.html#schema_Schema-virtual

[sour seal]

Another use case for transform() is setting $locals values on populated documents to pass parameters to getters and virtuals.
So they mention getters on populated virtuals 🤔

[mighty elk]

No, you scrolled past the populate virtuals docs :D

Look at the code example for the quote you sent

[sour seal]

Yeah... i might search for some better tutorial about virtuals and population or a cheat sheet, because i still don't understand them as much as i would like to

The things i understand is:

• virtuals are ad-hoc fields, which are not stored in db, so they are meant to be used to calculate value based on other fields (or arbitrary values). We use .virtual() to create them. They support custom getters and setters (even multiple of them, because first parameter of get/set is the value received from a previous get/set or null if it's the first call)
• population is a mechanism to merge/associate more than one document with each other and it uses .virtual() to actually create an ad-hoc field, which will expose field(s) taken from other documents

So population is somehow based on virtual, because virtual field is needed to create a population. But they setup is somehow different (e.g. selection instead of getters/setters regarding population) and that's what confuses me

[mighty elk]

Populates don't need to be virtual fields

You can have regular ObjectId fields stored on the document that can be populated the same way

Populate Virtuals combine the non-storing of Virtuals with a pre-configured population operation

You can and very often do make populates without Populate Virtuals

An example is a collection of roles and a collection of users with roles, on a user you would store the ObjectId of the role(s) with a ref to the role collection

[sour seal]

Can every field be populated then or does such populat-able field need to be somehow decorated with an indicator to other document(s), which are about to be populated/associated? 🤔

[sour seal]

https://medium.com/@nicknauert/mongooses-model-populate-b844ae6d1ee7
After reading this article i have a question: is there any particular difference between populate field and virtual-populate field, except first actually stores a field with ObjectId(s) and second one doesn't store anything (because it's virtual) in database? To me using virtual-populate in such case is just saving a little space in db, because we are not storing field with ObjectId (which is not such "heavy" i guess) - but is virtual populate it worth it in such simple case?

I guess virtual population might be better if we want to do some transformations?