lazy loading 3rd party module fails | NestJS | Page 1

sick tundra Oct 26, 2022, 6:32 AM

#

NestJS fails to lazily inject third-party dependencies. I've tested with both nest-mongodb and @golevelup/nestjs-rabbitmq, here is a reproduction with the former: https://github.com/timm-stelzer-e-farm/nestjs-dependency-injections-issue. Running start:lazy fails with:

UnknownDependenciesException [Error]: Nest can't resolve dependencies of the DatabaseService (?). Please make sure that the argument fooDb at index [0] is available in the DatabaseModule context.

Potential solutions:
- If fooDb is a provider, is it part of the current DatabaseModule?
- If fooDb is exported from a separate @Module, is that module imported within DatabaseModule?
  @Module({
    imports: [ /* the Module containing fooDb */ ]
  })

    at Injector.lookupComponentInParentModules (/<path>/node_modules/@nestjs/core/injector/injector.js:241:19)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)

Would expect both start:eager and start:lazy to work as expected.

I created an issue at https://github.com/nestjs/nest/issues/10458, which was closed, which I guess means its on my end? Would appreciate some pointers.

GitHub

Issues · nestjs/nest

A progressive Node.js framework for building efficient, scalable, and enterprise-grade server-side applications on top of TypeScript & JavaScript (ES6, ES7, ES8) 🚀 - Issues · nestjs/nest

sick tundra Nov 1, 2022, 3:53 PM

#

Bump, this would genuinely improve our code, we currently have to statically define root modules for half-a-dozen use-cases.

soft hornet Nov 1, 2022, 4:08 PM

#

Looks like it's probably related to this: https://github.com/nestjs/nest/issues/9622

tulip drum Nov 1, 2022, 4:09 PM

#

Could you try the workaround mentioned in the issue above?

sick tundra Nov 2, 2022, 7:38 AM

#

I assume you mean https://github.com/nestjs/nest/issues/9622#issuecomment-1140164939 ?

#

I'm not sure how I would use that workaround in my case.

#

The dependency isn't defined globally, that's the point. Is should only be created on-demand.

#

It's specifically injected into the lazy module.

sick tundra Nov 2, 2022, 8:24 AM

#

I updated the repository to better reflect the issue. That should show that's its not the same issue as the linked one 🤔

tulip drum Nov 2, 2022, 10:27 AM

#

What if you call await context.init() before get?

sick tundra Nov 2, 2022, 10:33 AM

#

Same issue.

soft hornet Nov 2, 2022, 3:41 PM

#

By the way, the database you're trying to inject is global: https://github.com/erikc5000/nest-mongodb/blob/master/src/mongo-core.module.ts#L24

sick tundra Nov 2, 2022, 3:45 PM

#

oh, by default, ugh

#

i see

#

i'm still not sure how i can implement that workaround in my case, though

soft hornet Nov 2, 2022, 3:47 PM

#

this.dbService = this.modRef.get(getDbToken(dbName)) would probably do it in the onModuleInit

sick tundra Nov 2, 2022, 3:47 PM

#

let me try

#

https://github.com/timm-stelzer-e-farm/nestjs-dependency-injections-issue/blob/feat/workaround-1/lazy.ts

#

doesn't seem to work

#

file:///<PATH>/nestjs-dependency-injections-issue/lazy.ts:22
        await this.test.insertOne({qux: 42});
                        ^
TypeError: Cannot read properties of undefined (reading 'insertOne')
    at DatabaseService.insert (file://<PATH>nestjs-dependency-injections-issue/lazy.ts:22:25)
    at main (file://<PATH>/nestjs-dependency-injections-issue/lazy.ts:54:14)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

soft hornet Nov 2, 2022, 3:53 PM

#

Okay, now I'm not familiar with mongo on pretty much any level. How should the db.collection('test') get created?

sick tundra Nov 2, 2022, 3:54 PM

#

that's implicitly created on access

#

thats a mongodb driver feature

#

onModuleInit is never called, do i have to do that manually?

#

removed the collection, now i'm only testing the db itself: https://github.com/timm-stelzer-e-farm/nestjs-dependency-injections-issue/blob/feat/workaround-1/lazy.ts

#

same result

soft hornet Nov 2, 2022, 4:09 PM

#

You know, it's funny. The docs say that lifecycle methods actually don't work on lazy services and modules, so the workaround wouldn't be valid automatically, you'd have to call dbService.onModuleInit() manually

#

And yep, when I set the correct connectionName in the getDbToken() and call db.onModuleInit() manually everything works as expected

sick tundra Nov 2, 2022, 4:27 PM

#

Interesting, thats what i did and it didnt work

#

Grocery shopping right now, will try later

soft hornet Nov 2, 2022, 4:27 PM

#

  onModuleInit() {
    this.db = this.moduleRef.get(getDbToken(connectionName), { strict: false });
    this.test = this.db.collection("test");
  }

sick tundra Nov 2, 2022, 5:36 PM

#

oh, i gotta pass connectionName, right

#

yeah, that seems to work

#

brilliant

#

i'll try to implement this for our work code base, but thanks so much already @soft hornet!

tulip drum Nov 2, 2022, 5:45 PM

#

jay is the best!
@sick tundra you can close this post now if you want to 🙂

sick tundra Nov 2, 2022, 6:07 PM

#

Haven't tested with the "real-world" code yet, will close it if its working 🙂

sick tundra Nov 2, 2022, 7:18 PM

#

works with mongodb ❤️

#

now i'm trying to figure out how to solve the same issue with @golevelup/nestjs-rabbitmq

#

yeah, doesn't work, but different issue 😐

#

Nest can't resolve dependencies of the DiscoveryService (?, MetadataScanner)

#

That one isn't Global though

#

https://github.com/golevelup/nestjs/blob/master/packages/rabbitmq/src/rabbitmq.module.ts#L33

soft hornet Nov 2, 2022, 7:31 PM

#

Is that @golevelup's DiscoveryService?

sick tundra Nov 2, 2022, 7:31 PM

#

import { DiscoveryModule, DiscoveryService } from '@golevelup/nestjs-discovery';

#

yup

#

do i need to inject that as well?

#

tbh, i'm not sure how to get the module token for @golevelup/nestjs-rabbitmq

soft hornet Nov 2, 2022, 7:33 PM

#

Hmm, probably not? The thing it's trying to inject is ModulesContainer which is kind of an internal provider. What's the full error you're getting?

sick tundra Nov 2, 2022, 7:33 PM

#

i assume its https://github.com/golevelup/nestjs/blob/master/packages/rabbitmq/src/rabbitmq.module.ts#L35

soft hornet Nov 2, 2022, 7:33 PM

#

And is this specifically for a lazy service?

sick tundra Nov 2, 2022, 7:33 PM

#

yes, lazy

soft hornet Nov 2, 2022, 7:34 PM

#

Ah, I think that's an issue here then

sick tundra Nov 2, 2022, 7:34 PM

#

UnknownDependenciesException [Error]: Nest can't resolve dependencies of the DiscoveryService (?, MetadataScanner). Please make sure that the argument ModulesContainer at index [0] is available in the DiscoveryModule context.

Potential solutions:
- If ModulesContainer is a provider, is it part of the current DiscoveryModule?
- If ModulesContainer is exported from a separate @Module, is that module imported within DiscoveryModule?
  @Module({
    imports: [ /* the Module containing ModulesContainer */ ]
  })

    at Injector.lookupComponentInParentModules (</node_modules/@nestjs/core/injector/injector.js:241:19)

soft hornet Nov 2, 2022, 7:34 PM

#

I believe some of the internal modules like ModulesContainer are technically global. Let me see if I can find that, cause ModuleRef isn't 🤔

#

Ah, yep. It comes from the InternalCoreModule which is @Global() https://github.com/nestjs/nest/blob/master/packages/core/injector/internal-core-module.ts#L16

sick tundra Nov 2, 2022, 7:36 PM

#

https://github.com/golevelup/nestjs/blob/master/packages/discovery/src/discovery.module.ts#L11 ah yeah, that's global

#

right, soo, i have to inject that? 😄

soft hornet Nov 2, 2022, 7:37 PM

#

No, I think until the bug around this is fixed you've hit a wall

sick tundra Nov 2, 2022, 7:37 PM

#

damn

#

no workaround?

#

but 50% is cool, at least i was able to fix the mongodb part 🙂 when you say "fixed", you're referring to https://github.com/nestjs/nest/issues/9622?

soft hornet Nov 2, 2022, 7:41 PM

#

Yeah, this PR: https://github.com/nestjs/nest/pull/10005

#

no workaround?
Not that I can thnk of at the moment

sick tundra Nov 2, 2022, 7:43 PM

#

then thanks so far, i'll close this and subscribe to that pr 😄

chilly bough Nov 24, 2023, 8:50 PM

#

soft hornet By the way, the database you're trying to inject _is_ global: <https://github.c...

@soft hornet I don't suppose you have Erik's contact details by any chance, would love for him to accept a PR to upgrade this package to nest v10. On a side note, why is Mongoose supported directly by NestJS, but MongoDB isn't? I can understand why specifically for things like TypeORM since it would probably be a pain to add support for all the SQLs 😛

#

It would be super nice if we could somehow have modules which can be pulled in on any version given that they're compatible. It's annoying that things break all the time and require people to update (I do understand why btw, just wondering if there is a work around 😛 )

soft hornet Nov 24, 2023, 11:33 PM

#

chilly bough It would be super nice if we could somehow have modules which can be pulled in o...

Technically, that's what peerDeos should allow for. If they're set up correctly, it should be fine

soft hornet Nov 24, 2023, 11:34 PM

#

chilly bough <@524281921115652106> I don't suppose you have Erik's contact details by any cha...

Probably for a parity-esque approach for ORMs and ODMs (SQL and NoSQL)

#

As for contact info, nope. You can check if he added it to his repo though

chilly bough Nov 25, 2023, 12:55 PM

#

soft hornet Technically, that's what peerDeos should allow for. If they're set up correctly,...

most modules I use don't use peer deps, I wonder why?

chilly bough Nov 25, 2023, 12:55 PM

#

soft hornet As for contact info, nope. You can check if he added it to his repo though

yeah already checked, nothing... found his reddit though, but no luck

soft hornet Nov 25, 2023, 9:23 PM

#

chilly bough most modules I use don't use peer deps, I wonder why?

Because they probably don't understand the why it's necessary. I really should write up a page on making packagesfor NestJS

chilly bough Nov 27, 2023, 11:14 PM

#

soft hornet Because they probably don't understand the _why_ it's necessary. I really should...

That would be big help, the peerDeps in this package look like this

#

    "peerDependencies": {
        "@nestjs/common": "^6.0.0 || ^7.0.0 || ^8.0.0 || ^9.0.0 || ^10.0.0",
        "@nestjs/core": "^6.0.0 || ^7.0.0 || ^8.0.0 || ^9.0.0 || ^10.0.0",
        "mongodb": "^6.0.0",
        "reflect-metadata": "^0.1.13",
        "rxjs": "^6.0.0 || ^7.0.0"
    },

#

Can you think of any reason why it shouldn't just be

#

    "peerDependencies": {
        "@nestjs/common": ">=6.0.0",
        "@nestjs/core": ">=6.0.0",
        "mongodb": "^6.0.0",
        "reflect-metadata": "^0.1.13",
        "rxjs": "^6.0.0 || ^7.0.0"
    },

#

It's unlikely the module system is going to change in NestJS future versions right? Shouldn't this just be the recommended approach?

soft hornet Nov 27, 2023, 11:18 PM

#

chilly bough ``` "peerDependencies": { "@nestjs/common": ">=6.0.0", "@nes...

There were actually breaking changes between v7 and v8 in how dependencies were referenced

chilly bough Nov 27, 2023, 11:18 PM

#

Yes, I remember, but ideally... you'd just install a version of this package supported for that version of nest

#

wouldn't it make more sense that way?

#

Because it's annoying how a majority of packages just don't break with new versions of nestjs, but the maintainers just don't bother updating them even with PRs

#

so you have to clone the repo

#

republish it to npm

#

and litter npm with the newer version thats actually maintained by someone which the maintenance most of the time is just package versions and minor bug fixes

soft hornet Nov 27, 2023, 11:25 PM

#

chilly bough Because it's annoying how a majority of packages just don't break with new versi...

Then they shouldn't be package maintainers. Or they should hand off the package to someone else

chilly bough Nov 27, 2023, 11:26 PM

#

I agree, but like I said, some packages don't break between nestjs versions, so surely there is a way around this?

soft hornet Nov 27, 2023, 11:26 PM

#

chilly bough I agree, but like I said, some packages don't break between nestjs versions, so ...

You can still install outside of the peerDep range. Usually you'll just get a warning from your package manager

#

The whole point is peer deps is to get the package manager "hey, this is necessary, but it'll be installed by another package/command, so don't install it with this package"

chilly bough Nov 27, 2023, 11:28 PM

#

yeah, hmmm. I'll give it a go now, see what happens

#

im pretty sure it failed last time

#

but i'll check

soft hornet Nov 27, 2023, 11:29 PM

#

Knowing what specifically is failing will improve this process tremendously

chilly bough Nov 27, 2023, 11:29 PM

#

meh, ill give it a shot now, probably nothing burger and I'm making it up

#

we'll see 😛

#

PS D:\projects\hrefsmedia\craftsman> npm i --save nest-mongodb
npm ERR! code ERESOLVE                                                                     
npm ERR! ERESOLVE unable to resolve dependency tree                                        
npm ERR!                                                                                   
npm ERR! While resolving: craftsman@0.0.1                                                  
npm ERR! Found: @nestjs/common@10.2.10                                                     
npm ERR! node_modules/@nestjs/common                                                       
npm ERR!   @nestjs/common@"^10.2.8" from the root project                                  
npm ERR!                                                                                   
npm ERR! Could not resolve dependency:                                                     
npm ERR! peer @nestjs/common@"^6.0.0 || ^7.0.0 || ^8.0.0 || ^9.0.0" from nest-mongodb@6.4.0
npm ERR! node_modules/nest-mongodb                                                         
npm ERR!   nest-mongodb@"*" from the root project                                          
npm ERR!                                                                                   
npm ERR! Fix the upstream dependency conflict, or retry                                    
npm ERR! this command with --force or --legacy-peer-deps                                   
npm ERR! to accept an incorrect (and potentially broken) dependency resolution.            
npm ERR!                                                                                   
npm ERR!                                                                                   
npm ERR! For a full report see:
npm ERR! C:\Users\Kieron\AppData\Local\npm-cache\_logs\2023-11-27T23_30_26_113Z-eresolve-report.txt

npm ERR! A complete log of this run can be found in: C:\Users\Kieron\AppData\Local\npm-cache\_logs\2023-11-27T23_30_26_113Z-debug-0.log

soft hornet Nov 27, 2023, 11:32 PM

#

That's specifically an npm issue, because of how it tries to handle peer deps and not at all related but NestJS (at least at runtime). You can use --legacy-peer-deps to get around it and bother the package maintainers to fix their package

chilly bough Nov 27, 2023, 11:32 PM

#

Yeah I have, he accepted the PR to update versions and said "I will make a release soon"

soft hornet Nov 27, 2023, 11:32 PM

#

Yarn and pnpm also behave differently here, just fyi

chilly bough Nov 27, 2023, 11:32 PM

#

idk why you just wouldn't do it all in one go

chilly bough Nov 27, 2023, 11:33 PM

#

soft hornet Yarn and pnpm also behave differently here, just fyi

damn package managers

soft hornet Nov 27, 2023, 11:33 PM

#

chilly bough idk why you just wouldn't do it all in one go

Depends on publish flow

#lazy loading 3rd party module fails