Optional Imports in Dynamic Modules

[stable gulch]

Hi,

We run our nest server in multiple environments and would like the ability to optionally import modules based on certain configuration parameters. For example, we would like certain modules to only be imported and enabled in dev until they are mature enough to push to prod.

I have seen this issue: https://github.com/nestjs/nest/issues/9868. But this suggestion requires us to use process.env directly which we would like to avoid given the way we have our env and configuration set up.

It seems a little weird that if our configuration or feature flags were hosted in an external service we could do this easily, but cannot if our configuration is managed by a nest dependency.

If this works:

static async registerAsync(): DynamicModule {
  const enabled = await loadExternalFlag();
  if (enabled) {
    return { /* the full module */ };
  }
  return { /* an empty module */ };
}

It seems strange that we are handicapped by the fact that our config is managed by a nest module and cannot do something like:

static async registerAsync(@Inject() config: ConfigService): DynamicModule {
  const enabled = await config.get('flag');
  if (enabled) {
    return { /* the full module */ };
  }
  return { /* an empty module */ };
}

Is there a way around this that allows us to import and use an imported provider?

[junior herald]

You cannot use config module for it, because it lives in the very DI tree that is being constructed. Chicken and egg problem, you see. You don't have to use process.env directly, but you sipmly can't use a provider to decide to load other providers. It has to come from outside the DI

[stable gulch]

I don't know exactly how the DI tree is constructed, but I would think that you could depend on and inject providers that live higher up in the tree.

If a service that is part of this module can depend on the config module service, why can't the module itself depend on it.

I understand if this is a current limitation.

But this seems like it drives behavior counter to the best practices of nest, as it would mean we need to manage these flags in an external service or via globals like process.env.

[junior herald]

Configuration is kind of special. It inherently IS a global concern, so it does not go against best practices. It would be great if NestJS provided some static configuration object, that you could access outside of DI

[stable gulch]

Got it. Well thank you for the response. It helps us move forward.

We have some logic for loading configuration that we can probably just call statically when we need to use it like this, to keep access consistent with other places in the app.

[warped cargo]

By the way, something that does exist is a getter on the ConfigModule (I think, I'll double check the location) that's like await ConfigModule.envVaraiblesLoaded so you can actually use process.env with deterministic outcome inside of an async registration method (i.e. async register or async registerAsync)

https://github.com/nestjs/config/blob/master/lib/config.module.ts#L39

[random gate]

I don't know if I fully understand what you need, but you were unable to build your module with https://trilon.io/blog/nestjs-9-is-now-available#Configurable-module-builder ?

To me it looks like the problem is with the provider injection, in this case you can add the inject property with your ConfigService and let NestJS sort it out for you before initializing your custom module.

You can return the empty module or the actual implementation, but I think it might be a problem if some service depends on it (outside the module).

I do this with NestJS 8 and load environment variables from Parameter Store before initializing the database and this same logic works great.

[stable gulch]

@warped cargo thanks, I saw that suggested in the github issue posted in the original message as well. I think that works if we were just using the plain ConfigModule but we have some additional logic on top of this such that it doesn't give us much more assurance that what we need is valid. Additionally that would still require us to use process.env which we are trying to avoid for our use case.

@random gate as far as I understand, this isn't really an issue that the configurable module builder helps solve. Even with the configurable module builder, there is no way to inject other services into the registration functions that would allow us to conditionally add an import to the DynamicModule.

But I may be misunderstanding what you are suggesting given the example you listed. Do you have any reference I can look at for your parameter store / database initialization?

[warped cargo]

I'll try to think of something you might be able to do. The registration methods can be async, so there might be a solution that can work. Where would you prefer to get the config from?

[stable gulch]

Ideally nest handles the injection for us completely. The solution we have gone with for now is just a separate async method that loads a feature flag config. Since this function happens outside the scope of the DI tree, we can just pass the result of the function to the module registration.

We have something like this:

// flags.ts
function async loadFlags() {
  // Load and validate a separate config.
}

// my.module.ts
@Module({})
export class MyModule {
  static forRoot(flags: FeatureFlags): DynamicModule {
    if (flags.myFlag.enable) {
      return { /* full module */ };
    }
    return { /* empty module */ };
  }
}

// app.module.ts
const flags = loadFlags();

@Module({
  imports: [
    MyModule.forRoot(flags),
  ],
  controllers: [],
  providers: [],
})
export class AppModule {}

But our preference would be the ability to use our existing config module so that we can leverage the same config loading and validation logic rather than having a separate one:

// my.module.ts
@Module({})
export class MyModule {
  static forRoot(@Inject() config: ConfigService): DynamicModule {
    if (config.enable) {
      return { /* full module */ };
    }
    return { /* empty module */ };
  }
}

[warped cargo]

More than likely that @Inject() won't ever work, because nest doesn't call that method, you do. The reason why parameter decorations work in controllers and resolvers is because that's where nest goes from frequent specific code to dev written code

That said, I might be able to contrive some sort of example. Maybe not with dynamic module loading, but at least with provider swapping with a noop implementation if a flag of not enabled

[stable gulch]

That would be awesome if there was an example of this. I think the use case is somewhat valid to be able to conditionally import modules in a dynamic module, using a conditional coming from somewhere else in the DI tree. That said, I'm not sure if that's even feasible.

[warped cargo]

Right. Modifying the imports would be difficult, but like I said I could said least think of a work around where you use either the actual service or a noop service in place if the service isn't enabled

[stable gulch]

Yea, I thought about that as an option. The module we want to omit is a combo of a controller, typeorm module, and a few others. So plugging in a noop for all of those seems a little more complicated than if the module was just a simple service provider.

We essentially want a whole section of routes and the infra setup (db connection) to not run / be set up in our prod environment while we build it out in dev.

[random gate]

Here's what I do:

@Module({})
export class CustomModule {
  static registerAsync(options: ModuleOptions, providerToken: any): DynamicModule {
    return {
      module: CustomModule,
      imports: [
        ...options.imports,
      ],
      exports: [
        providerToken,
      ],
      providers: [
        {
          scope: Scope.DEFAULT,
          provide: providerToken,
          inject: options.inject,
          useFactory: (...args) => {
            const options = options.useFactory(...args);

            return new Service(options);
          },
        },
      ],
    };
  }
}

And I use like this:

CustomModule.registerAsync({
      imports: [EnvModule],
      inject: [EnvService],
      useFactory: (env: EnvService) => {
        return {
          accountId: env.CF_ACCOUNT_ID,
          bucketName: env.CF_PRIVATE_BUCKET_NAME,
          accessKey: env.CF_ACCESS_KEY_ID,
          secretKey: env.CF_SECRET_ACCESS_KEY,
        };
      },
    }, CustomProviderToken)

And EnvService is like this:

providers: [
    {
      provide: EnvService,
      useFactory: async parameter => await EnvService.factory(parameter),
      inject: [ParameterStoreService],
    },
  ],

But I misunderstand what you need, this only works with services, I don't think you could disable controllers or other modules and maybe that's what you need but it seems impossible now.
Perhaps what you need could be solved if imports and controllers could be like the providers, supporting create them with useFactory.

[floral plank]

Reviving the thread. As @stable gulch suggested, we have been also trying to omit X module along with its controllers based on Y flag passed in ModuleOptions into registerAsync. Is there any way to make this work without process.env?