How to use Redis as a publisher and subscriber on NestJS project | NestJS | Page 1

abstract fjord Oct 29, 2022, 4:14 PM

#

There is something i don't understand in the NestJS documentation (https://docs.nestjs.com/microservices/redis), at the beginning it says to create a micro service, but it mean to instantiate 2 servers an Express or Fastify one or whatever, and a micro service for Redis ?
How am i supposed to do from their ?

async function bootstrap() {
  const app = await NestFactory.create<NestFastifyApplication>(
    AppModule,
    new FastifyAdapter()
  );

 await app.listen(3000, '0.0.0.0');
}

bootstrap();

I'm stuck since few hours on that

Documentation | NestJS - A progressive Node.js framework

Nest is a framework for building efficient, scalable Node.js server-side applications. It uses progressive JavaScript, is built with TypeScript and combines elements of OOP (Object Oriented Progamming), FP (Functional Programming), and FRP (Functional Reactive Programming).

cinder wyvern Oct 29, 2022, 4:15 PM

#

Are you trying to create a hybrid application (RPC and HTTP request handling in the same server) or are you wanting to make two separate servers?

abstract fjord Oct 29, 2022, 4:20 PM

#

I try to make a cluster of NodeJS/NestJS http micro service using for exemple port 3000, and i am using websocket on another port like port 3001.

For each Websocket Gateway i need to get every users informations on each servers.

For that i use Redis since i use it for cache and optimization.

But i don't now how to use Redis on my Fastify and websocket server for sharing data accross my cluster

cinder wyvern Oct 29, 2022, 4:22 PM

#

Okay, to get this straight: you are not making a microservice in the terms of publishing and subscribing to events or messages via Redis, right? You just want to use redis as a database?

abstract fjord Oct 29, 2022, 4:31 PM

#

Imagine a Gateway with an event who handle the client who disconnect from your server, you need to have the information shared between every nestjs server.

So i use Redis to publish and subscribe data like this exemple below:

const options = {
  cors: {
    origin: ["localhost:4200", "*"],
    methods: ["GET", "POST"],
    credentials: true,
    transport: ["websocket"]
  },
  namespace: 'ws'
};

@WebSocketGateway(3001, options)
export class CryptoGateway implements OnGatewayConnection, OnGatewayDisconnect {
  
  constructor(
    private clientRedis: RedisClient,
  ) {}

   @WebSocketServer() server: Server;
   listUsers = [];

   async handleDisconnect(client: any) {
    // A client has disconnected
    this.users--;
    await this.clientRedis.publish('userClientIdDisconnect', JSON.stringify(client.id));

    await this.clientRedis.subscribe('userClientIdDisconnect', (clientId: string) => {

      for(let i = 0; i < this.listUsers.length; i++) {
        if(this.listUsers[i].clientId == clientId) {
            this.listUsers.splice(i, 1);
            break;
        }
    }

  }
}

cinder wyvern Oct 29, 2022, 4:32 PM

#

Okay. This is not necessarily a microservice in the sense of Nest microservices. This is making using of redis directly, so no need to spin up a server for listening to redis events. You're technically already doing that

#

If you were to use decorators like @EventPattern() or @MessagePattern() then you'd need to use NestFactory.createMicroservice() or connect a microservice to a hybrid application, but that's not what you're doing

abstract fjord Oct 29, 2022, 4:33 PM

#

I already have a redis docker imagine running o

cinder wyvern Oct 29, 2022, 4:34 PM

#

Okay, good. That means you have a redis database you can communicate with. Just like you save data to postgres or mongo or whatever you use

abstract fjord Oct 29, 2022, 4:34 PM

#

Exactly

abstract fjord Oct 29, 2022, 4:35 PM

#

cinder wyvern If you were to use decorators like `@EventPattern()` or `@MessagePattern()` then...

And btw, in this case, how would i use a Redis micro service with my fastify server ? I should make a new project using a different instance (createMicroservice()) ?

cinder wyvern Oct 29, 2022, 4:36 PM

#

abstract fjord And btw, in this case, how would i use a Redis micro service with my fastify ser...

Well, if you want them on the same server we call that a hybrid application: https://docs.nestjs.com/faq/hybrid-application
If you want two separate servers, different process IDs, different dependencies, etc, you'd just make a second server, either in a monorepo or a separate repository, and control them separately.

abstract fjord Oct 29, 2022, 4:37 PM

#

Is that what i need then ?

#

It's ok with a proxy server like nginx since redis is only available in local

#

If i can give you a recommendation, it's to add this FAQ to MicroServices area

#

just a link with an information or warning text

cinder wyvern Oct 29, 2022, 4:42 PM

#

It sounds like you don't need anything microservice related. You're using redis to publish and subscribe to messages directly, but not as a microservices server would and that's okay. You're just making use of the database's capabilities

abstract fjord Oct 29, 2022, 4:44 PM

#

How that ? Like managing Cache with Redis ?

#

If it's what you mean, how am i supposed to listen this one other servers side ?

cinder wyvern Oct 29, 2022, 4:44 PM

#

abstract fjord Imagine a Gateway with an event who handle the client who disconnect from your s...

Exactly like you showed here. Everything is looking like you've already got it set up

abstract fjord Oct 29, 2022, 4:46 PM

#

I didn't tested yet , i was wondering how to inject Redis to Publish and Subscribe event

cinder wyvern Oct 29, 2022, 4:46 PM

#

Create a redis client provider, inject the provider. Should be pretty straightforward

abstract fjord Oct 29, 2022, 4:46 PM

#

Is this correct ?

import {RedisClient} from 'redis';

constructor(
    private clientRedis: RedisClient,
  ) {}

cinder wyvern Oct 29, 2022, 4:47 PM

#

Do you have a provider created for the RedisClient class?

abstract fjord Oct 29, 2022, 4:49 PM

#

No it's what i want to make for reusing "clientRedis"

cinder wyvern Oct 29, 2022, 4:50 PM

#

Okay. So do you know how to make a custom provider? Causethat's step 1

abstract fjord Oct 29, 2022, 4:52 PM

#

it's a service inject in a module

#

but i don't know how to make it

cinder wyvern Oct 29, 2022, 4:56 PM

#

Here are the docs for creating a custom provider: https://docs.nestjs.com/fundamentals/custom-providers

abstract fjord Oct 29, 2022, 4:56 PM

#

I mean for the Redis client

cinder wyvern Oct 29, 2022, 4:56 PM

#

Using redis or ioredis?

abstract fjord Oct 29, 2022, 4:58 PM

#

making a module like this:

import { DynamicModule, FactoryProvider, ModuleMetadata } from '@nestjs/common';
import { Module } from '@nestjs/common';
import IORedis, { Redis, RedisOptions } from 'ioredis';

export const IORedisKey = 'IORedis';

type RedisModuleOptions = {
  connectionOptions: RedisOptions;
  onClientReady?: (client: Redis) => void;
};

type RedisAsyncModuleOptions = {
  useFactory: (
    ...args: any[]
  ) => Promise<RedisModuleOptions> | RedisModuleOptions;
} & Pick<ModuleMetadata, 'imports'> &
  Pick<FactoryProvider, 'inject'>;

@Module({})
export class RedisModule {
  static async registerAsync({
    useFactory,
    imports,
    inject,
  }: RedisAsyncModuleOptions): Promise<DynamicModule> {
    const redisProvider = {
      provide: IORedisKey,
      useFactory: async (...args) => {
        const { connectionOptions, onClientReady } = await useFactory(...args);

        const client = await new IORedis(connectionOptions);

        onClientReady(client);

        return client;
      },
      inject,
    };

    return {
      module: RedisModule,
      imports,
      providers: [redisProvider],
      exports: [redisProvider],
    };
  }
}

Would work, but how could i reuse that on my Gateway or whatever other Services

#

I guess, i let you see what you think

cinder wyvern Oct 29, 2022, 5:03 PM

#

Honestly, if you don't plan to publish the module, I'd just do something like

@Module({
  providers: [
    {
      provide: 'REDIS_OPTIONS',
      useFactory: (config) => optionsFromConfig,
      inject: [ConfigService],
    },
    {
      provide: 'REDIS_INSTANCE',
      inject: ['REDIS_OPTIONS'],
      useFactory: async (options) => new Redis(options);
    }
  ],
  exports: ['REDIS_INSTANCE'],
})
export class RedisModule {}

And inject it with @Inject('REDIS_INSTANCE')

abstract fjord Oct 29, 2022, 5:05 PM

#

what is the instance of new Redis(options) ?

cinder wyvern Oct 29, 2022, 5:05 PM

#

import Redis from 'ioredis'

abstract fjord Oct 29, 2022, 5:07 PM

#

and how you inject 'optionsFromConfig' ?

cinder wyvern Oct 29, 2022, 5:07 PM

#

That's just a place holder cause I don't know what options you use or how you grab them from config

#

I can't write everything perfeectly. You should be able to interpolate the idea and use it for yourself

abstract fjord Oct 29, 2022, 5:08 PM

#

yes of course, i still ask because i don't know the type of the variable

cinder wyvern Oct 29, 2022, 5:08 PM

#

config is of type ConfigService

abstract fjord Oct 29, 2022, 5:09 PM

#

so something like this ?

optionsFromConfig: {
   host: configService.get('REDIS_HOST'),
   port: configService.get('REDIS_PORT'),
},

cinder wyvern Oct 29, 2022, 5:12 PM

#

Yeah, you'd just need (config) => ({ host: 'hostval', port: 'postval'' }), but otherwise you got it

abstract fjord Oct 29, 2022, 5:17 PM

#

ok i try it

#

to be sure, how you use it in your service once you imported the module like this

imports: [
    RedisModule,
]

cinder wyvern Oct 29, 2022, 5:27 PM

#

Yep

abstract fjord Oct 29, 2022, 5:27 PM

#

Yes but in my Gateway how i inject that ?

cinder wyvern Oct 29, 2022, 5:30 PM

#

Like I showed earlier, @Inject('REDIS_INSTANCE')

abstract fjord Oct 29, 2022, 5:30 PM

#

Yes i was writing something like that to be sure alright

#

Thanks you

abstract fjord Oct 29, 2022, 6:55 PM

#

I have an error: TypeError: ioredis_1.Redis is not a constructor

#

with this lib: import { Redis } from 'ioredis';

#

it's not the good import for me

cinder wyvern Oct 29, 2022, 6:59 PM

#

abstract fjord with this lib: import { Redis } from 'ioredis';

That's not the import I showed or that ioredis's documentation shows. Did you even check the docs?

abstract fjord Oct 29, 2022, 7:00 PM

#

about custom provider ?

#

the issue is on this line : useFactory: async (options) => new Redis(options),

#

it's ioredis for what i saw

#

you meant ioredis yes i just saw

#

again

#

but it's the same

cinder wyvern Oct 29, 2022, 7:36 PM

#

Using import Redis from 'ioredis' gave you the same error?

abstract fjord Oct 29, 2022, 7:39 PM

#

cinder wyvern Using `import Redis from 'ioredis'` gave you the same error?

no it's working now, it's because i did import { Redis } from 'ioredis' , my mistake sorry

#

i now try to find why connection is refused to my redis container

abstract fjord Oct 29, 2022, 10:04 PM

#

i am stuck on this error [ioredis] Unhandled error event: Error: connect ECONNREFUSED 127.0.0.1:6379

#

i can't connect to my redis container, i am on the same network

#

I try to connect like this:

useFactory: (config) => ({ 
            legacyMode: true,
            socket: {
                host: 'redis',
                port: '6379'
            }
        }),

#

and this is my redis container

redis:
        container_name: redis
        image: redis
        command: ["redis-server", "--bind", "redis", "--port", "6379"]
        #mem_limit: 256m
        ports:
            - '6379:6379'
        restart: always
        networks:
            - projetpb-network
        volumes:
            - redis-data:/data

cinder wyvern Oct 29, 2022, 10:26 PM

#

Are you running the server locally or through docker?

abstract fjord Oct 29, 2022, 10:28 PM

#

through docker of course

#

i try to rebuild them

#

and i will try if it's not ```
{
provide: 'REDIS_INSTANCE',
inject: ['REDIS_OPTIONS'],
useFactory: async (options) => new Redis(options),
}

#

if it's not options in the new Redis(options) instance if it's the good connection

cinder wyvern Oct 29, 2022, 10:37 PM

#

Well, it's trying to use host localhost

abstract fjord Oct 29, 2022, 10:37 PM

#

ok that was it, but there is something i don't understand

#

it's seems to work i don't have anymore error

#

but there is something i misunderstood

#

@Module({
    providers: [
      {
        provide: 'REDIS_OPTIONS',
        useFactory: (config) => ({ 
            legacyMode: true,
            socket: {
                host: '199.199.199.199',
                port: '6379'
            }
        }),
        inject: [ConfigService],
      },
      {
        provide: 'REDIS_INSTANCE',
        inject: ['REDIS_OPTIONS'],
        useFactory: async (options) => new Redis(6379, "redis"),
      }
    ],
    exports: ['REDIS_INSTANCE'],
})
export class RedisModule {}

I don't need the first provider ? otherwise why the options value wasn't working ?

#

i just tried a fake host, just to see if my error was different

cinder wyvern Oct 29, 2022, 10:40 PM

#

Got a reproduction

abstract fjord Oct 29, 2022, 10:40 PM

#

what do you mean ?

#

oh ok

#

sorry i am tired ^^

#

ok i try to make one later since it will take me time i debug first

#

ok i understand what happened

#

it's jsut because it wasn't formatted as the expected object

abstract fjord Oct 30, 2022, 12:36 AM

#

If i Inject that in my constructor, redicClient is undefined:

@Inject('REDIS_INSTANCE') private clientRedis,

i miss something, somewhere ?

cinder wyvern Oct 30, 2022, 1:43 AM

#

Not unless new Redis() returns undefined. So long as that's in the constructor it should work (unless you instantiate the class yourself, that would be a different story)

abstract fjord Oct 30, 2022, 3:26 PM

#

yes this:

constructor(
    @Inject('REDIS_INSTANCE') private clientRedis: Redis,
  ) {}

shouldn't be undefined

#

but it is when i use it like that:

await this.clientRedis.publish('userClientId', JSON.stringify(client.id));

abstract fjord Oct 30, 2022, 4:59 PM

#

that was just the "return" who was missing in the code you showed me:

@Module({
    imports: [ConfigModule],
    providers: [
      {
        provide: 'REDIS_OPTIONS',
        useFactory: (config: ConfigService) => (
            {
            port: config.get('REDIS_PORT'),
            host: config.get('REDIS_HOST'),
        }),
        inject: [ConfigService],
      },
      {
        provide: 'REDIS_INSTANCE',
        inject: ['REDIS_OPTIONS'],
        useFactory: async (options) => {return new Redis(options)}, // the return here
      }
    ],
    exports: ['REDIS_INSTANCE'],
})
export class RedisModule {}

cinder wyvern Oct 30, 2022, 5:00 PM

#

Shouldn't need the return if it's a direct arrow function () => new Whatever(), but glad that adding it made it work for you

abstract fjord Oct 30, 2022, 5:01 PM

#

without a return your function is of type "void" so it make sense if it's undefined

cinder wyvern Oct 30, 2022, 5:02 PM

#

Do you have the {} in there in both cases (with and without the return)

abstract fjord Oct 30, 2022, 5:03 PM

#

you mean without the {} it should work without the return ?

cinder wyvern Oct 30, 2022, 5:03 PM

#

Yes. Just as I wrote it. (options) => new Redis(options)

abstract fjord Nov 5, 2022, 2:56 PM

#

yes thanks you

#

it's not really related to this post, but i am replicating websocket's client's id throw redis pub/sub so each server can send event for a specific client id, but client id is connected for one nodejs instance, since i am replicating it, it can't work like that.
Do you have an idea how i can handle that ?

#How to use Redis as a publisher and subscriber on NestJS project