How to use createMicroservice factory in a AWS Lambda?

[ember falcon]

I'm using NestJS to create a microservice, hosted on AWS Lambda.

lambda.ts:

async function bootstrap(): Promise<INestApplicationContext> {
    if (!cachedApp) {
        cachedApp = await NestFactory.createMicroservice(AppModule);
        cachedApp.init();
        // cachedApp.listen() is a NO GO since we're in a serverless lambda! 
    }
    return cachedApp;
}

export const sqsHandler = async (event, context: Context) => {    
    const instance = await bootstrap();    
    // How can I parse the 'event' to the instance??
};```

The function sqsHandler is triggered every time an event is push to a SQS queue.

How can I parse the lambda event to the nestjs microservice instance and get the result?

Lots of examples online shows how to use the NestFactory.createMicroservice with listen(), this is not a solution since this code is executed in a serverless AWS Lambda.

Also, using a stand-alone nestjs app is also not an option, since we need to implement a custom transport strategy.

[cosmic breach]

If you want to process sqs event as an HTTP request, you can use libraries that convert SQS event to HTTP request like my library: https://serverless-adapter.viniciusl.com.br/docs/main/adapters/aws/sqs#use

In this case, you will get an http post request to /sqs.

But if that's not what you want, you should use standalone mode and get the instance of the service that will handle the SQS event and then process it.

You said you implement a custom transport strategy, can you explain that?

In theory having a custom transport strategy doesn't change anything, you just need to have the service that handles the custom transport logic outside the custom transport strategy module so you can instantiate it when using standalone app, and then just call the same service that will be called by custom transport, that makes sense?

[ember falcon]

Thanks for your answer!

Looking at this thread https://github.com/nestjs/nest/issues/3960 I conclude that the "right" way to handle events from different data source, ex SQS, is to create a custom transport layer (see this answer https://github.com/nestjs/nest/issues/3960#issuecomment-771647374), as described here: https://docs.nestjs.com/microservices/custom-transport

Using a combination of microservice factory and custom transport layer is really awesome.
Only (BIG) problem: It does not work in a serverless setup, as described in my question.

Your solution is workable, but it feels like an anti pattern/hack to use a http layer to process SQS events.

There ought to be a different way, right...?

[cosmic breach]

I think the solutions kamil suggests should be deployed outside of lambda, so an SQS transport layer could work smoothly as it does with Java and other languages.

If you are deploying apis inside the lambda, you don't need to define a custom transport because it's not needed, the lambda will only be called if some SQS event needs to be processed.

And yes, it looks anti-pattern, but I use it a lot, especially when I want to handle both SQS events and HTTP requests in the same code.

The best way to not feel too anti-pattern is to use standalone mode, so you create a service to handle sqs events directly and then just instantiate the API, get the reference to that service and process it.

But using serverless and custom transport layer might work, but to me it seems more anti-standard than using http layer.

Like, you could instantiate your custom transport, create and expose an Observable, and every time a request comes from Lambda, you emit a new sqs event for that observable and then the nest will be treated as if it came from somewhere else.

[ember falcon]

Alright, makes sense.

A follow up question if you dont mind:

A standalone app works great on lambda, but using a service to process events seems like a trade off compared to using controllers with function decorators.

What would you recommend: Using a service to process events (standalone app) or converting the SQS event to a http event, thus using controllers + decorators?

[cosmic breach]

If the code you deploy to this lambda is only meant to be used to handle SQS events, go with standalone, your cold start will be less and performance will be better.

If you want to expose some http endpoints using API Gateway, but also want to handle SQS events, use libraries that convert your SQS event to http, so you could handle both with the same codebase

[ember falcon]

Perfect, Ill take a look at your library. Thank you so much for your insights and help!

[ember falcon]

Quick question:

I have implemented your library in my code, everything works as expected.

Is it possible to route the SQS event based on the SQS topic?

So, if topic is "foo", the foo function is called.

@Controller('sqs')
export class AppController {
  constructor(private readonly appService: AppService) {}

  @Post('foo')
  foo(@Body() event: SQSEvent) {
    // TODO: Implement
  }

  @Post('bar')
  bar(@Body() event: SQSEvent) {
    // TODO: Implement
  }
}```

Thanks!

[cosmic breach]

It's possible but not support by default, what you can do is create another class, extend SQSAdapter and override the getRequest method, you can copy all the code but you only need to change how path is defined, maybe set by event.Records[0].type.

But I don't recommend doing in this way because you loose the ability of processing items in batch, but if your code is not expected to process in batch, is okay.

/**
   * {@inheritDoc}
   */
  public getRequest(event: SQSEvent): AdapterRequest {
    const path = `/sqs/${event.Records[0].type}`; // I don't know where type could live but is inside Records, maybe you need to parse the data property.
    const method = getDefaultIfUndefined(
      this.options?.sqsForwardMethod,
      'POST',
    );

    const [body, contentLength] = getEventBodyAsBuffer(
      JSON.stringify(event),
      false,
    );

    const headers = {
      host: 'sqs.amazonaws.com',
      'content-type': 'application/json',
      'content-length': String(contentLength),
    };

    return {
      method,
      headers,
      body,
      path,
    };
  }

You can override the methods of entire library, so changing the behavior is very simple if you read the source code and identify what needs to be changed.

The source code of the adapter: https://github.com/H4ad/serverless-adapter/blob/main/src/adapters/aws/sqs.adapter.ts#L92

[ember falcon]

Awesome, your the best. Ill make sure the whole team stars your project on monday. Thanks!

[thick storm]

Guys, What's the intend for this question???

Do you wanna use AWS SQS Send/Receive and Consume in Nest.js as Microservice?

[cosmic breach]

Yes, but he wants to do it inside the lambda, which is not possible to run microservice.
If he is on-premise, this option looks pretty good, but inside the lambda you can process it directly instead

[thick storm]

Thanks, You mean Nest Microservice can't be used in AWS Lambda. Is it right???

[cosmic breach]

It can be used, but because the way the lambda works makes the nest microservice useless, because you can't run lambdas for more than 15 minutes and your lambda will only activate if they have something calling. Nest microservices need to be online and waiting for the request to appear and the lambda only works when something triggers the lambda out there, so you can't wait without having a trigger, does that make sense?

[thick storm]

So Make sene. Nice!!!

Thankss!!!

So, I already ask about it in this link #1038091892061110303 message

Any advice about implementing Parallel SQS Service in Nest.js??