Running transactions across modules

[tepid ocean]

Let's say I have two modules: PostsModule and CommentsModule and I want to create a post along with its comments in a single route (I know this is a weird scenario but I'm taking that example just for the sake of clarification). Let's also assume I have createPost method in my post service and inside of that service I create a post and call createComment method of comment service. Both of these services are called inside the same route.

Since we are performing two different write operations in a single route, I have to wrap everything within a transaction. I know that one solution is to create query runner in that route and use the entity manager from it to insert into multiple tables like:

const queryRunner = dataSource.createQueryRunner();
await queryRunner.connect();
await queryRunner.startTransaction();

await queryRunner.manager.insert(Post, {/**/})

await queryRunner.manager.insert(Comment, {/**/})

However with this solution I'm not using comments service. I'm doing everything inside the post service which would break the single responsibility principle. I want to call comments service methods to create a comments but still wrap all those operations in a transaction. Does anyone know how to use transactions across different modules? Thanks in advance

[atomic slate]

Either you pass the queryRunner around modules, or use something like https://github.com/odavid/typeorm-transactional-cls-hooked (which does not work with modern TypeOrm) to propagate the transaction. You can also build your own solution around AsyncLocalStorage

[tepid ocean]

Thanks for the suggestion. I had already seen transactional-cls-hooked but I'll look into AsyncLocalStorage

[tepid ocean]

For those who might have the same problem, I have implemented the following interceptor that uses AsyncLocalStorage as mentioned by @atomic slate

import {
  CallHandler,
  ExecutionContext,
  Injectable,
  NestInterceptor,
} from '@nestjs/common';
import { AsyncLocalStorage } from 'async_hooks';
import { Request } from 'express';
import { Observable, catchError, concatMap, finalize, throwError } from 'rxjs';
import { DataSource, EntityManager } from 'typeorm';

export interface IStore {
  manager: EntityManager;
}

@Injectable()
export class TransactionInterceptor implements NestInterceptor {
  constructor(
    private dataSource: DataSource,
    private als: AsyncLocalStorage<IStore>,
  ) {}

  async intercept(
    context: ExecutionContext,
    next: CallHandler<any>,
  ): Promise<Observable<any>> {
    const queryRunner = this.dataSource.createQueryRunner();
    await queryRunner.connect();
    await queryRunner.startTransaction();

    const store = {
      manager: queryRunner.manager,
    };

    return this.als.run(store, () => {
      return next.handle().pipe(
        concatMap(async (data) => {
          console.log('Comitting');
          await queryRunner.commitTransaction();
          return data;
        }),
        catchError(async (e) => {
          await queryRunner.rollbackTransaction();
          throw e;
        }),
        finalize(async () => {
          await queryRunner.release();
        }),
      );
    });
  }
}

Thanks a lot @atomic slate . You saved me a ton of time. I really appreciate it

[blazing rivet]

Thanks for sharing @tepid ocean 🙂
So you'd just use that interceptor on your route then?

(I'm quite new to Interceptors so my question my be perfectly dumb :x)

I was thinking about adding an AsyncLocalStorage context on each request through a middleware so I could enter a transaction context whenever I wanted and make that available in said context.

In turn, my repositories would know about said context and use the appropriate transaction object when needed. (I'm using prisma so the api varies a little)

Basically the same idea but applied globally and transaction would be started from within the controller's handling (use-cases in my case, but that's irrelevant to the discussion)

[atomic slate]

This guy did it: https://github.com/Papooch/nestjs-cls/issues/42

[blazing rivet]

Ah perfect @atomic slate ,thank you very much!

I implemented exactly this in my previous work, but I am very new to nestjs so all the lingo is still very confusing to me, but this should help.

I have experience with als and proxies so it's a matter of understand how all those nestjs parts fit together.

[tepid ocean]

That is correct. I inject als in my repository as well and decide which entity manager to use with a method like that:

  protected getRepository(): Repository<T> {
    const manager: EntityManager =
      this.als.getStore()?.manager ?? this.dataSource.manager;
    return manager.getRepository(this.EntityClass);
  }

[blazing rivet]

I think I'll explore the proxy way as I intended originally as it'd remove the need to do that in repositories.

If I manage to inject a proxy service for prisma instead of prisma directly it'd be much easier and transparent.

[rocky ore]

I could add a magnitude of complexity to that, how to do transactions across microservices?

I know I know this is out of scope of this thread

[atomic slate]

Btw if you go with the nestjs-cls library, it has a built-in support for proxies (called Proxy Providrrs). You might wanna check that out:)

[blazing rivet]

I had a quick look when you shared the link, it does look promising but I'm still trying to wrap my head around all the things in NestJS 😅
So many things to learn

[blazing rivet]

As far as I can tell, unless you have something in between the services allowing for sharing the connections it isn't possible.

A transaction is linked to a connection so unless you can resume said connection (but so many timing issues), I believe it's not really possible.
That's one of the trade offs with microservices I guess. But maybe I'm wrong, I have never looked at it

[blazing rivet]

Maybe we're not talking about the same thing though 🤔

I mean Proxy as in the JS STL Proxy class.

So I basically have a Prisma service at the moment, like this:

@Injectable()
export class PrismaService extends PrismaClient<Prisma.PrismaClientOptions, Prisma.LogLevel> implements OnModuleInit {
  private readonly logger = new Logger(PrismaService.name);

  constructor() {
    super();
  }

  async onModuleInit() {
    await this.$connect();
  }

  async enableShutdownHooks(app: INestApplication) {
    this.$on('beforeExit', async () => {
      await app.close();
    });
  }
}

And this is used in repositories through injection.

My ideal would be that this injectable service would now be a Proxy that would know whether to return PrismaService or the result of prisma.transaction if it is available in the ALS store.
And I'm not too sure how to do this in the context of NestJS's DI injector yet 😅

[atomic slate]

Yep, we're talking about the same thing 🙂 Proxy providers in nestjs-cls are implemented using JavaScript Proxies. Just a bit abstracted away behind the whole AsyncLocalStorage mechanism.

[blazing rivet]

Would those allow me to do what I tried to describe above 🤔 ?

[atomic slate]

Absolutely, you would just decorate it with @InjectableProxy and register with thr ClsModule.
If you want to do it manually, you'd need to define the proxy and implement all the access traps and use the proxy instance as the value of the PrismaService provider

[rocky ore]

yes, I think you have to have something like a queue that is able to build the transaction for you

[blazing rivet]

Makes sense theoretically.
That part I can't figure out is how to return the proper prisma instance from that proxy regarding the injection process

I think my struggles here are mostly about NestJS and my understand of how all moving pieces fit together though, so I might have to spend some more time with that first

[atomic slate]

Yeah, I think it's key to first understand how the DI and modules fit together. I actually haven't seen proxies used much within the ecosystem (apart from internally by libraries), they're still a very little known feature