TypeORMError: Driver not Connected during Shutdown | NestJS | Page 1

tender wind Sep 19, 2023, 10:37 PM

#

I'm currently working on a Nestjs app that relies on a mix of Bull queues, scheduled tasks (via cron) and various async generator loops for iterating among fetched items from an external API. These features are all working rather nicely together but I can't seem to make a graceful shutdown work as expected. The main problem boils down to, whenever the app is shutting down, despite the TypeORM datasource having been configured with keepConnectionAlive: true, when the queues are closing and any of their corresponding event listeners are then called (e.g.: onQueueCompleted, onQueueError, onQueueFailed), where some housekeeping chores need to be performed against the database, I inevitably get a TypeORMError: Driver not Connected exception on the console, barring any of these chores from being performed, ultimately leaving some data behind in an inconsistent state.

I've been looking around on the interwebs for almost two days now, but I couldn't find any definitive solution to this problem. I hope someone here can help me figure this out.

spice cargo Sep 20, 2023, 10:11 AM

#

my guess is that this handler forces the disconnection: https://github.com/nestjs/typeorm/blob/36ccf21bb0200c3b1376cef20f6192fad19accc6/lib/typeorm-core.module.ts#L141-L152

unfortunately, I don't see how you could disable/delay that. Only maybe doing the queue cleanup in earlier lifecycle event.

tender wind Sep 22, 2023, 1:51 AM

#

Thanks for pointing that out. I have now moved the cleaning up of resources to the onModuleDestroy event handler. Before, I was using mainly the beforeApplicationShutdown event. Things are looking better now, in that the TypeORM error seems to be gone. However, at almost every shutdown, the app hangs for quite a long period (minutes). It does eventually end, releasing the terminal, but something is definitely not right.

tender wind Sep 22, 2023, 10:17 PM

#

I guess I spoke too soon. The driverError: Error: Connection terminated error is presenting itself again. Oh boy...

spice cargo Sep 26, 2023, 12:50 PM

#

These are unfortunately quite difficult to debug (or at least I don't have a simple trick).
When this happens to me, I try commenting out modules one by one until the process exits correctly, to figure out which module is causing the hang. At the moment, bull with redis is somewhat tricky, as it doesn't close the queues when exiting repl.

Also, this bit is somewhat useful for identifying what's hanging:

process._getActiveHandles().forEach((handle) => {
  console.log(handle);
}

You could drop that to setTimeout in some shutdown lifecycle event to log what's not closed. If it's a socket with port 6379, it's redis.

#TypeORMError: Driver not Connected during Shutdown