Questions about code optimization with TypeOrm

[umbral frigate]

For some of my API routes, I have many many other functions being called to obtain data regarding parameters sent in and those functions each have their own database query call.

Example of a selection of code that uses this functionality.

const [server, activeUnit, unitList, callList] = await Promise.all([
    this.patrolService.getServer(serverId),
    this.patrolService.getPatrolUnit(user.userId),
    this.patrolService.getPatrolUnits(serverId),
    this.cadService.getActiveCalls(serverId)
]);

Example of my .getPatrolUnits(serverId):

async getPatrolUnits(serverId: number): Promise<PatrolUnit[]> {
        const units = await this.patrolUnitRepository.findBy({ serverId: serverId });

        for (let i = 0; i < units.length; ++i) {
            units[i] = await this.getPatrolUnit(units[i].userId);
        }

        return [...units];
    }

This loops through all units found in the original query, and then makes a function call using that userId to get more data about the user.

And inside of each .whatever(param), they call to the database which can sometimes result in other .whatever(params) being called inside of each of those. Is there a more streamlined way of doing this, or even more efficient?

[brave crane]

First question would be - do you notice a degraded performance with this solution?
That being said, there certainly is a way to optimize your read queries using the query builder instead of trying to mold everything into entire entities, even though you only need a single property. That's why a lot of complex apps use a separate read model, using which you can express complex queries easier.

I see one simplification straight away - instead of looping over ids and awaiting them, you can use Promise.all, or even change the query to retireve all items at once with a single query

someRepository.find({ where: { id: In([1, 2, 3, 4]) } });

[umbral frigate]

First question would be - do you notice a degraded performance with this solution?
Response times on some endpoints could be anywhere from instantaneous to 2-8 seconds, which obviously isn't wanted.

[umbral frigate]

Not sure this would work for what I am trying to do necessarily, as inside of my await this.getPatrolUnit(units[i].userId); I am just obtaining more information to add to the object for the user

async getPatrolUnit(userId: number): Promise<PatrolUnit> {
        const patrolUnit = await this.patrolUnitRepository.findOneBy({ userId: userId });

        if (patrolUnit && patrolUnit.userId > 0) {
            patrolUnit.department = await this.departmentRepository.findOneBy({ departmentId: patrolUnit.departmentId });
            patrolUnit.beat = await this.beatRepository.findOneBy({ beatId: patrolUnit.beatId });

            if (!(patrolUnit.currentCallId === 0 || patrolUnit.currentCallId === null)) {
                patrolUnit.currentCall = await this.cadService.getCall(patrolUnit.currentCallId);
                patrolUnit.currentCall.callType = await this.patrolCallTypeRepository.findOneBy({ callTypeId: patrolUnit.currentCall.callTypeId });
            }

            return patrolUnit;
        } else {
            return null;
        }
    }

Unless I am overthinking the method you mentioned

[brave crane]

Right, you can still fire the requests that don't depend on each other concurrently with Promise.All. That will already give you a considerable speed-up

const patrolUnits = await Promise.all(units.map(({ userId }) => getPatrolUnit(userId))

This trick can be done in other places, too, for all queries that don't depend on the result of the previous one.

const [department, beat] = await Promise.all([
  this.departmentRepository.findOneBy({ departmentId: patrolUnit.departmentId }),
  this.beatRepository.findOneBy({ beatId: patrolUnit.beatId }),
])
patrolUnit.department = department
patrolUnit.beat = beat

[umbral frigate]

Ohhhhhhh I see. That makes more sense.

Ill try updating my code base to utilise this.

Thank you

[brave crane]

Other option is using in-memory LRU-cache for certain repeated queries. For example, if you query for getPatrolUnit for an user multiple times in a short period of time, it doesn't make sense to hit the database every time. You can use this nice decorator library with the lru-cache adapter https://github.com/joshuaslate/type-cacheable and put the @Cacheable() decorator on methods that should be cached (for example for 1 second)

But keep in mind that both of these approaches will increase the memory consumption of the application

[umbral frigate]

Hmmm, using this option on some other selections of code for like searching information via a search bar on my website, with the old method it used to show all the data because of my multiple awaits, but using this method, it looks like that information is skipped over now and finishing before those processess complete

Oh wait stby maybe not