Monitor Outgoing HTTP Requests

[fallen needle]

Our service relies on many third party apis, and it is hard for us to keep track of how often the apis get called, how often they succeed and fail - if they fail then with what status code.
I am wondering if someone knows an easy way to visualize something like this with NestJs!

I would greatly appreciate help, thanks a lot!

[crimson whale]

Can't you just log calls and responses ? https://docs.nestjs.com/techniques/logger#using-the-logger-for-application-logging

[fallen needle]

I would like to visualize the data instead of having to scroll through an endless stream of logs. Any idea how I can easily achieve something like this with NestJs? 🙏

[rose hinge]

Do you need to trace which exact calls failed with what exact code, or do you just want some statistics? If the latter, then the tool for that is called Prometheus and you can plug it into your app to generate all sorts of metrics, which you then scrape and visualize using some observability tool (like grafana)

There's nothing Nest specific to it. Maybe except using a nest-specific integration of Prometheus

[fallen needle]

I want some general statistics, like API X was called Y times, and failed Z times. Out of the Z failed requests it failed A times with status code BCD. Do you think Prometheus is a good choice for that?

[rose hinge]

Yes, definitely

[fallen needle]

Ok, thanks a lot! I have 0 experience with Prometheus. Do you know a good place to get started with Prometheus in a NestJs application in this context (tracking the API stats)

[rose hinge]

Here's a great intro video https://youtu.be/STVMGrYIlfg

Anything else is just a google search away

[fallen needle]

Got it, thanks a lot for helping me out!

[fallen needle]

I have been playing around with Prometheus and Grafana the last few day, I really like it!
Do you know what would be a good approach if I would like to trace each API request? Like how it was called, what it returned, and what error it threw in case it failed.

[rose hinge]

That's what OpenTelemetry is for. There's a module called nestjs-otel which integrates with Nest. If you already use grafana, you can ise Grafana Tempo to display traces

But if you want to know the payloads of each request, too, then you need to log them yourself. It's not a good practise to store payloads in traces

[fallen needle]

Thanks a lot, I will check it out!

[bronze berry]

Potentially this is a good use case for Sentry, which also has a SDK specifically for nest (and uses OpenTelemetry under the hood to trace your api calls): https://docs.sentry.io/platforms/javascript/guides/nestjs/