Supporting 1000x calls per second for httpService (best practises)

[median charm]

Hi there, I would like to see if anyone has recommendations on best practises when implementing a NestJS backend, using the axios based httpService and wanting to support high frequency external API calls from the system? I have recently run into issues and believe that im am getting client side 'ECONNRESET' errors.

So far I have started trying this (see screenshot) and seek your opinion on whether this is a good approach or if there are other/better things I can do to support this kind of http call frequency.

[pulsar aspen]

I'd suggest that if the external API is down, you don't only retry, but rather fail after a number of attempts to your own clients/ users with a "We have a server issue currently, please try again later" message. Otherwise, you'll probably end up with back-pressure issues and your whole system might crash.

Also, you might want to look into exponential backoff and the circuit breaker pattern.

Lastly and more of a side note, if you are having issues with throughput and there is any way to cache the results for requests to the external API, you would probably want to look into that too.

[median charm]

Thanks @pulsar aspen, so I have not yet gotten to the bottom of why I am getting the ECONNRESET error quite frequently, but have confirmed that when I am getting it, the external API is not called (confirmed that with the API vendor). So this instance of ECONNRESET errors that I am experiencing are max sockets reached or any other client side (my side) issues, that I am trying to solve. Makes sense?