Microservice autoscaling

[proven belfry]

Hi guys im new to nest and nest microservice and decided to use microservices, but the issue is ive built microservices with tcp and the devops team told that it wont work in terms of autoscaling, so what do i do now? should i move everything to gRCP?
can someone tell me how to handle this?

[hollow flare]

this is what AI comes up with
Event-Driven Systems: Apache Kafka (for durability and partitioning).
General-Purpose Messaging: RabbitMQ (for flexibility and reliability).
High-Speed, Low-Latency: NATS (with JetStream if persistence is needed).
Synchronous APIs: gRPC + service mesh (e.g., Istio).

So which use case are we looking at?

[vale glacier]

What specifically with your TCP endpoint/ service wasn't autoscalable? Did your devops team tell you that? Because, the issue usually isn't the service directly or how it communicates per se, but rather either the issue comes from its deployment environment or its dependencies. We need more info to help.

[proven belfry]

Thanks for the response you two,

Soo the team told there TCP since its stateful will cause trouble in load balancing and auto scaling since its hard to change the established connection to a different pod, i don't have 100% understanding on this since I'm not a dev ops and just started to learn it.

So the problems according to dev ops are:

1. TCP being not stateful
2. in EKS its easy to auto scale HTTP requests but for TCP they say its not a good practice or will cause a lot of trouble

And if for what I'm looking , im just trying to develop a b2b application that also has a feed in it , this might get heavy traffic in times so the devops planned for microservices and nest, i built the microservices without knowing these complexities , now i just need some guidance of what would best suit this situation , thanks.
@hollow flare @vale glacier

[vale glacier]

Those two points don't help me make sense of the concerns for autoscaling.

At any rate, why not just start off with a normal app and make it Restful? Don't even use the microservices route.

If you need realtime data in your feed, you might also go with websockets and HTTP/ Restful in a hybrid app.

Or, take the dive into the deep end and use gRPC with microservices.

It really depends in the end on what you need to get done. 🙂

[proven belfry]

so you mean even with a tcp auto scaling and load balancing can be done without any problem?
And yes im now actualy planning on moving to gRPC, but also for that internet says its hard to do load balancing without using any support tools, so im still confused how to proceed

[vale glacier]

So, if your app can handle connections being round-robin routed to any pod running the microservice (at any scale), TCP will work fine and scale fine. It's why I was wondering what the issue is and those two points don't really change this fact in any real way.

gRPC runs on http/2, which means the connections can be long-lived. This does complicate the networking decisions for sure. But, there are known solutions for it, like using a service mesh.

I'd also suggest just sticking to a monolith until you see performance degradation. You can scale a monolith horizontally too. 🙂

[hollow flare]

so it is this one right? Synchronous APIs: gRPC + service mesh (e.g., Istio).

[hollow flare]

I agree, until you hit a wall with the monolith that would justify better trade-offs with individual microservices. Sometimes you can single out a specific service too and do not need to break the whole app into services

[terse merlin]

But doesn't gRPC usually run over TCP too?

[proven belfry]

Yeh I understand but the scale of the application is actually big , hence the micro services

[proven belfry]

So by using a round robins algorithm even TCP can work well ? Even the load balancing? I also wanna know like if I use 2 ports on a microservices where one runs on TCP for internal nest microservices communication and a grcp port for external microservices communication like a python service just so that all the microservices stays connected since idk if python can connect using TCP but grcp connection can be configured in python, what's your thoughts on this?

[proven belfry]

Yeh

And uses https 2 as well

[vale glacier]

Well, if you consider HTTP as just a protocol layer higher over TCP, then yes. But, it needs HTTP/2.

[vale glacier]

So by using a round robins algorithm even TCP can work well ?

Sure. Round Robin is a type of load balancing. So, not sure what you mean by "even the load balancing".

If you connect one service via gRPC to another service, then you'd be using the client in that service, so you wouldn't need a second port.

[proven belfry]

Thanks for yours response Scott, really appreciate it,

No i was just trying to confirm if the round robins is actually the best/recommended way? because one thing I actually came here for is to know how these micro are usually handled and deployed .

For the extra gRPC connection yes, i thought to make things less complicated for the internal communication with TCP and only expose few APIs for other services in gRPC which would be like 2-3 APIs every service

[vale glacier]

Round robin is what you have as the standard and only method with k8s services. If you need something else, you'd have to build out an internal network (like with a service mesh). That might be the concern your devops team have. More work.

For the extra gRPC connection yes, i thought to make things less complicated for the internal communication with TCP and only expose few APIs for other services in gRPC which would be like 2-3 APIs every service

This didn't make much sense to me. If you go with a communications path, you should stick to that one path. You shouldn't mix and match, as it only adds a lot of complication. Instead, stick to one. If you need something that one communication method can't do (like I mentioned using websockets with HTTP for adding live data feeding), only then look for a different method.