Managing multiple pod discovery | Runpod | Page 1

wet tulip Mar 2, 2024, 9:20 AM

#

Hi, if I want to put a load balancer/queue system for multiple pods, is there some premade app I can use for that? I was thinking of something like Kubernetes but its not compatible with Podrun. Or is this not the use case of podrun?

haughty bayBOT Mar 2, 2024, 9:20 AM

#

To help others find answers, you can mark your question as solved via Right click solution message -> Apps -> ✅ Mark Solution

tough raven Mar 2, 2024, 9:22 AM

#

What is Podrun? Do you mean RunPod? Kubernetes is not compatible with RunPod and you can't use a load balancer. If you want to scale up and handle many concurrent requests, you should use RunPod Serverless, not Pods.

wet tulip Mar 2, 2024, 9:39 AM

#

Yeah RunPod. I want to use some open source abstraction without lock-in

#

https://www.anyscale.com/ray-open-source

There is this, might be a good approach but I see no Runpod compat

Anyscale

Ray Distributed Computing - Anyscale

Ray, the fastest growing open source framework in distributed machine learning, makes it radically simple for developers to scale their most compute-intesive workloads from their laptop to any cloud.

tough raven Mar 2, 2024, 9:46 AM

#

wet tulip https://www.anyscale.com/ray-open-source There is this, might be a good approac...

Its not compatible with RunPod, RunPod Pods aren't designed to be load balanced etc, they are meant to be stand-alone. If you want to do this, you will have to figure out a solution yourself, its not something RunPod offers or can support you with.

#

As I mentioned, you can use serverless, it has queing, load balancing, auto scaling etc all built in, so either use that or you are on your own.

#Managing multiple pod discovery