error: "ComfyUI server (127.0.0.1:8188) not reachable after multiple retries."

[flint tangle]

After several days of working API without fails, it started to return the following error :"ComfyUI server (127.0.0.1:8188) not reachable after multiple retries.".
What can be wrong?

Also noticed that a lot of my workers are throttled recently. The region is EU-RO-1. Execution timeout in serverless is set to 1200.

And the following is set in the handler.py:

COMFY_API_AVAILABLE_MAX_RETRIES = 500
COMFY_API_AVAILABLE_INTERVAL_MS = 50

After assumption that this happens due to the lack of GPUs, I tried to swtich from 5090 to two 4090 GPUs and it was not running at all. It was just idle all the time, despite 3 queued requests waiting for more than 5 minutes.

Currently all my workers are throttled and the question is, how can i rely on the service that does not guarantee the aviability of the workers for my API? I had working API for a week and due to higher demand apparently I do not get any more GPUs that suit my usage.

[umbral remnant]

To help others find answers, you can mark your question as solved via Right click solution message -> Apps -> ✅ Mark Solution

[bitter fjord]

unfortunately just the nature of renting serverless access to scarce resources. no service would be able to guarantee access to one or a couple specific cards reliably without having the renter commit to a long-term rental period or jacking up the price. the norm/expectation is to have multiple backups, which runpod conveniently supports

this issue is much worse if relying on a network volume since you become region-locked, significantly reducing the supply of available GPUs. in my experience, you can't both utilize a network volume for serverless and be picky about GPU selection

(not associated with runpod)

[vital lark]

you can use multiple regions + multiple network storage in one endpoint
but they won't copy oover the files to the other regions automatically

but it simplified the setup

[vital lark]

you add more workers

[vital lark]

top up your balance to like $200 maybe, then get more workers, add up the max workers there

*in serverless page

if you ustill get throttled much, try adding another region for backup

[flint tangle]

What is the point of adding more workers? For me 1-2 is enough, i always get extra 3 with are throttled most of the time

What about the main problem of the post? The error

[vital lark]

so that you have extra workers when its throttled, or if it doesnt work switch region, add active workers, or have more worker running duration ( request processing)

[vital lark]

you correctly found the variables just add those two, modify then build your image to runpod
COMFY_API_AVAILABLE_MAX_RETRIES = 500
COMFY_API_AVAILABLE_INTERVAL_MS = 50

or use github integration (if you havent)

[vital lark]

it could help so that not all of your workers get throttled

[flint tangle]

This was already set, the problem appeared after

[vital lark]

i think you misunderstood my statement to "add" those variables, its not like that.

I meant, to add the VALUES of those variable (meaning to increase)

COMFY_API_AVAILABLE_MAX_RETRIES = 1000

for example like that

[crisp bronze]

I have the exact problem. Have you solved it?

[vital lark]

Did you try anything that doesn't work?

[crisp bronze]

What do you mean?

[vital lark]

Did you try anything to solve it?

How did it go?

I think I've given the solution above.. If you have tried it and failed, please report back with your comfyui logs

[sour quarry]

@flint tangle @crisp bronze Could you try one thing for me? create a new network storage volume in EU-RO-1, put fresh data there, and see if that helps fix the issue. I suspect the data in your current storage might be corrupted and cause the comfyui server unable to start

[flint tangle]

The problem persist after some retries

[flint tangle]

If that is already on exactly this volume? I had also backup one, but this resulted in unexpected errors that i havent seen before