Pod stuck when starting container

[uneven hinge]

Yesterday I updated my serverless endpoint with "New release" button. However when the new request came the worker stuck when trying to start container and sucked the remaining funds from my account. In the logs I see multiple worker exited with exit code 0 errors

Probably something wrong with my container, but would be nice if after multiple failed attempts to start container the worker stopped automatically and didn't drain money.

[fair wolf]

maybe its on your program ( not sure )

how does your handler file looks like, or describe how it works maybe

[tropic lance]

Returning 0 is reporting success. If you want it to count as a failure it will need to return a number other than 0.

Suggest you open a ticket with RunPod (under help/contact on their site) they are generally pretty good about those kinds of things.

[uneven hinge]

It is just starting ComfyUI and executing prompts from incoming requests

if __name__ == "__main__": wait_for_service(url="http://{}/prompt".format(COMFYUI_ADDRESS)) print("ComfyUI is ready. Starting RunPod...") runpod.serverless.start({"handler": handler})

[uneven hinge]

Ok, thanks. The weird part is that it kept trying to execute single request for many hours, logs look like this

2024-10-09 06:55:45.855[n5bwvtgs4p6x3t][error]worker exited with exit code 0
2024-10-09 06:55:29.577[n5bwvtgs4p6x3t][error]worker exited with exit code 0
2024-10-09 06:55:13.271[n5bwvtgs4p6x3t][error]worker exited with exit code 0
...

and so on

[tropic lance]

I think I see the problem, You should NOT be querying ComfyUI in main. That should happen in your handler function. In main this is before you have any request.

Does your handler function have similar code or does your handler function call another function with similar code as below?

# The top level element "prompt" is required by ComfyUI
data = json.dumps({"prompt": workflow}).encode("utf-8")

req = urllib.request.Request(f"http://{COMFY_HOST}/prompt", data=data)

[uneven hinge]

yes, it has exact same code and some other stuff like uploading generated images to s3

[tropic lance]

Are you using this one? https://github.com/blib-la/runpod-worker-comfy

I use that one and this is my complete main:

# Start the handler only if this script is run directly
if __name__ == "__main__":
    runpod.serverless.start({"handler": handler})

[uneven hinge]

nice, i'll have a look at it, thank you

[tropic lance]

wait... that is for Serverless Worker, not pod.

[uneven hinge]

yes, i'm using serverless