ComfyUI + custom models & nodes

[open surge]

I've read this here, and tried it: https://github.com/runpod-workers/worker-comfyui

But im still not sure if I did it correctly.
So I made a docker file based on one of the versions and add the things I need:

# start from a clean base image
FROM runpod/worker-comfyui:5.3.0-base

# install custom nodes using comfy-cli
RUN comfy-node-install comfyui_ipadapter_plus
RUN comfy-node-install ComfyUI_yanc

# download models using comfy-cli
RUN comfy model download --url https://civitai.com/api/download/models/789646 --relative-path models/checkpoints --filename realvisxlV50_v50Bakedvae.safetensors

Is this enough? also when I set it up on runpod, each run takes time, and it's not working all the time (queue mode), so I feel like I did something wrong

[spark knoll]

Each run will take a bit of time as the workers have the overhead of downloading the image itself. After the initial rampup you should have the same amount of cold start as everyone else. If you want you can share your endpoint ID and I can take a look to see if you've tripped any errors I can view.

[open surge]

yea ok, write it here?

[spark knoll]

Yes, here or in private whichever you're more comfortable with.

[open surge]

c9q99ujckyd710

it needs credentials anyway

[spark knoll]

Yeah just some people are more paranoid.

[open surge]

if they dont know how it works 🤓

one of the last requests I did ran for 12 minutes and was on queue and nothing happened

so canceled it manually

btw, if i'm changing to load balance, I need to change something in the handler?

[spark knoll]

Yes, load balance is more for a standard web server than what our handler does.

Also also, change your allowed CUDA versions to 12.6, 12.7, 12.8, and 12.9

You'll stop seeing the error you had, where a worker was spawned to try to handle that job but it was throwing:
requirement error: unsatisfied condition: cuda>=12.6, please update your driver to a newer version, or use an earlier cuda container: unknown

[open surge]

where do I do it?

[spark knoll]

Manage, Edit Endpoint, Advanced, "Allowed CUDA Versions"

[open surge]

done 👌

[spark knoll]

You should be good

[open surge]

so my endpoint is good?

[spark knoll]

Yes, you won't have that error anymore

[open surge]

ok, i'll check it now
and my problem wasnt the error lol

its the long queue time

[spark knoll]

Your long queue time was because of this error, sorry

[open surge]

ohhh

ok cool

[spark knoll]

I can see like 7 hours ago is when you had the long queue time?

In the admin logs it's a bunch of "failed to start container"

[open surge]

this one probably

[spark knoll]

Yes

Then I see a1b10813-3406-460e-a25b-2a2e2c8470f2-e1

And no more after that

[open surge]

tested one now, looks great

thanks 🙏