N/A

[ashen hill]

While Comfy is already at 6.0:

This blocks us from offering some new image models to our SaaS clients, can we urgently get it updated for Serverless?

[proven finch]

@ashen hill will do the update today and let u know!

[ashen hill]

Ugh that would be amazing, we're holding off a new launch because of the Comfy version that doesn't work

[proven finch]

sorry for this, will do it asap

[ashen hill]

no worries, thanks for the quick response, will the devs also add in z-image? Comfy's update was specifically to make this ai image work

This model is almost as big as flux dev 1, the one currently in the template, but its significantly faster, its basically the new hot thing right now and we would love to offer it for our SaaS clients

[proven finch]

would z-image-turbo be fine? https://docs.comfy.org/tutorials/image/z-image/z-image-turbo

@ashen hill so the images are updated on docker hub and after some time also on the hub, but if you create the image yourself, you can use the new base-image already: https://hub.docker.com/r/runpod/worker-comfyui/tags?name=5.7.0

also we have the model store now, which will make it obsolete to bake models into the images. we are currently making sure that this also works with comfyui

[ashen hill]

Correct, that's the only one available right, Base and edits will be released later

[ashen hill]

But won't that have extremely slow start up time?

I initially thought (Correct me if I am wrong, my main dev is on holidays) that I could have the models stored on Network volume and use cards on the same DC to run queues on Comfy, is that doable?

That's when I ran into Comfy old version which doesn't support Z image

[proven finch]

the model store works like this:

• central storage per data center
• a worker is requesting a certain model, so we sync this onto the machine where the gpu is running
• once the model is there, it will have at least the same speed as the model from the docker image

the more people that use a certain model, the higher the chance that it is already there. and the faster we can load the docker images, as they are way smaller

so there is a certain tradeoff, but famous models will be there instant AND iterating on the docker images / updates is way faster

[proven finch]

yes that is doable, but the network storage is way slower than having the model directly on the same machine as the gpu is (model store or model baked into the image)

[ashen hill]

How's that in terms of speed?

Z image is all about fast generation, so hopefully you get my point why it matters to our clients? With all due respect

[proven finch]

loading the modal into vram from the network storage takes longer than having the model in model store or baked into the image

the larger the model, the longer it takes

[proven finch]

yes totally got you, right now baking the model into the docker image makes the most sense until we have model store support also for comfyui

when we do have the support, then that will be faster

i will make sure to add a new image that also contains z-image-turbo, that is not hard to do

[ashen hill]

Yes!! So is it negotiable to have it as an image like Flux dev 1 is right now?

[proven finch]

yes totally, what ever the community needs we try to do

[ashen hill]

if it's not out of the scope of your work of course

[ashen hill]

Thanks!!

So basically I'll have it here right? And it becomes just as fast Flux dev serverless?

[proven finch]

nope, you will have it as a docker image on dockerhub and then you can use it like that or build on top of it.

the hub currently does not support different docker image versions, we can only host one there. we could have a new entry for comfyui with that image model, but this would make it even harder to choose which one is correct. we are working on making this ui more "nice", meaning that you will have one "comfyui" there and then can choose which model you want, so that we don't have multiple comfyui listings

[ashen hill]

So the rest of this is not backed in on serverless right? Meaning there's gonna be that annoying boot time just like network storage? With all due respect

Maybe my question isn't precise, but after you are done adding the z-image-turbo image, will that also include the z-image files to run? meaning my users will still continue to "queue online" and runpod serving the results, right?

[proven finch]

yes exactly, you will then have to create a new serverless endpoint with the new docker image for z-image-turbo and then everything else works as you have it right now

[ashen hill]

Perfect!!

Quick question, since we're using a custom workflow that our users just drag and drop into their Comfy, is it possible to have the custom nodes also installed into the image as well?

[proven finch]

jep, that is what you can do as explained in here: https://github.com/runpod-workers/worker-comfyui/blob/main/docs/customization.md

[proven finch]

and this should help u doing it, but has some bugs right now, but we are working on those https://discord.com/channels/912829806415085598/1426122736601661550

[ashen hill]

Currently trying to do it myself yeah

Some issues here and there trying to walk through them

[ashen hill]

Hi Tim - Are you able to help me with our image on Runpod?

[soft orbit]

Sorry coming from peanut gallery. Can you link more info on this? Sounds awesome

[mystic snow]

Can do!

https://docs.runpod.io/serverless/endpoints/model-caching#enabling-cached-models

[ashen hill]

N/A