what am i doing wrong, serverless workers optimization

[narrow hemlock]

To help others find answers, you can mark your question as solved via Right click solution message -> Apps -> ✅ Mark Solution

[wispy atlas]

i dont know this discord is dead

[obtuse wyvern]

Sometimes just downloading from GitHub can be rough. If you can help it, downloading in parallel can be useful

[wispy atlas]

is there like a blueprint, for best cold start optimization

i want to keep the min workers at 0

i have models already on network volume, in same datacenter location

but its like 40gb of models

[keen storm]

i didn't know sending this could achieve something

But maybe you can try opening a support ticket too if you'd like

[keen storm]

just have a constant flow of request coming in

the more requests the better your flashboot start will be

[keen storm]

Try to put the models into hf then use the model cache feature ( model field in your endpoint settings)

i think its not yet well documented yet but worth trying to optimize cold starts

[spiral sparrow]

@keen storm Is the model cache feature different (and more efficient) than downloading the models from the Dockerfile with the Flashboot enabled?

[keen storm]

Yes it should be faster

[keen storm]

You don't have to bake in models meaning faster push and build and easier but I think it's not well documented for usage. You can try it

[spiral sparrow]

@keen storm When using the cached model feature in the endpoint setting, should I remove commands to download associated huggingface files (model, textencoders,vae, etc) from my Dockerfile ?

[keen storm]

Yep, you can access it inside your worker on a specific path

[spiral sparrow]

@keen storm What should be the specific path? There is no information in the online documentation.

If I use cached model and remove the initial download from Dockerfile, ComfyUI cannot find my model files in /comfyui/models/

[keen storm]

Right..