I'm currently using the https://github.com/ashleykleynhans/runpod-worker-a1111 along with a network volume. I only use a single model with the sd text2image endpoint and I don't need the UI. Right now, I'm experiencing an 80+ second delay for cold startups on the first request. Do you have any suggestions on how to optimize this (without 1 constant active worker)? Thanks in advance!
#Best tips for lowering SDXL text2image API startup latency?
keen laurel
keen laurel
Or is it better to run comfyui api, is that one faster to boot?
winged shadow
Maybe .. But for me I experience like 2 mins for comfyui cold start
keen laurel
Maybe .. But for me I experience like 2 mins for comfyui cold start
Ok, maybe not then! 😅
winged shadow
It may vary depending on your custom nodes
keen laurel
Do you know how long it takes for it to become cold?
winged shadow
No time specified soo no one knows other than you with your tests
Or other people that tests (idk it might be different too from yours)
keen laurel
Alright, thanks!
idle stirrup
I'm also experimenting with both as well. Some says the long latency may also caused by using network volume. I haven't really tried package models in the docker image myself as I kind of need a lot of models and loras, not sure if it's worth packaging in the docker image