#HDD-Goblin || HDD-Disk Model loading - 10X faster! (For real)

1 messages · Page 1 of 1 (latest)

peak leaf
#

I was initially presenting my own extension here. But this one is simpler:

It preloads models into system RAM and loads from there. Speeding up HDD loading.

Set up --disable-mmap in ExtraArgs (Server > Backends)

How to install:

Look in:

\SwarmUI\dlbackend\

You should find a ComfyUI folder in there. Then the install path becomes:

\SwarmUI\dlbackend\ComfyUI\custom_nodes\

Open a terminal in that custom_nodes folder and run:

git clone https://github.com/nonnonstop/comfyui-faster-loading.git

Then restart SwarmUI and it'll be active automatically.

hexed lotus
#

There's simpler hacks to tell comfy to load into ram first, "comfy faster loading" or something was the node pack iirc

peak leaf
hexed lotus
#

Yeah that

peak leaf
#

Interesting, maybe that is superior. I’ll give it a read. Thanks

peak leaf
# hexed lotus Yeah that

That one works better overall and needs no Ramdisk. I edited my initial post to advertise it instead 🥲

marble needle
#

One thing to consider is that comfyui-faster-loading works on safetensors files only, not gguf.

Once the recently hyped comfyui "dynamic-vram" transition seemed to get stable I approached cautiously and...

...I've been having a lot better load times using the default settings than before with default settings.

So, I finally gave up trying out various additional hacks, working along the lines of comfyui-faster-loading above. Note that I have lots of RAM available, so that might be one reason it works for me, even though the wins are supposed to be for the RAM-needy users.
https://blog.comfy.org/p/dynamic-vram-in-comfyui-saving-local

But also note that WSL support will not come for the new default technique, it will fall back to the older methods. Windows and Linux are supported only.

Dynamic VRAM in ComfyUI: Saving Local Models from RAMmageddon

A new memory system that makes it possible to efficiently run the largest models on the smallest memory.

peak leaf
#

It’s definitely a niche issue. I got 128gb RAM and 24GB VRAM.
But buying terabytes SSDs storage for the models is the thing that’s the real annoying part. Compared to the other resources it permanently takes up expensive storage space and LLMs are also getting bigger each year.