Invoke is killing my SSD 😭

[worn slate]

My MacBook Pro (M1) has only 16 GB RAM (yeah, cheaped out on that...damn) and every time I use Invoke, it eats a ton of RAM and causes a lot of swap.

Recently I noticed that the swapping occurs at every image rendered, thus causing 30 Terabytes (!) of data being written to the drive in a matter of days, maybe a week or two (within one uptime).

I can't use the software like that. DiffusionBee is working differently, I know, but at least it doesn't eat 12 GB+ for that app alone.

Is there any way to reduce the RAM consumption for me?

[limber gyro]

@full crow is there an option to not cache models in RAM, but only in VRAM?

[bright viper]

what do you mean swapping occurs at every image rendered?

what is getting swapped

ooh .. is this a unified memory thing .. ? @limber gyro

[limber gyro]

I'm guessing swapfile. RAM fills up and the OS uses a swapfile to temporarily move less recently used data from RAM to disk.

My guess is that the models being held in memory (even though they've been pushed to the GPU's VRAM) might be causing memory pressure.

[worn slate]

Yes, I am talking about the RAM swapfile. I am not sure if not caching models would help, because I got swapping already with only one default model being loaded

[worn slate]

I just tried to quit all possible programs, even system stuff like Spotlight which is usually on, and loaded up Invoke. It jumped to 11 GB for the Python process, writing 2 GB of swap data to the SSD just for starting up 😩

[balmy tapir]

1. It does not use swap heavily if you generate 512px.
2. M1 always uses swap.
3. I use SD since august, from 512 to 1280px. My SSD is ok.

[balmy tapir]

Also, you can use 2gb models instead of 4.27gb

this is on start

nothing

[halcyon crane]

is there an easy way to compress the normal 1.5 into 2.13 gb?

[worn slate]

Thanks for the input, @balmy tapir! So you never encountered Activity Monitor displaying ridiculous write amounts on the drive tab? I got 80 GB after 5 days of use - sure, but 30 TB is a whole different level 😅 Can I shrink model files? Won't I lose quality then? Or are they being unpacked to RAM when loaded anyway, so it won't matter?

[balmy tapir]

I never noticed that amount. But if you do 768 and more — that’s expected I believe.
You can shrink models to fp16 (2.1gb), then you’ll lose some quality (I do this for trained models only). But the change is minimal.

[worn slate]

Is there a model shrinking guide somewhere? Does this also work with the SD-inpaint model? I think I would try shrinking models. The resulting fp16 also increase the processing speed, right? Hmmm 🤔

[balmy tapir]

Not the speed, just some free RAM/VRAM. I believe I’ve seen the 1.5 somewhere at this size. But can’t provide links, sorry. And the fp16 checkbox was in the dreambooth colab I used.

[worn slate]

I will look around, thank you! 👊👍

[hybrid phoenix]

I believe the M1 / M2 architecture uses the very same RAM for both CPU and GPU. 16GB is just not enough to run Stable Diffusion without generating a swap file. The best you can do is set up a second user account with every extra feature turned off, then run InvokeAI in the most efficient web browser you can find, and run no other apps at the same time. I have the M1 Pro PowerBook (16GB / 10 CPU cores / 16 GPU cores) and the M1 Ultra Mac Studio (64GB / 20 CPU cores / 48 GPU cores) and on the Ultra those 48 cores use up over 40GB of RAM during generation, sometimes leading to swapfiles. (Interestingly, not 3x faster than the M1 Pro, but definitely faster.) When I got the Mac Studio I thought I would never have any use for its full power, and already I want to upgrade to whatever comes next and hurry up with the one that's 10x faster Apple.

[sturdy hemlock]

So can the new M2 Macbook Air run Invoke AI well?

[hybrid phoenix]

I have heard that it can, helped by the slightly faster clock and slightly improved architecture. New optimizations just announced by Apple should make it even more fast & efficient with macOS 13.1.

[worn slate]

Definitely looking forward to the native solution by Apple which apparently is very memory efficient and produces almost zero swapfiles.

I made a test comparison between A1111 (Pytorch), InvokeAI (Pytorch) and Diffusion Bee (some CoreML?) and A1111 is terribly inefficient, causing about 500 MB of swapfile write to the SSD PER ITERATION. So generating one image 512x512 writes about 18 GB of data to the SSD. Ouch. Invoke also writes a lot of data, but much less. I counted about 3 GB for a generated image. DiffusionBee? Almost nothing.

[indigo kiln]

I only have an 8GB M2 MacBook Air, so I don't expect much. But in DiffusionBee, I get about 1s/it, while with InvokeAI, I get only about 20s/it. Ouch!

So I hope that those recent Apple optimzations make it into InvokeAI some day soon!

[balmy tapir]

did you tried coreml version?

[indigo kiln]

I don't know what that is.

[balmy tapir]

these recent apple optimisations 🙂 https://discord.com/channels/1020123559063990373/1048037638063538257

[indigo kiln]

There isn't currently a way to use those optimizations with InvokeAI, is there?

I haven't tried out Apple's fork yet. If I need speed, I just use DiffusionBee. It's fast enough. If I want certain features I use InvokeAI and must be very patient.

[balmy tapir]

nope, at least, not right now. Different model files, etc. But that implementation doesn't need RAM at all (you have 8gb only).

[indigo kiln]

I guess I'm not that motivated to fiddle a lot for nominal gain. I doubt that I'll get much better performance than DiffusionBee, and I won't have a GUI, which DiffusionBee has, and can do outpainting via a canvas, and inpainting with a brush tool.

DiffusionBee uses FP16, so it takes up a lot less ram. If I try to turn on float16 in InvokeAI, it causes it to crash.

[left vine]

20s/it... what size ? I get 512x512 @ 4 s/it on a 8Gb M1 mac Mini ?

[worn slate]

I think it also depends heavily on the model used and the prompts and what is being rendered. I had huge fluctuations in performance, for example in img2img where the source image was rather complex (many details) which had increased rendering time.

[past axle]

am i doing anything wrong? Cause i get 10+ gb by python 3.10, while using 2gb model in invoke. And big swaps, yeah...

And i generate 512px only

[balmy tapir]

How big? Python at 10gb is ok, that screen is true only for the start of Invoke.

[indigo kiln]

512x512. I switched to DDIM and it usually settles down to 10s/it.

I probably just have too much other junk running, I have a lot of menubar things installed on my Mac, etc. And I'm using Chrome, which is a hog. I should probably just do batch generation of images via the command line, or shut down Chrome, and everything else, once it's started generating a batch of images.

Which sampler do you use?