zealous shore
Getting "MPS backend out of memory" error when trying to run SDXL model. M1 Mac with 16GB RAM. InvokeAI set to use 12GB RAM.
Using following options looks to help, but should they be needed with latest version of Invoke AI?
export PYTORCH_ENABLE_MPS_FALLBACK=1
export PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0
half scroll
Well, to be honest, asking InvokeAI to use 12gb of a 16gb machine where you definitely do not have 12gb free for InvokeAI ... in theory it's behaving correctly by saying it's out of memory. You're creating a situation where you're using your SSD for VRAM otherwise. It'll be slow and not great for wear.
zealous shore
so not a bug but a feature? 🙂 I dont understand these export settings that well, but they seem to circumvent the problem
lilac elbow
PYTORCH_ENABLE_MPS_FALLBACK probably doesn't help with the memory, its there to support some functions some bits of SD use what are not supported by Pytorch using Apple Sillicon's GPU.
PYTORCH_MPS_HIGH_WATERMARK_RATIO basically set's how much of the RAM available pytorch allows itself to use, normally its value between 1 and 2 which is 1x to 2x the 'recommended' max value. 0.0 means 'just got for it' and allows it to use as much as it wants.
If the memory allocation gets to high MacOS will have to start using the SSD for swap space, especially if you're using for Mac for other stuff at the same time.
Try using fp16 if you're not already for model and vae, you may need the fp16 fixed vae for SDXL depending on the model.
hmmm I also wonder if the model cache counts in the MPS allocation in which case the cache could be a bad thing without the Ratio being set off or higher, I mean models in the cache probably get paged out anyway if you don't have the RAM. Might be worth bringin that cache value down.