spark sandal
Today Apple released significant optimizations for CoreML specifically to support SD - can these optimizations be integrated into InvokeAI? At first glance, it would appear to improve performance dramatically.
https://github.com/apple/ml-stable-diffusion
https://machinelearning.apple.com/research/stable-diffusion-coreml-apple-silicon
rigid vault
yes, the team is working on the diffusers implementation
btw, that core-ml works great
spark sandal
Excellent!
rigid vault
1.96it/s for the 1.5 model, 2.35it/s for the 2.0 model, usual is 1.11it/s for MBP 2021 16gb (python version)
spark sandal
Nice - almost 100% improvement for 1.5.
rigid vault
Nice - almost 100% improvement for 1.5.
more interesting, no RAM required
so no swap
spark sandal
Wild. Also APIs for swift to release on iOS devices.
spark sandal
import StableDiffusion
rigid vault
I tried maple fork on iphone Xs, it…works, techically (https://github.com/madebyollin/maple-diffusion)
but now it will be better
can't wait for the new fast 2.0 model
spark sandal
I've been out of the loop on 2.0 - this space moves so fast - something about only needing 4 steps?
rigid vault
I've been out of the loop on 2.0 - this space moves so fast - something about on...
yes, and Emad said <1 second to render because of all this
spark sandal
Insane.
Any place you'd point me to read about what distilled means in this context?
nm - found this: https://arxiv.org/abs/2202.00512
rigid vault
swift version, model 1.4 (a bit slower to render, but fast to start it at all)
2.0 model swift
blazing atlas
Will converted models work with InvokeAI?
rigid vault
Will converted models work with InvokeAI?
someday, yes. The team is already working on it (diffusers).
blazing atlas
someday, yes. The team is already working on it (diffusers).
So how to use it? With Maple Diffusion only?
rigid vault
So how to use it? With Maple Diffusion only?
right now? Go to their github, install it as is. Any model, but ddim (I think) only and 512px only.
celest obsidian
This is going to be a game changer
leaden lily
Really excited for this!
inland bison
Absolutely no swap, and no python at all.
What about resolutions?
rigid vault
What about resolutions?
Right now it’s like a proof of concept. 512 only.
inland bison
Right now it’s like a proof of concept. 512 only.
I just wonder what's the lower bound for x768 with this approach
rigid vault
Yeah, me too. I expect x2 for all.
undone owl
more interesting, no RAM required
OH EM GEE
Can't wait to get Invoke et al from "proof of concept" to release version 😅
Looks like there is a reason to upgrade to Ventura after all
barren gale
did you guys have to install the macos 13.1 beta for this?
rigid vault
did you guys have to install the macos 13.1 beta for this?
yes. Why not?
barren gale
yes. Why not?
was trying to avoid a restart for my box. no problem
sharp flax
I didn't see any flags to use the downloads you already have with python -m python_coreml_stable_diffusion.torch2coreml which is a little annoying for me and for huggingface.co.
rigid vault
was trying to avoid a restart for my box. no problem
you'll notice invokeAI speed boost too on 13.
barren gale
is this correct - split_einsum is slower than original attention implementation, even though einsum uses the ANE and original the GPU?
inland bison
you'll notice invokeAI speed boost too on 13.
For current versions? How so?
rigid vault
For current versions? How so?
maybe Metal3 update? I don't know, to be honest.
hazy lotus
https://huggingface.co/blog/diffusers-coreml handy link with pre-converted models
cobalt quiver
yes, and Emad said <1 second to render because of all this
If you check the thread, I believe the < 1s figure was a typo, corrected to < 18s
rigid vault
If you check the thread, I believe the < 1s figure was a typo, corrected to < 18...
For 4 steps? Hm.
inland bison
Boo, I can have 18 seconds already
inland bison
If you check the thread, I believe the < 1s figure was a typo, corrected to < 18...
Pretty sure it was 8s corrected to 18s
strong python
just reading about this now; can't wait to try in the morning! and yeah if this makes its way to InvokeAI i'll be thrilled, haha
strong python
on a m1 pro 16gb 16 core, split_einsum is WAY faster
Step 50 of 50 [mean: 1.67, median: 1.70, last 1.81] step/sec
when i use just cpuAndGPU on this setup it's much much slower, like 0.55 rather than 1.70
hm also it doesn't work, so something else is going on. disregard
inland bison
> Step 50 of 50 [mean: 1.67, median: 1.70, last 1.81] step/sec
V2?
strong python
yeah, that was 2.0 base
barren gale
M1 max 32gb ram 24 gpu cores
with cpuAndGPU, original attention and using swift
fastest i could get it to go
strong python
nice
barren gale
that's on sd2 btw
strong python
i can't seem to get the python command to work because it keeps assuming 1.4 though i have 2.0. but the swift one works fine
i wanted to use the python one because it has args for various schedulers and the swift one doesn't seem to
barren gale
i was able to use sd2 with the python launcher
theres a command to tell it to use sd2
strong python
i did see the model-version flag but i must be using it wrong
barren gale
try using the name used in huggingface. with the prefix too
--model-version stabilityai/stable-diffusion-2-base
strong python
ah. i was using apple/ - thanks!
bleh, it's yelling me about auth token. ah well, the swift one works
lean raven
I tried this command after downloading the converted 1.4 from hugging face
But on first run it downloaded about 5gb of files, and then when executed consumes 8g+ ram.
Any idea what I am doing wrong here ?
lean raven
I am going to try the swift command and see if it gets better
native creek
I tried this command after downloading the converted 1.4 from hugging face
```py...
Yep, that sounds like the same as it does on my mac, well after I'd updated to macOS 13.1, but the good thing is that there is no memory pressure while running the the sampling so a chunk of that 8Gb is swapped out at some point and stays swapped out.
lean raven
tried running the swift for the first time
swift run StableDiffusionSample "a photo of an astronaut riding a horse on mars" --resource-path models/coreml-stable-diffusion-v1-4_original_compiled --seed 93 --output-path /Users/jibril/stablediffusion/output
I got this error message
error: 'ml-stable-diffusion': Invalid manifest
<unknown>:0: remark: did not find a prebuilt standard library for target 'arm64-apple-macos' compatible with this Swift compiler; building it may take a few minutes, but it should only happen once for this combination of compiler and target
/Users/jibril/ml-stable-diffusion/Package.swift:4:8: error: no such module 'PackageDescription'
import PackageDescription
I have the command line tools installed, do I need to install xcode as well?
celest obsidian
Is there any news on integrating this with Invoke?
sharp flax
The instructions in this article work well to get you set up with a Gradio-based interface in the browser, so the model only needs to load once. https://blog.devgenius.io/a-step-by-step-guide-to-unlock-the-power-of-stable-diffusion-on-your-mac-51056584672c
The appearance of the UI should be familiar…
cobalt quiver
Another coreml diffusers frontend: https://github.com/TheMurusTeam/PromptToImage
This one also includes a coreml version of Real-ESRGAN for upscaling.
balmy obsidian
Any idea when this will be supported in Invoke? Are we talking weeks or months?