#SDNext WebUI on Intel ARC

1 messages · Page 5 of 1

proper cradle
#

try removing intel-extension-for-tensorflow

#

It should work without it

#

Also are you using Fedora?

#

Because Fedora uses an ancient build of Intel Compute Runtime that is not compatible with IPEX

green pagoda
#

A few days ago, so it should be fine.

proper cradle
proper cradle
green pagoda
#

ImportError to be exact.

proper cradle
#

did it also remove tensorflow?

#

install it with;

pip install tensorflow==2.13.0
keen marsh
#

Is this common? I get some distortion with some samplers although I honestly don't understand most of the new sampler settings in sd.next, seems like it's more for developers. For example DPM SDE with inpainting is just giving a mask atm, (just testing them out.) I think UniPC is working even with the error. Also, this could be a native windows thing.

Screenshot_20231020_120709.png
green pagoda
#

Let me nuke the install and try again.

green pagoda
#

Yeah, no dice, I guess I'll try Docker.

keen marsh
green pagoda
proper cradle
green pagoda
#

Docker just halts with ```ansi
fatal: destination path '.' already exists and is not an empty directory.
fatal: not in a git directory
/usr/bin/startup.sh: 4: ./webui.sh: not found

proper cradle
#

Directory you've passed to docker isn't empty

green pagoda
#

Which directory.

proper cradle
#

-v ~/docker-mount/sd-webui:/sd-webui \

#

~/docker-mount/sd-webui

green pagoda
green pagoda
#

I don't get it, it literally doesn't exist prior to running the command.

#

Removing that line makes it start to work.

green pagoda
proper cradle
onyx moth
#

Disty can you recommend some setting to use with hypertile? is 256x256 ok?

proper cradle
onyx moth
#

have you played with FreeU?

#

I haven't seen any difference using it

proper cradle
#

40 steps 512x768, ESRGAN 4X + Anime6B, 10 steps 1024x1536 hires, HyperTile 256, Diffusers:
Time: 11.76s

09759-3831344792-SD1.5-sotemix_v10_e354a0f87d.jpg
proper cradle
proper cradle
proper cradle
onyx moth
#

disty the issues I was having before with dpm ++2m karras seem to be gone mainly

#

not using SDE

onyx moth
#

Latent upscaling goes OOM

onyx moth
#

do the sd latent upscalers work with arc?

proper cradle
#

They work fine

#

They just use more VRAM compared to other upscalers

#

Nothing ARC specific

fickle plume
#

anyone tried out if the latest driver still breaks things on native windows?

chrome bone
#

isnt the latest 4900? if so, yes it doesnt work with ipex

fickle plume
#

welp, thanks for the info at least

keen marsh
#

I really hope they figure this out soon, with my luck the DX 11 fix will come and I can't update cause Ipex is broken lol.

onyx moth
proper cradle
#

Try decreasing Torch GC Threshold to %80 or less

#

Also what resolution?

onyx moth
#

I think I was trying to 2x 1024x1536
also, I have zero idea what Torch GC Threshold is im sorry

proper cradle
onyx moth
#

after setting it to 80 (it was 90 before)

onyx moth
#

seeing this 20:38:36-114778 WARNING upscaler is not loaded

ember orchid
#

All I'm thinking of starting up a regular live steam for GenAI. Weekly. For this to work, it would involve all of you. Thoughts are as follows
I host with a cast of regulars who are interested in being on the stage with their mic or camera
Flow would be I introduce a segment, where I have something to share / show on it, then I turn it over to you all to do the same
Segments would be: "AI News", "Feature or Technique", "Show-N-Tell"
For those who want to participate regularly, we'd discuss separate, who wants to show or discuss what on these topics ahead of the show

Two questions.
👍 If you like the idea
🖐️ if you'd like to participate, be a regular or semi regular participant

Separate link to thread on this topic
https://discord.com/channels/554824368740630529/1167519361918042112

proper cradle
#

LCM works with OpenVINO on SDNext dev branch

#

4 steps LCM_DreamShaper on my R7 5800X3D: Time: 8.86s
(DDR4 3200 MHz CL18 RAM)

01830-Diffusers-SimianLuo-LCM_Dreamshaper_v7_c255dc8f16-1699678274.jpg
#

0.64 it/s

image.png
proper cradle
keen marsh
#

Does it also work in ipex?

keen marsh
minor plank
#

Does the 3d Cache make a difference here?

proper cradle
proper cradle
#

Cache helps

minor plank
#

Raw Bandwidth I presume over latency

ember orchid
ember orchid
#

Yes

#

If you want to use IPEX on Windows you need to use older drivers. Not sure how far you need to go back but before 4885 for sure

keen marsh
#

I am on 4676, that was the last working version for native ipex.

#

Wsl might still work, although I did have issues with it on newer drivers but that could have been issues with the dev branch of sd.next

proper cradle
#

Use SDNext dev branch and just use the built in model downloader

#

Models / Huggingface

#

And download this: SimianLuo/LCM_Dreamshaper_v7

#

SDNext has native Diffusers backend

#

No need for a standalone app as an extension

ocean cipher
ember orchid
green pagoda
#

I use Fedora without issue.

restive parcel
#

fedora uses latest kernel usually, so it works well

green pagoda
#

I OOM on system memory when I try to use SDXL. Any recommendations to reduce system memory usage?

fickle plume
#

32gb should be enough? You can disable live previews

torn knot
green pagoda
fickle plume
#

I think you'll want to get more. Even for regular gaming
DDR4 RAM now is fairly cheap

keen marsh
#

sdxl needs more system ram, I'd recommend as much as you can afford tbh. I have 32gb now but wish I had 64 tbh.

#

Also if you have any of the medvram, lowvram features turn them off, and turn off the load model on cpu when not in use stuff.

#

although if you have an a750 medvram is needed

green pagoda
ember orchid
# proper cradle No

Cloned the dev repo to a new directory and installed.
Downloaded the model from the webui, then set backend to Diffusers. Restarted
But model is not loading. Any thoughts?

image.png
proper cradle
#

Activate the venv SDNext uses. (source venv/bin/activate in Linux. Don't know how to do it in Windows.)
pip install git+https://github.com/huggingface/diffusers
And start the webui with --experimental so it won't downgrade diffusers

ember orchid
#

I may need to wait till it merges. FYI in the dev branch I noticed Sample Steps missing from the UI. I couldn't figure how to get that to show

image.png
keen marsh
#

To activate venv in windows go to where the activate.bat file is and open it in the cmd prompt iirc

keen marsh
proper cradle
keen marsh
#

I will give it a shot then, I always get issues in Dev branch so I went back to master.

proper cradle
#

clean install is recommended

fickle plume
#

Is it normal that diffusers needs internet to work? 🤔

green pagoda
#

wat

fickle plume
#

Had an internet outage and sdxl would not work, trying to connect to something but it couldn't

green pagoda
#

Sounds like a bug

keen marsh
#

I think there was some sort of issue with sdxl needing to be online, but it was fixed I thought?

proper cradle
fickle plume
#

Mm, am I missing some convenient way to update?

proper cradle
#

--upgrade

paper horizon
#

good news. IPEX is working again with Driver 4952

grave condor
#

no reinstall required?

#

there go my excuses to not work on the project right now.

ember orchid
keen marsh
#

Finally can update my drivers!

keen marsh
restive parcel
#

oh sweet, I've been hanging back long enough on driver updates

proper cradle
#

RTX 4090 can get 24 FPS with TAESD VAE and batch size 24

#

A770 can get 5 FPS with TAESD VAE and batch size 5

proper cradle
proper cradle
proper cradle
proper cradle
fickle plume
#

Well, that is definitely very fast

#

Presumably above a batch size of 3 there isn't much improvement?

proper cradle
#

So i can't call it FPS

spiral junco
#

LCM+TAESD on A770 can get to 100 images in about 40s batch size=1

ember orchid
restive parcel
#

local 3d gen?

paper horizon
proper cradle
proper cradle
ember orchid
restive parcel
#

ahh, I'll wait on it then. It's only fun when I can run the stuff on my own hardware GGNodders

spiral junco
#

@proper cradle for LCM on SD.NEXT have you looked into setting same seed for image generation?

#

the older pipeline doesn't have generator built in the pipe so image generations are always random

spiral junco
#

tried to use newer pipelines with generator implemented, but ran into issues.

fast scaffold
#

Value error while installing deforum extension in auto1111
how to solve this?

#

Anyone know???

#

How to get over this?????

ember orchid
#

LCM is nuts. This 512x1536 grid took 1 second to create, at 8 steps.

00223-LCM_Dreamshaper_v7-close_up_male_face_rugged_looks_black_hair-grid.jpg
#

I'm liking 8 steps for LCM. 4 steps often results in cloudy results in eyes and details

fast scaffold
keen marsh
#

Never tried it, check the deforum GitHub

proper cradle
keen marsh
#

I believe he is using sd.next judging by the folder name, I think he just confused the two.

proper cradle
proper cradle
#

Dev branch merged to master

ember orchid
grave condor
#

according to the changelog, yes. and it's also been included in the newest diffusers release yesterday

ember orchid
proper cradle
#

git pull should be fine but it won't delete the old unused codes

#

New install should clean up a gigabyte of unnecessary space

proper cradle
proper cradle
#

SD 1.5 LoRas works fine with LCM

#

No Lora / LyCoris LoCon

00554-1576233277-Diffusers-SimianLuo-LCM_Dreamshaper_v720models-Dif.png 00553-1576233277-Diffusers-SimianLuo-LCM_Dreamshaper_v720models-Dif.png
#

No Lora / Standard LoRa

00577-2959740748-Diffusers-SimianLuo-LCM_Dreamshaper_v720models-Dif.png 00576-2959740748-Diffusers-SimianLuo-LCM_Dreamshaper_v720models-Dif.png
proper cradle
#

Overtrained character Lora i made is actually a decent Anime style lora

00601-2563027909-Diffusers-SimianLuo-LCM_Dreamshaper_v720models-Dif.png
#

No Lora

00603-2563027909-Diffusers-SimianLuo-LCM_Dreamshaper_v720models-Dif.png
proper cradle
#

LCM released the model conversion code

#

Converted my anime model to LCM

#

6 steps text2img + 3 steps 2x hires
Time: 3.71s

00962-3306754967-Diffusers-Disty0-lcm_sotemix_models-Dif.jpg
proper cradle
#

It's decent if i don't use chaotic prompts:

proper cradle
#

New LCM sampler added

LCM model + 4 steps LCM sampler

01670-2805530966-Diffusers-Disty0-lcm_sotemix_models-Dif.jpg
spiral junco
#

are you aware of this?

#

just came out today

#

4 step for SDXL, it's crazy

proper cradle
#

Coverted my SD 1.5 Anime model

spiral junco
#

woah.. this is game changer

proper cradle
#

Existing loras are still compatible

spiral junco
#

at this point we can technically turn any ldms to lcms with the lcm lora fused

#

1.5 and SDXL

grave condor
#

how fast would these run on CPU?

proper cradle
#

Converted models has the same speed

#

Lora method is slower

grave condor
#

when I find the time, I might look into turning my control net space into all the new and fast stuff.

proper cradle
#

Using TAESD VAE (uncheck full quality under advanced drop-down) will decrease the quality but it will cut down another 2-3 seconds

spiral junco
#

4 step, LCM scheduler , + SD 1.5 LDM model, + LCM 1.5 Lora

image.png
ember orchid
#

LCM to SDXL

00235-LCM_Dreamshaper_v7-close_up_male_face_rugged_handsome_face_flowing.jpg 00008-nightvisionXLPhotorealisticPortrait_v0743ReleaseBakedvae-rugged_handsome_male_eyes.jpg
#

1s for 512x512 image on left, then img2img at 1080x1080

proper cradle
#

LCM SDXL is fast but there is too much quality degradation

Time: 18.53s

02503-3418877994-Diffusers-local-lcm_infinianimexl_models-Dif.png
proper cradle
spiral junco
#

I think 6 steps is a sweet spot

#

and guidance_scale=1

#

also with LCM, 1024x1024 curse issue seems to be disappeared lol

proper cradle
ember orchid
#

My next video showing SDXL and LCM should publish next week

proper cradle
ember orchid
#

Very cool. My unpublished video is now already out of date, but cool

restive parcel
#

That does tend to happen quite rapidly in the AI space DinaKEK

green pagoda
ember orchid
#

Well, means just more video content😊

pastel geode
#

is there something wrong with using sdxl refiner because I started getting pixelated results just recently after upgrading it to the main branch. Using driver version 31.101.4887 btw.
The photo is without and with refiner respectively.

00007-sd_xl_base_1.0-warm_color_palette_fantasy_style_A_picture_of.jpg 00008-sd_xl_base_1.0-warm_color_palette_fantasy_style_A_picture_of.jpg
spiral junco
#

please use driver 4952

#

4887 has known issues with the AOT package

pastel geode
#

alright. Thanks for the tip :>

#

will i have to reinstall automatic or will it work just fine after upgrading my driver version?

spiral junco
#

it should just work

pastel geode
#

updated the driver but the results are the same. Any ideas?

spiral junco
#

what’s the vae set to

spiral junco
#

the corruption looks like related to vae decode or attention

chrome bone
#

just use cpu vae to see if its vae related

keen marsh
pastel geode
#

ok ill do those when i get to it

spiral junco
#

let's continue conversation here @proper cradle . my bad for sending the questions in the wrong channel

proper cradle
#

SDNext has it's own lora handler

#

I don't really know what is going on in there

spiral junco
#

have you tried using regular base model + lcm lora + lcm scheduler and generate correct outputs?

proper cradle
spiral junco
proper cradle
#

I don't know where is where anymore

spiral junco
#

haha

#

yeah project is huge

green pagoda
#

What else would you expect from Hugging Face

proper cradle
#

Diffusers method works fine

#

Something is borked with the built-in method

spiral junco
#

should be fixed now in dev branch 🙂

proper cradle
#

Also noticed that OpenVINO Lora loading is broken with Torch 2.1.0 (using openvino-nightly).

#

It Loads fine the first time and then it get's broken.

#

Caching is disabled

#

Using diffusers lora method

#

openvino-nightly==2023.3.0.dev20231114

#

@spiral junco does this happen on A1111 as well?

proper cradle
#

This is with unloading the entire model and loadig it again.

#

Downgraded PyTorch in SDNext

spiral junco
#

i’ll do a quick test

#

might need to unload lora and clear_cache() to gc the fused model that is already loaded in GPU

proper cradle
#

compiled_cache
partitioned_modules
Are tied to the model in SDNext with the last commit

#

They get removed with the model reload

#

Tied to CompiledModelState*

fickle plume
#

I wonder if the 1024^1024 issue might be something more concerning 🤔
Using the sdxl_vae, sdxl 1.0 and refiner, any prompt, any sampler, other settings default and this very specific seed 1279636787 it shows up in 920x1080, 928x1080, 936 and so on until 1000x1080 where it stops
Tested a very large chunk of those resolutions
What if this affects training or other things?

00040-sd_xl_base_1.png 00033-sd_xl_base_1.png 00026-sd_xl_base_1.png 00031-sd_xl_base_1.png
#

I have memory of encountering this same long horizontal line long ago when I was using A1111, with other lower resolutions around 800x600

#

I've been trying to get non-disfigured results on anime characters and changing the resolution off the typical 1080^2 helped a bit but then I randomly stumbled upon this

proper cradle
pastel geode
fickle plume
pastel geode
fickle plume
# pastel geode 4953

Show the advanced settings
You have face restore on which for something like this should be off anyways

#

Also, generate 1080x1080

#

512x512 seems to fry it, sdxl's mainly intended for around 1024x1024 anyways (though, see above, that exactly won't work with arc for now)

proper cradle
#

512x512 on SDXL is like 256x256 on SD 1.5, not good.

#

Other than the resolution being too low, what is your secondary pass settings?

pastel geode
#

base: 1.42it/s
refiner:4.64s/it
at 1080x1080

image.png
#

Heres the diffuser settings as well in case you need.

image.png
proper cradle
#

Increase the refiner steps

#

It's running for only 1 step rn

pastel geode
proper cradle
#

Refiner start 0.8 means start at the %80 of the refining process and do only the last %20

#

And %20 percent of 5 is 1

#

Matching steps to the base steps will be better

fickle plume
#

Trying it out now and the refiner settings seem broken
30 regular steps + 30 refiner steps at 0.8 refiner start results in 121 regular steps and 0 (?!) refiner steps and then an expected completely broken image
Neither 0 nor 1 refiner start actually has the refiner run afterwards

#

Setting to 0.5 has the correct number of regular steps, but only 7 refiner steps

#

Upping the refiner steps to 90 with 0.5 results in 22 refiner steps which is some very weird division

fickle plume
#

With 30 base steps, 0.67 refiner start -> 61 base steps, 0.75 -> 91, 0.8 -> 121, 0.9 -> 271
So, the resulting base steps are (1 / (1 - refiner start) - 1) * base steps + ~1
As expected, .99 results in 2971 base steps

image.png
fickle plume
#

As for the refiner... With 90 steps, 0.9 -> 0, 0.85 -> 1, 0.8 -> 3, 0.75 -> 5, 0.67 -> 9, 0.6 -> 14, 0.5 -> 22, 0.34 -> 38, 0.25 -> 50, 0.2 -> 57, 0.1 -> 72... I can't come up with a formula for this one
After some graphing calculator action, refiner steps seem to be roughly refiner steps * refiner start^2 - refiner steps * refiner start * 2 + refiner steps
Interesting math

fickle plume
#

However I'm unsure which of the 2 (30/5 with 0.8 -> 24/1 or 30/5 again) it should be

proper cradle
#

Forwardet this to him

#

He replaced my calculations and i don't know what he is trying to do

fickle plume
#

maybe sleepy math

#

I think the intent was to get the steps to remain the same regardless of refiner start

#

But I'm also very confused

keen marsh
#

I know sd next now just bypasses that res btw

proper cradle
#

SD 1.5 can do 768x1280 base resolution really well:

05657-577626628-Diffusers-Disty0-sotemix_models-Dif.jpg 05631-180487637-Diffusers-Disty0-sotemix_models-Dif.jpg 05617-2838116320-Diffusers-Disty0-sotemix_models-Dif.jpg
fickle plume
ember orchid
fickle plume
#

Generating faster is nice, but I'll be honest, I want higher quality and am willing to wait longer for it

#

Good to see that at least it seems everyone values controlnet/controlnet-like features a lot though as that has them too

proper cradle
#

Tried Animatediff

00014-2063030091.gif
onyx moth
#

oh wow, thats great

#

1.5 works with animatediff right?

#

might have to try that soon. I pretty much went straight to SDXL after setting up sd next

keen marsh
proper cradle
spiral junco
proper cradle
#

openvino_fx is now in OpenVINO package?

spiral junco
#

no it's not

#

it's only in webui, and sdnext?

proper cradle
#

Yes

#

SDNext uses openvino_fx from the official script

#

As a torch.compile backend

spiral junco
#

what's the method you used to unload lora?

proper cradle
#

Send the model to meta device, set it to none, load a new model, clear openvino caches, compile

#

Also new Diffuses could've broke it too

#

Dev branch PyTorch works but needs caching to be disabled completely

#

It wasn't doing this before

#

It caches 300 small files and produces garbage output if caching is enabled with a Lora

spiral junco
#

yeah caching has issues

#

testing with standalone scripts now and see if I can reproduce the issue

#

it seems to be working fine with standalone script. but lora loading and activating is a bit different. it needs cross_attention_kwargs={'scale': 1.0} to be passed as arg for pipe

#

used the lcm lora for testing, every other run is initialized with new DiffusionPipeline and load_lora_weights

image.png
proper cradle
#

Can you try running without a lora after loading a lora in the first run?

#

That was the issue with the mainstream PyTorch 2.1

#

Old nightly PyTorch 2.1 works fine but caching has to be disabled

spiral junco
proper cradle
#

Do not load a lora at all

spiral junco
#

ok, then load lora with the same pipe for the next iteration?

#

I'll try it

proper cradle
#

Load a lora in the first run, generate,
Unload / Reload the model and do not load a lora in the second run, generate

proper cradle
spiral junco
#

i'm on windows, not sure if that makes a difference

#

I'll try it on sdnext

proper cradle
#

I am on Linux

proper cradle
#

Manually update to the mainstream one

#

Also tied OpenVINO cache to shared.compiled_model_state on dev branch

spiral junco
#

dev branch seems to uninstall openvino even tho I launched with --experimental

#

20231102 nightly package is no longer available . the released 2023.2.0 is available now so we don't need the nightly package anymore

#

nvm.. I was actually on master branch

proper cradle
#

Dev branch should uninstall nightly one

spiral junco
#

forgot to checkout lol

#

🤣 cloned a new repo for test, forgot about I was on master branch

#

yeah I'm getting the same error

image.png
#

I think I found the root cause

#

installing the old torch-dev package and see if that's the case

proper cradle
#

But caching has to be disabled

spiral junco
#

yeah if we don't disable caching it will generate a lot of partitioned IR files

image.png
spiral junco
#

@proper cradle it looks like it's related to the lora loading method in sdnext

#

some ops not supported in openvino, causing it to split graph into partitions (this is why we see 100+ garbage files generated when cache enabled)

spiral junco
#

aten._assert_async.msg looks cuda specific

keen marsh
#

So whats up with this Microsoft olive stuff with arc? I just read 2x performance but compared to what exactly? Is this even used anywhere?

proper cradle
#

DirectML improved from basically unusable to somewhat usable.
DirectML is the worst case scenario.

grave condor
#

onnxruntime should be a better experience, but I believe the openvino execution provider doesn't support gpu yet.

onyx moth
fickle plume
#

hence why all the comparisons gotta be relative :^)

#

or I'd say something like 4-8x slower than ipex IIRC

proper cradle
proper cradle
#

I have no idea where this SymInt comes from

#

Only happens when we use a lora

#

And doesn't go away

proper cradle
#

Built-in lora method was hijacking the model even if we used the diffusers lora method

#

Disabled the hijacks and everything is working fine with stable Torch 2.1.1

proper cradle
#

SD WebUI running on a mid range Android phone 🙂

PXL_20231121_135355815.jpg
#

It is slow but it works
MTK Dimensity 1080 CPU + 8GB RAM

00000-LCM_SoteMix-masterpiece_best_quality_highres_1girl_solo_BREAK_flowers.jpg
#

Tried OpenVINO but it OOMd

fickle plume
#

mm, took a while huh?

image.png
proper cradle
#

Tooks 5-6 mins average

fickle plume
#

still nice

keen marsh
#

Wonder if openvino would work with 12gb ram phone? If so should be pretty fast

spiral junco
spiral junco
proper cradle
#

IPEX hijacks aren't applied for OpenVINO

#

I get garbled results if i apply attention hijacks with OpenVINO

#

Or use diffusers attention slicing

spiral junco
#

when we see multiple cache files generated it’s usually related to unsupported ops for torch compile openvino backend at the moment. if we can find where it is calling from then we can make a request to get it supported

fickle plume
#

Their examples look way better than the various other video generators

#

IDK how related this is to sdnext but I feel this is our main AI thread

chrome bone
#

thats interesting.. i just hope they stop stealing models and rename them. give credit where its due

fickle plume
#

the only stolen model i've heard of is the leaked novelai model, which I don't think stability have touched or claimed or renamed or whatever

chrome bone
#

there is nothing "stable" about stable diffusion. researchers call this LDM

keen marsh
chrome bone
#

https://www.youtube.com/watch?v=-1OfG6DaGF4

YouTube
Forbes
The AI Founder Taking Credit For Stable Diffusion’s Success Has A H...

Stability AI became a $1 billion company with the help of a viral AI text-to-image generator and — per interviews with more than 30 people — some misleading claims from founder Emad Mostaque.

In this video, reporter Kenrick Cai shares how a Forbes investigation was able to dive into these claims to separate fact from fiction, and how the compan...

▶ Play video
green pagoda
fickle plume
#

I have no idea who the founder of stable diffusion is, or if he (or the company)'s claiming the denoise-noise-until-image approach as his own
I'll watch the video

fickle plume
#

oh boy

fickle plume
green pagoda
#

I had just finished going through the end of an abusive working relationship at the time, and their lack of documentation on the arrangement they seemed to have when I reached out to fix it gave me cold feet.

#

By the time they tried reaching out to me again, I had discovered that I was being represented to Series A investors as an employee before I was even began my work for the contract.

#

Which was enough for me to not want to interact with them again.

fickle plume
#

I see, this is pretty enlightening

#

I feel as bad for you as I can in an online chatroom without doing too much about it

#

:(

#

I had expected a bit better from that company given the open source-ness surrounding it

green pagoda
green pagoda
green pagoda
#

The abusive relationship sent me into a spiral, destroying my semester and pushed back my graduation date another year (if I ever finish).

#

I had a pretty rough start to the year, lol

#

But I've co-founded a company and we'll soon be launching our first product, so things aren't looking so drab for me looking forward.

fickle plume
#

Well, good luck with your product, feel free to post the launch around here if it's related I guess

#

Unless that's too much self-promotion 🤔 idk

green pagoda
fickle plume
#

Well, good luck either way

keen marsh
broken grail
#

I've had an install that's sat for a while. Is there anything I need to do for updating other than git pulling/submodules & deleting config.json to get fresh updated defaults?

fickle plume
#

It's probably better to make a new install; and on that note, I recommend having a dedicated folder for the models outside sdnext's folder and using --models-dir, or at least not deleting the models folder and then putting it back inside

keen marsh
broken grail
#

got it

onyx moth
#

I'm just wondering if I'm doing it wrong but every reinstall I start with wsl --unregister ubuntu and then go through the steps

#

there's a better way isn't there?

chrome bone
#

lol why

#

you just need to remove SDNext

onyx moth
#

is it go cd, cd automatic, --reinstall? or ./webui.sh --reinstall?

#

or ./webui.sh --use-ipex --reinstall

fickle plume
#

Why are you reinstalling so often

keen marsh
#

I've never once did an --unregister ubuntu command, you can just reinstall normally. You can also just delete the folder and then do another git checkout if you want a fresh instal for the latest version. You can delete the folder in windows explorer like normal, or download the GUI nautilus in linux and use that if you prefer. https://learn.microsoft.com/en-us/windows/wsl/tutorials/gui-apps

Run Linux GUI apps with WSL

Learn how WSL support running Linux GUI apps.

onyx moth
#

When you say reinstall normally, that's what I thought was normal

keen marsh
proper cradle
#

New HDR center feature on dev branch is really good with BGs.

10688-2504742871-Diffusers-Disty0-sotemix_models-Dif.jpg
#

This is with SD 1.5. Haven't tried SDXL yet.

fickle plume
#

can you explain what it do

#

I kinda doubt it makes the resulting image into an exr

proper cradle
#

Corrects the latents before VAE

fickle plume
#

SDXL off vs on

00364-sd_xl_base_1.0-digital_illustration_of_a_dog_playing_basketball_indoors.jpg 00365-sd_xl_base_1.0-digital_illustration_of_a_dog_playing_basketball_indoors.jpg
#

It is... different? Maybe a bit better?

keen marsh
fickle plume
#

In general I'm kinda unsatisfied with how SDXL does art, let me try a photograph or something

keen marsh
#

Yeah its a bit static in my experience, even with the trained models. Photos are also a bit too static for me as well, it does adhere to prompt better but also lacks randomness so you can get the same face with the same prompt without a seed etc.

#

You have the add more prompt detail i think so less happy accidents

fickle plume
#

For me it's there being too much disfigured/incoherent stuff, like the background or the balls

#

It does seem to have a more sensible effect with this

#

idk what happened to my owl pics

fickle plume
#

owl pics again,
center
off
maximize

00371-sd_xl_base_1.0-photograph_of_an_owl_with_a_platinum_crown.jpg 00370-sd_xl_base_1.0-photograph_of_an_owl_with_a_platinum_crown.jpg 00369-sd_xl_base_1.0-photograph_of_an_owl_with_a_platinum_crown.jpg
#

it does progressively become more 'phone hdr'

#

though i wonder if that's what the acronym is supposed to mean in the ui

keen marsh
#

Anybody tried out the new video diffusion model?

restive parcel
#

what hardware does it even run on?

fickle plume
#

Their HF repo makes no mention of VRAM or any specific Nvidia GPU... So that is indeed a mystery

#

oh god

image.png
#

hmm, probably still doable on arc

image.png
ember orchid
#

Hey @proper cradle or @spiral junco is there an update to SDNext for LCM that uses the Lora method with any SD 1.5 or SDXL model, rather than the workaround for converting that model to LCM? I'm still on the work-around

proper cradle
ember orchid
proper cradle
#

Simple git pull should be enough

keen marsh
restive parcel
#

oh, might be manageable on arc at some point then

#

or does it already work with say ipex?

keen marsh
#

I believe it is working in comfyui but I am not 100%. I came to ask cause I don't know much about yet.

tall grove
fickle plume
#

why are they getting so fast suddenly???

#

Next, an additional test was completed with the same method for image quality. In these blind tests, SDXL Turbo was able to beat a 4-step configuration of LCM-XL with a single step, as well as beating a 50-step configuration of SDXL with only 4 steps.

#

This is nice, not even sacrificing quality for it

tall grove
#

Additionally, SDXL Turbo provides major improvements to inference speed. On an A100, SDXL Turbo generates a 512x512 image in 207ms (prompt encoding + a single denoising step + decoding, fp16), where 67ms are accounted for by a single UNet forward evaluation.

#

oh its 512x512

#

if its a lot faster and generates similar quality to sdxl at that resolution it might get more popular, idk if the 512x512 limitation can be messed with but it seems cool

#

sdxl slow adoption mostly cus its hard to run anyway

fickle plume
#

hmm, 512^2 is a bit sad

tall grove
#

it functions a bit like a GAN cool

keen marsh
#

Non commercial license though

tall grove
#

tbh thats fine for us

#

they have to make money i guess and if they can still release it then whatever

keen marsh
#

Its fine to play with if your just having fun with it, maybe they will release a real open source version at some point.

tall grove
#

they've released the weights havent they?

#

7gb model jeez

keen marsh
#

Non commercial

tall grove
#

well its still open source right

keen marsh
#

So just a toy to experiment with

#

Which is cool, not what I am looking for though

tall grove
#

they all were stated as for research purposes anyway

keen marsh
#

Nope, they aren't

#

Sdxl and sd1.5 are full commercial license

tall grove
#

huggingfaces pgae says that

keen marsh
#

Sd2.x as well I believe. Never used that one

tall grove
#

i guess its they're business model to do this

keen marsh
#

This is the first non commercial release afaik. Maybe the video haven't looked into it much yet

tall grove
#

idk how one would make money off releasing models without this style

keen marsh
#

They license to corporations supposedly

tall grove
#

hopefully more of the industry adopts this method of ai development instead of closed source stuff

#

apparently they have 2 more models to release idk what

keen marsh
#

Video is non commercial as well

tall grove
#

have to see how easy this model is to tune and whatnot

keen marsh
#

I hope people read the new licenses cause I would have assumed it was the same as the other models.

keen marsh
#

Are people doing anyting with the video models yet?

tall grove
#

havent seen much idk if you can even mess with that but i guess people are trying]

keen marsh
#

Its out so they probably can, just probably requires pro hardware to train

tall grove
#

probably wont attract much stuff as it seems like a hard to run model

keen marsh
#

People also license their models out too(they were anyway), so the non commercial thing might slow things down some too

ember orchid
# keen marsh Video is non commercial as well

Which part is non-commercial? Distribution of model is non-commercial or output ie images are non commercial?

Currently US says any AI output is not copyrightable so the output is currently free and fair use regardless

keen marsh
#

It just says personal non-commercial use so I assume its everything. Also you can still use fair use commercially. Its kinda ridiculous, but I understand the apprehension they have about copyright.

ember orchid
# keen marsh It just says personal non-commercial use so I assume its everything. Also you c...

Got it.

Per models themselves maybe the non Commercial use of Turbo SDXL means, you can use it but you cant put it into a tool and sell access to that tool. You can't (yet) make money from their model

Per AI output, I cant remember where I heard this, but it appears Google Search may have set precedence on some of this per Google search, scraping copywritten material, making it available for another use. This has gone to court multiple times and seems Google practice is considered fair use, regardless of source material.

My understanding what is not allowed is to take the output of a google search and try to protect that as your own thing. The output is math based, with limited user control, thus cannot be copywritten. So it could be Google has set the stage for everyone to freely use AI to produce whatever they want, but what they produce is also freely reusable. This is why I think AI generative images need to be more like the watercolor brush in Photoshop or procedural water shader in Unreal... as tool and process part of a larger canvas but not the process itself.

green pagoda
green pagoda
#

They are now starting to get scared of being sued, most likely because they know that they don't actually have a fully good-faith fair-use defense for most of their products.

#

Not to mention they want to commercialize it.

#

Pretty much “we will sue you if you use this in production”.

tall grove
#

tbh the idea of copyrighting ai models is a bit of a weird one

green pagoda
tall grove
#

tbh i see them more as tools, and mostly that the idea of AI models becoming a lawsuit hell seems like a rather unpleasant future

keen marsh
#

Might be why lamma leak didn't seem to spark any legal stuff afaik. Ai output is fairuse so you can sell it, you can copyright a collection for say a comic but you can't copyright the images themselves unless you prove it's mostly human, which is murky since you can use controlnet or image to image with sketches. To me teaching an ai with content available online is fair use, I fear that the debate may end up with corporations copyrighting an art style. We all learn from others and use copyrighted work to study art, I see it as the exact same thing. Its just ai is much much better. The real thing is in how its used by the artist imo.

#

I could just imagine getting sued for all the comic books i copied as a kid lol

green pagoda
keen marsh
#

I don't really know the specifics but companies usually hit the lawsuit button on any code leak

tall grove
#

they can probably do it if ur using commercially

#

but idk about anything else

keen marsh
#

They put out the license for free I think after the leak, iirc

tall grove
#

meta seems to be the most open out of the large companies rn

keen marsh
#

Yeah, surprisingly

#

I wonder what the legal area will be on models fully trained on ai output? Even if the original was trained on human output?

tall grove
#

tbh training is a weird concept to put any legal ramifications on

keen marsh
#

Thats what they are doing though lol

tall grove
#

has anything substantial been done yet i havent checked recently

#

with china existing its unlikely they will stifle it much

#

i do recall companies shamelessly lobbying for stuff like that tho

keen marsh
#

Afaik they are still having lawsuits and senate hearings etc. I do understand the fear, but I honestly cant see the legal claims tbh. Unless wr can copyright style, then we are all in trouble

tall grove
#

ye there exists lawsuits but i cant really see them working out

keen marsh
#

It all depends on the mood of the courts, I don't really have faith in the judicial system (in the US) atm

tall grove
#

depends on how the defense is done

keen marsh
#

Its kinda all opinion based

tall grove
#

but it can be quite easily spun to how it mimics human learning or whatever

keen marsh
#

They tried to sue ed Sheeran for using a comman chord progression, and it went to trial. (Not ai related exactly but this is what I am afraid of)

tall grove
#

they will have to copyright style to copyright ai models so its hard to see it happening

#

well idk i dont really have an opinion as its a hard thing to answer

keen marsh
#

The ones hurt the most will be artists it they do, and moat don't seem to realize this.

#

Yeah, but its all out of our hands. But this tech is the future regardless

#

I am still excited about it

tall grove
#

atleast in the UK they said they wont regulate this space for now lol

#

tho with AI models the fact that it can produce questionable stuff is also a whole other issue and it becomes complicated and blah

keen marsh
#

Yeah, I wish there was more self regulation in the community

fickle plume
#

The way I see it, AI will keep improving more and more, more people will lose more jobs regardless of how ethically it's trained or not
Any attempts at stifling it, especially with copyrighting style or whatever else will just mean whoever implemented those laws gets left behind. People already buy a lot of their products from China and no one cares too much about the laws there, and the Fairphone remains niche, while Nike remains popular. I also have to wonder if people can come up with good laws that apply to AI but don't specifically refer to it, like copyrighting style in general.
IMO, AI development should be subsidized and AI use allowed and taxed as heavily as reasonable, to the extent that people don't just give up on using/developing AI or are not too discouraged. UBI seems inevitable, something will have to fund it.

chrome bone
#

its just lesser evil. having expensive new drug is better than having no new drug at all. and patent expires anyway

green pagoda
chrome bone
#

maybe we can soon add emad amongst these ppl

fickle plume
#

What's that, an access token? Nice, I don't want to pay for Bloomberg anyways

keen marsh
#

I don't think he is anywhere near those people, but definitely has some 'questionable' actions. Say what you want though, stability ai seems to be the only real open source platform for images a sell could kill it

#

Don't want ai only in the hands of big corporations like they are trying to do

green pagoda
fickle plume
#

I'm already very annoyed at DALLE's censorship, I'd rather there be less closed source stuff

keen marsh
#

I don't mind some censorship, but its way overboard

fickle plume
#

Yeah... Nevermind other goodies like controlnet, training your own loras, or even just some basic negative prompts

keen marsh
#

Yeah, I also don't trust Microsoft

#

Probably all just tracking data to sell me ad's lol

tall grove
#

they are desparately trying to make it profitable so they'll do anything

keen marsh
#

Basically, I have no interest in SVD or Turbo anymore.

#

If I pay, I'd rather use midjourney, Gen 2 or something more capable off the bat.

tall grove
#

seems like a thing where if u make millions in profit from the model you need a license

keen marsh
#

For now

#

They also want to charge a monthly fee

ember orchid
keen marsh
#

I mean, I have nothing against that it's just don't advertise free and open source and talk about democratizing AI. It's basically just SAAS, all the talking they did was just smoke until they could make money off us. And I am also weary about any future changes.

#

When people talked about AI displacing jobs, I would say we had free and open sourced models that anybody could use. But now that it's getting down to the point where you really can run this stuff off a phone, the open source and free to use is getting canned lol.

#

I wonder if a community platform like Blender could ever make it in the space?

ember orchid
#

Blender is a rare beast

#

Its an amazing business model an it would seem, at least over the last number of years, leading tech companies like Epic, Intel, Microsoft, NVidia are happy to fund the Blender Foundation

keen marsh
#

Yeah, I wonder how Blender convinced everybody it was a good idea. Usually everybody just wants to make as much money as possible for themselves.

ember orchid
#

My understanding it was this perfect storm of - expensive 3D tools for part of the game pipeline (Maya) vs accessible tools for the other (Unity, Unreal) where Blender was a good enough tool during the app economy to fuel interest in game devs, tech artists etc to try it, use it in part of the pipeline. It grew the user base to a massive amount, where tech companies saw a way to be relevant with an audience that valued an accessible pipeline

fickle plume
#

Blender recently had a meaningful amount of donors pull out, so things aren't super smooth
Though that could be just a post-covid slump

ember orchid
#

I was just about to wonder that

#

AI is sucking every dollar right now

green pagoda
ember orchid
#

Since I started with Direct Modeling Ive always found it hard to go to CAD and BIM Models, but I do appreciate it. Revit is a nice tool. SketchUp seems to be a happy medium being Direct Modeling like but works with parametric CAD and BIM

grave condor
#

I got my start in Fusion 360 since Autodesk offered a really good free tier for students while I was doing my 3D printing stuff. but it's been degraded ever since.

Isn't there quite a bunch of stuff that's almost good but then really buggy? Like FreeCAD or OpenSCAD?

green pagoda
tall grove
#

seems u can merge sdxl models with sdxl turbo? Im trying out a new release that runs in sd next without any changes and it seems to generate 4 steps in https://civitai.com/models/215418/turbovisionxl-super-fast-xl-based-on-new-sdxl-turbo-3-5-step-quality-output-at-high-resolutions about 2 seconds depending on the sampler used on an a770. It naturally lets you do 1024*1024 images which is cool

TurboVisionXL - Super Fast XL based on new SDXL Turbo - 3 - 5 step ...

Wow guys, you have to try this for yourself! 😲😲 Super fast generations at "normal" XL resolutions with much better quality than the (freshly releas...

ember orchid
green pagoda
#

I have not, actually.

#

They do look pretty cool, but it obviously isn't good enough for doing production engineering with.

ember orchid
# green pagoda I have not, actually.

I was watching a video few weeks ago on this. Guy explains all the things you need to do to make Blender CAD friendly, from an addon that provides a more precise unit system, to BIM support, and operations that are more CAD friendly

green pagoda
fickle plume
#

I do doubt how well it can do CAD
Blender doesn't support faces with holes and doesn't work well with concave faces, and I really wonder how any addon can work around that
Though funnily enough, there's a commented out toggle for the data structures to support face holes... I wonder how many errors or crashes uncommenting it would cause

fickle plume
#

Huh, sampler and samples makes way more of a difference for SDXL than it used to for SD1.5

#

In the past it was basically unipc all the way and everything looked almost the same except unipc and 1-2 others looking a pinch better and unipc resolving with less samples

#

20 samples

xyz_grid-xyz_grid-0000-sd_xl_base_1.0-drawing_of_wann_holding_a_cup_of_coffee.jpg
#

Mm, finally small enough to embed, and also horizontal enough to not take up much space either

#

99 samples

xyz_grid-xyz_grid-0001-sd_xl_base_1.0-drawing_of_wann_holding_a_cup_of_coffee.jpg
proper cradle
#

I use Euler a or Euler most of the time

tall grove
#

SDXL turbo only seems to work with DPM SDE karras or whatever that one was, it produces quite high quality images for how long it takes so its probably likely that that model will become more popular to use

#

guess u'd need more steps on euler i havent checked that tho

sterile hearth
#

Just a quick question as I do not have time to debug today, but: I got sdnext and comfyui to run under ubuntu 23.04 , but both are prone to crashing and start to produce black outputs after a few generations. Is this the status i should get used to with sd on arc or do I have to sacrifice a few more hours getting it right? Ty!

keen marsh
sterile hearth
#

Ty for the heads-up, and the frequent crashing 🥺?

proper cradle
#

SDXL without FP16 Fixed VAE?

sterile hearth
#

No. 1.5 for now.

proper cradle
#

Black images shouldn't happen with SD 1.5

#

Try ypdating your packages and kernel

sterile hearth
#

Ok. I'll report back after. Prbly on the weekend.

sterile hearth
#

I did a fresh install of ubuntu 23.10.1, sudo apt update, sudo apt upgrade. Then I follwed Disty's guide on technopat. Everything went fine, but at the end I got this: "ImportError: libmkl_sycl.so.3: cannot open shared object file: No such file or directory" (I have the "Traceback (most recent call last) dump". but I'm not sure if I am not already cluttering the chat...)

fickle plume
#

2023.2 or 2024

#

look in /opt/intel/oneapi (each of the subfolders there should have folders named after every version you have installed)

#

and activate the venv in automatic's folder and do a pip list | grep torch and show what it says

sterile hearth
#

Ok. I got 2024.

fickle plume
#

do you know how to activate a venv?

sterile hearth
#

Yeah. Just takes a sec

fickle plume
#

outdated ipex

#

@proper cradle New basekit uses ipex 2.0.120+xpu (also new itex too)

proper cradle
#

I don't see any new ipex release?

fickle plume
#

They made a new page and stuff

#

#1162821590153699438 message

#

Yes, the url says 110

proper cradle
#

No Python 3.11

#

Pass

fickle plume
#

🤷 either old basekit or new ipex

#

otherwise it errors out

#

@sterile hearth If you want to use the old basekit, you uninstall the current one you have installed (e.g. with synaptic? for easy searching) and get the old one
wget https://registrationcenter-download.intel.com/akdlm/IRC_NAS/992857b9-624c-45de-9701-f6445d845359/l_BaseKit_p_2023.2.0.49397_offline.sh

fickle plume
sterile hearth
#

ok I'll see if I can manage that 😄 ty

proper cradle
# 
export TORCH_COMMAND="torch==2.0.1a0 torchvision==0.15.2a0 intel-extension-for-pytorch==2.0.120+xpu --extra-index-url https://pytorch-extension.intel.com/release-whl-aitools/"
#

This will fail if you use PyThon 3.11

#

Python 3.10 is already at security only support status

fickle plume
#

yes, do this instead

#

@sterile hearth

sterile hearth
#

ok. so I get python 3.10 in the venv and run the above command?

proper cradle
#

AnimateDiff on Diffusers backend with LCM
Time: 6.29s

00045-3443683961-Diffusers-Disty0-lcm_sotemix_models-Dif.gif
fickle plume
#

That's pretty strong flickering
Gif issue?

proper cradle
#

LCM does that too

fickle plume
#

Otherwise looks surprisingly coherent

proper cradle
keen marsh
#

Is it something to do with vae and/or seed variables?

#

I still haven't messed around with any of the new stuff, lcm or animated diff

tall grove
#

has anyone tested sdxl turbo on this type of thing?

#

i think they also released an sd 2.1 thing

onyx moth
#

Do turbo models work with sdnext right now? Is that only on dev branch for now?

keen marsh
#

Only see one post about on the github, probably just download and see. You will likely have to manually adjust the cfg and steps

tall grove
#

merges work

#

if u got to civitai there are some

#

atleast one i tried had worked

proper cradle
#

SD 1.5 at 768x1280 base + 2x hires to 1536x2560
No edits.

16841-3434697352-Diffusers-Disty0-sotemix_models-Dif.jpg
spiral junco
#

@proper cradle have you tried StableVideoDiffusionPipeline?

proper cradle
#

And attention slicing garbles the outputs

spiral junco
proper cradle
#

Battlemage should have 64 bit suupport, dropping 64 bit is really hurting Alchemist

onyx moth
#

So svd is a no go for a770 then.. what about animatediff?

spiral junco
#

got svd partially working

generated.gif
#

can't generate higher res videos tho.. had to play around with chunking

ember orchid
#

Woohoo

restive parcel
#

its crazy they didn't include 64 bit on arc, for some reason half my troubles are caused by it not being available, not just in ai annieSnap

fickle plume
#

I'm still of the opinion that fp64 emulation can get far, though I guess you can't reasonably emulate 64 bit pointers to get over 4GB... or is that actually possible with ok speed

#

Blender's Cycles is emulated 64, the performance seems ok

#

Sad how specific that is though, I wanted to try out Luxcore...

proper cradle
broken grail
#

wait, does the 770 actually have a 32 bit address bus? how'd they hell do they make that work

green pagoda
#

Saves on register sizes.

broken grail
#

ahh

green pagoda
#

Also would explain its inability to operate on 64-bit types.

#

It's actually a pretty decent idea if the card will be overwhelmingly used for graphics, since very large contiguous allocations are pretty unlikely in that case.

keen marsh
#

is there a good way to back up all my settings etc when doing a clean install of sdnext?

proper cradle
#

back up config.json and ui-config.json

keen marsh
#

Thanks, appreciate it.

coral mulch
#

So I got SD.Next running proper once again, however there's one thing that is confusing me.

#

What is the hands down fastest configuration I can currently use?

#

I really want to learn to use the LCM sampler, but it causes blurry images on the FP16 base model. I need to figure out how to use it.

proper cradle
#

Fastest:
Use IPEX with Diffusers backend
Use SD 1.5 with LCM Lora baked in
Use LCM Sampler with CFG Scale set to 0
Turn on HyperTile and turn off memory optimizations

coral mulch
#

Why SD 1.5 over something like SDXL Turbo?

proper cradle
#

SD 1.5 is 2x faster

coral mulch
#

SDXL turbo was designed (also) for 512x512 image generation.

#

I see.

proper cradle
#

Can someone thest if this release has AoT for Windows?

#

We have torchauido now

#

Interesing:

ImportError: libmkl_sycl.so.3: cannot open shared object file: No such file or directory
#

I already have OneAPI Basekit 2024.0

#

Welp, i activated the wrong venv

keen marsh
#

I may try it out today, still need to update the entire install for sd.next

proper cradle
#

Linux support is in SDNext dev branch

#

needed a few small fixes

#

1024 curse is still here

safe plume
#

How's the performance?

proper cradle
#

Startup times are really fast

#

ITEX needs to update for OneAPI 2024.0 too

safe plume
#

So how does that compare with openVINO backend?

proper cradle
#

IPEX and OpenVINO has the same generation speed

#

For startup, we are comparing a few seconds to a few minutes

safe plume
#

Few minutes for openVINO?

proper cradle
#

yes

safe plume
#

Hmmm, so IPEX good i guess

#

Kinda sad that the next version probably won't be out until like well into Q1 2024

grave condor
#

So the windows wheels got aot now?

#

I am slightly worried to update before seeing any other people with success or issues

proper cradle
grave condor
#

patch notes say Arc on Windows is now officially supported. Any claims Intel makes tomorrow during the "ai everywhere" show have some value I'd say

#

But the whole zero level and base toolkit mess is still horrible

keen marsh
grave condor
#

no idea, haven't tried it myself. And might not do so as I hope to have a new system in like a few weeks

fickle plume
#

I'm not risking my waifu generating setup

#

Maybe we can annoy the ipex team on their github if none of us want to check

keen marsh
#

Only issue I have is no access to older versions on windows of one api. I may check it out later today

proper cradle
#

2024.0

spiral junco
#

I repacked the ipex packages with OneAPI 2024 dlls, no need to install oneAPI toolkits

spiral junco
#

tested it on A770 🙂

proper cradle
spiral junco
#

no they don't

#

we can unpack the wheel files and repack them with those dlls tho

proper cradle
#

Welp, ipex doesn't see mkl if we install it with pip:

OSError: libmkl_intel_lp64.so.2: cannot open shared object file: No such file or directory
image.png
proper cradle
#

Does installing mkl with pip work on Windows?

#

pip install mkl==2024.0.0 mkl-dpcpp==2024.0.0

spiral junco
proper cradle
#

Is there a common file with Intel ARC GPUs?

#

detecting OneAPI won't work with this method

#

Current autodetect looks for sycl-ls or OneAPI root

proper cradle
#

And updated Windows wheels too

spiral junco
#

not sure about linux, but I packed these into ipex/bin and repack wheel

spiral junco
#

btw we can use torch.compile ov backend with the latest IPEX 2.1.10 packages

#

with cache disabled, gets nice perf gain too

green pagoda
#

I couldn't get torch.compile to work—it looked like it was only supported on Flex series?

spiral junco
green pagoda
#

ah, okay.

spiral junco
#

are you using SDNext --use-openvino with a fresh install?

#

it should work

green pagoda
#

No this was on other raw PyTorch code.

coral mulch
#

SD.Next with the latest IPEX is fine.

#

Very fine.

coral mulch
proper cradle
#

SD 1.5

20682-113564188-SD1.5-SoteLunar_bcfed58392.jpg
proper cradle
proper cradle
#

There are a lot of AMD GPU + Windows users using OpenVINO with SDNext.

keen marsh
proper cradle
#

Works on Intel, AMD and Nvidia

#

OpenVINO performance is on par with Olive / Shark WebUI on AMD

keen marsh
#

Oh wow, I will check the benchmarks on sd.next

#

multi-vendor support is kind of a big deal IMO.

#

still can't seem to find any data.

proper cradle
proper cradle
chrome bone
#

just curious if its possible to get MTL vpu and gpu working in tandem to increase inference speed lol

proper cradle
#

Probably possible with OpenVINO

proper cradle
proper cradle
#

Wait, i get this error with old IPEX too

#

something else is broken

#

MKL from pip was bad

#

Using the system OneAPI works fine

coral mulch
#

I have aliases set up in my WSL environment now

#

so I can type two things in and instantly get sdnext running

#

Same with ComfyUI

fickle plume
#

two?

#

why not one

coral mulch
#

One for ComfyUI and one for SD.next launching (so actually three)

#

Also, I suck at code.

fickle plume
#

i dunno about comfyui but sdnext has its own venv

coral mulch
#

my aliases are literally just a bunch of &&

#

Lol

# 
alias comfyui="cd ~ && cd automatic && cd venv && cd bin && source ./activate && cd ~ && cd ComfyUI && python main.py --lowvram --use-pytorch-cross-attention"
alias sdnext="cd ~ && cd automatic && ./webui.sh"```
#

I could've just set the proper directory LOL

#

But I made a bunch of &&s to go to that directory, which is incredibly inefficient

#

What I should've done is this

# 
alias comfyui="cd ~/automatic/venv/bin && source ./activate && cd ~/ComfyUI && python main.py --lowvram --use-pytorch-cross-attention"
alias sdnext="cd ~/automatic && ./webui.sh"
#

I added these to my bashrc basically

fickle plume
# coral mulch ```alias oneapi="source /opt/intel/oneapi/setvars.sh && conda activate IPEX" ali...

&& will run the thing after if the thing before ran successfully, it's an AND operator, while it's not wrong I'd be using ;
chaining 5 cds to go down folder by folder is pointless, just do a single source ~/automatic/venv/bin/activate or whatever
activating a conda environment is completely pointless in your case as sdnext activates its own venv, and you activate a venv for comfyui, the conda environment will not be used

#

sdnext also activates the oneapi environment for you on linux

#

all you need for sdnext is just a single
alias sdnext="~/automatic/webui.sh --use-ipex"

#

and for comfyui, you probably need just
alias comfyui="source /opt/intel/oneapi/setvats.sh; source ~/automatic/venv/bin/activate; python ~/ComfyUI/main.py --lowvram --use-pytorch-cross-attention"

chrome bone
#

why is comfyui using sdnext venv though o.o

fickle plume
#

I'd also like to hear an answer to that

proper cradle
#

It will fail if you are launching it with python launch.py

#

Trained SD 1.5 at 1024x1536 DoggoHype

21275-3093070649-SD1.5-SoteMixV2_cf585bf6df.png
keen marsh
# coral mulch so I can type two things in and instantly get sdnext running

I just created a .bat file and then made a shortcut for my desktop, so I just double click and everything is up and running. Sometimes I have to go in and edit to reinstall or something though. Did the same thing in native linux, was a more involved process though especially since I had never used linux before.

keen marsh
proper cradle
#

Windows should work fine

keen marsh
#

But still need to setup and install the oneapi environment right? in windows I mean

proper cradle
#

no

#

SDNext will install MKL and DPCPP

#

That should only take 0.5-1 GB instead of 20GB

keen marsh
#

awesome, thanks.

minor plank
#

@proper cradle Please stop posting these pictures, she's too pretty. I can't take it 🤖

fickle plume
#

he wants more giraffe pics

#

post the giraffe pics!

#

3 torsos!

broken grail
#

Fresh install. Any settings to change from defaults?

fickle plume
#

set the live preview display period to -1

#

that disables it

#

minor boost in speed and less vram

broken grail
#

generic sdnext question: How do I load model settings from .yaml files, or check that they're being loaded? If anyone knows where the model yaml config is Is like to take a look at it

proper cradle
#

Use Diffusers models

#

And model config in the repo will be used

#

Otherwise, it will be the default SD config file

proper cradle
#

Fixed Stable Video Diffusion DoggoHype

#

Apperantly i was slicing the wrong thing with 4gb workarounds

#

Has no idea about anime but it works

00224-3509243765-Diffusers-stabilityai-stable-video-diffusion-img2vid_models-Dif.gif
ember orchid
proper cradle
#

IPEX 2.1 doesn't lock up after a OOM error anymore DoggoHype

#

SVD is really VRAM hungry :/

proper cradle
#

1024x1024 curse %95 fixed with SD 1.5

#

But still here with SDXL

#

Still here if you look close enough but not nearly as bad as before (SD 1.5)

22328-1358241330-Diffusers-Disty0-SoteMixV220models-Dif.png
keen marsh
proper cradle
broken grail
# proper cradle Use Diffusers models

so if I have a safetensors, I need to convert it and its configuration to a diffusers model?
is there a way I can slide a line of code in somewhere to load a scheduler config instead?

broken grail
#

ah alright, I dug around and found the code

#

it looks like tbe original backend can load configs, but default can't

proper cradle
#

I cahanged mine from SD 1.5's default PNDM to Euler a

broken grail
#

is there a way to do that without reupload on hf

proper cradle
#

Edit the model you've downloaded

#

or converted locally

#

Diffusers models are in models/Diffusers

broken grail
#

couldn't figure out how to convert it locally so set up some hacky code and got config loading working, worth pring?

proper cradle
#

OpenVINO on CPU under 8GB DoggoHype

image.png
#

PyTorch model was sitting in the system RAM doing nothing

#

Deleting it doesn't work so i downcasted them from fp32 to fp8

proper cradle
#

ARM CPU still needs 16 GB

chrome bone
#

i remember reading somewhere pytorch doesnt support cpu fp16 so it is inherently more memory intensive

green pagoda
#

Most hardware can't do either binary16 or bfloat16

proper cradle
#

But my CPU supports bf16 so i didn't really see much improvenment over bf16

proper cradle
safe plume
#

I guess we have to wait for openVINO 2024 or IPEX 2.2+XPU

grave condor
safe plume
proper cradle
#

INT8 is supported with IPEX on dev branch DoggoHype

#

But it's a autocast

#

Expect VRAM savings but it will be slower to run.

#

last 2 options

image.png
#

OpenVINO uses the Compress Model option too but it's CPU only on OpenVINO

#

600 MB savings with SD 1.5 and 2 GB savings with SDXL

safe plume
grave condor
#

hm? The PR or the progress

safe plume
#

Jumped quite a bit in a couple of days

#

Was at 31% when you posted it originally

proper cradle
#

dev branch merged to master

onyx moth
#

If I were to --unregister ubuntu basically do a super clean install that would get me the latest versions of things like torch and ipex right? Yes I am basically a caveman with sticks, rocks, and a computer.

proper cradle
#

just start the webui.sh with DISABLE_IPEXRUN=1 ./webui.sh --reinstall

keen marsh
proper cradle
#

These are the IPEX and OpenVINO changes:

image.png
#

IPEX 2.1 fixed torch.linalg_solve so performance in original backend isn't terrible anymore

#

Diffusers is still faster tho

keen marsh
#

1024 curse was fixed?

#

I see "remove 1024 workaround" 🤔

proper cradle
#

OpenVINO is being used by AMD users more than ARC users.

#

And CPU too

#

1024 curse is still here

keen marsh
#

ah okay

safe plume
keen marsh
#

Lol Its likely because we have ipex and rocm only works in native linux, we have wsl and native windows. Openvino requires more system resources than ipex

chrome bone
#

who needs openvino when you have ipex

#

though i can think of a time you will want to use it - multiple device support

keen marsh
#

Openvino is easier to setup, so there is that.

proper cradle
#

OpenVINO pretty much "just works" out of the box

grave condor
#

as a torch Dynamo backend?

#

or do you still have to use the OVIR model and convert them yourself? I stopped using ov because my ram and storage weren't happy with it

ember orchid
#

OpenVINO is faster and supports more hardware. The issue is added compile time at first run or change in sampler or resolution.
Also some features may not be supported. HiRes Fix, LCM LoRa were issue in the past. IPEX tends to be work on emerging features sooner

proper cradle
#

Memory usage is much better after compile now

#

Also rewrote my IPEX library to ged rid of CondFunc in dev branch

proper cradle
grave condor
#

I have to see if it's possible to use openvino through accelerate, as I believe there is dynamo backend support

proper cradle
#

Only thing that's a dealbreaker with OpenVINO for me is no HyperTile support

#

HyperTile doesn't play well with compile in general

#

But HyperTile is too OP with SD 1.5

proper cradle
ember orchid
#

For those interested in trying Fooocus, I added instructions to this thread on the IPEX fork of the repo. It's a streamlined simpler webui, supporting SDXL and LCM fast LoRa. The outpainting feature is very nice and intuitive. Advanced features allow for a lot of customization and configurations to step outside the defaults. Props to @spiral junco for the heavy lifting
#1175245000322322483 message

upper pine
proper cradle
#

only intel-compute-runtime / level-zero-gpu is needed

upper pine
#

aight. Could that be the reason after a couple of generations sdnext would crash though?

proper cradle
#

Try disabling ipexun

#

Linux:
DISABLE_IPEXRUN=1 ./webui.sh

#

Windows doesn't use ipexrun

upper pine
#

after generating one image, this happens

image.png
#

that's why I am mentioning. I am figuring this is a skill issue on my end, I just wanna make sure I didn't miss anything

proper cradle
upper pine
#

Gotta do some stuff, but if you need anything from me to situate this situation, do let me know. Once again, thank you for all you do

steady gate
#

Hello I have installed SDNext in a virtual Ubuntu environment on my computer and it is running but when I try to generate an image (SD1.5 model) the shell crashed out right after what looks like the model loading. I could use some help troubleshooting.

tall grove
#

Jim if you are in windows please just run it natively. Just get sdnext and then specify ipex on run. Unless there's a reason you need openvino ipex is normally good to use

steady gate
tall grove
#

Sdxl should work, maybe try diffusers backend but it's been a while since I've used this

steady gate
fickle plume
#

Read the guide

#

Well, this guide specifically

steady gate
fickle plume
steady gate
#

VICTORY!

spiral junco
#

happy new year everyone!

proper cradle
proper cradle
#

Found a way to delete unused weights without actually deleting them with OpenVINO DoggoHype

image.png
proper cradle
proper cradle
#

OpenVINO is faster than IPEX now.

LCM 4 steps: Time: 0.40s

03095-516425538-Diffusers-Disty0-LCM_SoteMix20-home-dist.png
#

Under 10 seconds on CPU DoggoHype

Time: 9.99s

03104-3348996537-Diffusers-Disty0-LCM_SoteMix20-home-dist.png
upper pine
#

if I try using --use-openvino, it will fail to load a model. A constant line I see coming up is NameError: name 'get_raw_openvino_device' is not defined

proper cradle
upper pine
#

I figured, no clue why I tried that

#

but the error screenshot provided is what happens after a few images are generated in sdnext

proper cradle
#

Is it working fine without ipexrun?

upper pine
#

same thing happens, but doesn't throw any specifics other than segmentation fault (core dumped)

#

this is after things are loaded

proper cradle
upper pine
#

exactly the same issue I am having

#

because this seems to be a recurring issue with other users, should I post anything that may help with development or should I just wait for the time being?

proper cradle
# 

RUN apt-get update && \
    apt-get install -y --no-install-recommends --fix-missing \
    ca-certificates \
    wget \
    gpg \
    git

RUN wget -qO - https://repositories.intel.com/gpu/intel-graphics.key | \
    gpg --dearmor --output /usr/share/keyrings/intel-graphics.gpg
RUN echo "deb [arch=amd64,i386 signed-by=/usr/share/keyrings/intel-graphics.gpg] https://repositories.intel.com/gpu/ubuntu jammy client" | \
    tee /etc/apt/sources.list.d/intel-gpu-jammy.list
RUN apt-get update

RUN apt-get install -y --no-install-recommends --fix-missing \
    intel-opencl-icd=23.35.27191.42-775~22.04 \
    intel-level-zero-gpu=1.3.27191.42-775~22.04 \
    level-zero=1.14.0-744~22.04

RUN apt-get update && apt-get install -y --no-install-recommends --fix-missing \
    python3.10 \
    python3-pip \
    python3-venv

RUN pip --no-cache-dir install --upgrade \
    pip \
    setuptools

RUN apt-get install -y --no-install-recommends --fix-missing \
    libgl1 \
    libglib2.0-0 \
    libgomp1 \
    libjemalloc-dev

RUN apt-get clean && rm -rf /var/lib/apt/lists/*

RUN echo '#!/bin/bash\ngit status || git clone https://github.com/vladmandic/automatic.git .\n./webui.sh "$@"' | tee /bin/startup.sh
RUN chmod 755 /bin/startup.sh

VOLUME [ "/deps" ]
VOLUME [ "/sdnext" ]
VOLUME [ "/root/.cache/huggingface" ]

ENV venv_dir=/python/venv
WORKDIR /sdnext
 
ENTRYPOINT [ "startup.sh", "-f", "--use-ipex", "--listen" ]```
#

Can you try this Dockerfile?

upper pine
#

I am on Arch Linux

proper cradle
#

Copy the texts, save it to a file named Dockerfile

#

Build the Dockerfile
docker build -t sdnext-ipex -f Dockerfile .

#

Run it
docker run -it --device /dev/dri -v ~/sdnext:/sdnext -v sdnext-venv:/python -v huggingface:/root/.cache/huggingface -p 7860:7860 --name sdnext sdnext-ipex

#

This should eliminate dependency issues

keen marsh
upper pine
tall grove
ember orchid
#

For clarity on options you can use

  • SDNext Win Native or Linux/WSL with IPEX or OpenVIN0
  • Fooocus Win Native IPEX
  • ComfyUI Win Native IPEX or Linux IPEX
  • A1111 Win Native or Linux/WSL OPEX or OpenVINO
#

You can also run Stable Diffusion directly inside of

  • GIMP via OpenVINO plugin
  • Blender via OpenVINO add-on
upper pine
#

I primary linux so I need to get stuff straightened out for Windows

tall grove
#

i never had much issues, I just gfot the sdnext thing, ran it with use ipex, and it seemed to work fine after that

#

though only diffusers run at full speed

proper cradle
#

Original / A1111 backend is CPU bottlenecked, that's why it has even worse performance on Windows

#

SDNext fixed some of those bottlenecks on it's A1111 backend but A1111 backend is still horrible in general

#

Diffusers is bottlenecked to around 9-11 it/s

#

SDNext with Original / A1111 backend is bottlenecked to around 7-8 it/s

#

A1111 should be bottlenecked to around 5-6 it/s but current implementation gives 3-5 it/s for some reson.

upper pine
keen marsh
upper pine
proper cradle
upper pine
#

(disclosure:I am using Nuullll's IPEX wheels because official wheels try to call [and fail to find] IPEX DLLs)

proper cradle
#

I still can't reproduce them.

upper pine
proper cradle
#

probably

#

I wasn't able to reproduce them on docker or native linux

upper pine
proper cradle
proper cradle
#

Benchmark for A770

image.png
spiral junco
proper cradle
#

Filling this now

image.png
#

OpenVINO*

#

OpenVINO will easily surpass IPEX in terms of speed if HyperTile worked with compile

safe plume
#

So anyone know where's Pytorch 2.2?

#

It was supposed to be released today

spiral junco
#

I'm mostly getting around 6 it/s

#

I think IPEX 2.1 has torch.compile IPEX backend as well

proper cradle
#

Triton release lacks support for XPU backend

#

I tried the Intel release but that also lacks support for XPU for some reason

spiral junco
#

let me check with them

green pagoda
safe plume
#

torch.compile is WIP ATM

green pagoda
#

afaict, torch.compile isn't ready

spiral junco
spiral junco
proper cradle
#

A1111 is CPU bottlenecked

spiral junco
#

hm... CPU utilization is like 20%

proper cradle
#

Single core

proper cradle
ember orchid
# spiral junco

Evertime I see the Intel Core Ultra badge, in my head I hear the Intel bong and an animation that flips it around and says "Intel Arc Inside"😁

spiral junco
#

😂 lol

proper cradle
#

Trying Intel GPU Max 1100 in Intel Devcloud;
4GB limit issues and 1024x1024 curse doesn't exist with GPU MAX

#

A770 / MAX 1100

03415-4183023294.png 00022-animagineXLV3_v30-masterpiece20best20quality20highres201girl20solo20scenery20flowers.png
#

Also noticed that SDNext is significantly faster than Intel's reference notebook

fickle plume
#

may the discord cdn rest in peace

proper cradle
#

Seems like my attention slicer helps with VRAM usage really well.
Quickly ran out of VRAM without it

image.png
proper cradle
#

Added IPEX_FORCE_ATTENTION_SLICE env variable

#

Had to use very low HyperTile sizes to not to run out of VRAM with 48GB VRAM

proper cradle
#

SDNext should be compatible with GPU MAX series out of the box in dev branch now

#

Attention slicing has heavy hit to the performance but memory usage is way way better

#

Attention Slice: GPU active 11388 MB

No Attention Slice: OOM with 49152 MB VRAM

proper cradle
#

Setting slice rate to 36-42 GB prevents OOMs while not sacrificing much performance

proper cradle
grave condor