✨｜sdxl

[uneven dove]

no, they're the same core architecture. the process node is no smaller. more CUDA cores, then, equals, more power.

[eternal fog]

Yes absolutely

[high skiff]

ok, I am not gonna argue this with you, that is not at all how that works lol

[uneven dove]

the only way to pull less power with more CUDA cores is for them to be literally closer together

[eternal fog]

8gb is struggling to run SDXL 12gb is not

[regal heath]

i wonder why tf nvidia made the ti with 8gb instead of the opossite

[eternal fog]

Fuck you, that's why

[regal heath]

which resolution usually is 12gb able to generate? i've seen people making up to 1024 with 8gb

[high skiff]

so then you think 1k cuda cores at 1GHz and 1k cuda cores at 2GHz use the same power?

[eternal fog]

I mean I can do 4k with 10gb.

If you are wanting to do any training like making LoRAs you'll struggle on 8gb.

[high skiff]

you force in more power into the GPU in order to overclock it and get more performance out of it

My 3080 is severely overclocked at start, meaning it draws a lot more power for a little more perf over a stock varient

[regal heath]

straight up 4k or using hirefix?

[elfin cobalt]

Doesn't actually matter.

[high skiff]

also, the 3090 chips are better manufactured, the 3080 gets the cut down failed chips of the 3090

Which means the 3090 has better silicon, and in general needs less core voltage to sustain clock speeds

[eternal fog]

If you can high Res fix you can do it normally as well. Just takes longer.

[uneven dove]

lowering the voltage = overclocking

😛

[visual glade]

I think the formula for power vs clock is a cube or something close

[high skiff]

You're joking, right?

[high skiff]

yes

you need way more power to get a little more clock speed

[eternal fog]

The 3080 is pretty much maxed out

[high skiff]

exactly

[eternal fog]

I'm limited by the power limit on my FE card

[elfin cobalt]

It's cube if you assume you need higher voltage to get higher frequencies. Which has been true historically, but..

[high skiff]

I thin pseudo is confused here is all

[visual glade]

higher frequency needs higher voltage but higher temperature also needs higher voltage

[high skiff]

yeah

[visual glade]

that's because the silicon needs more "push" to update itself faster

[high skiff]

and the better the integrity of the silicon chip, the less voltage it needs to sustain a specific clock

[elfin cobalt]

Drop down enough levels, it's due to the capacitance of the transistors.

[high skiff]

which is known as its efficiency

you can undervolt a chip to save power while not losing performance

[elfin cobalt]

Switching them faster means you need to move electrons faster, so you need higher current, so you need higher voltage to push the higher current.

But higher voltage means the capacitance will increase...

[visual glade]

that's only because they leave a margin of error

[high skiff]

for example, this GPU uses 1.2v to run at 1850mhz stock

but I use just 0.89v to run it at 1800mhz, and it uses way less power

[visual glade]

because as chips age they also need higher voltage

[elfin cobalt]

...so you need even higher voltage to compensate for the effect of higher voltage.

[high skiff]

yeah, higher impedence starts to happen

[eternal fog]

You can even gain performance doing that, because it also lowers power consumption which gives the boost more power headroom to clock higher.

[high skiff]

more heat = more resistance = more power needed = more heat

[boreal bough]

#🌶｜off-topic or #💬｜general-chat before a mod shows up again ^^'

[high skiff]

yup!

[elfin cobalt]

I never managed to wrap my head around impedance. Thinking in terms of subatomic particles works fine for me. 😛

[high skiff]

this is very much on topic, its about GPU clocks for SD lol

[boreal bough]

'sdXL'

[uneven dove]

that's why i said undervolting is an overclock, and he said, "are you joking?" but i guess when you say it

[high skiff]

but yeah, anyways, my current 3080 uses more power than the 3090 I am looking to get, and the 3090 is faster

[uneven dove]

that's workload-dependent, anyway

[high skiff]

undervolting is not an overclock, an overclock is an overclock

undervolting can help you overclock lol

[elfin cobalt]

Undervolting isn't overclocking. It just has most of the same implications.

[uneven dove]

undervolting is an overclock, ask AMD or NVIDIA support

[high skiff]

its not ._.

[uneven dove]

it helps you attain higher clocks than stock

[elfin cobalt]

You're literally not changing the clock. 😄

[uneven dove]

especially on something like the Vega56

[elfin cobalt]

It isn't the opposite either.

It's just orthogonal.

[high skiff]

they are just different

and it should be leveraged by a lot of people running SD as well

you can save a lot of power and lose no perf by undervolting

[uneven dove]

it's hard to know whether you'll freeze your whole system by undervolting, or else the mfg would do it for you

[high skiff]

but anyways, this all got derailed

the 3080 I have currently is both bigger and uses more power than the 3090 I am going to buy, and its slower

which makes sense why this 3080 LOVES to strangle my pathetic little 650 watt PSU lol

[urban fjord]

Is there anything other than ComfyUI that supports SDXL LoRA atm and runs well? The VAE decoding is much slower for me with LoRA which is kind of a problem when testing out LoRA training.

[lusty raptor]

i keep meaning to write a proper bug report, but with a primitive wired to a CLIPTextEncodeSDXL node, even if they're not attached to anything else- deleting either or removing the connection breaks the graph; it won't run at all

[elfin cobalt]

Idly notes that XL is ~4x faster than 1.5 at 2048x2048

...though the MPS library still crashes in VAE decode.

[high skiff]

I noticed that as well

[urban fjord]

I don't know if I've tried 2048x2048 in SDXL, but I cannot even run it in 1.5

[elfin cobalt]

I imagine it's because of tiling. It's just doing 4 tiles instead of 16.

Or however that works.

[high skiff]

SDXL slows down less and uses less VRAM than 1,5 and 2.1 when increasing resolutions

meaning, it requires less addtional

[elfin cobalt]

Makes it more practical for me to work on my bot on my laptop.

Although honestly I should just stub out ComfyUI.

[urban fjord]

Though personally I don't care too much about 2048x2048, but I'm sure there are use-cases where it matters.

[elfin cobalt]

Also, I just had ComfyUI suck down 60GB of VRAM. XD

[uneven dove]

Mac doesn't have certain precision levels:

The neural engine can't be used for training anyway. It only supports Float16, Int8, and UInt8, and is only accessible through CoreML and MLCompute. PyTorch uses neither of these APIs for training.

bf16 is a no-go

[elfin cobalt]

"Error: NDArray dimension length > INT_MAX"

[uneven dove]

wonder if you're hitting that, lmao

[fresh path]

I hate pytorch (amd user)

[elfin cobalt]

So 4GiB I guess

[lilac wren]

Me and my 2070 Super, after being told this sad news

[elfin cobalt]

Oh, I'm just trying inference.

[uneven dove]

@fresh path sorry mate. what platform did better? JAX? Flax? ONNX? DirectML? TensorFlow? Some other garbage no one uses?

[high skiff]

hey, this looks like the first image I made in SD 1.0 back in the yonder years!

[elfin cobalt]

But honestly I don't need full-res pictures from this.

[high skiff]

my first image in SD 1.0 lol

✨ WETNESS ✨

[fresh path]

I think most including comfyui uses directml

im not sure what the perf is meant to be

but it was alright

[uneven dove]

DirectML occurs via PyTorch.

[lilac wren]

I'm disappointed, I did a big sorting of my 16k images, but I still have the 11th.

[autumn forum]

If it’s the opposite how did I undervolt and overclock my cpu at the same time😂

[elfin cobalt]

It isn't. We've been over that. 😛

[uneven dove]

people are opinionated on this stuff and most don't have a background in electrical engineering 😄

[uneven dove]

that's a software-specific feature

the core concept of undervolting, will inherently overclock a modern GPU that uses thermal headroom to decide its max clocks

[warm hazel]

1.0 isn't scheduled until next week right?

[uneven dove]

the same reason cooling off a 7800X3D makes it go way faster..

[high skiff]

oh well yeah, if your GPU was thermally throttling, then it will

[uneven dove]

thermal throttle by design at stock settings, yes

[high skiff]

yeah, most well designed products don't do that 😅

[uneven dove]

whut? of course they do

[high skiff]

no

[uneven dove]

maximum strength tylenol was designed so that it killed half of the people that took it. and then, they backed the dose off a tad

[high skiff]

GPU's sit wellll below their throttle temps

[elfin cobalt]

Throttling isn't a firewall anymore

[uneven dove]

it's just a modern approach, man. if the GPU can go faster, it will

[elfin cobalt]

It's more of a gentle curve..

[fresh path]

what melts at 100 degrees thats so bad anyway?

[elfin cobalt]

The solder.

[uneven dove]

lmfao

Indium

[high skiff]

yeah lol

[uneven dove]

100 degrees C is going to increase the risk of electromigration as well

[fresh path]

true

[uneven dove]

electrons move along the junctions and it literally degrades them

[fresh path]

the transistors themselves probably go way over that

[high skiff]

this is the way I have to run my GPU right now for it to mostly not turn my PC off

[elfin cobalt]

And modern GPUs/CPUs are so fast, they need to account for the speed of electricity. They can't instantly throttle up, because then they'll brown out; it's like suddenly removing an obstacle to water, the water doesn't instantly fill the new channel.

[uneven dove]

yeah it always freaked me out how the temp we see of a device is an average and not real

[lusty raptor]

hi is this the sdxl channel?

[lilac wren]

Nop

Sausage channel

[uneven dove]

this is #gpu-go-brrr because they deleted that channel

NOW WE HAVE NOWHERE

thanks for rubbing it in

[fresh path]

yes

[uneven dove]

oh, i added png info to all my bot's generated images

[high skiff]

having extremely adjacent to SDXL conversations seems pretty reasonable to me lol

Its not like we are talking about cars, its a convo on how the hardware required to run what we are all doing works lol

most people here would benefit a lot from undervolting their GPU's

same perf, much less power

[uneven dove]

don't do it

sytan just wants your computer to hang

so he's the only one

[fresh path]

but i want more perf

i want h100 in my gaming rig

[elfin cobalt]

No you do not. 😛

[high skiff]

oh god, an H100 in a gaming rig

thats a cursed thought lol

[elfin cobalt]

Do those even have graphics units?

At all?

[high skiff]

WOOOO

its happenningggg

[elfin cobalt]

I think the A100 has, like, ten.

[high skiff]

I am gettinga 3090 for $600

A nice one too

[elfin cobalt]

(Or maybe that was the W6000)

[uneven dove]

you don't need them because you'll be too busy trying to make ends meet by working multiple dead-end jobs after splurging on the H100, there won't be enough time to play games

[elfin cobalt]

No no, just sell your house

[uneven dove]

i'm already homeless so i guess this plan isn't for me

[ionic dragon]

is unipc the worst sampler?

[visual glade]

some people say it's the best

[warm hazel]

https://stable-diffusion-art.com/samplers/

[uneven dove]

those people are "inpatient" in a "psychiatric intensive care unit" and we wish them the best

[urban fjord]

I tried it but never saw any real benefits to it. Though I never saw any real benefit of all the different samplers using 1.5 either. Some of them worked some of them didn't and I just disabled those who didn't work.

[uneven dove]

ControlNet works really well with UniPC for some reason, so, i just use it for that. literally, only for that.

a zebra made of crochet

the sarengeti spaghetti valley

[ionic dragon]

i generated images using unipc, i just got blanks images, i dont why

wait

i used unipc with exponential scheduler

i just got all blanks for every generation using that combo?

[small birch]

has anyone run Zeroscope V2 XL? and if so have u ran it with under 15.3GB VRAM?

[visual glade]

use uni_pc with normal

it might mess up with the others

[small birch]

comfy do you support anything like it? (txt2video)

[visual glade]

once a decent one comes out I might add support

[small birch]

do you know Zeroscope V2 576w ?

and the XL ?

https://civitai.com/models/96563

looks way better than is it Gen2 the runway one?

im gonna try it with auto1111 but if you had a model/support for this i would rather stick in comfy

[ionic dragon]

i am doing a comparison so i am using all combos, so is it with my pc or is it that unipc doesnt work at all with exponential scheduler?

[visual glade]

yeah that's very possible

I made it so that "normal" matches close enough the default uni_pc scheduler

the others might not work very well

[small birch]

whats the VRAM on that?

[high skiff]

24 GB

[small birch]

nice one

[high skiff]

yeah, need it for playing with SDXL tuning

[small birch]

run this for me https://civitai.com/models/96563 lol

[high skiff]

I don't have it yet

currently trying to secure the payment for it

sending $700 to pay pal is apprently "sketchy" lmao

[small birch]

my 2070 only has like 9gb VRAM 24GB total it says

[quasi remnant]

even renewed/refurbished 3090s are still going for $1000, so as always be careful of scams

[high skiff]

its a verified seller on hardwareswap

[small birch]

4070 was going to be my choice

[high skiff]

and we are doing it through paypal good and services as well, so I have buyers protection

[ionic dragon]

@visual glade do you know any tool to attach images, so i can replicate a1111 grid?
i am doing comparision

[small birch]

but i discovered its cheaper to get a prebuilt on Scan than it is get the parts

[glad fulcrum]

What are the possible resolutions?

[high skiff]

he actually uses SD< and when I told him I am using it to research SDXL more he was like "Dope!"

he was also able to run some SD on it to give me some speed numbers, said he also runs vicuna 30B on it

[small birch]

good, my laptops doing me good so far, but next big job i'll get the 4070 machine

[quasi remnant]

for text models you should only expect 20b

for 1000 token memory

[small birch]

hopefully the war hasnt started by then

[quasi remnant]

30b would be a lot of CPU ram usage i think

[small birch]

u heard the whole 'we will blow up the chip factory' and bring the people/tech to US if you come to 'the island'

[rustic temple]

could be smaller bit quantized

[quasi remnant]

hmm

yea maybe, not sure how that works

[small birch]

Zeroscope V2 XL (txt2video)
The XL models use 15.3GB of VRAM when rendering 30 fps at 1024x576

[quasi remnant]

back when i was running text models the above kinda thing was what i was using with like

gpt-neox 20b

took a lotttta vram and had to offload some to cpu

maybe stuff's better now

[small birch]

i dunno, is it worth me trying this model

it looks so good

hey i can always upscale, but only so far you can get with that

[quasi remnant]

uhhh idk, never tried txt2video stuff except deforum way back

[glad fulcrum]

Anyone talking sdxl? What are the lowest possible resolutions?

[rustic temple]

4bit can cut memory requirements in half, pretty impressive

[quasi remnant]

the image data is trained on 1024x1024

[small birch]

SDXL is running super fast for me

[glad fulcrum]

Ok. Thanks

[quasi remnant]

any resolution that adds up to a megapixel is recommended

[small birch]

thank Comfy it seems to be his UI helping it

[rustic temple]

don't get too cozy, 1.0's coming in 5 days 😄

[quasi remnant]

i don't see current setup changing much

unless

[small birch]

yeah

[boreal bough]

768x768 is barely ok. 640x640 can be made to work in a bad way
but by design 1024x1024 is how it should be run

[quasi remnant]

Is that style stuff used by the bot anything we're going to have access to as like a separate kind of thing we load in?

[rustic temple]

aye, but might not respond to prompts all the same though

[quasi remnant]

or is the style just additional pre/post prompt?

[small birch]

i think its just going to get more pathways

[patent badger]

final date for sure?

[warm hazel]

Whats the resource load on SDXL? Like is a 3090 able to handle it?

[quasi remnant]

3090 handles it fine

[small birch]

im using a 2070

[quasi remnant]

i can batch 2 images on 3090

and it hits 23gb vram

[rustic temple]

hasn't been any sign of it getting pushed back

[small birch]

i can get an img in 15 seconds

[quasi remnant]

yea

[uneven dove]

oh it's going to get released as 1.0 on that date most likely even if it's a terrible checkpoint

[quasi remnant]

15 sec for 60 stepped image here too

give or take

[boreal bough]

a 3090 is strong enough to run 2 instances of it simulataneously. you're good to go XD

[quasi remnant]

it's insane how far we've come

these were the speeds i used to get on like 512x512

[small birch]

i could cook eggs on my laptop but it runs good

[ionic dragon]

i dont think its gonna be terrible checkpoint

[uneven dove]

SAI has already stated in advance that they're "looking to provide regular updates to the SDXL weights" which to me indicates 1.0 is gonna be kinda rough

[warm hazel]

Considering 90% of you guys don't remember 1.1 it would be good to revisit it to see what that was like.

[uneven dove]

@warm hazel i love the patronism, keep it up 👍🏽

[ripe flax]

hello, at inference time, what process is done? it just take the text and a random noisy image? and it comput the denoising unet?

[warm hazel]

Not you specifically.

[grizzled warren]

and SDXL appears to be faster than 1.5 at good resolutions, because it doesn't have to use highresfix

[quasi remnant]

i think the checkpoint is fine; when i roll in bot it really feels like 1 image = 9 gens in 0.9 worth of quality most of the time lol

idk what changed but the release candidates seem real good

[boreal bough]

but there weren't any mentions of license changes for 0.9, right?

[uneven dove]

that's still just guessing on our part, not any official guidance

[uneven dove]

really? they don't follow my prompts.

[boreal bough]

to be seen in a few days anyway

[quasi remnant]

i first roll my prompts in 0.9 to get to a point where it's pretty consistent, then i roll in 1.0

and the 1.0 gens come out a lot better

[uneven dove]

i don't do any kind of weird cherry-picking like that. i go straight to 1.0 and try to gen what i want. it does not work as well as 0.9 for me

[quasi remnant]

well i mean

you still have to learn the prompting style of sdxl lol

[uneven dove]

... i know how to prompt sdxl

it's OpenCLIP style prompting, i helped Sytan figure it out.

[quasi remnant]

i'm not saying that, more if you can get a good prompt in 0.9

[regal heath]

why do i get some black rectangles when using hirefix?

[boreal bough]

"trashpanda is amazed by the quality of the release candidate and holding his head in glee"

[quasi remnant]

oh yeah and cuz of the bots restrictions

namely the above

you have to be aware of the filtered words

at all time

if they're any part of the tokens they're cut out

[uneven dove]

@quasi remnant my point is that you shouldn't have to focus on where the prompt is made, first. that's a weird comparison method that the model authors designed because it favours their newer work. i'm running the weights locally. i have contributed a new workflow to the img2img pipeline for Diffusers, i'm pretty familiar with the internals.

[regal heath]

anyone know why this happens?

[eternal fog]

Usually it's because the VAE Fails to run through properly

[regal heath]

when i try to upscale it again it just ignores

[eternal fog]

What Interface are you using? Comfy, Auto1111, something else?

[quasi remnant]

with 1.0 you only get 1 pic at a time; in the time that 1 pic is made in 1.0 i have 6-12+ pics made in 0.9

[regal heath]

auto1111

[quasi remnant]

once 1.0 is out you'll be able to test it a lot more

[uneven dove]

you get two per gen in the bot channels and over 6-8 gens you can be pretty certain you've seen their 4 test models' outputs.

[eternal fog]

Try --no-half-vae as an argument

[quasi remnant]

so i prefer to run prompt engineering tests first then pass it through there, it does come with quality results

you get the same seed, sometimes it's the same pic just micro-changed

with more details

so i consider it 1 pic

sometimes the 2 are completely diff

[uneven dove]

that's all weird test methodology you seem to be doing specifically to give 1.0 an advantage

[quasi remnant]

sometimes one is just more detailed ver of the other

[regal heath]

already using

[regal heath]

this one

mine is like this ```@echo off

#set PYTORCH_CUDA_ALLOC_CONF=garbage_collection_threshold:0.9,max_split_size_mb:512
set PYTHON=
set GIT=
set VENV_DIR=
set COMMANDLINE_ARGS= --xformers --disable-safe-unpickle --allow-code --xformers --skip-torch-cuda-test --no-half-vae --api --listen --update-check --medvram

call webui.bat

[fresh path]

@echo on

[quasi remnant]

if it works it works /shrug i'm impressed with the results of 1.0 and that's all i personally care about at the end of the day

[uneven dove]

i care about a lot more than that, lol

[quasi remnant]

once i finish a prompt, it works a hell of a lot better in 1.0 most of the time

[small birch]

Are these models just going to get more and more VRAM hungry or will we get lower as we develop?

[uneven dove]

considering they're nuking half of the model i'd say more efficient

[boreal bough]

both, which means they'll try to cancel them out around consumer hardware

[vital wolf]

I tried using hunblemikey's workflow, upon using it says empty latent ratio custom sdxl, I tried using comfy manager, but couldn't find it, can anyone help?

[quasi remnant]

wdym?

[small birch]

fair, im looking for the open source render farm comfy mentioned yesterday

[tender timber]

Does anyone have performance results for Intel ARC cards and SDXL, or even AMD? How do they compare to a 3060 or a 3090?

[small birch]

incase that could help

https://80.lv/articles/draggan-a-new-method-for-manipulating-generated-images/

this is interesting

[uneven dove]

a spaghetti fall

[stone fossil]

A ramen ship.

[eternal fog]

Does anyone know of a good tool that can use BLIP-2 to auto tag images?

I've got taggers for Booru style tags, but nothing for plain english

[uneven dove]

i have a bghira/SImpleTuner project with interrogate.py that does this

[eternal fog]

Is that going to be any good for tagging stuff for SDXL

I tested using a dataset I already had that used Booru style tags and it wasn't brilliant

[high skiff]

Welp

[boreal bough]

did it have a trigger word? did you make sure to keep 1 token, shuffle rest?

[boreal bough]

reply meant for this*

[fresh path]

I just realised all the space pics are really good because the training data is made by billion dollar telescopes

[uneven dove]

try to get a phoenix nebula

it perfectly combines the concept so that a phoenix shape arises out of the space dust

[boreal bough]

omg. I can finally get love for lacerta!

the lizard star constellation

[hard fractal]

This is how many images the applied ML team at SAI has made in Comfy while testing SDXL.

This doesn't include the massive grids we make. Or other teams at SAI testing it out.

This is just the applied finetuning team.

[eternal fog]

Can't in SDXL, as I can't train the text encoder, so it disables that option.

Cache the outputs of the text encoders. This option is useful to reduce the GPU memory usage. This option cannot be used with options for shuffling or dropping the captions.

[boreal bough]

@hard fractal do I get an achievement if I can get (1.5 equivalent) finetune levels of improvement with a 43mb lora (trained on 5k images) on 0.9base? xD

[uneven dove]

dang that's only like 7x more images than @waxen berry made

[compact flax]

Nuking half the model? What are you referring to?

[uneven dove]

the refiner going away

[compact flax]

I see, so what takes it's place?

[uneven dove]

nothing

[ionic dragon]

1.0?

[uneven dove]

if it happens, probably the biggest mistake 🙂

refiner is the coolest part of SDXL

[eternal fog]

They need to fix it blurring stuff in img2img if they are removingthe refiner I think

Otherwise any sort of high res fix will just break stuff

[uneven dove]

that's probably implementation details downstream

not for SAI to fix

[eternal fog]

I mean wasn't comfy working with them to implement it?

[uneven dove]

i've put work into Diffusers to fix that issue, so, it's already happening

do you mean the latent upscaler that makes things look blurry?

[eternal fog]

No, someone was talking about this earlier as well.

when you do img2img, it washes out loads of details on the original image

So if you are trying to do a pixel upscale highres fix type thing, you lose loads of detail and it smooths it all out

[uneven dove]

yeah, that's because it adds noise to the completed image

[eternal fog]

It wasn't an issue in 1.5 or 2.1 though. Or at least I never had that issue.

[uneven dove]

it depends on your settings and your prompts. the hires. fix in SDXL base needs a lower CFG value

it's just a problem for A1111 or your ComfyUI settings, and not an issue with the model itself.

[eternal fog]

You see I've been seeing people say the opposite in here

That it needs higher CFG

[uneven dove]

that'll lock it in too much to a single outcome

[eternal fog]

I've tried multiple settings and it always washes out detail on the 2nd pass

[uneven dove]

lower CFG on the refiner gives it more freedom to fix the image

[sharp robin]

No. SDXL won’t do i2i

We have all tried

[uneven dove]

then you haven't tried hard enough lol my implementation overrides the model for img2img with the SDXL base model

[eternal fog]

Do you have a Comfy workflow that can consistantly use Img2img with low denoise without washing out details?

[shy kelp]

no he does not

[sharp robin]

Tried cfg from 4-12 doesn’t help. Tried the manual denoise from all ranges

Doesn’t work

[boreal bough]

base + overfitted lora + refiner at 0.2~0.3 denoise

[uneven dove]

@eternal fog idk how many times i have to tell people i don't use ComfyUI

[eternal fog]

That's using something extra with the LoRA. I mean with the base stuff

[boreal bough]

lacks face details

[sharp robin]

Tried the one step less on refiner don’t work. Tried base and refiner no go

[shy kelp]

refiner was trained on leftover noise from image generation. it dosent work on added noise

[boreal bough]

you could try to prompt fix that - but at cost of introducing huge bias

[uneven dove]

ptx0/s1 is my fork of the SDXL base model on HF hub, and ptx0/s2 is the SDXL refiner. they're forked so I can change their scheduler_config.json to use DDIMScheduler by default for convenience.

that prompt_variation runs when a img is pinged to the bot with a prompt to guide diffusion with

it just uses ControlNet if i don't provide text with the img

in fact the base model is so good at img2img, especially compared to the refiner

[eternal fog]

You have ControlNet that works with SDXL?

[uneven dove]

the refiner doesn't seem to add "enough" changes to really impact it much

[shy kelp]

yeah bro we know its better without the refiner. this shit is the easiest shit in the world in comfy, but it still dosent preserve detail

[uneven dove]

@eternal fog i experimented with making a Controlnet Tile model and can't figure out how it's trained. i use 2.x and 1.5 controlnet models for SDXL outputs instead.

[uneven dove]

that's why you cut the img2img output from the base, halfway, and feed into refiner.

[ionic dragon]

Wouldn't it makes the images off?

[uneven dove]

oh it certainly has side effects

[shy kelp]

yeah tried that. even worse

[uneven dove]

not enough that i notice

@shy kelp idk what to tell you lol it works in Diffusers

i'm using DDIM / ddim_uniform equivalent

[shy kelp]

diffusers?

[eternal fog]

This is what I have at the moment. The left is just the base with a 2x pixel upscale. The right is after the 2nd pass.

This one looks ok, the refiner has added detail to the eyes, but if you look elsewhere, it's pulled some detail off the skin, although this image isnt' that bad.

[uneven dove]

and base model at low CFG is even better

i friggen love SDXL base cfg 1.7 outputs

[boreal bough]

• no refiner, or refiner at 0.15~0.25 denoise

[uneven dove]

very realistic looking details, high risk of deformities. roll the dice more

[sharp robin]

You got a comparison? Let’s see it.

[uneven dove]

the code is open source 😄

[shy kelp]

ur telling that if i just use ddim+ ddim_uniform sampler it will work?

[boreal bough]

has the requirement of no short prompts to work properly - so you need a few supporting style tags (anything as long as its not nothign)

[ionic dragon]

Dude, for real?
1.5 with something around 5 gives shit images mostly pixelated and some random images which have no context

[boreal bough]

you can just use a clip query to get a string, then use that string - that makes it work

[uneven dove]

yep

[shy kelp]

i think hes talking about upscaling

[sharp robin]

[regal heath]

is there any workaround to use RAM as complement for VRAM?

[eternal fog]

looks a bit iffy for me with CFG 2

[regal heath]

cool

[eternal fog]

Left is CFG 2, Right is CFG 10

[ionic dragon]

@uneven dove what do you think of training a model with 20million images?
Not gonna train for sure, ask our of curiousity

[regal heath]

depends on the images i would say

[uneven dove]

i'm using DDIM uniform, like i said

lower CFG has higher realism from base with this scheduler

idk what to tell you, sorry if yours doesn't work that way

i do 20/20 between base and refiner, and refiner adds no extra noise. i use a higher CFG on the refiner to bring the washed out look to life a bit more from the base.

[sharp robin]

This will give an entire different image

[eternal fog]

One of the devs put on reddit I think that dpmpp_2m_sde was the "best" in their testing, so I've been trying that.

[uneven dove]

@eternal fog that testing data had people upvoting deformed images. i know, because i upvoted deformed images

[ionic dragon]

Yeah it does, but even 20million doesn't cover many subjects.
Just wondering how many did MJ use?

[eternal fog]

DIMM actually makes the washout even worse for me

[uneven dove]

i think that's a waste of time? why would you want to train on 20 million images? what are they of?

[regal heath]

would be a waste tho

[eternal fog]

Left is Pixel Upscale, Right is after an img2img pass over it.

[regal heath]

i think 10k is already an absurd ammount

depending on what you need

[boreal bough]

its a matter of prompting. if you use any clip model to extract a prompt from a picture, then throw that prompt back into that node, it will work pretty well

[uneven dove]

depends on the model. for fixing 2.x i needed about 200k images from real cameras

[sharp robin]

Is this i2i?

[boreal bough]

also this for negative "Deformed, unrealistic, bad quality, grainy, noisy, plastic, hazy, low contrast"

[eternal fog]

It's not though, because on the same prompt some schedulers are fine and others aren't.

I think the issue is that you can get very different results for very very small changes.

[uneven dove]

that is the base model generating and feeding its 50% denoised output to the refiner, not a completed image as input, no

[quasi remnant]

where'd they say they were getting rid of the refiner? i thought just the bot genning images doesn't use refiner

[ionic dragon]

I ain't training, just some random thought.
Training on multiple subjects

[eternal fog]

Just had a quick check too, DIMM high CFG Deep fries the image. on the SDE ones it makes it look better.

Probably why low CFG works on DIMM

[ionic dragon]

Like an all round model.
Like making SDXL double capable

[eternal fog]

But it looks shit on SDE

[uneven dove]

i can run up to CFG 20 on the Diffusers implementation of ddim_uniform and it doesn't burn.

[royal fern]

I often have troubles focusing with xl

[uneven dove]

i'd try higher, but my bot has a hard limit that i'd have to remove first

[inner ruin]

anyone trained LoRAs yet?

[uneven dove]

yep, they work on the base model

[eternal fog]

I have, but I'm getting weirdness with the training

[inner ruin]

which script/params did you use?

[eternal fog]

Like it worked last night and did training

Then I put the exact same parameters in today with no changes and it OOM

[uneven dove]

khoya-ss is the one people are having success with but darien-distro has Easy Tuning Scripts for SDXL repo that optimize things further

[eternal fog]

Do you have a link for this? googling darien-distro doesn't bring anything up.

[boreal bough]

#✨｜sdxl message
and
https://github.com/derrian-distro/LoRA_Easy_Training_Scripts/tree/SDXL

[uneven dove]

oh

i typo'd

[inner ruin]

nice thanks! did you discover good face parameters yet?

[uneven dove]

it is incredibly biased on picking facial features it seems... if you have abnormalities in any faces, suddenly, they are everywhere

[boreal bough]

while there are methods - I'd recommend against them, as they will automatically be invalidated by 1.0 release - so further research/implementations of them are kinda moot

everything else goes though

literally train anything except
• face
• body parts
• eye accessories

[inner ruin]

I thought 1.0 was pretty similar to 0.9? Like similar enough that the scripts should be easily adaptable?

[boreal bough]

yep, for literally everything except the 3 things I posted above

faces will be much easier mind you, not the other way around

[inner ruin]

why is that?

[uneven dove]

this was my best 1.0 result so far and it didn't do my prompt 😦

[sharp robin]

hopefully they get rid of the refiner step

that was a mistake

[inner ruin]

ufff! And that's 1.0?

[uneven dove]

it's a technological requirement to do certain levels of image detail @sharp robin because they stuck with the ancient VAE architecture

[boreal bough]

try it yourself in #1100170312106127410 normal prompts work well

[inner ruin]

I'm assuming @uneven dove was displaying a 1.0 LoRA

[uneven dove]

nope. that's the bot here

[clever verge]

Any tips prompt on framing a photo of a human to not show the fingers/hands? Like the one above.

[sharp robin]

they have talked about removing it already, so I assume it is not that big of a requirement

[fair crow]

I've tried training Loras using the diffusers sdxl dreambooth script, and they all come out with either weird lines or without the subject at all. Has anyone had luck with that script at all? I suspect it's borked.

[uneven dove]

they're able to do that because they can continually just point to previous base models that were worse, and say, "but 1.0 is so much better than 1.4, 1.5, 2.0, 2.1"

it's like telling someone you doubled your sales, but it's just from 1 to 2.

[sharp robin]

if they can do 1.0(no refiner) better than .9+refiner its a W

[uneven dove]

technical details indicate that is extremely unlikely

you can keep hoping for it but i and many others are likely to use the refiner forever, even if SAI abandons it

[inner ruin]

I haven't tried it, did you try kohya?

[fair crow]

Not yet - diffusers was easier to setup for my env

[old anvil]

hey, did you see that the sdxl support is being added in the a1111?

https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/11757

[uneven dove]

yawn

[fair crow]

Ohh interesting - I'm currently mostly running stuff on modal without a ui but might be worth setting up a notebook to check this out!

[uneven dove]

DDIM, PLMS, UniPC samplers do not work for SD XL
lol

fail

once again he can't fix his samplers

i wish he'd just throw them all away and use Diffusers

honestly just use Vlad's fork

[pure hazel]

1.0 weights aren't out yet right?

[uneven dove]

correct

[visual glade]

last time I checked a1111 didn't even have img2img working on unipc

now I understand why vlad switched to diffusers, that was the right choice

[pure hazel]

Diffusors is far more advanced.

[stone fossil]

Open up.

[old anvil]

I definitively need a better gpu 💀

[uneven dove]

is that batch size like 150?

mine took 300 seconds per it but i was on an A100-80G doing a full general fine-tune on 2.1-v

[uneven dove]

his SDXL PR says DDIM, UniPC, and PLMS are not working with SDXL

that's not either yours or my experience

[visual glade]

that's cause he didn't bother abstracting things out

[uneven dove]

you don't have to bother with code abstractions when you have genius on your side

[visual glade]

in comfyui I wrote the code in a way that I could just swap out things and it still work

[uneven dove]

yeah, Diffusers did that too, which was an interesting thing to discover for me, how much is being abstracted away and "normalised" under the hood

hadn't looked at that much til this week

kudos to Auto for reproducing the SGM results though, guess that's what happens when you lift code letter-for-letter

[visual glade]

that's easy when you just use the code straight from it

[glad fulcrum]

Why?

[uneven dove]

less technical debt and easier to extend, a more quickly growing ecosystem full of various disciplines

[pure hazel]

Better architecture, far more advancedments and features integrated

[uneven dove]

ask Vladmandic why he switched to it

gave him native Kandinsky support in his fork of sd-webui without any extra labour on his part

[visual glade]

oh I checked the reference images on his pr, there's a lot more differences than there should be

[uneven dove]

i was going to say that

it's probably the optimizations he's doing

Euler is so simple there's no way he could have screwed that up?

[visual glade]

it's not that, I do more optimizations in comfyui and my images match extremely closely

[tribal lantern]

help the poor model a little by adding some fluff 😉

[uneven dove]

what the hell, did you do a 1.0 strength i2i?

[tribal lantern]

#1101178530865352815 message

[vale eagle]

I think someone already done this (use one node to run both base+refiner). But I added a little more to loop the base+refiner process multiple times. https://civitai.com/models/108594?modelVersionId=116911

[uneven dove]

OHHHHHH

mine sucked because of the "coon" again omfg

you dropped an o

this bot is the damn worst

[tribal lantern]

yeah, saw that in other chat

[upbeat summit]

"let's party together" I made this 2 days ago 😄

hi all!

[high skiff]

new GPU is in the mail

[visual glade]

nice hope it works and everything

[tribal lantern]

lol that raccoon was really a lucky parameter and seed roll, the eyes through those sunglasses amazing

[high skiff]

oh nice, thats my workflow 😅

[uneven dove]

omg i didn't even think of it being DOA that's so unsettling, thanks comfy

[high skiff]

very very interesting to see the difference from the loops

[high skiff]

if it is, I am covered :p

worst case is I get my money back

[visual glade]

Here's how closely sgm and comfyui match with the right settings

that's the kind of difference there should be when using euler even with max optimizations or whatever

[uneven dove]

i'm so disappointed in you, comfy /s

[tribal lantern]

is this the 450 dollar one? if so you're lucky... don't know wtf happend with used prices in europe, few weeks ago i was looking and they went for 650euro on ebay, it's bloody 800+ now

[uneven dove]

yeah, Euler is truly wild.

[vale eagle]

Yeah, thanks for sharing the nice workflow.

[high skiff]

no problem! Gonna be updating it soon

[visual glade]

yeah euler is easy, at least the default sampler was not SDE cause that might have been a problem

[elfin cobalt]

Is there a big difference between looping the diffuser, and just running SDE+Karras for a bunch of extra steps?

[high skiff]

Vs my method with my prompt split and the same seed

[visual glade]

oh my pic is without the refiner

[visual glade]

what do you mean?

[elfin cobalt]

It was my understanding that that combination effectively adds a little bit of noise at every step

[uneven dove]

are you being stochastic

[visual glade]

you mean the advanced sampler in comfyui?

[copper dock]

are the two images from the bot 1.0 vs 0.9? or different versions of 1.0?

[uneven dove]

it's a waves hands around mysterryyyy

[visual glade]

it could be a lot of different things

[copper dock]

i see 😅

[soft zealot]

nope I'm still conflicted.

On the left is a raw SDXL output, on the right is that image run through Proton. Personally I lean towards favouring the RH image

[elfin cobalt]

While that is what I'm using, I think I just mean dpm-2m-sde-karras. Never mind.

[pure hazel]

What's proton?

[elfin cobalt]

And Proton?

[soft zealot]

I meant Photon, that was a typo lol

https://civitai.com/models/84728/photon

[eternal fog]

I think the left one is better, other than the awful shit SD does for earrings

[soft zealot]

if you check my workflow you will see Im outputting the SDCL image to an IMG2IMG step

[old anvil]

1

[uneven dove]

[old anvil]

gtx 1650 moment

[eternal fog]

wow ok that's a first lmao. SDXL just made NSFW without me telling it to

Oh I see what it's done lol. I was testing img2img and the top is almost skin coloured. I think it decided that the top wasn't there

[shy kelp]

sdxl’s imminent release makes it look like a good time for me to get into this. Is vlad’s fork the one I should be using or is there another app?

[eternal fog]

A lot of us here are using ComfyUI, but if you just want a simple generation tool it seems like Vlad is decent

This was done with img2img, seems like it works decent enough. There does seem to be a point where it just switches though. From being similar to the input, to not looking like it at all.

[visual glade]

if you want the full experience get ComfyUI and this workflow: https://github.com/SytanSD/Sytan-SDXL-ComfyUI

[eternal fog]

Yeah ^, I'm using that workflow too, just modified slightly for upscaling

[upbeat summit]

ahm... nope

[eternal fog]

I think my attempt to force a style with prompt weighting has kind of locked in this face for some reason

[uneven dove]

dont threaten me with a good time

[eternal fog]

I've been playing around with Img2Img a bit, it can be very particular on what prompt you use and the exact step settings. Even just 1 thing being slightly off you get a mess. But when it works it works reasonably well.

[uneven dove]

that may be too much skin for this server lul

i say nothing, it is up to you 😛

[soft bone]

@boreal bough how am i supposed to see training progress/errors with this Kohya distro ui?

[eternal fog]

I mean it's normal attire for the beach so...

That's a completely different step though lol

[elfin cobalt]

Anyone else find the text rendering in ComfyUI a bit weird?

[shy kelp]

thanks, I'll try that first

[elfin cobalt]

It's blurry... and disappears entirely well before it'd be too small to read, if it wasn't blurry.

[visual glade]

update it, some dpi issues were fixed recently

[high skiff]

Turns out my 3080 I have now is bigger in almost every way to the 3090 I just bought

insanity lol

3090:
12.5 inches long
4.8 inches tall
2.3 inches thick

3080:
12.7 inches long
5.5 inches tall
2.3 inches thick

[shy kelp]

whats proton?

[soft bone]

this distro Kohya ui that caith sent aint workin:(

[high skiff]

3080 is 1565 grams
3090 is 1702 grams tho
So the 3090 is a lot more dense

will definitely be getting a bracket

[boreal bough]

oh damn. right one passes the vibe check. 👍

[soft bone]

i have the biggest 3090 ever made and no bracket lmao

[floral island]

ketchup incident. heh.

[uneven dove]

yeah i just meant it as a general heads-up 😛 i liked the pics

[boreal bough]

you dont yet. unless derrian implemented it already - cause that's from yesterdays patch on kohya

documentation is still lacking though

will prob work in a day or two

[soft bone]

okay but when I start training the button greys out and nothing is happening, no resources being used

[boreal bough]

oh that

um, you should be able to see in the cmd thats open in the background

kohya should be running in that

[soft bone]

tis completely blank

[boreal bough]

did you add it to the queue, before pressing start?

[shy kelp]

sdxl could probably do a better job if it could do inpainting

[soft bone]

got it, thanks. ngl i dont see how this is easier than kohya ui but queue system is neat

nvm CPUAllocator error as usual with any batch over 1

[quartz sequoia]

@visual glade even though im still learning ur ui i would like to give you a high five for the memory optimization🙌

[fading kayak]

anyone know if this node works like multidiffusion tiled vae from a1111?

[soft bone]

I get this in any lora config on any install of kohya. 5800x 32GB 3090 isnt good enough?....

[civic sigil]

Is there a way to choose the tile size for vae encode/decode

[visual glade]

thanks I did a bit of memory optimizations because I wanted SDXL to be well received

[visual glade]

it's easy if you change the code but not from the UI

[civic sigil]

You think its good for it to match the grid when you are doing grid method animations?

[eternal fog]

Is it something that would be open to creating a custom node for?

I dunno how much is exposed to the nodes

[quartz sequoia]

would it be possible to add a preset workflow drop down menu, something with recommeded settings from you or stability, and like community list?

[visual glade]

it's super easy if you create a custom node, just look at the node for the tiled VAE

[quartz sequoia]

i really like the drop img workflow to it and it just works

but sometimes it messy to look through that

[shy kelp]

alright so people who claim that img2img works, what settings?

[uneven dove]

i wonder if they'll do the inpainting model, and if they do, if it will be done the same way as before, with 5 new extra channels

[eternal fog]

I can show you mine but the nodes are a mess.

And depending on the image input you have to change the steps around or it doesn't work

[fading kayak]

this has been working for me https://github.com/SeargeDP/SeargeSDXL

[shy kelp]

yeah well it works, it just does a shit job. so how do you get to preserve detail

[eternal fog]

Do you have an example of what your wanting to do?

And I can give it a go

[fading kayak]

hah as soon as I said it was working my results look like this

[eternal fog]

Like the source image and your idea

[shy kelp]

like img2img upscaling. some people say its not shit.

with the right settings

so whats the right settings

[eternal fog]

img2img upscaling, I still can't get that 100% right, it still does stuff like remove skin texture.

I think I'm close, but something always goes a bit off.

[shy kelp]

im using DIMM and DIMM uniform and still looks like shit

[uneven dove]

oh, i use the refiner model for upscaling. not the base. the base does this

broken as all hell

[shy kelp]

you'll ever got it to work 😂

[eternal fog]

Yeah I get that with the base, but then with the refiner I find if you do too much denoise it goes mental, and too little nothing happens

And there is a very small window between the two

And sometimes there is none

[shy kelp]

alright so how do you do the refiner, what settings

[uneven dove]

i was using the base as img2img for artsy stuff like, comic book versions of people. that, it does really well.

[sharp robin]

no proof yet of img2img working

[uneven dove]

i did a PIL upscale to 2048x2048 and then put it in the refiner with a lower guidance than i use for typical denoising and a high strength like 0.6

[eternal fog]

I posted an actual image 2 image above, not an upscale though

[eternal fog]

How many steps on the refiner? Because when I do that it melts.

[civic sigil]

I thought you said you prefered the base model

[sharp robin]

this?

yeah results not even close

[uneven dove]

upscaled from refiner vs original

#🌠｜show-and-tell message

[eternal fog]

It never really was with img2img.

I think people have been spoiled by controlnet.

Because I kept thinking it was nowhere near, but then I went back and that's what it does.

If you make the denoise too low, to try keep it the same, it just blobs

I'll try a pass now with low denoise, I still have the same image loaded.

[uneven dove]

i had to put the refiner strength pretty high for it to destroy the image

like .9?

it doesn't do very good image transformations at all

and by that i mean, CHANGE the image

it just adds more detail

i will try and find those settings. sec

ptx0/s2 is my refiner model fork

.3 strength, 7.5 guidance 10.0 aesthetic score 1.0 nascore

[boreal bough]

in soviet russia. bear trains you

[shy kelp]

whats guidance? you are using controlnet?

[uneven dove]

no controlnet, just SDXL refiner there as an image input from the discord attachment which is Image.resize'd to the 2048x2048

[eternal fog]

CFG

[sharp robin]

@shy kelp Automatic1111 image2image works, tho does not support refiner yet and takes ages but it is seeming like it is working

30 step .40 denoise

[uneven dove]

yeah that's using the traditional img2img SEdit method that adds new noise.

[shy kelp]

like like it didnt upscale 😂

[gentle mirage]

Oo you have a bot to run multiple models through discord? That's really neat dude

[uneven dove]

also make sure you're using a different seed

[mellow tendon]

I just use a 3rd SD 1.5 model (like Juggernaut , EpicRealisim or photon) for upscaling it has better skin textures and gets rid of some of thr blurry bokeh background that is overdone by SDXL.

[shy kelp]

also whats the point if its not using the refiner at all 😂

[uneven dove]

@gentle mirage any model on huggingface Hub

[shy kelp]

then its not even sdxl

thats just half of sdxl

[gentle mirage]

Is the code for the bot open source

Id like to try if i can

[shy kelp]

this is probably the way to go rn

[uneven dove]

it does language models and so on, too. there's also Bark TTS. yes, it's bghira/discord-tron-master on GH

[gentle mirage]

🤝

[soft bone]

Bot 1 isnt using refiner rn

[sharp robin]

this was after the image was already upscaled, i just want i2i working well first upscale comes second, tho will test it out, just takes long to do this,

[uneven dove]

it's not got great support for low VRAM systems since i run it on an 80G but i've done some work for that lately and it runs basic stuff alright on my 8G laptop GPU

[sharp robin]

likely to come soon

[gentle mirage]

Do u have A100s lying around?

Dayum

[shy kelp]

i hope not

[uneven dove]

just one.

[sharp robin]

if you read the paper it was only meant for the last 20% of the gen or .20 denoise

[gentle mirage]

Cool

[uneven dove]

200 steps

[shy kelp]

you should go check how much the last 20% can actually change the image

[uneven dove]

you guys know anything about single-order vs multi-order samplers etc?

[shy kelp]

i thought people were advising to use it at 0.35 tho?

i actually like it better at 0.20

[uneven dove]

most people aren't using SEdit, they're using the new partial-diffusion workflow

SEdit expressly does better with lower noise levels whereas the refiner seems fully capable of continuing the base model's denoising at higher percentages

https://tripleback.net/public/partial-diffusion-comparison.png

at the top middle is the base model's output when it's fully denoised at 15 steps

the left column is SEdit and the right column is Partial diffusion

the rows go from .1 to .9 strength for the refiner, where it does 90% of the denoising work

you can see somewhere around 60% strength the deformities start kicking in like an acid trip but it's still quite valuable results for certain styles

the columns btw do 20 steps total, i did 15 at the top because it seemed fairly representative of the base model's contribution

[eternal fog]

This is about as close as I can get it where it changes enough to follow the style, but doesn't change the image completely.

Any less denoise it barely looks different, any more it starts changing stuff.

[shy kelp]

and then the rest of the world will start realizing that img2img dosent work with the refiner

[soft bone]

@boreal bough m8 your lora config is trying to allocate 54 gigs

[eternal fog]

This is just slightly less denoise and it barely changes from the source

[uneven dove]

is this channel just going to be scantily clad women for a few hours now?

[eternal fog]

Nah I'm going to change the image

[uneven dove]

thx

[sharp robin]

we dont know if the implementation of auto can do it or not yet, but at least the base model does some, im trying to be positive, best case for me is they just figure out a way to do away with it like they said they may be able to

[lament rune]

We now return to our regularly scheduled episode of "Raving Racoon Rampage"

[uneven dove]

fuck yea bud can you get a rakun attackin' old David Attenborough?

[shy kelp]

when they said that?

[soft bone]

still havent seen one source for that claim

[uneven dove]

the claim that the refiner is going away?

lmao

it's pretty common knowledge

[soft bone]

Not even Sytan knew about it and he's obviously very active. so no

[lament rune]

I tried to make the attack and instead they became friends because he's David Effin Attenborough

[boreal bough]

XD you should you're not running with all optimizations off?

my full setup, with batch 8 should run you around 18gb vram + overhead

if you do batch 1, it should fit under 16gb vram

[soft bone]

batch size 1 allocates the exact same bytes. heres the config @boreal bough

[uneven dove]

no worries fam jam

i gotchu

[lament rune]

sick em!

[uneven dove]

what the hell

[eternal fog]

@shy kelp What about that ^?

[sharp robin]

like 2-3 days ago

[shy kelp]

the refiner is going away? why? for 1.0 release or in the future?

[shy kelp]

what was said

link link link

[uneven dove]

i really hope it doesn't because i like the refiner, so, the news that they're trying to compress the quality of the model into the base, was pretty frustrating to hear

[boreal bough]

I didnt have tensorboard on, but other than that, identical

[uneven dove]

it's on Twitter and i can't seemingly look at stuff on there anymore

but if you have a twitter account still, go to EMastoque's page

[lament rune]

Grrrr I asked Emad personally for the ability to fine tune the refiner 😄

[shy kelp]

well it would mean we can do img2img upscaling so im all for it 😂

[uneven dove]

i guess you pissed him off and he just chose to delete it

@shy kelp why would it mean that though

[lament rune]

great. just great

[boreal bough]

when you configured it, did you follow this for accelerate?

- This machine
- No distributed training
- NO
- NO
- NO
- all
- bf16

[uneven dove]

they didn't test any of that to begin with for SDXL, why would it change now

[lament rune]

I heard they are eliminating the refiner because they want to go green and dont want to use oil in their pipeline. Very noble.

[shy kelp]

because the refiner is the reason why you cant img2img upscale and get the same quality as the original pass

[uneven dove]

it's because of the noise schedule

[sharp robin]

U already have refiner. So if model can do away w out it. Please do

[uneven dove]

read the original paper they used for their training framework, the part about the 'ideal denoiser' and how it works by making everything look smooth like soft serve ice cream

[soft bone]

it never asked actually. regular kohya did but not this ui

[boreal bough]

when you click install.bat, it should have gone through that :/

[uneven dove]

you can probably fine-tune a new noise schedule into sdxl but it'll be hard

[soft zealot]

random thought:

Could you take the output of a first base+refiner pass and feed that into a second IMG2IM Base+refiner pass???

[visual glade]

even if the refiner "goes away" people are still going to use it

[soft bone]

i ran install from terminal per the guide. I'll try it again

[shy kelp]

well thats because the refiner works on the detail right

[visual glade]

if we say "you don't need to do this" people are going to do it anyways

[shy kelp]

but the refiner dosent do a good job with detail unless its the first pass, so there it is. not good at upscaling.

[boreal bough]

@golden quarry
you still have an error here on the sdxl branch readme.md
fp16 leads to NAN, and based on people in the last two days, enough people followed that part XD

[uneven dove]

the base model is trained on the full 1000 timesteps

[eternal fog]

This is what I've been doing, but I've just found somethign I think is an odd behaviour I'm investigating

[uneven dove]

tbh thats sort of what a research community does

[visual glade]

yeah that's expected

[soft zealot]

person1 "you can't do that"
person2 "hold my beer!!"

[soft bone]

torch 2? yes
triton? no
cudnn? yes

thats all install.bat asks me

[uneven dove]

like someone said earlier, "Google innovates, StabilityAI releases, and the community fixes the mess"

[boreal bough]

oh damn :/

but that explains the vram issue

[visual glade]

stability does innovate though

[uneven dove]

lol by copy-pasting all of the research that came before them

don't stray too far from the Imagen paper! you'll get hurt!

like the technical paper was a long winded way of saying "Karras scheduling is pretty cool, and Hires. Fix works well, and ensemble pipelines are great!"

didn't even put it up against any serious community fine-tunes like epicRealism or any ControlNet pipelines or even a basic finetune from earlier gen with hires. fix

[vale vapor]

based on tests i've seen, it looks like img2img on sdxl might not be that great...unless maybe 1.0 has like the proper things to make it work that 0.9 doesnt but as of right now txt2img always gives better results than img2img which is opposite of how sd1.5 operates.

[urban fjord]

I don't see much point in judging img2img before the control-tile model is out as that changes everything.

[uneven dove]

oh, and reference_only.

[soft bone]

negative. just ran accelerate with regular kohya, set it up exactly as you said, used same parameters, and same error

my pc is broken i guess

[urban fjord]

Can you see the accelerate settings without editing it? Because I'm pretty sure I did not run with bf16 and my stuff trained. But maybe I would need less VRAM with it.

[uneven dove]

bf16 is needed

[soft bone]

idk but you can run install.bat and choose option 4 "run accelerate manually"

[boreal bough]

uff. rip ❤️

[uneven dove]

NaNs mean you had a division by zero, which means some number underflowed so that the precision isn't enough to represent the value

[sharp robin]

Was epic realism an actual big finetune?

[uneven dove]

@sharp robin it works without negative prompts and is generally the winner in that category in any comparison spaces on HF Hub

[urban fjord]

Yeah I know how to change them but I want to see what I was actually running. Though in the training config you can also set precision, what's the difference?

[sharp robin]

Amazing

Thanks for that info

[uneven dove]

you gonna loop SDXL through it? 😄

[golden quarry]

Whoops! Yeah I forgot to update that part, I'll do it soon!

I'll also update the install scripts to set bf16 default

[uneven dove]

dang it Derrian 😂

jk jk

[golden quarry]

Oh cmon I was trying to just rush it out! I didn't have the weights for a long time, so I couldn't test it until I got them

[inner ruin]

yo let me tell you that watermark ain't invisible

[soft bone]

@golden quarry recognize this at all? im using same system and config as caith

[golden quarry]

Well I certainly don't get that issue, but it's a system ram oom

If you aren't caching latents to disk, you might want to

I have 32gb ram, so I can just cache to ram

[soft bone]

guess i'll have to close my 26 browser tabs🤔

[golden quarry]

But it uses about 28gb total

[soft bone]

i have 32g as well

[golden quarry]

Well that definitely could be it lel

[urban fjord]

So caching latents to disc is necessary to save normal RAM but has nothing to do with VRAM?

[golden quarry]

Yeah

[soft bone]

thanks for the help, i couldnt tell what memory it was talking about

[golden quarry]

Cashing latents does conserve some vram, but it uses more memory instead

[soft bone]

but sometimes it tries to allocate 54 gigs

[urban fjord]

I don't have a big enough dataset where RAM is an issue, but I had some issues regarding cached latents on disc as I was using the same dataset several time with different settings. Though I can sadly not remember the exact issue I had.

[boreal bough]

5k images only goes up by like 2gb vram... so... yeah

last time I realized that dataset size even has a real impact, was on my 30k dataset attempt

had to tone down batch size for that

with caches its getting better

[urban fjord]

Where do you get your datasets?

[boreal bough]

having a few images at 4000x6000 or bigger will cause issues though. make sure to keep your images at or below 2048x2048.

[uneven dove]

yeah it sucks but violating the license is worse imo

[boreal bough]

an issue called data hoarding XD

[urban fjord]

To get that many it sounds like you need some decent source.

[uneven dove]

"i bought my dataset from Shutterstock" - first day on the internet kid

[urban fjord]

I thought the training script had a max size for images so high-res images didn't matter, they'd just be resized down.

[uneven dove]

depends on the training script

some of them just crop the fuck out of it

[boreal bough]

hydrus network -> all booru sites can be auto scraped
jdownloader 2 -> any image board, or collection on flickr
pixabay/CC sites -> good 'normal' images, to balance whatever dataset you have
google/torrent -> backup solution if its a rare source (overwatch artbook or similar)

[golden quarry]

@boreal bough updated the install guide and installer to set bf16 instead of fp16

[uneven dove]

just download LAION data with "NSFW" set to "Definitely"

guaranteed it's not in there yet

[soft bone]

its finally working goddamn

[boreal bough]

yeah... everything is in the laion database, to the point where its legit terrifying

[soft bone]

@boreal bough r u sure you meant 1e-3? thats like 2 minutes training time for 10 images

[boreal bough]

I'm aware of how insane it sounds... but it still hasn't failed me once 🙈

[soft bone]

didnt you say thats 30 minutes for 45 imgs??

[boreal bough]

only things that ruined things so far, was messing with dim/alpha, or captioning without trigger words (or having words like "feet" or "foot" in the captions - any body parts are taboo and will break all anatomy knowledge of the original base)

[boreal bough]

10 min for 30~50 images

[soft bone]

im trying without captions rn. some ppl on reddit had success

[boreal bough]

it can definitely work... but it's bad practice with sdxl, as you're guaranteed to get significantly worse results. hell, even booru tagging real life images has better results than no captions

[soft bone]

i use blip for my app

only problem is when it specifies someone as bald or black and removes that info

[boreal bough]

XD

[uneven dove]

by 'no captions' you mean 1.0 dropout?

[soft bone]

i mean no caption files

[uneven dove]

does it just use a trigger word then

[soft bone]

folder name, yeah

[uneven dove]

ooh ok

my trainer prepends the folder name an image is in, to its caption, which is optionally its filename

i do that so that when i'm captioning data, i do not end up with too many similarly captioned images

[soft bone]

i think captioning tools will have the biggest effect on future training. being able to tell the tagger which types of descriptors to include or omit

[uneven dove]

oh, that's easy

[inner ruin]

I found I need fewer images to get a decent LoRA with SDXL. Did y'all experience that too?

[soft bone]

i've seen the tool thats supposed to do that but for scaling on any type of human, it gets messy

[soft bone]

great news for me:)

[inner ruin]

I might be hallucinating fwiw

[boreal bough]

yes - but it backgrounds need to be watched out for

unless you're talking styles, those are super easy to get now

[soft bone]

super bad news for me. I wonder if background removal for subject training even helps anymore

[inner ruin]

how do you mean?

[inner ruin]

nah I'm still doing faces, they work fine for the validation images in wandb so far, but once I use it for inference it kind of doesn't truly capture

[shy kelp]

is there a way to make the background not blurry

[boreal bough]

you can simply caption the background correctly, then either not prompt for it, or even negativ prompt it
but if you ignore tagging it, the other words absorb it

[shy kelp]

even if I use negative in any of the bots it still shows up blurry

[boreal bough]

many prompts cause blur. what are your positive prompts?

[shy kelp]

ok lemme try less prompts

[boreal bough]

anything that usually has blur causes it, like "cinematic" <- obviously cinematic shots have insane bokeh

[grizzled warren]

does it help with eyes?

[shy kelp]

look

prompt:beautiful ginger woman style:Photographic negative_prompt:blurry, blurry background

blur

[eternal fog]

I imagine the photographic style includes blur

[shy kelp]

ohhh

whats a good style for photorealistic no blur

[eternal fog]

Anything like photographic, movie still all that stuff have some depth of field effect

[shy kelp]

ohhh

[eternal fog]

Try not using a particular style in the bot and stuff like sharp maybe

Apparently for some reason "taken by iPhone" gets rid of blur

Or so I've heard

[grizzled warren]

might also get rid of sharpness ;D

i'd put bokeh in the negative, but chances are the style overrules it

[soft bone]

its looking like I'll need 100+ epochs for faces

[shy kelp]

what is bokeh

[soft bone]

https://tenor.com/view/bokeh-effect-yellow-gif-24342376

[shy kelp]

oh

[grizzled warren]

also, a lot of relatively bad retouching techniques involve blurring the skin, because they rely on dumb frequency separation

[shy kelp]

its still blurry ugh

[uneven dove]

looks like DeepFloyd

[grizzled warren]

it's an effect created by shallow depth of field when you open the aperture in a photographic lens.

well... keep in mind the bot doesn't use the refiner yet

[uneven dove]

DeepFloyd ginger

[grizzled warren]

and the base model might not be trained to output a lot of fine facial details because of the refiner's existence

[soft bone]

pseudo says refiner is going away

[shy kelp]

its xl 1.0

[uneven dove]

i hope it ain't, but the one we have will likely exist for a long time

[shy kelp]

oh

[uneven dove]

@visual glade are you allowed to demonstrate img2img with 1.0 for us yet?

[shy kelp]

even clipdrop is giving blur

[grizzled warren]

what makes you think it does? The original announcemet regarding the bot here? Or I missied something?

[uneven dove]

discussions from this channel with the devs, and Emad's twitter

[grizzled warren]

oh

well

I hope the refiner will be compatible with 1.0

[uneven dove]

the base model wasn't like, trained to incompletely denoise images, if that's what you meant earlier

[grizzled warren]

cause it does help at times

[grizzled warren]

that's not what I meant

I meant it could not be sufficiently trained to output extremely fine details, like skin textures and whatnot, because the refiner can actually handle that.

[uneven dove]

but it can do them

base is a little derpy but i have no idea where you guys are getting this "can't do details" from

[grizzled warren]

and partially noisy output merely is an option that helps to save a little bit of time with the refiner. neither refiner not the leftover noise is mandatory

[eternal fog]

Base on it's own can do loads of detail, it just fucks up small things like eyes

And then the refiner usually sorts those out

[uneven dove]

eyes are pretty complicated

i see no issues here

[uneven dove]

Model: SDXL Base
SDXL Refiner: Off
Resolution: 1152x768

maybe you're running it like guidance 7.5?

i use no negative prompt, btw

[eternal fog]

Talking about eyes, this is what the refiner can do to help

[uneven dove]

which one is the fixed one? the glowy on the left or the triangular iris on the right? lmao

[eternal fog]

I mean the ones on the left are clearly broken

On the right they aren't perfect, but they look like actual eyes

[inner ruin]

funny that when I add a LoRA the image quality takes a big dip – everything looks a bit shittier

not sure why. Still messing with parameters though but default diffusers looks bad

[uneven dove]

here's a set of funky eyes i got but they're not that bad as the ones you had on the left

[elfin cobalt]

Well, are you sure the LoRA's good?

[inner ruin]

haha no, but if you have parameter suggestions I'd appreciate it

[uneven dove]

default diffusers set up uses Euler 😦

[upbeat summit]

the colors are really nice

[inner ruin]

ahhh should I use something else?

[uneven dove]

i use DDIM

prompt= a stunning portrait of a 1985 movie elder-teen standing in line at a grocery store
seed= 1505469711
guidance= 8.2
guidance_rescale= 0.0
steps= 20
resolution: 1184x664

SDXL Refiner:
strength 0.65
num_inference_steps 30
guidance_scale 7.5
aesthetic_score 10.0
negative_aesthetic_score 2.8```
^ for the SEdit / img2img style diffusers, these are fine parameters

[eternal fog]

What does setting Aesthetic score to 10 actually help with

I noticed setting it lower on horror type pictures helped a bit with the feel

[uneven dove]

this is what i got for those

[grizzled warren]

guidance - you mean Aesthetic score guidance on the refiner? positive or negative? or an actual CFG scale on the base being so low?

[elfin cobalt]

Not very much right now, but AIUI it's basically what the RLHF was trained into.

[uneven dove]

we're talkin about the base, Erilaz

[elfin cobalt]

So you get pictures more like the average of what people liked. ...in 1.0.

[uneven dove]

naw the RLHF was for their internal data and charts

the LAION aesthetics score is what went into the conditioning input value.

[upbeat summit]

man, pseudo. you are showing screencaps of movies from my favorite era 😉

[uneven dove]

Elder Teens R Us?

[urban fjord]

Thanks

[upbeat summit]

hehe

[uneven dove]

i love it when they get a 40 year old to play a high school student. it makes it so much more relatable and immersive

[west breach]

Morning

[grizzled warren]

oof. never even though of unironically setting the CFG scale on the base that low. thanks, gotta try that

[uneven dove]

yeah when you do CFG 1.7 through the base and then 7.5 through refiner you can add prompt keywords like 'colourful' and 'balanced contrast' to help bring it back to life

[eternal fog]

I've found cfg low on the base, high on the 2nd pass / refiner seems to work well

[uneven dove]

6.0 isn't low enough on the base though imo

you can't really straddle that uncanny valley there. it's either 1.7-3.0 or 7.5-8.0

you freaks and your middle CFGs can keep your horror shows 😄

[eternal fog]

I've been running 12 on the refiner and it seems to make the details pop a bit more

[uneven dove]

lmfao

yeah i bet it burns them to a crisp

[eternal fog]

But I'm using dpm++ 2m SDE at the moment, which might be different to what DIMM does

Nah it doesn't, not at all

[uneven dove]

stochastic samplers are a bit different, more forgiving but less accurate

[eternal fog]

I can try it on DIMM and see if it deep fries

[upbeat summit]

I'm back to dpmpp_sde_gpu. still my favorite

[eternal fog]

It's slower though, I want SPEEEEEED

[upbeat summit]

yeah, that's true. but the image just converges nicely for me

[elfin cobalt]

Sell your house, buy an H100

[eternal fog]

implying I own a house

[prisma peak]

dpmpp_2s_ancesterial is too slow. High quality, but very slow. With V1.5 I could create an image every few seconds

[elfin cobalt]

I'm back to using dpm_adaptive a lot, actually. It seems to do a better job getting rid of leftover noise.

I should try using that as a third pass maybe

[prisma peak]

But then again, V 1.5 was low quality at times

[eternal fog]

Yeah, on DIMM it deep fries

[prisma peak]

What would be a good choice? I want something that is fast, but not compromising too much on quality. It doesn't have to be super high quality.

I want to do more AI animations

[elfin cobalt]

dpmpp_2m?

[uneven dove]

you have to use the same sampler on both models

just in case that's what that is

[eternal fog]

I was, but the CFG was 12

[west breach]

this is what the refiner is getting at 13/20. It's got a lot of work to do

[eternal fog]

DPM doesn't care about it, DIMM seems to

[uneven dove]

@west breach like hell. if you squint, they mint

[sharp robin]

What ur trying to do?

[uneven dove]

partial-diffusion

[west breach]

I'm just showing what is going on. I don't think this is leaving the refiner to help sharpen up the end result, it's left with way more work than that