✨｜sdxl

[boreal bough]

yep! cause that uses 5 separate files. but the core tagging process uses only the single one 🥲

[heady vale]

yeah I stop at 40

[glad fulcrum]

I need SDXL generated images of WOMAN for my training on my gf

[boreal bough]

WOMAN

[west breach]

why not generate them then?

[glad fulcrum]

is there any link with that done?

well I don't know how many should I use either now that I think about it

[nimble heart]

going around asking for woman is a good way to end up on a list buddy

[boreal bough]

there's no guide for sdxl yet - on how to do it properly.
There are a few bad guides for teaching only the face, and nothing else.
Or you can train the body - but get other faces on the body 🤣

#priorities

[trim orbit]

just go for it and figure it out on the fly. its actually kind of easy

[nimble heart]

when you say "get other faces on the body" I just imagine elon musk's head on some plastic doll asian girl body

bonus points is she has absolutely stuffed up milkies

[boreal bough]

I'd joke about using 2 loras - but that's still a bad idea unless you've got your captioning down to a science

[nimble heart]

sometimes I wonder what those civit.ai guys put in their datasets

[lean knot]

We can't even train batch size 2 without gradient checkpointing?

Damn

[boreal bough]

?
all depends on many variables

[lean knot]

At least I've had no luck

[boreal bough]

like what vram, what kinda training, and more

[lean knot]

24GB of VRAM

512,2048 res

[heady vale]

GPT 9000 is alive

[azure oxide]

aw caith you never responded to the sec tutorial guy, i had my 🍿 ready

[lean knot]

From my quick testing it seems about batch size 1 without gradient checkpointing and batch size 4 with it

[boreal bough]

#✨｜sdxl message

[nimble heart]

it depends on sampler too. Some sample twice per iteration which kinda throws off the numbers

[boreal bough]

link broken XD

[lean knot]

That's weird, I'm using use_8bit_adam too

[boreal bough]

10~12 is the theoretical maximum on 24gb vram with adamw8bit
I use 8 cause I still like using my pc

[lean knot]

with gradient checkpointing in your case? Or how?

[boreal bough]

#✨｜sdxl message

these settings should get you there.
except you can now use fp16

(fp16 wasn't working in 0.9 - when I wrote that message)

[lean knot]

Thanks

[lean knot]

I'm not seeing big differences. I'm using bf16 and catching latents to disk, unet only

I guess I should stick to gradient checkpointing

But even that was around 23GB of VRAM and bs 4

[boreal bough]

bs 8 should give you around 18~20gb vram without overhead

[lean knot]

on a linux PC that I access through ssh

[boreal bough]

sure you vram cache is empty, and you're not running comfy or A1111 in background?

[lean knot]

And has a usage of 99MB of VRAM without anything on it

it's really weird

[nimble heart]

is it just me or are the 2M samplers totally broken

[glad fulcrum]

Is this okay, ? or I need to uninstall those modules? I don't understand what modules are that and if I can brake something..

[boreal bough]

check your install. you probably missed something like xformers, diffusers, bits and bytes, or something

also cudnn

[lean knot]

dim or network alpha matter much for memory?

It may be that

[nimble heart]

this for me or him

[lean knot]

I'll check the logs

[upbeat summit]

I don't use them. wasn't so happy with it - but probably just my settings. I had some nice images in the beginning with 2M but I'm on dpmpp_sde_gpu normal / karras now

[boreal bough]

was meant for dream

[nimble heart]

I don't mean like "not nice" I mean like totally broken

[boreal bough]

since linux and servers often dont work with the default install

[nimble heart]

dpm++ non-2M version

[upbeat summit]

ok... steps, cfg or broken 😄

[boreal bough]

what in the world XD

[nimble heart]

all settings same except sampler

[upbeat summit]

it almost looks like you had return with leftover noise at the end on

[boreal bough]

I'm on dpm 2m, and getting good results

[nimble heart]

I don't though

[boreal bough]

using the timestep for extra details on skin & background

can be removed though, since difference is very minor

[vast narwhal]

That's lack of steps

[lean knot]

Enable xformers for U-Net
import network module: networks.lora
create LoRA network. base dim (rank): 64, alpha: 32.0

Seems to be using xformers and bitsandbytes

I'll do a test run with bs 4

[boreal bough]

cudnn working? cause without that bigger batch sizes dont work

[lean knot]

mmm it's using cu117

It should be cu118 for torch 2 right?

[nimble heart]

but fucking how? Literally using the default workflow with a 2M sampler. No advanced node, no refiner, no return noise, etc

[lean knot]

CUDA SETUP: Highest compute capability among GPUs detected: 8.6
CUDA SETUP: Detected CUDA version 117
CUDA SETUP: Loading binary /home/bot/miniconda3/envs/kohya/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda117.so...

[boreal bough]

checks out :/

odd

you using kohya with gui?
send me your config file. I'll check if some weird setting is added or missing

[boreal bough]

try mimizukaris setup with 2m, see if it happens there as well

you can apply my minimal changes from the screenshot afterwards, to get a bit more performance out of it

[lean knot]

I'm still with the old command line and variables. The only think I had to change was the catching latents, because if you use 1024,1024 by default the max res seems to default to 1024 anyways

accelerate launch --num_cpu_threads_per_process $num_cpu_threads_per_process $repo_path/sdxl_train_network.py
--pretrained_model_name_or_path=$pretrained_model_name_or_path
--train_data_dir="$image_folder" --output_dir=$output_dir --logging_dir=$log_dir --output_name="${train_name}-${versionname}"
--train_batch_size=$train_batch_size --unet_lr=$ss_unet_lr
--max_train_steps=$max_train_set --lr_warmup_steps=$lr_warmup_steps
--use_8bit_adam --xformers --mixed_precision=$mixed_precision --persistent_data_loader_workers
--network_dim=$network_dim --network_alpha=$network_alpha
--shuffle_caption --keep_tokens=1
--caption_extension=".txt"
--lr_scheduler $lr_scheduler --min_snr_gamma=5 --network_train_unet_only
--resolution=$max_resolution --min_bucket_reso 512 --max_bucket_reso 2048 --enable_bucket
--save_every_n_epochs=$save_every_n_epochs --save_model_as=safetensors --save_precision=$save_precision --seed=$seed
--network_module=networks.lora

[boreal bough]

here's mine for comparison

accelerate launch --num_cpu_threads_per_process=2 "./sdxl_train_network.py" --enable_bucket --pretrained_model_name_or_path="A:/models/SDXL 1.0/sd_xl_base_1.0.safetensors" --train_data_dir="A:/Datasets/Concepts/nier/2B" --resolution="1024,1024" --output_dir="A:/Datasets/Concepts/TRAINING" --logging_dir="A:/Datasets/Concepts/TRAINING" --network_alpha="1" --save_model_as=safetensors --network_module=networks.lora --unet_lr=0.001 --network_dim=8 --output_name="2B_v1" --lr_scheduler_num_cycles="40" --learning_rate="0.001" --lr_scheduler="constant_with_warmup" --lr_warmup_steps="50" --train_batch_size="8" --max_train_steps="35800" --save_every_n_epochs="2" --mixed_precision="bf16" --save_precision="bf16" --seed="1234" --caption_extension=".txt" --cache_latents --cache_latents_to_disk --optimizer_type="AdamW8bit" --max_data_loader_n_workers="0" --keep_tokens="1" --bucket_reso_steps=64 --mem_eff_attn --flip_aug --shuffle_caption --gradient_checkpointing --xformers --network_train_unet_only

since I also use cli

oh right, buckets can escalate if you have too many of them

no issue if you have a few buckets, but each one does add a small amount of vram

[polar epoch]

When using loras and SDXL, would you need lora between both base and refiner?

[boreal bough]

I try to do all my loras with up to like 2~4 buckets - since that has multiple benefits

[candid walrus]

do you have your scheduler set to karras? that helped me on that type of artifact

[boreal bough]

base only. refiner is currently not compatible

[nimble heart]

karras fucks up the refiner when using return noise

[trim orbit]

loras help the base a lot and the refiner often isn't needed. if you want to use one with a refiner you'll need to specifically make one. i've used the refiner on loras too for the last 5 steps and its actually fine

[lean knot]

Did you get to check if it catched the latents with the correct resolution? Because I had to add min bucket size and max_bucket_size

[candid walrus]

i'll have to take your word for it as i'm only using the base atm

[lean knot]

otherwise the upper max was 1024 for some reason

Now it's this with repeats:
bucket 0: resolution (704, 1408), count: 10
bucket 1: resolution (704, 1472), count: 5
bucket 2: resolution (768, 1280), count: 70
bucket 3: resolution (768, 1344), count: 30
bucket 4: resolution (832, 1216), count: 320
bucket 5: resolution (896, 1152), count: 10
bucket 6: resolution (1024, 1024), count: 5
bucket 7: resolution (1152, 896), count: 10
bucket 8: resolution (1216, 832), count: 25
bucket 9: resolution (1280, 768), count: 5
bucket 10: resolution (1344, 768), count: 140
bucket 11: resolution (1408, 704), count: 60
bucket 12: resolution (1472, 704), count: 85
bucket 13: resolution (1536, 640), count: 40

[boreal bough]

eh yeah sry, that config from that job was without buckets. I also have to add them like you did

[candid walrus]

how much vram are you hitting with this?

[boreal bough]

on my specific setup - 19+overhead

[nimble heart]

1st pic is using the default steps "Kitten". Can't really see artifacts but just reducing the steps to 30 without changing anything else for the 2nd pic you get artifacting really bad. 3rd pic is unipc 30 steps.

[trim orbit]

the artifact is that the cat is brain damaged or hd a stroke

[boreal bough]

XD

[upbeat summit]

more like infected with VAE

[nimble heart]

I feel like the mimizukaris workflow uses so many steps the artifacts just slowly fade away

[boreal bough]

yeah. umm gimme sec - I have an amazing image I made to proof you right

[nimble heart]

but when you use an actual sane step count like 30 it shows fucked up

[trim orbit]

it got the VAEds

[upbeat summit]

hmm... I'm thinking. I had this too a week ago... I'm looking for the metadata

[boreal bough]

bit big - but it shows exactly what you're experiencing

I just take the performance hit and go for 66 steps now

[nimble heart]

yea I think 2M samplers and LMS is bugged in comfyui

[dapper current]

For 2M are you using Karras or Exponential?

[nimble heart]

maybe lms is just shit tho idk never used it

[dapper current]

I always do with those

[nimble heart]

normal

[dapper current]

Try Karras

Especially 2M_SDE

[nimble heart]

karras ruins return noise for the refiner

pretty sure exponent does too

[dapper current]

You have to set both base and refiner to the same scheduler

[nimble heart]

Y E S

and it's fucked up

with karras

[dapper current]

Hmmm

[boreal bough]

lms was always a step hungry sampler

[nimble heart]

I'm not crazy I've tried just about every combo by now

[boreal bough]

keep in mind, multiple setups used to rely on 100~150 steps in 1.5

[trim orbit]

ok well, somethings broken, so like, double check

[dapper current]

I'll give it a whirl when I'm back at a PC

[nimble heart]

I use my Principled node which physically does not allow you to have different samplers/scheduler for refiner/base

[boreal bough]

its why euler a, and other ancestral setups became so popular, because they run damn fast

[candid walrus]

so what is everyone using for sampler/scheduler on a base/refiner setup?

[boreal bough]

#✨｜sdxl message

check yourself!

xD

[nimble heart]

euler/normal. debating switching to either ddim/normal or dpm ++ sde / normal

[vast narwhal]

Use this workflow

[nimble heart]

No

Caith just confirmed the results my node is definitely programmed correctly

[boreal bough]

20 steps 😂

[candid walrus]

sad racoon

[nimble heart]

I'm not about to render a bunch of 90 step test images or whatever to show that the artifacts are invisible if you put in 3x the horsepower

gonna go eat

[boreal bough]

fyi, I rendered 2500 test images to check what works best with my loras 🤣

[candid walrus]

yeah it looks like _sde without the 2m is the best at the low steps when using the refiner

[lean knot]

@boreal bough Thanks for the help! It seems like I can do batch 8 with gradient checkpointing now. Added this:

--cache_latents --cache_latents_to_disk --gradient_checkpointing --mem_eff_attn --bucket_reso_steps=64

[vale eagle]

I always use 100% base on 2m sde karras

[boreal bough]

if you run base -> refiner
then you're essentially handing the refiner a chaotic unfinished image
meaning refiner does more - which has its positive and negative parts.

but for many artstyles & photos, refiner is just way too biased, so 2m sde stops being an option, since we want the base to do exactly 80% of the work

[vale eagle]

I think refiner kinda limited the output to its biase. Why would we use the refiner when the target is out of the refiner scope? If we throw away the refiner, the SDXL two stage architecture has no point.

Would we train the refiner to expand its scope to solve the biase issue?

[nimble heart]

If you're using 2m karras ofc the refiner is gonna look like garbage. The karras scheduler ruins the partial diffusion technique and 2m has noise problems anyways

if you're addicted to 2m karras coming from a1111 try unipc / normal. Does similar things but isn't broken

[boreal bough]

ah no. not what I mean, I was only referring to control. Refiner is an expert at the last 20% of image generation - so ideally we want him to focus on exactly that - not more nor less.
2M SDE causes irregelar noise patterns which the refiner wasn't trained for. So it can end up causes some glitches to persist, and at the same time cause certain refiner bias to pass through, when it does too much work. (but on a positive note - it will significantly add detail if it works - albeit the details won't be varied nor very original.)
"photograph of a brown haired woman" essentially always creates the same face for the woman, if the refiner is given enough steps & chaos to do its thing. (this is a bit of an extreme example - but the thing is that you won't know if bias is happening or not)

[nimble heart]

I prefer the refiner @ 15% tbh

sometimes it overrefines @ 20%

[lean knot]

I'm not up to the with the conversation, then which samplers are recommended?

[nimble heart]

turns nostrils into eyes

[lean knot]

I come from A1111 using sde karras

[boreal bough]

#✨｜sdxl message
decide yourself!

[nimble heart]

that's without the reinfer right ^

[midnight shuttle]

Seems there is a market for a really good GAN just for making people. SD is a diffuser which is somewhat good at many things but not excellent at anything. Instead of base and refiner maybe the next model should be art model + person model.

[boreal bough]

yep - but refiner only adds 'details', it doesn't like change composition

[lean knot]

https://tenor.com/view/afraid-scared-wtf-help-what-did-i-see-gif-20070435

[boreal bough]

that's what finetunes are for. SD is really good at people, just bad at generating the exact same person again and again

[nimble heart]

maybe you should open an issue on comfy for the 2M + SDXL compatibility issue

just upload that picture and it'll speak for itself

[boreal bough]

but is it a comfy issue? O: didn't even consider that till now

[nimble heart]

Maybe

Comfy would know if it is or isnt

[boreal bough]

thought it was just the fact the weights got readjusted + pruned model (less accuracy)

[autumn forum]

cool side effect of the lora im making. if you crank up the values. you get very liminal pictures...

[boreal bough]

noice

[nimble heart]

maybe if @visual glade is here they'd know why 2M samplers are broken on XL
#✨｜sdxl message

[nimble heart]

SDXL 0.9 breaks with 2M as well

so I assume its an sdxl config issue

[autumn forum]

also thanks to masslevel for helping me with this but my lora also improves coherence by a ton.

[boreal bough]

meanwhile I feel like my lora guide is turning into 90% FAQ and 10% guide 🤣

[midnight shuttle]

I don't see how it could be a Comfy issue. Seems to be more of a side effect of how the samplers work.

[autumn forum]

nothing wrong with that! faq can be a guide.

[vale eagle]

Do you released your guide? I am very interested in.

[nimble heart]

it's up to the maintainers to implement the samplers. IIRC some samplers on a1111 are kinda broken.

[autumn forum]

before and after. look at the tiles on the ceiling

[nimble heart]

not saying comfy's is but maybe 2m needs a compatibility patch for SDXL

[boreal bough]

yep - I'll post a prerelease version here to get a bit of peer review in, then it will go up on reddit & civitai as well

will take a bit though, as I'm doing a comprehensive guide - that mentions all the different kinds of loras that can be done with sdxl, and a full walkthrough of each type

[nimble heart]

always fun seeing commits like this

[boreal bough]

comfy: "I release and check sdxl chat if there are complaints."
also comfy: "there were no complaints again. nice."

[wicked frigate]

you mean 2m_sde?

[nimble heart]

both

[wicked frigate]

cause 2m_sde and lms are just bad samplers

[nimble heart]

dpm++ 2m and its sde version

[wicked frigate]

regular 2m should work well (with karras scheduler)

[nimble heart]

with normal scheduler it produces super broken images on SDXL.
karras scheduler doesn't work well when using the refiner

[boreal bough]

yes and no. half of the community is using a variation of sytans workflow, which isn't compatible with karras

[nimble heart]

sd 1.5 you can use dpm++ 2m normal without any goofy ass artifacts

so I assume it's an sdxl base model compat issue

[wicked frigate]

i mainly use dpm++ 2m, without issues

[brazen patrol]

my 3 mains so far with testing ate Euler 32+8 (all normal), 2m_SDE_GPU 48+12 and uni_pc_bh2 48+12

[wicked frigate]

i do not use sytan's workflow

[boreal bough]

same. but it assumes above 50 steps to get rid of the artifacts

[wicked frigate]

... i also rarely use the refiner at all lol

[nimble heart]

run it with the normal scheduler on sdxl base for like 20 steps

turns everything into stained glass

[brazen patrol]

I am using Comfy's basic Base+Ref

[rich coral]

Is there some sort of 'Interogate Clip' image reverse lookup thing that is more applicable to SDXL?

[boreal bough]

Vit-H model

works especially well for real life photos. works okay for most artwork/renders

[rich coral]

Hmmm, is that something I can use on a site?

[boreal bough]

can't help there - as I run it locally

[upbeat summit]

dpmpp_2m_sde karras 20 steps (just base)

[wicked frigate]

oh yeah. 2m x normal is pretty fucky. But karras is fine.

[vale eagle]

just use the base is ok.

[rich coral]

Oh well maybe I can do that too then, do you call it in comfy or a1111 or is it its own webui or pythony something something?

[west breach]

I tend to use simple for everything

[boreal bough]

a1111 + "interrogator" extension

[rich coral]

whoot, you rock!

[upbeat summit]

dpmpp_2m_sde normal 20 steps (just base)
uff 🙂

[boreal bough]

should look like that once you have it setup

[azure oxide]

wait why does karras break on sytan's?

[lean knot]

Shouldn't that be 40 to compare it to karras?

[nimble heart]

don't know. It just makes the refiner not work well

[boreal bough]

because of how noise works

[nimble heart]

mabye it denoises in a weird pattern that the refiner isn't built for

[boreal bough]

karras does things differently - which is good when it works alone, bad if you try to interrupt it midway

[brazen patrol]

So what is the test? 2m+karras at 20+5?

[nimble heart]

yea. I use karras on all my 1-pass stuff

never when refining

[azure oxide]

does sytan do anything funky with noise? his workflow looked fairly simple to me

[nimble heart]

his workflow stops the base before it finishes denoising and has the refiner finish the noisy image

which is how the refiner is supposed to be used iirc

[zealous horizon]

Rainbow colored nodes in Comfy 🙂 ❤️

Because those default colors just don't satisfy me

[nimble heart]

yea in the latent preview it always looks more or less fine till the last step

[boreal bough]

instead of 1+1/3
it does something like 2/3+1/3
and by doing that it both saves time, and adds detail as the 2/3 is just the right amount of chaotic left

[nimble heart]

think he reduced it so it's not 2/3 anymore for 1.0

only like 20% now

[rich coral]

awesome! works like a charm, thanks!

[boreal bough]

yeah now it's more like 4/5+1/5 - which checks out for how the refiner was trained

[boreal bough]

remember to use Vit-H in the clip selector!

[nimble heart]

20% is nice for photoreal

but on artwork it keeps turning my dragon nostrils into eyeballs

so 15%

[upbeat summit]

dpmpp_2m_sde
normal
38/40 steps

[nimble heart]

yea I think something near the last step rogers it

[west breach]

92% base, 8% refiner also works well

[nimble heart]

do 39/40

[boreal bough]

I've essentially gone the mcmonkey route and base only.
since that works best when messing with loras

only when I do film like prompts do I switch back to full setup

[azure oxide]

yea this happened to me a bunch as well

[upbeat summit]

dpmpp_2m_sde - normal - 39/40 steps
dpmpp_2m_sde - normal - 40/40 steps

[boreal bough]

XD I smell a conspiracy!

[upbeat summit]

I mean we had that issue in a1111 before

[nimble heart]

10/10 vs 10/9

[west breach]

try doing a few steps with a low cfg like 5.5, then passing to a base sampler with a high cfg of 9.5 - 10.5

[livid basin]

Hi, is there any advice on the VAE for SDXL 1.0?

Should we use the reverted VAE on hugginface?

Or use the default VAE in SDXL 1.0

[boreal bough]

yes this

0.9 vae

[upbeat summit]

use the reverted one which is the VAE of SDXL 0.9. the 1.0 VAE introduces some artifacts

[nimble heart]

uhhh I did dpm fast for 10 steps wtf

[west breach]

personally, I don't noticed the rainbow edges unless i zoom in past 100%

[upbeat summit]

it's beautiful

[queen ether]

Dors anyone know why sdxl makes a .npz file during lora training? What does it even do? Just curious

[west breach]

latent cache

[digital shoal]

any way to change font size on comfyui for my prompts?

[queen ether]

@west breach ooo

[nimble heart]

scroll wheel

[digital shoal]

that just zooms in and out, i was wondering if there was actually a way to change it

[brazen patrol]

2m_SDE Karras 20+5

[livid basin]

Thanks for the advice @upbeat summit @west breach @boreal bough

I'll overwrite my save file of SDXL with the reverted VAE on hugginface

[nimble heart]

post the actual output file not a copy-paste

[brazen patrol]

Since you mentioned dragons:

samurai with sword fights white dragon, epic, legendary, award-winning illustration

[livid basin]

Although I am surprised they haven't fixed the SDXL 1.0 checkpoint with the reverted VAE

Seems like it would make sense to just replace the VAE in the hugginface SDXL 1.0 repo as well, rather than just the VAE repo

[midnight shuttle]

Probably taking a few slow days after sprinting to get 1.0 released. They'll probably fix it once they definitely identify the issue.

It doesn't happen to everyone.

[livid basin]

Fair

The artifacts are not super obvious

[nimble heart]

not using the karras scheduler increases the detail the refiner makes tenfold

also used the non-2m sde since the 2m version is borked

[brazen patrol]

I don't normally use Karras. I was merely doing so to see if something strange happened

[nimble heart]

I'd say the dragon scales and his face looking like soup is pretty strange

[livid basin]

No love for the UniPC scheduler?

[nimble heart]

unipc is good

[west breach]

what does the Epoch setting mean here? I usually just set it the same as the max epochs

[nimble heart]

I didn't use it so the seeds would look similar

[livid basin]

Ah I see

[nimble heart]

unipc looks like this on that seed

[livid basin]

UniPC converges I think

[brazen patrol]

It is one of my top 3: euler, 2m_SDE_GPU and uni-pc-BH

[livid basin]

Does UniPC not converge when using the same seed?

As in, same seed, same pic

[nimble heart]

against itself yes. not against other samplers I don't think

maybe needs more steps

[livid basin]

Ah I see yah

I think UniPC looks similar to DPM++ 2M

[zealous horizon]

like so?

[vast galleon]

I got inpainting working if anyone wants to try. Just upload the image, take it to mask editor and save a mask, then run. Still rudimentary, but it works.

[digital shoal]

yeah :0

looks great

[upbeat summit]

thanks for sharing it. I will try it out for sure

[nimble heart]

50 euler 50 ddim 50 unipc 25 dpm2.

unipc uses dpm stuff under the hood iirc so they converge similarly.

[west breach]

what does the Epoch setting mean here? I usually just set it the same as the max train epochs

[vast galleon]

anyone know how to properly feather a mask?

[nimble heart]

vaseline

[zealous horizon]

I added it to my extensions collection: https://github.com/failfa-st/failfast-comfyui-extensions/tree/main
Enjoy

[livid basin]

Nice, seems like with enough steps UniPC & DPM++ 2M are very similar

[digital shoal]

bless your soul

[nimble heart]

that's dpm2 not 2m since 2m is broken on sdxl

[upbeat summit]

which one do you prefer?

[livid basin]

Ah, damn

I was using 2M with the diffusers library, and it seems to look fine

Perhaps broken in AUTO?

Or comfy

[nimble heart]

I'm using comfy

[upbeat summit]

so bright!

[west breach]

Can you rename a custom node without breaking an old workflow? E.g. If I improve a node and I want to use the same name, but don't want to break an old workflow that used the old version of the node

[nimble heart]

I've been using euler the whole time. Seems like the safest option.
Kinda want to play with the non-2m SDE see how the refiner likes it or probably doesn't

[brazen patrol]

so here is an image with 2M_SDE and 20+5

and this is with 25 and no refiner

I prefer the one with the refiner

[upbeat summit]

some of my workflows broke because a node was renamed. also sometimes when a parameter / widget is added, everything gets jumbled up

[nimble heart]

do a 9 batch of your 2m_sde (karras?) against non-2m sde normal so you can get some actual statistics going

[upbeat summit]

but I was going to suggest giving it an abbreviation or something. like HMN (humblemikey nodes)

[brazen patrol]

Look, as much as I sympathize with whatever issues you are having, I am not seeing it

[west breach]

I hoped there was a way to deprecate previous versions of the node without breaking things. Currently I'm adding V2 V3 to the newer nodes and just leaving the old ones as is

[nimble heart]

Look harder then.

[brazen patrol]

Certainly nothing that screams: refiner is broken, or 2mj is broken

[upbeat summit]

yeah I'm not sure if there is a better way to handle that. just experienced it from the user side

most just update their nodes

so if it breaks, it breaks

[nimble heart]

2M is objectively broken on XL
#✨｜sdxl message
and karras just makes the refiner worse but it has the nice side effect of making 2m not look broken.
try karras alone using euler and euler karras and it's very clear it breaks the refiner's denoising.

[upbeat summit]

WAS updated his save image node and than all my widget settings were like:
Quality = PNG
Format = 100
because he added a widget parameter

so the queue wouldn't run, before I fixed it

[brazen patrol]

So let me get this straight: Euler + Karras 20+5 and then plain 25?

[west breach]

yeah it seems to use the old order of the outputs even if you name the outputs

[brazen patrol]

(the second being with no refiner

[upbeat summit]

yeah that whole system could be improved. I guess some node guidelines could make sense - more standardization. but that's a lot of work. because that parameter jumble doesn't need to happen. there are ways to handle this

[west breach]

@upbeat summit with V3 you can use multiple styles and multiple loras by using the lora:lora_filename style:style-name syntax instead of selecting them. was starting to find the very long list of styles annoying to scroll, and you can only pick one

[boreal bough]

fun idea. comfyui could be turned into a factorio like experience if you're willing to program an insane amount 🤣

[upbeat summit]

that's great!

[west breach]

oh and you can select a image size or set a custom one

[nimble heart]

too late already did using your prompt. normal scheduler on left, karras on right, everything else is the same

all the small details look substantially worse on the karras

like look how the dragon's beard is soupy on the karras but clear individual strands on normal

scales are uneven in karras as well. Lumpy and mis-sized in some places

poor dragons were in the fukushima reactor meltdown

[brazen patrol]

Both with refiner?

[nimble heart]

yes

you can open the images in comfy to see

50 steps, 42 base 8 refiner euler sampler
only difference is normal vs karras scheduler

[boreal bough]

this just gave me a revelation... WE DONT NEED A REFINER TO REFINE XD
oh god why didn't I think of this sooner
Base + Lora -> Base + Lora (trained on only last 20% of steps) to effectively make a new refiner

[nimble heart]

but will it work

[boreal bough]

would insta solve face issues, and only doubles training time, rather than x6, while avoiding overfitting

[thin nova]

bold claims

[boreal bough]

it will work if we use it exactly like the refiner model - with denoise set to 20%

[upbeat summit]

@nimble heart the issue of samplers messing up an image on the last steps has also been round in automatic1111 for some time
https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/3483
https://github.com/AUTOMATIC1111/stable-diffusion-webui/discussions/10193

[nimble heart]

interesting

[autumn forum]

is there a purpose of fitting all training lora images to 1 size or if they are different sizes that okay becuase of buckets?

[crude sparrow]

whats the difference between the 2?

[boreal bough]

would also save vram funnily enough

[nimble heart]

top one borken get bottom

[boreal bough]

get 0.9. it fixed an error

[crude sparrow]

alr

whats the error

[nimble heart]

vhs lines

[crude sparrow]

dont think ive had it

[upbeat summit]

I would suggest getting the 0.9. the 1.0 VAE introduces artifacts to the image

[crude sparrow]

ok

[midnight shuttle]

They both contain the same unet. But the bottom one has an older version of the VAE. Some people were getting strange artifacts in images with the 1.0 VAE.

[boreal bough]

buckets work great. one size saves a bit of vram (but really not much) and makes working with big batch sizes easier

[upbeat summit]

used a SDXL init image?

[timid sonnet]

Yes sir

[crude sparrow]

should i still use the sdxl_vae?

[autumn forum]

mk cool. cuz i really dont want to resize these haha

[boreal bough]

you dont want bucket 1 to have 60 images
then bucket 2 & 3 with 1 image each

other than that, all good

[crude sparrow]

also what about these

[midnight shuttle]

The VAE is already contained in the model.

[boreal bough]

have at least as many images as your batch size, in each bucket

[midnight shuttle]

Look at the names and compare them with the other ones. I think you will understand.

[crude sparrow]

k

[timid sonnet]

@upbeat summit

[rich coral]

ahh very cool... I had some images I made with MJ using a text prompt and three image prompts. I ran the three image prompts through Vit-H and then copy/pasted all three results from the image prompts along with the text prompt into SD and it is giving me some really close looking results.

[upbeat summit]

your images look cool 🙂

[brazen patrol]

what is Vit-H?

[trim orbit]

i've heard that, but i've never really cared about it and my loras always come out half decent

[rich coral]

It is a model that you can select in an image interegator add on for A1111 that seems to work best for SDXL - https://github.com/AUTOMATIC1111/stable-diffusion-webui/discussions/7968

[trim orbit]

i think there's gradient checkpointing / batching code that solves those issues

[boreal bough]

it doesn't by defintion "break" anything, but it does duplicate those images until there are enough to fill up the bucket, introducing unnecessary bias

[trim orbit]

in my experience its safe to just throw all the images in a folder with whatever AR they got, an go for it

[brazen patrol]

ah cool. I am on Comfy due to performance @rich coral

[trim orbit]

yeah i looked into that. i think early versions of kohya did that before i came along

i can't find any evidence in the current code that it's doing that

i'm no expert coder but i can comb through and find out whats happening to some extent

[timid sonnet]

@upbeat summit check your messages

[upbeat summit]

it's evolving

[timid sonnet]

denoise is too high

[trim orbit]

https://github.com/darkstorm2150/sd-scripts/blob/main/docs/train_README-en.md#aspect-ratio-bucketing doesn't seem to mention the limit either. i've combed all through these manpages looking for that. i decided just to not care about it since i can't find any evidence of the theorized bias

my assumption is that early versions had it and it became engrained in the lexicon

[boreal bough]

kohya is releasing updates at a rate where no gui can keep up 😭

I can't just tell people "don't use the gui cause its old"

[trim orbit]

https://github.com/bmaltais/kohya_ss this stays up to date with the main branch

[boreal bough]

he's outdated :"D which is why I'm complaining

I don't blame him either

I mean kohya is damn productive

(dev branches - since main is ancient, and sdxl branch still has the typical flaws that are all fixed in dev)

[half ivy]

in simple terms, what is ascore?

[boreal bough]

a prompt (embedding) you dont see that is always applied to both negative and positive

[half ivy]

oaky im just not gonna play with the values and hold thumbs 😅

[rich coral]

I should be too. having to switch models in A1111 is painful but then again I do need SD Ultimate upscale... I don't think that is on comfy... although... hmmm, maybe it can do it

[trim orbit]

well i mean, if you mean dev branches, don't recommend those to people in the first place. lol. if people want those they'll find them. they should get experienced enough with main branch to have to want those

the kohya gui has a dev branch too. dev2

[boreal bough]

negative is perfect as is. positive value can be raised and lowered by 1, to achieve more real/creative results

[trim orbit]

i hadn't checked out the dev branches though. what's good about them for sdxl?

[half ivy]

i see o0

[boreal bough]

if they get the main branch, then sdxl training is partially broken

at the very least the sdxl branch

[half ivy]

so, a score positive 8 is more real but less creative than a 1

[trim orbit]

i dont know what partially broken means

[boreal bough]

funnily enough the other way around higher = more artworky, lower for more photo like

[half ivy]

tx

[boreal bough]

I saw one merge from sdxl to main, back when sdxl training still had a lot of issues, such a sample gen, clip completely breaking, lots of errors, and performance issues

[trim orbit]

i use main. works for me fine. no errors. finishes without nans. /shrug

i been doing it that way since about a week before 1.0 dropped

lora training.

an 2 TIs

[boreal bough]

dev has TI working, clip working (though I still wouldn't recommend it), sample generation, speedups, better bf16 support, full bf16 support

[nimble heart]

auto dev?

[trim orbit]

i have all that in my branch too.

full bf16 only works on linux with updated bits&bytes last i checked

https://github.com/kohya-ss/sd-scripts/commits/main i don't know what you mean by main is ancient either. it gets updated daily

[vast galleon]

No need for control net with this workflow. It creates an edge detection mask and blends with a handmade mask in mask editor.

[boreal bough]

it was before 2 days ago. I see that he's adding all the new updates now though. cool 👍

[trim orbit]

no but i mean, i've never left main and i trained a ti like, over a week ago

on the gui

while 0.9 was out

[boreal bough]

well TI was always working. it just wasn't working as intended

[trim orbit]

hmm. i still don't see any good reason for recommending a dev branch to people, or for saying the dev branch of the common gui isn't kept up to date.

it would be more encouraging to people to hear how accessible all these methods are now and today instead of saying they're not, when they are

[boreal bough]

I'm not gonna recommend dev to people. I'm currently running main to see if I run into any problems while I go through the 7 loras

but for context on how active it is
https://github.com/kohya-ss/sd-scripts/commits/dev

[nimble heart]

is it possible to reproduce a single image from a batch in comfy?

[trim orbit]

i'm not sure what this was about then. /shrug.

you don't need to make things complicated for new users. FOSS lets you see the development process. don't have FOMO over it

[half ivy]

/installmissingcustom nodes

[trim orbit]

i looked at that too. i just don't see much that's essential. and when i asked, you talked about TIs

[half ivy]

for this

[boreal bough]

I don't remember the exact situations, but multiple times in the last two weeks I got bugs in the kohya gui, that weren't present in the dev branch of kohya

so yes I'm pretty damaged from those experiences

[trim orbit]

are you sure it wasn't a configuration issue? you don't seem to have speant much time on it if you can't recall

[boreal bough]

my fix was to take the 'command' and then use it in kohya directly. which then worked as intended

[trim orbit]

switching branches usually comes with deleting the venv, right?

[boreal bough]

exact same setup

yeah, I always create everything from new

[trim orbit]

and why on the dev branch to begin with?

testing which feature?

[boreal bough]

cause I checked commits and often saw my exact issue addressed

turns out not just me having the issue

[trim orbit]

that's a big reset button that kind of allows a whole lot of confirmation bias in. a fresh venv could work elsewhere too. anyways, now it seems suddenly you do recall a lot about this process. see where i'm going with this?

moral of the story, why the dev branch?

[boreal bough]

yeah. cause I stopped writing on the side so I can properly focus on this conversation as you're clearly not letting it go

[trim orbit]

people recommding dev branches to newbs concerns me is all

[trim orbit]

i'm just saying, for me personally, it's been fine the whole time on the main branch kept up to date. same install i've always used. same venv even. it's kept on keeping on. i run the setup.bat file when ever i pull and the venv gets kept up to date. if there were a bug i were having, you'd best believe i'd figure out why before i deleted and rebuilt that venv. If only to avoid it in the future.

sometimes we can read "issues" and find our own. Then we delete the venv and rebuild it, albeit on a new branch, and it works again. 😮 . when really just staying on main and rebuilding the venv would've been fine.

i've had most issues when installing other guis and they screw with something in the global scope that breaks things.

i asked specifics about the issues you were talking about. that's all i ever wanted. not vague "i don't knows" and "lookmans"

[boreal bough]

fair enough

I did give kinda useless answers XD

[trim orbit]

i have a tendency to not beat around bushes. i'm bit of a bushwacker kind of guy

https://tenor.com/view/the-bushwhackers-bushwhackers-bushwhacker-luke-luke-luke-williams-gif-22093472

[boreal bough]

🤣

[trim orbit]

chicken signifies nothin it was just first good bushwacker gif i picked xD

[timid sonnet]

what setting do i use in khoya ss to train SDXL models? 1.5?

[cerulean bobcat]

SDXL in automatic1111

[crude sparrow]

offtopic but does anybody have any good controlnet workflows?

cant figure it out

(comfyui)

[trim orbit]

there is none for sdxl far as i know

hope i'm wrong

[crude sparrow]

thats in beta i think

was asking for pre xl modeles

[trim orbit]

this is #✨｜sdxl

[crude sparrow]

alr

[trim orbit]

sorry dont mean to argu i'll go

[timid sonnet]

@upbeat summit

[vast narwhal]

I really like the inpanting process, thank you. I want a node to do batch tries, you know where and which node i can change?

[vast galleon]

thank you! Was node suite has an image batch loader. Probably switch load image node with that.

[vast narwhal]

Not for the input but for outputs, i want several results at once

[vast galleon]

you would want to set up multiple samplers and image nodes each set to their own seed

[pallid sail]

Trying to use SDXL to make some cyberdeck concepts for a physical one I want to build. Fun to dig into what sort of future tech images live in the model

[pallid sail]

ooh, what resolution did you generate that at?

[late marsh]

Morning from the depths!

[vale eagle]

@visual glade Sorry for bother you, but I don't know who should I ping for the question. After having multiple conversations with different users, the custom fine tune of the base model should have custom refiner to achieve the best result. The lora which trained on base also should has a refiner version. Otherwise, the original refiner would ruin the base lora output. The original refiner has it limitations and it is not universal. How should we train the refiner? Would SAI release a proper training tools? Might be my precondition is wrong, the refiner is designed for universal and we just misunderstand the workflow. Please correct me if we were doing wrong with base+base Lora->refiner workflow.

[young mantle]

Hey could anyone tell me how I can use SDXL? Please

[late marsh]

ComfyUI, SwarmUI, SD.Next, A1111 and all those countless websites

[late marsh]

That water looks extremely good! ❤️

[young mantle]

Uhh, and would I get better results if I run it local?

[wicked frigate]

most customizable results local, yes

better/best are relevant concepts

[timid sonnet]

ty ty 🙂

[wicked frigate]

clipdrop/dreamstudio might be faster or have quality advantages from using experimental new models

[late marsh]

At least from a far 😄

[timid sonnet]

@late marsh

[wicked frigate]

comfy's an odd choice to ping for training questions. Kohya trainer is the go-to trainer rn for most things, idk about refiner

[young mantle]

I see thanks, could u tell me how to run it locally? Or a source where it's explained

[wicked frigate]

https://github.com/Stability-AI/StableSwarmUI/blob/master/docs/Basic Usage.md

[young mantle]

Thanks!

[visual glade]

you can train loras on the refiner, they should be trained on the last 200 timesteps

[vale eagle]

I don't know who should I ping.😢

[visual glade]

but really it depends on what you are training

[young mantle]

Damn that's fire

[vale eagle]

thanks for the answer.

[timid sonnet]

the second one is amazing, the third one has a perfect photo filter on it

[high skiff]

99.9% sure I just cracked high res fix

[timid sonnet]

like as if it was edited in Luminar AI

[high skiff]

@visual glade got a better upscale workflow, and it only uses stock nodes!

[late marsh]

Thanks! I have some basic guidelines for prompt but the rest of it is just from One Button Prompt script's randomness 😛

[timid sonnet]

wewll they wowrk lol

[late marsh]

Yeah

[visual glade]

cool

[wicked frigate]

ooo? do share

[high skiff]

base vs 12.5% fix vs 25% fix vs 50% fix

base image

50% fix

2048x res too

sending to some colleagues to test

[half ivy]

hi i am here

[half ivy]

why do ya'll homies put 10k?

[timid sonnet]

10k*?

[half ivy]

yea

[timid sonnet]

because we set the total steps with an integer

[vale eagle]

just end with total steps

[timid sonnet]

and we set the start and end steps as welwl wiwth an integer

[half ivy]

okay thnx thnx thnx

[upbeat summit]

epic scene

[visual glade]

it's at 10k because that's the default and it gets ignored if it's equal or bigger than steps

[half ivy]

k thats what i was looking for thnx

[young mantle]

hey which do you guys recommend me the most, ComfyUI or SwarmUI?

[wicked frigate]

swarmui lol

[half ivy]

dissecting all of yr workflows 😆

[wicked frigate]

i'm biased but i'm also right

[hardy cipher]

be a big dawg and get comfyui. your powers will become limitless

[wicked frigate]

do you know what swarmui is lol

[half ivy]

@young mantle

[wicked frigate]

do you know what swarmui is lol

[half ivy]

nope

[wicked frigate]

you should really look at what swarmui is

https://github.com/Stability-AI/StableSwarmUI

[half ivy]

i dont trust that link bro 😂

[wicked frigate]

here's a hint

[wicked frigate]

literally a link to the stability official github

[half ivy]

jkkkkiiinnnggg

[soft zealot]

that is looking sweeeeet

[hardy cipher]

tbh I actually don't know a lot about swarmui. I was an a1111 peasant until recently

[half ivy]

amen

[wicked frigate]

SwarmUI = the full power of comfyui, including node editor, with the ability to use an easier UI mode (similar to auto in a way) and bonus power tools like grid gen, and also multi-gpu support for ultrarapid generation

[soft zealot]

why isnt SAIs official Motto " This is Stable Diffusion, This is The Way"

[hardy cipher]

hmm. I'll have to look into that. so I can do all the comfyui stuff though? it's not restricted?

[wicked frigate]

yes correct

[hardy cipher]

awesome. I'll check it out

[hardy cipher]

I do miss some of the features from a1111. but it just got too restrictive. I very much prefer the ability to put together a workflow. but I do miss some of the plug and play aspects. and using both is a hassle since my video card can barely handle running one of them

[vale eagle]

Would we have official extension manager for swarmUI?

[half ivy]

i feel like im so close to figuring out something good with comfy, just need to push through the pain and frustration a bit longer

[wicked frigate]

not yet. Waiting on, uh, community extensions to be built first

[young mantle]

What's ur videocard?

[vast galleon]

Improved inpainting controlnet workflow. still a mess, but it works.

[wicked frigate]

swarm is literally perfect for you

[vale eagle]

Great. Waiting for it

[young mantle]

help

the "install-windows.bat" doesn't do anything

[half ivy]

i am here

[young mantle]

it just closes without going further

[young mantle]

From the SwarmUI

I'm trying to install it

[half ivy]

i am not here

[young mantle]

hahahaha

thanks anyway

[magic dawn]

Is there already a tool to train refiner? Does kohya already support this?

[young mantle]

@wicked frigate

Are u here?

[wicked frigate]

hi

[young mantle]

hi!!!!

[wicked frigate]

did you install DotNET?

[half ivy]

https://dotnet.microsoft.com/en-us/download/dotnet/7.0

[young mantle]

yep

I did

Do I have to reboot my pc?

[wicked frigate]

try running launch-windows.bat?

[hardy cipher]

6gb 3060, lol

[wicked frigate]

uh, maybe, i don't thiiink so though?

[high skiff]

new high res fix workflow I am working on seems to be fast, efficient, have controlable levels of fix/faithful, and also considerably fixes faces

[hardy cipher]

does the job though. to a degree

[high skiff]

its not perfect, but

[hardy cipher]

can't really train stuff. but easy to do online

[young mantle]

Download The Install-Windows.bat file, store it somewhere you want to install at (not Program Files), and run it.

I'm following this step

but

it doesn't work

[visual glade]

it should

[wicked frigate]

try running launch-windows.bat?

[young mantle]

I don't have it

where is it?

[wicked frigate]

is there a StableSwarmUI folder?

[half ivy]

hair changes color?

[young mantle]

nope, it doesn't install anything

[pallid sail]

Anyone had longer generation times using the base 1.0 VAE 0.9 combined model than using the normal base 1.0 model?

[wicked frigate]

can you... open a command prompt window, and run install-windows.bat through the command prompt window?

[young mantle]

lemme see

[young mantle]

'winget' is not recognized as an internal or external command,
operable program or batch file.
'git' is not recognized as an internal or external command,
operable program or batch file.
The system cannot find the path specified.
'.\make-shortcut.bat' is not recognized as an internal or external command,
operable program or batch file.
'.\launch-windows.bat' is not recognized as an internal or external command,
operable program or batch file.

[wicked frigate]

ooooooo Okay.
Uhm.
First of all, are you, uh, signed in to an up-to-date windows 11 (or 10) computer? 0.o

[young mantle]

yep but I'm in a modded one

[wicked frigate]

oh. Well. Your mods broke important things? So... maybe don't do that?

But uh

https://git-scm.com/download/win if you install git manually that should resolve that

[young mantle]

alr I'ma install it

and then what I do next?

[wicked frigate]

then run install-windows.bat again

[young mantle]

Which should I use as Git's default editor?

[wicked frigate]

eh doesn't matter you're extremely unlikely to ever use that part lol

[young mantle]

I prefer to be careful XD

[wicked frigate]

defaults are all good, only 'add to path' needs to be on

[visual glade]

so, how long until you just start shipping a standalone?

[wicked frigate]

.> i have two things externally installed, they are the easiest things possible to install, and yet still somehow they're constant issues. Standalone might be necessary .-.

they're installed through winget aka the free automatically enabled built-in windows installer magic tool that just does it magically. And somehow Zutamy modded that out of his windows install???

aaaaaaaaaaaa

[young mantle]

hahah fax

I installed it

lemme see what happens

[wicked frigate]

:D

[young mantle]

oh

it's working now

[wicked frigate]

:D

[young mantle]

now I have

a StableSwarmUI folder

that's what u did say

and another .bat in my desktop

[sharp flame]

Any Comfy users know the right way to use the base/refiner models to do an upscale or img2img? Wondering if there's a way to effectively manage denoise since there isn't a denoise parameter

Am I supposed to configure the steps a certain way so that it makes less changes to the original picture?

[visual glade]

lower the start step of the base

and give it an image instead of an empty latent one

I mean increase the start step

[sharp flame]

Will the base be doing anything prior to its start_step? For example, if there are 30 steps total and I want to start the refiner at 20, and I set the base to start at 20, will it be doing anything?

[late marsh]

I see you got an inspiration 🙂

[young mantle]

haha

[timid sonnet]

@late marsh they finally took him down

[young mantle]

everyone is inspirating now XD

[late marsh]

Oh noes!

[young mantle]

damn the cops are amazing on it

[young mantle]

I did it all finally XD

Now the models are downloading

thanks!

[upbeat summit]

phew - what an adventure 😄

[wicked frigate]

er -- set the refiner to start at 20, base starts at 0 still lol

[sharp flame]

But the Comfy man himself told me to set the start step of the base, not the refiner

[wicked frigate]

base starts above 0 only if you're using base for img2img

[young mantle]

how do u think the model will work on a RX580 8gb?

[visual glade]

on windows? very badly

[wicked frigate]

iiddunno. I assume y ou're about to find ou-

oof

[late marsh]

Well, I'm just happy if my images inspire others 🙂 Take a couple more!

Just the raw output

[young mantle]

damn urs are awesome:O

[visual glade]

no but it won't be adding noise either if it's not doing any steps

[sharp flame]

Thanks, so effectively it boils down to how many steps for the base and then how many steps for the refiner. More steps from the base = more changes.
Is there a reason that if you do no steps with the base, the refiner produces a very blurry image?

[young mantle]

which do u recommend? linux?

[visual glade]

for your card it's going to be very painful on both

[young mantle]

sad to hear

[visual glade]

but if you can handle linux that's probably going to be faster

[pallid sail]

Finally managed to get a LoRA to load. Seems as though Diffusers SDXL might not support LoRA scale yet as I get an error about the scale property not existing in the __call__ function when using StableDiffusionXLImg2ImgPipeline. It normally gets added via cross_attention_kwargs

[visual glade]

it's just going to be painful because that card isn't officially supported by ROCm and and not sure how well it works unofficially

[young mantle]

uhh sadly, well I'll see how it keeps going

[vale eagle]

Just tried. Kohya script doesn't support the refiner training yet.

[sharp flame]

Posting an example of an upscale where I ran the base for 1 step and the refiner for 10 steps

Its definitely similar but its not quite as much control as you can get from the normal method of setting a low denoise

[half ivy]

dayum

[young mantle]

Nice one

[young mantle]

I finally could install it XD

Now I'm downloading the models

[half ivy]

LG!!!!

[young mantle]

I'm gonna sleep meanwhile

thanks for everything guys

[half ivy]

knollingcase SDXL

[molten gull]

how do you get steps and end_at_step as parameter in the ksampler ?

and how/where do i get that brown step-control nodes from ?

[vale eagle]

convert input as weight

[molten gull]

i dont understand, how do i do this ?

[vale eagle]

right click at the node

[molten gull]

ok, i found, thanks

do you know what the guy in that example is using in the brown nodes called "steps" ?

[wicked frigate]

that's a Primitive

[molten gull]

how do i make one in my comfyUI ?

[sharp flame]

Would it make sense to add a parameter that sets the amount of noise added per step by the base refiner? Wondering if that would allow for a bit more control over how much the picture changes (functionally a denoise param?).

[molten gull]

can one of you tell me how to make a primitive number in comfyUI pls?

[pseudo summit]

hey, dumb question, but is it possible to train an SDXL Lora in Comfyui? (automatic1111 doesn't work for me)

[molten gull]

you can use this here: https://github.com/derrian-distro/LoRA_Easy_Training_Scripts

[late marsh]

Here is a few tutorials from the inhouse dev: https://www.youtube.com/watch?v=AbB33AxrcZo&ab_channel=ScottDetweiler

[upbeat summit]

it's just a primitive node. it does it if you connect it to the steps pin of a sampler

[molten gull]

how to i make one in comfyUI ?

[upbeat summit]

double click -> type primitive or right click -> add node -> utils

[molten gull]

thx, that worked

[hardy cipher]

there are also several node packages with all sorts of numerical nodes

I need to figure out how to use all that boolean nonsense so I can automate things

[hardy cipher]

nice

[charred zealot]

I'm using stablediffusion-webui with DreamshaperSDXL model - first it generated image of this quality. And after some time it generates only images of low quality (second example for the same prompt). I haven't changed any settings/prompts, already tried to reinstalled venv. All other non SDXL models work fine and produce same nice results as usual. Does anybody know what may cause this and how to fix it ?

[half ivy]

im sure ive experienced that too on comfy

[hardy cipher]

are you getting any errors?

[west breach]

did you try dragging the good pic into comfy to make sure it can't reproduce teh same pic with all the settings exactly the same?

[hardy cipher]

sometimes when my images do that I'll look and it'll be just an endless stream of errors in the terminal window

[high skiff]

@wicked frigate ok, its officialy

I think i found high res fix for SDXL

[charred zealot]

it looks ok in terminal window

[high skiff]

the results are so far, damning

[hardy cipher]

you should post what you're working with. might be easier to figure out

[upbeat summit]

sharpness looks really good. it might get a bit too "hot" if you check the color levels

[charred zealot]

https://github.com/AUTOMATIC1111/stable-diffusion-webui/ - I'm using this gui , not comfy

[hardy cipher]

yeah, I'd pass that back through and blend it with the original

maybe overlay or something

[high skiff]

2048x images, reliable, high resolution, consistent, mended deformities to a certain extent, all stock comfy UI nodes, running in sub 40 second times on my 3080 from initial prompt trigger

[west breach]

if you drag the good pic into the positive prompt field, then click the little blue square with a white arrow button under generate to populate the settings, then try running again?

[hardy cipher]

I like to get the difference between the original image and the over-sharpened image, then use the result to do a dodge or burn on the original

[high skiff]

I am messing with the post processing, as the img2img style work I am doing tends to wash out the levels, and I am trying to use the built in comfy UI tools for it

[high skiff]

its not sharpened, its just more detail in general, its 4x the pixel count

[upbeat summit]

nice!

[hardy cipher]

fair enough. looks sharpened to me. but I'm not that smart

[high skiff]

still trying to find the right color levels ATM

also, you can choose how much "fix" is in your high res

I have tested from 10% all the way to 50%

with no considerable deformities

[hardy cipher]

what do you think the fix is doing other than sharpening and adding detail?

[crisp owl]

I assume you'll be putting it in your github once determined what's best?

[woeful patio]

does anyone have a preset .toml file for this?

[high skiff]

Base 1024x crop vs 12.5% fix vs 25% fix vs 50% fix

[high skiff]

yessir, and on the comfy UI wiki, now that its all stock nodes <3

[hardy cipher]

I see

[upbeat summit]

crisp!

[high skiff]

so as you can see, you can choose how faithful to the original it is, or how much it changes it

[crisp owl]

https://tenor.com/view/bruce-almighty-jim-carrey-beautiful-happy-smile-gif-4874848

[hardy cipher]

how does it work?

[high skiff]

All of my best finds always come from a leap of faith lol

[high skiff]

effectively, upsacale, downscale, special diffusion ontop, post processing with image x image layering, and thats about it

not much more to share ATM as I find better values for it

[charred zealot]

sorry It's my bad - it was different model used, still it's a mystery to me why SDXL results are that ugly with the same prompt ((

[high skiff]

its even very good at fixing faces

[hardy cipher]

nice. I'm a bit tardy to the party I guess. I should probably scroll up and read

[high skiff]

these are both crops, but you can see here

not perfect, but its a lot better

fixed her shoulderblades very nicely as well

even more extreme example here

by changing how much "fix" is in the high res

[hardy cipher]

I really go nuts with these nodes. I've spent years doing random weird things with photoshop so I can bring all sorts of things out of images beyond just the ai aspect

not so hot in the bottom left, lol

[west breach]

we were all stuck on the A1111 workflow before, plus any extensions we added. now we can experiment with all sorts of crazy workflows

I need a workflow to make me a coffee in the morning 😄

[hardy cipher]

I just need to figure out how to do more cfg stuff with comfortui

scheduling, rescaling, etc

comfort, lol. close enough

[high skiff]

did I mention my new high res fix has the potential to help text considerably as well? Only tested one time, but after seeing all of this... I have hope

[hardy cipher]

you're going to be like that dude from csi miami when he has the techs upscale license plates

I used to make fun of that

[hardy cipher]

mr cyberpunk over here

[normal bridge]

I am using Lykon's Greg Rutkowsky LoRA, but using his way of upscale fix results in the upscaled and refined version looking much less crisp... any suggestions on how to get it as crisp as the base version?

[high skiff]

it is genuinely only a matter of tim elmao

[heady vale]

from pencil to 3d model

[naive goblet]

The refiner often produces a blotchy effect

[west breach]

looks like the base and refiner steps aren't lined up right in the 2nd pic

[hardy cipher]

I can't decide what to think of the refiner

I certainly can't say I know how to use it correctly

[normal bridge]

what do you mean

[west breach]

the blotchy look

[hardy cipher]

I guess that's more steampunkish

[heady vale]

the future of shooting your weapon.. lol

[west breach]

Projekt Red still have a few bugs to iron out

[sharp flame]

I'm not sure if there's a good way to use the refiner as an upscaling method, using it with the base XL model changes too much and using it standalone doesn't seem to work (just makes the image blurry)

maybe I'm not understanding something

Obviously I can just use the base model with a regular sampler and just lower the denoise, so I'll probably just do that for now and only use the refiner during the low-res image gen

[high skiff]

I am messing more with my upscale workflow... and Midjourney should be genuinely scared

for realism specifically

2048x2048

[urban breach]

i can feel the fur from here

[high skiff]

if all goes well, expect this 1.0 workflow out tomorrow

on my github, and the official comfy wiki

[west breach]

I found if you use the sd ultimate upscaler, you set the tile resolution to 512 or 768. The refiner model is more coherent at lower resolutions

[heady vale]

yep 768 is the sweet spot

[sharp flame]

What exactly does sd ultimate upscaler do? Its a tile based upscaler?

[high skiff]

yes

cuts the image into tiles at a more standard size, does them individually, then stitches them together

its what I first used for mmy SDXL upscale workflow, but my new method is faster and yeilds much better results

[sharp flame]

Would the sd ultimate upscaler be doing base+refiner for each tile?

[hardy cipher]

sytan, do you have any suggestions on how to implement the refiner model properly in comfyui? I can get it to work, but I have no idea if I' doing it "right"

[normal bridge]

went for the black magic path I see

[heady vale]

AFAIK it only uses one model, but the image has already been constructed so the refiner is used to fill in the small details

[high skiff]

only refiner

[naive goblet]

https://comfyanonymous.github.io/ComfyUI_examples/sdxl/ drag and drop the top image into comfy ui

[high skiff]

my method on my github uses my mixed diffusion, which uses part of the base model and part of the refiner to do one image, rather than running the refiner ontop as a second pass

a more refined version should be out late tomorrow, if all goes well

[hardy cipher]

thanks. you know, I went to that page and forgot I could drag images in like that. and couldn't figure out what was informative about looking at those bottles

[high skiff]

@boreal bough Wanna play with my beta upscale workflow, see how it works? :p

[half ivy]

yes

[normal bridge]

looks very comic like

[high skiff]

lol

[hardy cipher]

I've been using the searge nodes to do the sampling, but not sure if it's necessary

I feel like I've barely scratched the surface of what I'll be able to do with comfy. and I'm probably doing things very inefficiently at this point. but the nodes satisfy me so I must keep going

[woeful patio]

anyone know how to fix this within kohya?

No data found. Please verify arguments (train_data_dir must be the parent of folders with images) /

[ornate elk]

❤️ sdxl

[boreal bough]

use https://github.com/derrian-distro/LoRA_Easy_Training_Scripts/tree/SDXL
using that you can just use normal folder structures and can ignore the typical kohya folder structure

[heady vale]

wake up Samurai, weve got lego to build

[ornate elk]

whats ur cfg? seems like it's too high

also that's the first time I've ever seen a recognizable signature...that's definitely the cyberpunk logo

[heady vale]

that one did blow out a bit, same as Im using for all images so its OK

[ornate elk]

yeah I am seeing more variation in that effect

[hardy cipher]

I got a signature on one of mine earlier

[normal bridge]

whats your prompt on that I am curious

[hardy cipher]

hmm, let me check. probably some randomness,

[normal bridge]

my explosions are floating away :(

[woeful patio]

can you train loras with 8gb vram?

[hardy cipher]

that's one I made on dreamstudio. I like to mess with that on my phone when I can't get hardcore on my computer.
Positive
"Bach's infinite analog synth symphony in a subatomic singularity
Negative
Mundane, earthly, blurry, smudged, cgi, low resolution

[normal bridge]

wow interesting xD

[hardy cipher]

I thin I used one of the filters too. but I have to see which

[normal bridge]

ah don't bother too much, don't want to give you work ^^

[hardy cipher]

either way. it was either the line art or analog photograph style. those two might be my favorites

analog photograph can make some really uncanny stuff

rainbow goblins in analog

[heady vale]

tele-goblins

[peak dove]

I'm using Scott Detweiler's Base-then-Refiner setup at ComfyUI - here is the Base photo, and then the Base+Refiner Photo - much much more detail and definition in the Refiner!!!

I'm just going to add a second Refiner stage - so the whole ComfyUI process will be Refiner-Base-Refiner - don't know if it'l be any better ... but here's hoping 🙂

[high skiff]

first successsful 4096x upscale

extremely reckless on a 3080 lol

but on a 3090, very pratical

base 1024x image crop

vs the 4096x crop

so much detail

[half ivy]

stahp

[high skiff]

zooms into the atoms

16.666 million pixels

my kinda demonic number

curious how practical the middle stop is

[normal bridge]

somebody said atoms?

[high skiff]

trying 3072x3072

ok, still too high res for 10GB VRAM

maybe 2662

WOOO

looks like barely lmao

9.8GB VRAM

that is 7x the pixel count of base 1024x1024

[dense chasm]

it's interesting when run diffusers cuz it has two prompts chain to different text encoder and the combination can result in mind blowing images

[high skiff]

my mind is indeed blown, not sure if in a good way tho lol

[dense chasm]

LoRA training for sdxl is always CUDA out of memory even run with deepspeed VRAM saving.

[hardy cipher]

need to get an a100

[dense chasm]

idk,but the previous diffusers Lora traning can run under 8GB GPU VRAM.sdxl can run on A100 40G for sure

[hardy cipher]

I tried to get gpt4 to tell me how to build a laptop with dual a100s

[normal bridge]

hope you got a strong lap then xD

[ember cargo]

i use this trainer too and it's my favorite so far

[hardy cipher]

it started going on about how laptop cases were too small. I told it that I wasn't looking for reasons it couldn't be done, just to tell me what to do. I think I'd need to carry around an external lithium battery to keep it running

[ember cargo]

batch size 2, activate gradient checkpointing (that works with my 3090)

[woeful patio]

Does a higher batch size in lora training use less or more vram?

[hardy cipher]

surely more, right?

[ember cargo]

more

[woeful patio]

cool, thanks

[ember cargo]

this

[normal bridge]

my dude got some serious medical condition

[woeful patio]

is it even possible to train lora with 8gb of vram?

[hardy cipher]

that's just the moon's gravity distorting your view

[heady vale]

theyre doing the moon walk

[vital wolf]

What's strength model and clip for loras in comfy? Do I set value for both? I know how loras work in a1111, can someone 3xplain?

[ember cargo]

dunno, i used to have a 2070 super but never trained with it

maybe with some heavy gradient checkpointing

[woeful patio]

Yeah it just keeps giving me this

[ember cargo]

https://github.com/victorchall/EveryDream2trainer/blob/main/doc/LOWVRAM.md

try with 4 accumulation steps

go higher and se what works or not

[hardy cipher]

so concensus is we shouldn't be running loras on the refiner model?

[ember cargo]

i run lora on base model yeah, then i run the refiner for 5-2 steps

[woeful patio]

will do thanks

[ember cargo]

in your case always batch 1

[hardy cipher]

I just ran the same thing through without refiner loras. it's slightly different, but can't tell if better or worse, lol

[peak dove]

Detweiler's triple Refiner-Base-Refiner - works, but as I do a lot of art - I'm not convinced that the extra Refiner-B4-base does any good? 🙂

[hardy cipher]

where is this detweiler triple refiner?

[peak dove]

My RTX2070 is always maxed-out by A1111 (8Gb VRAM). ComfyUI is much better at memory management. SDXL is making RTX2070 GPU's look underpowered!

[copper inlet]

I managed to start the training on my PC (4070 12G) by setting
set PYTORCH_CUDA_ALLOC_CONF=max_split_size_mb:128

[lilac wren]

I have a 2070 Super, you have to use --medvram

[hardy cipher]

if you dropped the cfg by one or two I think it'd pull out more detail there

[peak dove]

Yes, --medvram is no use on my rig - always warned High Memory Usage - VRAM AVailable 8Gb - VRAM Used 8GB 🙂

[hardy cipher]

or maybe not, but maybe

[dense chasm]

idk,the diffusers official page sdxl lora training run on A100 40G

[woeful patio]

Even on 10 steps it doesnt work lol

[lilac wren]

Without medvram, you won't be able to go very high in resolution with A1111.

[peak dove]

I have abandoned A1111 for SDXL - now using ComfyUI 🙂

[vital wolf]

What's strength model and clip for loras in comfy? Do I set value for both? I know how loras work in a1111, can someone 3xplain?

[ember cargo]

then you might just can't, training at 1024 res eat a LOT of resources

[peak dove]

I have just implemented Scott Detweiler's triple-process of Refiner-Base-Refiner - and its producing some stunning results

[hardy cipher]

we know, bud

[vital wolf]

Share the workflow?

[pallid sail]

Wait what, how does that work

[hardy cipher]

I guess I should google that name

[crystal forge]

anyone have problems with ComfyUI running out of system memory? my 32GB has a tendency to run out and --gpu-only runs out of VRAM (RTX 3060 12GB)

[peak dove]

If I post a photo, and if it holds its metadata - you can drop it onto the ComfyUI Desktop?

[hardy cipher]

nice

[crystal forge]

yes, the .png has the metadata

[peak dove]

This was done using Refiner-Base-Refiner - I hope the graph comes up when you drop it onto your CUI Desktop?

Here are some more Triple-Process R-B-R

As a general Rule - the base photos look as if they need a good sharpening; while the refiner photos are crisp and clear

[pallid sail]

Are these use different denoising percentages for each step? Like 0%-20% then 20%-80% and 80%-100% or something similar?

[crystal forge]

no metadata for me at least, try to save and upload the .json file instead

[amber pecan]

why is it that the service can do portrait similarity and the SD has to be trained on the face?

[peak dove]

OK, will save the 'json

Detweiler Triple-Process-R-B-R https://drive.google.com/file/d/1LLIY5IGwGFQ7fwiAg1VpysPD6g2Xorpo/view?usp=sharing

Hope it works?

[ember cargo]

my compact LoRa workflow with Ultimate SD Upscale (you may need the aspect ratio custom node)

[crystal forge]

seems good (did not have SDXL 0.9 models downloaded, so had to replace with 1.0)

[peak dove]

Oh! Silly me, I should have used 1.0 as well --- I will try again 🙂

[hardy cipher]

hmm, I guess I'll try it out

[normal bridge]

anybody know why I get an error on the ksampler advanced?

[hardy cipher]

could anyone tell me what "clip text encode SDXL Refiner does?

[crystal forge]

I see you have batch size of 10, anyone know how to reproduce a single image (i.e. say I liked image nr 5, how do I re-run the workflow for only that image?)

[hardy cipher]

some of this stuff still goes over my head

[peak dove]

batch size 1 at Empty Latent Image - then at KSampler set noise seed to your own value, and change the setting (control after generate) underneath to 'fixed'

[heady vale]

drag the image into comfy. it will have all the settings for that image

[normal bridge]

nvm looks like WAS number nodes are a bit fucked

[lethal tusk]

What sampling method should I use for SDXL 1.0

[hardy cipher]

I think I read ddim might not work well. maybe just try a few out and compare

is it possible to modify existing nodes so I can do things like have seed numbers sync up in all the samplers? is there some obvious method I'm missing?

[peak dove]

KSamplerAdvanced

[peak dove]

Find its SEED and reuse that

[hardy cipher]

and what about iterative functions? is it possible to run it through the same sampler x amount of times? or is it easier to just copy the sampler and repeat it?

[lethal tusk]

im using a1111 not comfyUI

[lilac wren]

Just update A1111

[peak dove]

If you have Image Browser in A1111 you can find the SEED there

[hardy cipher]

gotta git pull a1111. it doesn't do that for you

[lethal tusk]

i got 1.5.1 the latest version

[vital wolf]

Dpm2sancestral, dpm sde, DPM 2m karaas, ddim, eular a

Dpm 2m sde

[lilac wren]

So you should have a error message, post it on pastebin ans send it there

[lethal tusk]

error message?

[lilac wren]

You were talking about Ksampler error?

[peak dove]

A1111 - go to Extensions/ find the URL and direct d/load Image Browser

[lilac wren]

TErminal?

[lethal tusk]

no

[lilac wren]

Ah, right

[vital wolf]

He was asking which ones to use

[lilac wren]

personnally i didn"t changed my habbits

[peak dove]

I got to admit I like 0.9 much better for artistic output - 1.0 is so faded and pastel-like 🙂

[hardy cipher]

if I'd known I could run it on my subpar 6gb video card I would have tried to get 0.9. but read it required 16gb

[vital wolf]

I have 1060 6gigs, and I could run sdxl 0.9 not pruned, I'm guessing so could you

[hardy cipher]

all good though. I'm figuring it out now. tbh I can make things at almost the same level with 1.5. but it takes a lot more, and there are a lot more misses

[woeful patio]

trying to create a more rugged look, not exactly what I'm looking for yet

[hardy cipher]

liminal spaces

[vital wolf]

What's strength model and clip for loras in comfy? Do I set value for both? I know how loras work in a1111, can someone explain? Man someone tell me

[upbeat summit]

well it's liminal for sure. very nice image degradation in the first one

[woeful patio]

Yeah using prompts like: hot pixels, extremely poor quality, low res, pixelated, worst photograph, dim light, Nokia camera photograph