✨｜sdxl

[spring fulcrum]

Family Photo 🙂

[clever verge]

Mesmerizing

[peak dove]

Thank you

[chrome flicker]

SDXL seems to do bad in anime character or scene generation, I hope it can have a special anime version with surprise performence like niji jour
ney。

[dawn field]

Sdxl today?

[peak dove]

2moro 26th

[paper phoenix]

Wednesday is 26th mate.

[kindred plinth]

Ideologically, I disagree

[peak dove]

When's day 26?

[paper phoenix]

same. I think days, rather being tied to factual labels and conventions should be allowed to speak their own truth. Tomorrow can be wednesday if it wants to be.

[peak dove]

What is closest Model I can use to get the MJ Niji look?

I want to get this look (this is an MJ Niji Creation)

[kindred plinth]

Exactly, forcing any specific day to use a single label no matter what is pretty fascist imo

[peak dove]

ComfyUI Prompt = beauty in the style of basquiat, rauschenberg, warhol, hambleton, haring, jessie makinson, richard burlet, inuyasha, john caple, victo ngai

[paper phoenix]

kinda looks a bit jamie hewlett

[peak dove]

n Ice

[indigo carbon]

also this is still without the refiner

its that thing from fortnite

[peak dove]

Niji -ish

[paper phoenix]

by niji ish do you mean the stylised background?

and geometry?

[indigo carbon]

time machine

[cinder elbow]

some questions:

1. what version is running on this discord? 0.9 that will be trained to become 1.0?
2. what are the settings that are used by the bot? steps, cfg... are them listed somewhere?
3. will 1.0 be released in few days or will be released 0.9?

thanks!!

[indigo carbon]

middle of nowhere

[kindred plinth]

1. they are running the latest version that they are still working on and isnt released yet
2. undisclosed as they are using the bot to test these parameters, each generation uses different settings probably
3. 1.0 will probably be released on the 26th

[proper sorrel]

https://huggingface.co/hakurei/waifu-diffusion-xl

[peak dove]

Hope this'll do something like Niji in SDXL?

[smoky patrol]

I love all of these

[peak dove]

Getting Niji-er and Niji-er (is that a word?!) 😄

[uncut steeple]

I kinda love this prompt in SDXL
(Forced Perspective:1.2) Unveil a microscopic society thriving within the creases of a crumpled piece of paper. The inhabitants, agile and minuscule, use the folds as paths, navigating the mountainous terrain with their ink-drawn vehicles. Their world may be minuscule, but it pulses with stories writ large

[barren olive]

hi can anyone help me to install art diffusion xl 0.9 on google colab?

[final sun]

Hello👋 Do you think that sdxl 1.0 would work with controlnet and deforum at the same time ?

[upper sage]

Best to head over to #🤝｜tech-support for help with installing/running models!
Lots of incredible people over there focused on exactly those questions to help you out ❤️

[indigo carbon]

happy birthday to someone

[molten breach]

Will there be a separate inpainting model of sdxl when it releases?

[trim orbit]

i made one in kohya's newest version and tested in in comfy. it works there. automatic tries to load it but fails.

[trim orbit]

deforum is just a script for setting up the next frame's generation so i see no reason why it shouldn't already work. People who are creating may just be holding off on releases because of the research license with 0.9.

controlnet will need new models trained

[trim orbit]

Cute thing to note about this project is they got permission to release it under the research license by Stability.

People who think stablity are trying to censor the model in order to oppress humanities and control populations, will conveniently ignore this massive gaping hole in the side of their argument. Or they'll see it and call it fine because it's gaping and they can't understand context.

[hoary saddle]

any chance you could remake that in 1280x512 version could ya? 😉

[ionic dragon]

sure

[hoary saddle]

was just trying to get something simliar on a1111 for the background of a blender scene i'm working on 🙂

[ionic dragon]

rendering at 1280x512 wouldn't get the best results as it wasnt trained at that res, please choose any on these

[hoary saddle]

cant even get an astronaut on deliberate lol

1344x768 would probably work fine

wanna stick it here

[ionic dragon]

@hoary saddle

[hoary saddle]

those are perfect! if you wouldn't mind me using one, i'll tag your social for credit and put it on the art itself friend

gorgeous

[ionic dragon]

yeah you can use, no need to tag me(i am uncomfortable sharing my socials)

[hoary saddle]

appreciate you kindly! came out great

[patent badger]

that's a lora of a dog? or just shiba inu prompt?

[indigo carbon]

prompt

no funny embeddings. SDXL is by far the best model even without silly embeddings.

[turbid osprey]

yo

does anyone know a fix to fatal: No names found, cannot describe anything.

[eternal fog]

If you want help you'll need to give a bit more information than that. We don't even know what your talking about.

[young tulip]

I've noticed that sdxl 1.0 seems to have gained very significant progress in the last couple days, no? Or is it just me? I have been judging many of the generated images every day and hopefully this will play a role in the improvement of the model. I'm sure sdxl1.0 will get better as we work together.

[vale eagle]

sdxl1.0 is better than sdxl 0.9 for sure. But I don't think they would continuously update the models during the past few days.

[sharp robin]

U see what u want to see. There was 3 models to choose from. Maybe there’s only 2 now.

And those two may be better than 3

[eternal fog]

Or they are tricking you and using SD 1.5 finetunes XD

[sharp robin]

But iirc they haven’t altered the models

[sharp robin]

Ez to test. Just tell it to give you text.

Would be hilarious tho

If they used 1.5 as a control of sorts

[azure oxide]

as of 4 days ago they havent changed anything, i dont think theyd change anything still

[azure oxide]

re:

[vale eagle]

I don't think there has 1.5 behind the bot anymore.

[eternal fog]

I was joking

But yeah people thinking stuff has changed just shows how bias works

They get a few images a certain way that they like

Then they get a different type of image

"It's horrible"

[vale eagle]

may be

[eternal fog]

You can go from a decent to horrible image quite easily in the same seed, just by changing things like CFG, Steps and resolution

[vale eagle]

It is quite hard to compare because you don't know what going on behind the bot. Even using the same prompt, you could get a better then bot image and you also could get a worse than bot image using sdxl0.9.

[hard fractal]

we're honing in some parameters.

removing ones that very clearly would produce bad images.

[eternal fog]

Is that parameters like the samplers, steps and cfg, or parameters in the models themselves?

[trim orbit]

but but but thats censorship! ||/s||

[indigo carbon]

WIP

[eternal fog]

Those people moaning about that sort of stuff i swear either haven't used the model. Or they are expecting some sort of hardcore porn generator for some reason.

[trim orbit]

they want the holosuite at quarks if you know what i mean

[clever verge]

Candy-style!

[clever verge]

I'm wondering if it's possible to write back to a field in a custom node, anyone know?

Selection here should ideally update height and width.

For flexibility reasons.

[wicked frigate]

man wouldn't it be nice to have a really convenient UI that uses comfy's workflows & engine but gave you any easy native link between aspect ratio/resolution with easy dropdowns n stuff, like this,

maybe with like sliders if you selected 'custom', yknow like the good ol

shame that doesn't exist and i just made these screenshots up, oh well

[wicked frigate]

both! model variant is one of the parameters that gets randomized

and workflow and etc

[upbeat summit]

I really like those!

[novel birch]

is there a new release date for SDXL yet?

[urban fjord]

There is a stage event the 26th, if it doesn't get released then we'll probably get a release date instead.

I hope the wait is worth it and the changes from 0.9 is actually significant.

[nimble heart]

be cool to automatically calculate the high res fix sizes by applying the aspect to the same megapixel count as 1024 ** 2. i made a python/gtk app to do that and it seems to do well.

guess I could probably just turn it into a node with 4 int outputs

[eternal fog]

I've made a multiplier for an aspect ratio node that was posted in here to do that.

[indigo carbon]

WIP

[eternal fog]

https://old.reddit.com/r/StableDiffusion/comments/158n1td/controlnet_sdxl_10/

In case people haven't seen

Shame it won't be ready for the 1.0 release though

[solar tusk]

amazing difference!

[nimble heart]

too late; already made my own

why have one way of doing things when you can have 17 incompatible competing standards

[eternal fog]

lol, mine was just a quick thing because I got sick of having to work out the multipliers

[nimble heart]

real baller move would be the ability to bake a custom node into a workflow so you can redistribute your workflow without them getting the big red

[eternal fog]

That's a little sketchy though

Embedding a python script into an image file

No thanks

[nimble heart]

it'd only be eval'd in comfyui

could also convert it to bytecode so it can't be accidentally interpreted

[eternal fog]

It would still make it more difficult for you to see what it did before running it, although it's not like most people actually look.

oooo

[nimble heart]

?

[eternal fog]

https://github.com/comfyanonymous/ComfyUI/commit/0240946ecff9527a1c39f705ae9d2324612f4cf7

[nimble heart]

git pull time

[eternal fog]

I'm guessing this is so we can do something similar to the [Thing1:thing2:0.5] that Auto1111 has

[nimble heart]

yea I was just asking comfy about that yesterday

or the day before

something

time

do you know if nodes can reference other nodes?

So you can for example make a node that wraps two KSamplers together

[eternal fog]

I think there's some custom nodes that lets you do things like that

[nimble heart]

I imagine you could shadow the input types and feed it through eval(f"NodeClassName{NodeClassName.FUNCTION}(...)")

long as you can import the class directly from somewhere

[prisma wyvern]

I'm running SDXL 0.9 ON Vlad 1111 and after 47min it produced black square 512x512 🤔👎 Now I'm trying 1024x1024 it says 1085min 😱 Anyone had same problem and how have you fixed it??? Btw I run it with different model and it made 1 image in about 40min 👍

[eternal fog]

What are you trying to run it on because that's obviously not working properly, unless maybe you are trying to run it off CPU?

[prisma wyvern]

Yep on cpu

[eternal fog]

Yeah, you aren't going to have much fun with that

[soft zealot]

any news on if any work on SDXXL has started yet ?👼

[soft zealot]

ooof

[prisma wyvern]

I'm not concerned too much with time, more confused why it produced black square??

[trim orbit]

precision problems. try using xformers

[thin nova]

does xformers actually change anything on cpu

i expect the only precision available would be float32

[soft zealot]

or theyve got that plugin that turns NSFW into lack squares ;o)

[prisma wyvern]

All I tried to test on was sticker, vector puppy🐶

Is it possible problem is with SDXL not running well on vlad or a1111?

[eternal fog]

Well it seems to work, woman crossed with frog/lizard man

[nimble heart]

I didn't see the actual parser for [token:token:time] in that commit

unless its earlier

[eternal fog]

It's a node

[nimble heart]

Ohhh

[eternal fog]

Two Text Nodes, 1 into it's own ConditioningSetTimeStepRange and then both those into a Conditioning Combine

However @visual glade I would say, these new ConditioningSetTimestepRange nodes. The values on them seem backwards.

In my head I'd have it so 0 is the start of the step and 1 is the end.

So for a full generation you'd have start on 0 and end on 1. Or if you wanted to stop half way through you'd have start on 0 and end on 0.5, but the way you've implemented it, it's backwards, where Start 1 means it will run all the way through all the steps and end 0 means it won't do any.

Might just be the way I think about it though.

[visual glade]

oh, yeah it's because the first timestep is 999 and the last one is 0

so that's why I see it that way

[eternal fog]

It's super confusing to me lol

Works pretty well though, even if I can't get into my head exactly where it's cutting in

[visual glade]

I guess I'll swap them then

[prisma wyvern]

Can I successfully generate images 256x256 on sdxl? When I run other models with 256x256 it takes less time 15-20min and than upscale with topaz

[eternal fog]

I mean it's up to you, I'm only 1 person, others might think the same way as you.

But having it the other way seems logical to me.

[visual glade]

no SDXL only generates images close to 1024x1024

it works with other res but 1MP is the optimal amount of pixels

[eternal fog]

I'm sort of thinking that you'd be like ok I want this conditioning to start at the beginning so 0 and i want it to stop at half way, so 0.5 on end

[prisma wyvern]

Which package will work better on CPU and with what settings? I have Invoke 3.0, vlad, a1111 and ComfyUI

[sour obsidian]

dont you dare

I am the other person that likes it going from high to low haha

[eternal fog]

Lol that just seems completely backwards to me

I guess you could add a toggle, but I don't know if that would confuse people even more

[nimble heart]

node could have a tooltip explaining why it's backwards

[thin nova]

the debate between "noising" timesteps vs "denoising" timesteps

[eternal fog]

It's not the reasoning that's the problem for me. It just doesn't fit conceptually in my head when the numbers are backwards, so I find it really hard to work out where exactly in the process things are changing over.

[nimble heart]

create two primitives and title them the opposite of what they feed into on the timestep node

[azure oxide]

didnt we end up flipping clip skip notation around because the first way made sense behind the scenes but the current way makes more sense

could be a similar case

[visual glade]

it makes more sense from how it works to have 1.0 as the start but I guess people are more used to 0.0 becing the start

[azure oxide]

ya i remember this was a good explanation of how clipskip worked

[eternal fog]

It just completely screws with my head the way it is now lol

[visual glade]

I set the number for "clip skip" negative in comfyui because that's the only way it made sense to me

[eternal fog]

That makes sense I think because your using less clip layers.

The only reason this is an issue is when I'm quickly trying to figure out where to make something cut in and for whatever reason my brain won't do it when it's backwards

[visual glade]

it's because in python and other languages -1 is the last element in an array, -2 the before last element, etc...

[eternal fog]

I think if your thinking about it from a purely mechanical stand point your numbering makes sense.

But I think if your just looking at the nodes and wanting to make sense of it procedurally, it's confusing.

I think it's either that, or it's the "Start" and "End" words that don't make sense

Because in the code you have "start_percent" and "end_percent"

[visual glade]

ok, I see the controlnet extension has it 0 to 1.0 so I guess I have to change it to match since that's the standard

[eternal fog]

yey, easy enough then lol

You can get some really cool effects with this though

[visual glade]

if you update now I swapped it

[soft zealot]

having random thoughts/ If there are 11 distinct steps from 0 to 10 and there are 10 distinct steps from 1 to 10 why is the halfway for both 5 ?

[eternal fog]

Cool

[inner ruin]

is clipdrop using 1.0?

[sweet wyvern]

do we have a release date yet ?

[lusty raptor]

i'm with you here, everything else in the UIs out there count up from zero

[nimble heart]

we're waiting for Emad to finish Tears of the Kingdom, so about 2 days or so

[eternal fog]

He's changed it so it matches, so all good.

[lusty raptor]

just saw that

[eternal fog]

Going to need some playing with but you can get some cool stuff with this

[nimble heart]

rule 63 saiyan leonidas

[eternal fog]

noooooooo lmao.

I mixed the dog and the woman and it turned him into fur trim on her top 😢

[eternal fog]

That new node is awesome, I can finally get weird coloured skin to work

[primal nimbus]

sd xl release on 26th event?

[heady vale]

the last announcement said "in a week or so", without a specific date

[shy kelp]

okay so where are we with the release candidates?

[autumn forum]

apperently we are close becuase the bots got updated and we were told to make sure we pick which we like better becuase the outputs will be almost identical to each other

[shy kelp]

where can I go to vote?

not that it will make much of a difference

[autumn forum]

the bots.. lol

[heady vale]

in the bot channels like #1100170312106127410

[shy kelp]

voted for a few but the images look close to each other.

[autumn forum]

i literally said it would be, pick the best in your opinion. we are deciding on which sdxl1.0 model we get

[lusty raptor]

it does, just gotta generate/vote on more than a few

[sudden cliff]

A node allowed this? What node?

[eternal fog]

ConditioningSetTimestepRange, Comfy added it today

[eternal fog]

Interesting

What a strange way to use a bot, what's the purpose here lmao

[upbeat summit]

yeah

[eternal fog]

Do they know this isn't midjourney

They can't even spell it properly

[upbeat summit]

Thank you for the clean-up

[trim orbit]

comfybox i finally got around to making it work. cool.

[upbeat summit]

this can be used to do prompt editing?

[eternal fog]

It lets you merge or change prompt mid sampling

So if you've used auto, similar to the [thing1:thing2:0.5]

[hard fractal]

oo, what's this new ConditioningSetTimestepRange do?

[upbeat summit]

yep prompt editing ; ) nice

[hard fractal]

oh that!

[eternal fog]

It's a lot more customisable than A1111 too

[hard fractal]

nice!

[hard fractal]

I'm so done with A1111

[upbeat summit]

one of the features I really like. did some crazy stuff with prompt editing TI embeds in SD 2.1. a must-have feature for me 😊

[eternal fog]

The nodes in comfy can be a bit cumbersome at times, especially when you start doing more complex stuff. But there's comfybox that can get around some of that. And from what mcmonkey has been saying I think he has something cooking in that regard.

[autumn forum]

i agree! i was a die hard a1111 fan with 1.5 but sdxl made me change and im glad i did!

[upbeat summit]

I just looked through the github commit. have you tried it yet? can you do token weighting within a TimestepRange?

like [(foo:1.2):(bar:0.75)]

[eternal fog]

No it's a node. So you set up two text conditioning nodes, put them through the time step nodes and then combine them.

[upbeat summit]

I see. so it's not working like {wildcards} in a prompt field. you need to build that for every range you want to add.

much more flexible but could get wild if you prompt edit the whole prompt I guess 😉

[eternal fog]

Yeah

[upbeat summit]

maybe a custom_node might be able to integrate it in the same field

if one would want that

[eternal fog]

It's a mess on my workflow, but like this

[upbeat summit]

I see. need to do some conditioning experiments I guess 🙂

[gleaming owl]

is SDXL for auto 1111 out yet? like, not via web tokens, but for me to use my own GPU with"?

[upbeat summit]

afaik only in the dev branch. so not officially

[gleaming owl]

damn

will be real soon though right? Like next few days or something?

[upbeat summit]

that only automatic1111 knows

[tribal magnet]

[urban fjord]

Maybe the new node is useful, but way too tedious to actually use, while I'm glad to see ComfyUI getting more features, the nodes is holding it back instead of allowing for more complex usage.

[upbeat summit]

I'm sure somebody will build a prompt editing node combining some of the features. I would if I would knew python better. A couple of experiments are already out there. a node interface is hard to beat if you want to be able to do custom workflows without writing code.

[urban fjord]

The problem is that you kind of need to write custom nodes to make it work well and now you're writing code.

[upbeat summit]

yeah and I think that's okay. you can build a really simple workflow but you can also build more complex ones and try out so many interesting ideas that you won't able to do in tool with a static workflow.

I mean it's great there are many different tools. I started with ComfyUI a couple of days ago and I really like that I can build a workflow to my liking

[urban fjord]

The issue is that it is horrible to use for any complex workflow, I've tried. It becomes rigid and failure prone. If you want to develop a custom workflow in a good way your only option really is code.

[upbeat summit]

sure you can always go more hands-on with the tech. but from the current time I've spend with comfyui I'm thinking differently about a lot of the stuff I've been doing before. I mean it's great that there are other tools so you can pick and choose what is important to your own workflow.

[high skiff]

Alright, I think I might be stopping my SDXL research indefinitely

[upbeat summit]

nothing is perfect and I can see limitations in every workflow

[urban fjord]

GPU failture Sytan?

[upbeat summit]

why?

[high skiff]

Yeah, it's back to failures

[upbeat summit]

damn

power supply?

[high skiff]

It was working fine, one of my temps was hot, I meticulously took it all apart and reapplied everything, even optimized the mounting pressure, now the card is just as useless as the day I got it

Rebuilt it 4 different times

Sick and fucking tired of this card

[upbeat summit]

is it your new or old one?

[high skiff]

New GPU

[upbeat summit]

hmmm... damn

[high skiff]

I really wish I could just sell this card off to somebody else and have it be their problem, but that would be shitty of me

[urban fjord]

Sure there are, but I cannot replicate any workflow in ComfyUI I've used in Auto1111 with it's basic nodes.

[hearty ginkgo]

But like u should

[high skiff]

So I should just scam somebody else by selling them a GPU I know is faulty? I couldn't be that shitty of a person

[urban fjord]

Return it to seller and get your money back

[high skiff]

Can't

[boreal bough]

paypal?

[high skiff]

Yeah, but I won't win

[nimble heart]

may as well try

[boreal bough]

180 days / as long as you have documented proof

[high skiff]

Already am, but it's just a waste of time

[boreal bough]

I think you wrote him about the issues

[high skiff]

I actually might be able to fight him now that he admitted negligence on his end

[upbeat summit]

if you bought it from a commercial seller the buyer has a good chance to get the money back in a paypal dispute. if you have send the money to a person it would be different

[hearty ginkgo]

Did you try a clean windows boot?

[azure oxide]

ya paypal usually sides with the buyer from what ive read and in my experience (twice) its true

[ornate elk]

is there a workflow for getting past the 77 token limit for CLIP?

[azure oxide]

idk if comfyui does it automatically but you can use the combine or concatenate nodes in there to achieve that

[ornate elk]

I am using automatic1111

[azure oxide]

it should be automatic in that case

[ornate elk]

it's strange, it's not an issue for anything but sdxl

noep

[ornate elk]

that's in launch.py?

[azure oxide]

oh its for sdxl and a1111 huh
ya thats a tricky one

[ornate elk]

ohh were you talking about 1.5 etc

[azure oxide]

nah the nodes im talking about are in the gui itself, you can create them

no just comfyui in general

[ornate elk]

gotcha

been using automatic1111 for about a year, but I've been away for a while...are otehrs catching up?

[azure oxide]

comfyui has always been toptier imo but from the looks of it, its coming out on top with how sdxl is turning out

[ornate elk]

orly I thought automatic has always been top dog

[azure oxide]

ya it was always the most popular

[ornate elk]

but again I've been out of SD practice for several months

[azure oxide]

a1111 was pretty slow to implement sdxl

a workflow for it i should say

[ornate elk]

it's not even really implemented

[azure oxide]

ya

[ornate elk]

it's clunky as

moreso than usual

[azure oxide]

we dunno how its gonna work* with the refiner model too so its gonna be weird

[ornate elk]

I mean I don't a1111 for teh elegance...I use it because it has so many things

wait is the "refine" process not actually a refiner model?

that guy

[azure oxide]

oh looks like he did find a way to implement it

from what i read in its early "sdxl compatible" debut, the refiner wasnt working

[ornate elk]

yeah. super annoying tho, you have to find the file and bring it back in by hand

like idk why you can't just copy it from the other side of the ui

[ornate elk]

with comfyui there's no 77 token limit for sdxl?

[spring fulcrum]

I really like ComfyUI a lot... I have been getting amazing results... It seems way more customizable and I think its the way to go

[ornate elk]

do you know if it has a 77 token limit for sdxl?

[upbeat summit]

it does

[ornate elk]

thanks

[upbeat summit]

here is more info by the developer of ComfyUI #✨｜sdxl message

[spring fulcrum]

So actually it can go above 75 tokens. It puts the info into blocks of 75 tokens and appends them..

It doesnt even average them either which is good because you dont see a drop in impact

[ornate elk]

I am very confused by that thread. First they say that it uses more tokens, then they say that SD can't handle more tokens, then someone asks about that, and there is no answer

[ornate elk]

I am talking about SDXL. @upbeat summit just said there's a token limit for it in comfyui

[upbeat summit]

the model has a 77 token limit. as other SD models before it. comfyui does split your tokens into blocks of 75 tokens and appends them

[ornate elk]

appends them how? if there's a 77 limit how can more be appended?

seems like it's not a limit if more can be added?

like, why would a user care about the difference between these things?

[upbeat summit]

there is some math to it. a1111 averaged your prompt out when you went over 75 tokens. comfyui does it differently

[ornate elk]

so this is implemented in sdxl as well?

[spring fulcrum]

most end users really wouldnt care or think about it... it just allows you to use longer prompts

[upbeat summit]

it works the same with sdxl in comfyui. I don't know what a1111 does with sdxl. but I would think they handle it the same as with SD1 and SD2

most users won't care. most users even use more than 75 tokens per prompt

[ornate elk]

yeah that's what I am trying to figure out here. I just want to know if I can use longer prompts, if there's an easy way to add this to a1111's implementation of sdxl

[ornate elk]

they do not, it's limited to 77 rn

(only for sdxl)

[upbeat summit]

well they will probably implement the same functionality they're using for the other model versions

[ornate elk]

off to dig thru the issues list ig

[upbeat summit]

afaik there is no problem using SDXL in ComfyUI right now. at least I'm using more then 75 tokens a lot 😉

[spring fulcrum]

I have some pretty long prompts in ComfyUI... All of the text from those 3 boxes count as 1 prompt

[ornate elk]

and maybe install comfyui...I assume there's no overlap between the massive amount of crap that needs installed

[spring fulcrum]

I actually think comfyui is easier to install and get working

[ornate elk]

oh and one other thing...there's a facility in comfyui to install it on a host and use any client right? just checking before I start to dig. my art slave lives in a closet.

ah yeah, I see, it's browser based, so this should be the case

[spring fulcrum]

you would have to set the command line arguement to --listen 192.168.1.43 for example so it doesnt use localhost 127.0.0.1 and you can point it at your monster in the closet

[hearty ginkgo]

depends on the your specs tbh

[ornate elk]

3080

[hearty ginkgo]

then you should be good

[queen ferry]

Will sdxl explode enough to make people shift from 1.5 to sdxl?

[ornate elk]

most of the annoyance of installing a1111 is just waiting for crap to download

[queen ferry]

Cause my database seems reaaaaally dumb right now.

[spring fulcrum]

I think so especially with custom models based on it. sure

[ornate elk]

tbh I never liked 1.5 better than 1.4

[queen ferry]

Damn.

[ornate elk]

but I like to make weird looking shit, not glistening perfect waifus

[queen ferry]

Alright time to make a lora list for SDXL now

Sigh.

[upbeat summit]

well it depends how experimental you are and if you want to try building your own workflows, but you can build some really interesting processes in comfyui that help you build very weird looking and cool shit 🙂

[ornate elk]

nice, I am keen

[queen ferry]

So comfyui is better I'm told?

I have been using a1111 for too long too long to know

[honest flint]

it depends on what you are wanting, comfyui is way more flexible, a1111 is just point and click

[ornate elk]

and yeah I like to get experimental. here's a barbie inspired one

[honest flint]

its like a dslr vs a phone camera

[ornate elk]

or rather barbenheimer

gen2 does some cool shit with these sometimes

[honest flint]

https://imgur.com/a/0m56ABF comfyui tends to start looking like this

[ornate elk]

im not a stranger to node graphs that look like this...making me wonder why comfyui isn't just a part of nuke lol

[boreal bough]

SDXL in nuke would be a dream come true

now that's a true node setup

[upbeat summit]

give it a try. I've been an a1111 power user for the last year. there are some QoL features and workflows you may have to rethink - some stuff is certainly different or not available yet, but I really enjoy building my own workflows. but that is just how I use it and work with - mostly txt2img.

[formal jasper]

Is rtx 3060ti compatible for sdxl??

[ornate elk]

nice. yeah I've always gone for bulk which is why I am a bit hesitant. What I like about diffusion most is the ability to rapidly give me many options

but I will try

[honest flint]

for me i mainly use a1111 for my goofing off, training prep, etc.... comfyui i tend to use if i have a few images I need to need to iterate on and get super high quality proofs

[upbeat summit]

yeah. I've always wanted a node based tool for AI images after using nuke, blender etc

[ornate elk]

substance

bet adobe's working on a diffusion implementation in substance

but it will be crippled like all commercail models

[upbeat summit]

goofing off you can do 😉 you can still do the training in a1111 or kohya_ss. also you can always use both tools

[upbeat summit]

make it part of the adobe cloud subscription damn it

[honest flint]

comfyui is a bit combersome to use when im goofing off... sometimes i just want it to work and a1111 does a good job of that

[ornate elk]

I mean, it's kinda pointless to use their diffusion stuff. it will cockblock you constantly

[ornate elk]

this will help justify having two different ~50gb installs of SD i guess

[honest flint]

i just symlink the model directories between the two

[upbeat summit]

photostop's AI implementation is great for fixing things, in- and outpainting, but for creating images it's not so much fun.

[ornate elk]

venv alone was like 25g last I checked

[thorn lion]

noooooo

[upbeat summit]

just build yourself a goofing-off workflow 😉

[honest flint]

goofing off mode... dont want to dig through folders looking for the workflow json... i just want to load it up and click go

[ornate elk]

yeah. but if I want to build on something like this it's a nonstarter

[boreal bough]

photoshop is great for real life work situations - since that's what its supposed to do

[knotty lotus]

👀

[ornate elk]

not if you're working on something even slightly pg13 for a movie

[honest flint]

if comfyui ever gets a collection of preset workflows similar to A1111 scripts like prompts from text box, outpainting, tiling, matrix's, simple render, i could switch over

[boreal bough]

ph'nglui mglw'nafh ❤️

[upbeat summit]

sure that's why it's great we have different tools to use. but drag and dropping a json or image from your favorites folder doesn't take too long? but I get what you mean.

[knotty lotus]

[ornate elk]

actually I never make lovecraft references. didn't find it was useful, goes into hokey territory all the time

even though it's the feel I want often

[boreal bough]

yeah. 'lovecraftian' in the style prompt is the most I go for

[cunning geyser]

Who does loras for sdxl?

[cunning geyser]

As of right now?

[ornate elk]

i find mashing together a bunch of weirdos works better than adjectives. or in conjunction anyway

[cunning geyser]

Just want to ask a time estimate for making a lora if you got 12 gb vram.

[boreal bough]

12 works on some gpus & environment setups, but not all. 16gb vram works on all pcs

[ornate elk]

that is dope as shit dude, you got an ig or something like that?

[cunning geyser]

12 gb vram nvidia

[ornate elk]

love me some pop surreal feels

[upbeat summit]

https://twitter.com/masslevel (or X? whatever it's called now) but you can find most of my images on this discord right now. just search for:
from:masslevel has:file

[cunning geyser]

So if i had a 12gb vram nvidia it'd take 30 mins right to make a decent lora?

[upbeat summit]

the word is that SDXL 1.0 will be much easier to fine-tune compared to 0.9

[boreal bough]

assuming you have bf16 support (depends on your specific nvidia card - google is your friend), if all works out, then yeah. about 15~20min to train a good lora

#✨｜sdxl message

^use these settings to get it to work

offset noise = 0 (cause then it auto sets it to the correct value)

[urban fjord]

I've managed to train LoRA with fp16 too.

[boreal bough]

its working now? 😄

[urban fjord]

Last time I tried yeah.

[boreal bough]

noice

[urban fjord]

Though my settings gives me 4 s/it because I run out of VRAM.

[boreal bough]

though I'm curious to check kohya changelog if it just ignores the flag XD since the recent code updates have been a bit hacky

with fp16 working, it should work on all 12gb vram cards though, so thats good

[cunning geyser]

tyvm

So this is how I make a lora in generally 15-30 minutes?

[hearty ginkgo]

What gpu u have?

[urban fjord]

RTX 3060

[boreal bough]

• good captioning is still relevant
• Dataset size of 30~60 to avoid typical issues. 10 or more will work fine if you know what you're doing. under 10 works if you really know what you're doing.
• set repeat to 1, do 40 epochs. 20 epochs will prob be your 'ideal' lora. so save every few epochs

[hearty ginkgo]

Lucky u got 12gb I got 8 and got the 3070 😭😭😭😭

[cunning geyser]

wait what

Can you explain that last one

dafuq is setting to 1 and 40 epochs do

Dont we want an image to repeat usually around 20 times?

[upbeat summit]

does anybody know a comfyui node that has a text input and saves the field in the metadata? I want to save a processed wildcard prompt in the file.

I build a metadata .txt generator to save the processed prompt in an image companion file, but I would like to have it in the image's metadata as well.

right now only the unprocessed prompt is saved and the processed prompt is lost.

[boreal bough]

ultimately, total step count is what matters. so 10 repeats with 1 epoch = 1 repeat with 10 epochs
just much easier to tweak settings if you keep repeat at 1.

[cunning geyser]

..

So wait i just want 20 steps?

I do not think I follow at all with this logic, since this seems like so little steps

[boreal bough]

image count x repeats x epochs = steps

[honest flint]

it depends on what you are doing, some lora's i had to bake for 200 epochs, some needed 20

[cunning geyser]

OHHHHHHHHHHHHH

OK OK my bad

[boreal bough]

keeping repeats low, only helps you save the model more often. so you can save it like every 2 epochs, to later compare results

[cunning geyser]

So basically I don't need to overdo it with steps like 1.5?

I can just do 20-200 steps and im fine with lower end based on complexity of design of character/idea?

[boreal bough]

exactly

1.5 needed precise acupuncture.
sdxl can take shotgun treatment XD

[cunning geyser]

So 20 for a simple char -> 200 for a stupidly high accessory char or style based gen

[honest flint]

the 200's tended to be multiple object lora's

[cunning geyser]

Yeah like asset based world morphs is what i mean

or style based on artstyles in case

Gotta make sure it do be accurate or like it

Tyvm for the help on that.

Kind of have to fall behind since I have to work on the lora database/spreadsheet of all 1.5 loras

So i'm trying to ge tin as much sdxl prep i can while listing

[boreal bough]

as long as you stick with the settings I gave you, same steps regardless of complexity. steps only need to go up if you use significantly bigger datasets - which can be used for training multiple concepts into 1 lora

[hearty ginkgo]

How fast do you generate images like not create lorasa but comfy ui images

[vale eagle]

Just call twice the common_ksampler in the custom node. I was made a custom_node like that. https://civitai.com/models/108594/sdxlmixsampler-comfyui-jnode-workflow-included

[high skiff]

Things may have potentially just changed, I have a possible opportunity to get a different 3090 relatively soon depending on my luck in sourcing sellers

[cunning geyser]

Wait now i got a question

[urban fjord]

Fix your sentence please, but about 20 seconds for 20 steps for a 1024x1024 image. 2-3 seconds more if I also use the refiner.

[cunning geyser]

Wait question

Though this is moreso for lora training

How do people do a certain tag for the dress of a character vs a tag for the character itself

Cause its weird how some people can 'tear' them apart and make one usable on another character.

[boreal bough]

have a dataset of multiple people wearing the dress, then do a trigger word for the dress, shuffle the rest. Make sure the 'person' is mentioned in the shuffled tags

if you want to train just the dress from one person, then do the trigger word for it, do not mention the person, nor any body parts, then ideally you'll get just the dress in the lora is sdxl

if it works on the first try, good. else tag the clothing pieces that didn't get absorbed into the trigger word. (like specific boots)

fyi, if you tag the boots from the start,then it will almost never be absorbed into the trigger word

[vale eagle]

As I know, the trainning dataset separated into multiple subset. 1_dress, 1_swimming_suit, 1_fullbody something like this.

[boreal bough]

it can be done with just one folder though without issues

just helps organize, or have irregular repeats

my biggest sdxl test, was 5k images trained into a 41mb lora.
it successfully learned over 100 concepts - including multiple dresses

would not recommend though, as it took my rtx4090 a total of 18 hours, and was still a bit undertrained. will rerun it on 1.0 release for a full 40 hours

[vale eagle]

It is amazing.

[urban fjord]

Nice

Do you know how much of it was the LoRA and how much of it was helping SDXL doing things it already knew?

[honest flint]

its going to be interesting tossing my 1million image dataset into a lora, dunno how well thats going to work tho, might have to do what dreamshaper did and just release a custom refiner model

[boreal bough]

I used Vit-H + the typical (but not ideal) preprovided flavor chain
due to that, all the concepts trained either already existed, or were 'retrained' from similar enough concepts that it got all the details right.

I know that you can bend it to your will with enough steps, to adapt to any wording you want, but for now I'm willing to use the existing clip knowledge since it makes things infinitely easier

[urban fjord]

At 1 million images it sounds like you just want to do a full fine-tune and not a LoRA.

[honest flint]

lol, yeh

[boreal bough]

I'll also need a custom refiner model, and I hate it
I'm genuinely dreading that the refiner retains it flaws.

[honest flint]

at least with 0.9 most of the concepts in my dataset are grossly undertrained or just absent so still want to do it... originally built the dataset for 2.1 and had a model mostly ready to release but decided to wait with this new shiny around the corner

[boreal bough]

refienr is extremely biased when it comes to faces, eyes, and face accessories - which makes cosplay loras especially hard, when they have eyewear like 2B from Nier Automata

hopefully we'll get refiner lora support - as that would solve the issue as well

[boreal bough]

are you training at 1e-3?

oh, or you mean finetune

[honest flint]

2.1 was a full finetune and pretty low lr (1.97e-6 mostly then 3 epochs of polynominal descent to -9)

lora's i usually do at 1e-5 tho

[boreal bough]

sdxl lora doesnt need to be that low anymore. would highly recommend to try your datasets first at 1e-3 - since not one of my attempts at that rate has failed so far

also unet only

[thorn lion]

Why not use prodigy?

[boreal bough]

my original argument was, that I'd try other methods once one of my datasets failed
since a lora now takes me around 10 min to train on rtx4090, and works flawlessly, I've yet to try other methods XD

[thorn lion]

If I use the search bar and your name, will I find that you have mentioned the method you're using?

[boreal bough]

#✨｜sdxl message

• good captioning is still relevant
• Dataset size of 30~60 to avoid typical issues. 10 or more will work fine if you know what you're doing. under 10 works if you really know what you're doing.
• set repeat to 1, do 40 epochs. 20 epochs will prob be your 'ideal' lora. so save every few epochs

[thorn lion]

You got gud at dataset prep. I'm going to assume that's why you can run with a static LR. Is my assumption accurate? Consequential assumption would be that shitty datasets require LR adjustments but there should be an optimal LR if your dataset is of a minimum quality. How to quantify dataset quality is a different question but you musta got gud at qualifying the quality. Assumptions correct?

[thorn lion]

And no need to perform the labour for me like that in the future. Otherwise, other people will get the idea that they don't have to put any effort into learning, thereby abusing your efforts and contributions.

[nimble heart]

made a python script that centers ComfyUI workflows on the canvas if anyone might find it useful https://github.com/Beinsezii/bsz-cui-extras/blob/master/workflows/centerizer.py

[upbeat summit]

had a use for this a couple of times already. thanks for sharing it!

[urban fjord]

Having everyone reinvent the wheel is a waste for everyone. But it is also good to challenge assumptions and see if what people are doing is actually the best or if it only works the best in their usecase.

For instance, if you find settings that is very good at replicating your face that doesn't mean it's good for general concept training.

If you want a model to learn from drawings without turning everything into a drawing you might need different settings. Though the settings Caith posted looks pretty much like the ones I've used for 1.5 with the exception of dim.

[boreal bough]

ironically enough, sdxl is much more forgiving, even when using bad settings, or outright wrong practices.
You know you got your settings/practices right, once you can use your lora at weight 1, and 4/5 generated images are of 'good enough to post' quality <- mostly just a matter of getting your captions right

If you wanna do good manual tagging, I can highly recommend hydrus network. It's high effort to learn the first time, but it scales well. I do manual tagging for around 30~500 image datasets using that app.
above that size I switch to automated workflows.
below that size I just use vit-h to write me captions only, and then add the remaining tags manually in a text editor (Sublime 3) like a barbarian

[rancid tundra]

AUTOMATIC1111 WebUI now supports SD XL

[boreal bough]

7 mins XD you're fast

[urban fjord]

Does it run as well as in Comfy?

[heady vale]

oh yeah. just updated and 1.5 is released

[boreal bough]

I am both impressed and equally disturbed XD

[rancid tundra]

I was searching on now how to install SDXL in Automatic 1111 and just saw an update. It's funny and so random, lol

[boreal bough]

bet he was in a rush since it's almost 1.0 time

oh god how much I longed for this

[cunning geyser]

.

MY SHEET, IT HAS BECOME COMPLETE

now it doesnt matter wtf they are everything in the sheet works in a1111 now

gee gee

[nimble heart]

I've read your words 3 times and I still can't understand them

[cunning geyser]

Oh its

trust me its a passive ability

I confuse everyone

[boreal bough]

...so does A1111 not use the refiner, or am I too oblivious to see it

[nimble heart]

lemme check

[vale eagle]

It should have it

[boreal bough]

I mean it works if you manually switch model, then img2img it, but at that point... yeah XD

whelp. its definitely not a good sdxl experience

[nimble heart]

off to a good start

[boreal bough]

🤣

I predict a massive influx of new 'halp me' users in the next 12 hours

[autumn forum]

im gunna try the new update of a1111. if it doesnt work first try. then yeah i can see that being true lmao

[boreal bough]

hopefully sdxl gets full released with their internal inference tool / or a very big shoutout to comfy
cause A1111 is gonna make a lot of people sad

I did a fresh install for the new version, since a lot changed

[dusk mica]

a1111 is also super slow for sdxl, if compared with comfy

[nimble heart]

ripgrep for 'refiner' only showed that the UI has the configs to load the model. didn't see anything about 2-stage processing

[boreal bough]

it works if you do the whole process manually

[nimble heart]

kind of. is there a way to end early?

so you can do base 20/30

[boreal bough]

gen image, take half finished image into img2img, then change to refiner model, wait an eternity, then do the refining part

[nimble heart]

but t2i can't return noise

[boreal bough]

10/10 user experience 🤣

[boreal bough]

yep. img2img workflow only

[nimble heart]

damn

so they're right back to square one

when sdxl first leaked

[boreal bough]

but that was already known

[nimble heart]

and no one knew wtf they were doing

[boreal bough]

but that it would be 'released' without 2 stage process is... well...

guess he just wanted it 'finished' in time for

[nimble heart]

wonder if vlad implemented noise return yet

he uses Diffusers or something doesnt he. the shrek guy and sytan made a PR to get it implemented there

[boreal bough]

technically seen, its in diffusers now thanks to pseudo. so vlad should automatically get it due to using diffusers?

[nimble heart]

is it an automatic thing or does he still have to manually use noise return

from the screenshot of the PR psuedo sent it almost looked like he just added the ability to noise return

[boreal bough]

the function is there. but if you call it, or give ui access to enable the function is still his thing

[dusk mica]

whats noise return?

[nimble heart]

doing this. Makes the refiner work better

[dusk mica]

i see

[boreal bough]

rather than fully making 2 images, you send the half completed image into the refiner model directly. Has the advantage of massive speed increase, and due to the noise, you add just the right amount of chaos to literally add details to the final image

[nimble heart]

denoise half in base, return early with leftover noise , refine the last half

or really it should be like 2/3 1/3

[sand idol]

@spark pewter I've been trying to get the 8gb training to work. My command is:

accelerate launch --num_cpu_threads_per_process=2 "./sdxl_train_network.py" \
  --enable_bucket \
  --pretrained_model_name_or_path="<path to sd_xl_base_0.9.safetensors>" \
  --train_data_dir="<stuff>\img" \
  --resolution="1024,1024" \
  --output_dir="<stuff>\model" \
  --logging_dir="<stuff>\log" \
  --network_alpha="0.5" \
  --save_model_as=safetensors \
  --network_module=networks.lora \
  --unet_lr=0.0001 \
  --network_train_unet_only \
  --network_dim=8 \
  --output_name="last" \
  --lr_scheduler_num_cycles="10" \
  --cache_text_encoder_outputs \
  --learning_rate="0.0001" \
  --lr_scheduler="cosine" \
  --lr_warmup_steps="260" \
  --train_batch_size="1" \
  --max_train_steps="2600" \
  --save_every_n_epochs="1" \
  --mixed_precision="fp16" \
  --save_precision="fp16" \
  --cache_latents \
  --optimizer_type="AdamW" \
  --max_data_loader_n_workers="0" \
  --vae_batch_size="2" \
  --bucket_reso_steps=32 \
  --min_snr_gamma=5 \
  --gradient_checkpointing \
  --xformers \
  --bucket_no_upscale \
  --vae="<path to sdxl_vae.safetensors>"

I got the train command to be as similar to yours as possible.

[nimble heart]

50/50 is way too much refine

also pointing out the noise return only seems to work properly on certain samplers

like euler and ddim

without karras

which is extra bonkers that ddim errors out in auto

[boreal bough]

and where's my 2nd clip model! XD

[nimble heart]

tbf I still just use one text input fed to both clips

hard enough prompting for one as it is

[autumn forum]

confused why a1111 sdxl says sdxl doesnt support ddim but comfy it works very well..

[nimble heart]

someone else said that ddim isn't implemented properly in a1111

supposedly multiple samplers are semi-broken

[boreal bough]

its soooo good for cinematic photos though. just throwing all the still and style references into the 2nd clip only

[autumn forum]

ah

[nimble heart]

guess I'll have to play with it once emad finishes Tears of the Kingdom and drops 1.0

[boreal bough]

'2 stage process patchwork code of sdxl'

[dusk mica]

i mean only one and a half day left

[boreal bough]

it's gonna release, while I'll be playing DND 🥲

[nimble heart]

flowcharts already make my brain turn off, flowcharts in Simlish is just mean

[nimble heart]

procedural dnd npcs on a small screen

[autumn forum]

is this what stuff looks like to people that are illiterate?

[chrome flicker]

train anime lora for sdxl should include clip model?

[boreal bough]

?

[chrome flicker]

now it seems that sdxl not have too much train on anime character, so if we want to use it train anime character model , it means we should train also clip model because of danbooru tag is not in sdxl?

[boreal bough]

UNET only for default sdxl0.9 model

if you're using the waifu xl model, you'd have to try if the text encoder messes things up or not

[nimble heart]

someone should train an LLM to convert booru tags into actual prompts

[chrome flicker]

train two clip model is enough i think

[boreal bough]

Waifu XL is already being trained. use that as a base, then you won't have to touch the clip model.

[chrome flicker]

really? I just know stability AI release a common sdxl model, also a waifu XL model to release?

I see, waifu XL is now training based on sdxl 0.9, maybe a further work to train after sdxl 1.0 release. Because 1.0 may have different structer

[boreal bough]

https://huggingface.co/hakurei/waifu-diffusion-xl
here is their first test for 0.9.

1.0 will also have a full waifu diffusion xl release - but it will take a bit of time to train

[chrome flicker]

yeah, It looks so

[molten gull]

when i start a1111 i get this error:

any idea what i must do here ?

[boreal bough]

disable the extension

its cause A1111 added proper metadata support - but that also means that all/most metadata extensions need to be updated to work with the new version of A1111

[molten gull]

how do i tell in A1111 to first use the base model and then the refiner ?

[rancid tundra]

Seems like you need to manually select the SDXL base model, generate it like 50% and transfer it to img2img. Now use SDXL refiner model to further improve the details

[boreal bough]

automatically? you don't.
manually? Generate image, then send to img2img, then switch model to refiner, then generate it again in the img2img tab

[sweet wyvern]

the 1.0 model gonna be this tedious too ?

[molten gull]

hmm, that's a little bit clunky, isnt it ?

[boreal bough]

if you want a good sdxl experience, use comfyui

[rancid tundra]

Yep, it is

[boreal bough]

A1111 full release of sdxl, really isnt a full release

[vale eagle]

I am trying to produce the same result using comfyui and a1111 with same parameters. But they are just different

[molten gull]

can i train embeddings with a1111 and sdxl ?

(or with comfyUI and sdxl ?)

[boreal bough]

using kohya? yes
but text encoder is complicated.

If you're fine with training a lora, those are super easy to train in sdxl, as long as you have enough vram

[molten gull]

i dont really want a lora, i want an embedding, and i want to save an image every 5 steps

[boreal bough]

then no

[molten gull]

well, i would be fine if i could train a lora and save an image every 5 steps, but that somehow doesnt produce images in the same way as embeddings would back then

[boreal bough]

keep in mind, sdxl is not SD1.5
so many experiences made using one will not transfer over to the other

[molten gull]

true

[boreal bough]

SD1.5 was using 1 clip model, which was easy to train
SDXL uses two separate clip model - which makes everything much more complicated if you want to retrain them

[molten gull]

i m not even really sure about the theory of how to do an embedding here

do you have to train a lora both on the base and refiner, too, by the way ?

[boreal bough]

for now, we can only train lora for base

[molten gull]

is that the reason why the end images are always a bit blurry/unfinished ?

[boreal bough]

they shouldn't be blurry

A1111 does that, but that's because its not fully implemented

nothing to do with SDXL

[molten gull]

what would be a good set of training images ? all of a very similar appearance ? and how many pictures ?

[ebon magnet]

Isn't SAI working together with a1111?

[boreal bough]

you should probably get sdxl working properly in comfyui, before trying training.
but once you're ready
-> #✨｜sdxl message

[molten gull]

i do have it working in comfyUI

[boreal bough]

the images really shouldn't be blurry though

[vale eagle]

A1111 with hires fix. Not bad.

[molten gull]

with comfyUI from yesterday

not from a lora training though

my question wasnt really regarding settings though, but more like "what kind of images should i take to create a good lora"

like, if you want a photorealistic portrait, is it better to have people from all angles, are all the same angle, with different poses or rather same, rather 6 pictures or 100 ...

[boreal bough]

@molten gull the only blur here is bokeh in the cat & burning man image, which I specifically prompted for

[molten gull]

i quite probably made something wrong with creating the lora itself, yes

[boreal bough]

the 2B image, is from a lora I made

[vale eagle]

Same prompt same setting in comfy

[molten gull]

but what kind of images would be good to use for a training @caith ?

[boreal bough]

seeds aren't cross compatible. but are the images 'generally' the same?

[boreal bough]

if your lora introduces blur/background blur, then odds are high you forgot to caption the background in every image - which results in the other words learning to recreate the background

[molten gull]

if you want a photorealistic portrait, is it better to have people from all angles, are all the same angle, with different poses or rather same, rather 6 pictures or 100 ...

[vale eagle]

No.

they have similarity

[boreal bough]

"photorealistic" should also never be used - not for training nor prompting. since a photo is never tagged with 'photorealistic' on any website. only drawings are marked as photorealistic, when they try to look real. so that prompt alone would cause blur, since its a 'drawing' prompt

[vale eagle]

I kindly think a1111 more emphasis style

[boreal bough]

more angles = good
dataset size =
1~9 <- hard
10~29 <- good enough, if you tag properly
30~60 <- ideal. Will help correct some mistakes you could make while tagging.
60+ <- also good, but training takes longer. good if you want to teach multiple different things at the same time.

Tagging:
• always caption the background!
• have a trigger word, shuffle the rest
• **dont **tag the parts that you want to show up when you use your trigger word
• **do **tag everything else, especially background

[vale eagle]

prompt: a lineart painting of a banana. No refiner using comfy. Steps: 20. Sampler: Euler. CFG: 6.5. Seed:1

[nimble heart]

That's way too much of a difference. Using 1.5 models I can 1:1 reproduce images in both a1111 and ComfyUI

Something's different

[vale eagle]

left is a1111 right is comfy

[boreal bough]

maybe A1111 uses style prompt with a different weighting?

also, sample size of 1 is too small. at least 9 grid to make it a fair comparison

[vale eagle]

Opening both ate up my memory. LUL

[inner ruin]

I'm training with attention masking on human faces now which works well. The annoying thing is that 1.5 LoRAs (and 2.1) extrapolated the person's body really well but SDXL LoRAs get your body wrong sometimes or lose you entirely if the prompt is complex.

[boreal bough]

anatomy words are basically off limits. then everything is fine.

(during training)

[inner ruin]

how do you mean?

[nimble heart]

oof

[craggy agate]

It's supposed to be tomorrow

My bae

Bad

I thought it was the 26th 😭

[boreal bough]

feet/neck/torso/ears/hands/legs <- words such as these shouldn't be used during training. If you do, you mess up the anatomy, and bodies quickly turn into lovecraftian horrors. You can train anatomy, but it's done differently from normal training.

at least, dont casually try to train anatomy while doing anything else, like training clothing, faces or artstyles

[inner ruin]

I don't do that. I just label [trigger] fe/male person

I built custom attention masking on the kohya script to disregard the background so no need to caption that

[boreal bough]

and it still works in sdxl?

[inner ruin]

attention masking? yeah. I made it for sdxl

[nimble heart]

@vale eagle if you're comparing images seed by seed in a1111 and comfyui make sure you change a1111 to use cpu for seeds

[inner ruin]

the loras look good too but they don't infer your body shape which I find annoying (before I could use the same lora for portraits and random shit)

[vale eagle]

oh, where could I set it

[nimble heart]

settings -> stable diffusion

[inner ruin]

I think it might be because I also trained TI in 1.5

[boreal bough]

ah, that makes sense

[vale eagle]

But I run my comfy with --gpu-only

[nimble heart]

left comfyui, right auto

[vale eagle]

Should it be the same?

[nimble heart]

yea it's just the seed

--gpu-only affects model loading

[boreal bough]

I'm still hoping that the weights in the final 1.0 are more balanced - since that would solve your issue, and mine as well

[inner ruin]

hell yeah, man. I sure hope so

[nimble heart]

I hope the official 1.0 vae is fixed for fp16

[inner ruin]

but I doubt it lol

[boreal bough]

prob not XD

[inner ruin]

I think we just haven't figured out how to do LoRAs right yet

[boreal bough]

but it will make lora training a real pain. since faces dont train at the same rate as literally everything else

[nimble heart]

make a separate face lora and run them in series

[inner ruin]

blame your brain for being so good at noticing imperfections in faces

But SDXL gets similarity insanely well.

adherence is the pain

[boreal bough]

I've got it working pretty well. but it takes around 600% of standard lora training time :/ so I'm not amused

face/body shape/and clothing were trained in this lora. I can call each of them separately. I even got the eyepatch working, which was my greatest pain so far

[vale eagle]

Thanks for help. I got the same image.

[inner ruin]

yeah at some point it's just not fun to wait around 5 hours to train

[vale eagle]

left is a1111. right is comfy

[boreal bough]

congrats.
Harmony achieved

[nimble heart]

small differences still, probably just Xformers or autocasting differences

[digital tinsel]

blob:https://discord.com/e85d1cbb-3355-4c8f-aa5c-af6795f930a2 Add a suitable background to this table

#1072220168534642768 Add a suitable background to this table

[vale eagle]

It confirmed the refiner doesn't have involved in t2i using a1111

[nimble heart]

and i2i is the "wrong" way to use it lol

[half ivy]

any1 got an sdxl img2img workflow?

[nimble heart]

Like from a .png type image? or a hiresfix?

[boreal bough]

didnt read properly. img2img

[half ivy]

inpaint
control net

these are the things i seek

[nimble heart]

controlnet no go

[vale eagle]

a1111 vs comfyui base vs comfyui base+refiner. prompt: a banana in an anime style

[half ivy]

keep getting weird crashes

[nimble heart]

yea controlnet doesnt work on sdxl yet

[half ivy]

lol wasted the whole morning then 😂

[nimble heart]

inpaint should work fine in a1111 or comfyui

[half ivy]

im talking strictly sdxl

[nimble heart]

yea

inpaint is just i2i with a mask so it should work fine

[half ivy]

only txt2img is sdxl

[nimble heart]

No idea what you're trying to convey with that screenshot

[half ivy]

have a controlnet setup for comfy (not sdxl)
same for highresfix
same for img2img
same for infinite zoom
same for inpaint
I have sdxl fo txt2img
wondering if theres any sdxl workflows for these that are out there

or at least wanting to know if its possible for these to be done with sdxl, controlnet we know not..

[boreal bough]

txt2img?
https://github.com/SytanSD/Sytan-SDXL-ComfyUI

highresfix?
https://civitai.com/models/81540?modelVersionId=115129

[nimble heart]

don't know what infinitezoom is but the rest are possible 100%. just gotta add the nodes for them

I have a hiresfix workflow too
https://github.com/Beinsezii/bsz-cui-extras/blob/master/workflows/sdxl-upscale.json

[boreal bough]

those are the only ones I can link to premade stuff. rest, while possible, we have no link for. so either search, or needs to be selfmade

[half ivy]

happy to try make them, super fun process, just dont wanna waste a morning like i did today on control net 😂

[nimble heart]

they work fine

[half ivy]

k cool thanks for bumping the links, will check them out now

[nimble heart]

controlnet uses models trained specifically for 1.5

so the models need to be retrained for sdxl

same for loras

and inversions

[vale eagle]

SAI seems to try to improve controlnet for SDXL (reduce the size of it). Hear from reddit

[sour obsidian]

got some ideas in testing for it

working on the old bulkier types too in case the new variant has some issues

[half ivy]

deforum type affect

[nimble heart]

idk what deforum is either

[nimble heart]

there plans to be a dedicated inpainting model again?

[sour obsidian]

no special inpaint model at launch, model does a pretty decent job on its own though it may not mean there wont be something to improve it further in the near future

[sweet wyvern]

the refiner, is that gonna be separate ?

if so, doesn't the refiner count as an inpaint model?

[nimble heart]

looks like masked inpainting kinda kerfuffles the refiner in comfyui

[heady vale]

are you meant to inpaint with the base or refiner model?

[nimble heart]

base I assume

refiner can't i2i very well so I'd imagine inpaint would be worse

[sour obsidian]

yea still a second one, but as Beinsezii said the base is likely going to get you better inpaint results as inpainting is typically a full generation task and the base is a much smarter model. That said, go ahead and try all the things 🙂

[soft bone]

are we still going to be able to get fine details in skin with the base alone? are we getting the same base that SAI was previously going to release by itself? Cuz I want to train face loras and 0.9 base cant seem to do photorealism alone

[nimble heart]

refiner sure has a field day with tattoos and jewelry

[sour obsidian]

you should be able to get sharper detail with the base alone with 1.0 compared to 0.9, its still not quite up to spec with the dual combo for now but alone it should be much easier to get finer details like that with the base than on 0.9. Ill be able to dive in more on the 26th

[soft bone]

excellent thank you

[thorn lion]

I wrote my own gradio app for tag/caption editing and I have an automated workflow that does 85% of the work. I could get it to do 100% of the work at an acceptable quality by using papers/tech that stands on the shoulders of BLIP/CLIP but I need a decently sized high quality high variety training dataset with near perfect captions/tags. It may be stubbornness but, because I'm not using (I think they're generally called) booru tags, I typically classify anything related as unrelated to my purposes.

I could see a benefit to using a combination of single word tags and phrase based captions together but, since I am only observing a divide in protocol and not directing it, I have to choose a side.

Like, I've read a lot of tutorials (for v1.5/v2.1) and the advice for training a drawn person and for training a flesh person is similar but different. Slightly different optimal hyperparameters, from step count to batch size to data augmentation options. The adaptive optimizers are gonna hand wave a lot of that complexity away but it was quite annoying for someone to assert with such confidence that their way is the way when it couldn't apply to......a lot of diverse situations.

Example: there was a Forrest Whittaker LORA that someone trained with flip augmentation on. 🙃

[thorn lion]

The thing I'm struggling with would be inventing a standardized flexibility/complexity test to determine if the LORA subject can do everything from stand and look at camera to...uhh...a yoga pose while riding a bike with skis down a muddy mountain side being chased by a bear(the bike has skis. Not the subject or the bear. Let's give the bear a Sherlock Holmes hat though) in the style of some obscure not well known artist that is within the base model /knowledge/

[nocturne dove]

I was just training a spider-gwen lora, and I think I need a few more screenshots from spiderverse (I have a few movie stills, a few comic art images, a few cosplay, and a few of emma stone because why not?). In this first attempt, sdxl is like "you still want a little spider-man though, right?" Silly images 🙂

Oh well, live and learn. Don't caption anything as "spider-man pose" that's not explicitly spider-man unless you want to train it a lot longer.

[sweet wyvern]

it's not an AI, it's priditctive algorithm, so ofc if you type "spider man" in there you're gonna get something related to spider man in return, and i bet you most of the images in the data set contains a spider man suit.

[nocturne dove]

Sure, I was hoping to get this kind of pose trained as "spider-man pose" and I didn't think about how strong "spider-man" waould be in the model. It's an easy fix to the training data.

[sweet wyvern]

you're gonna want that pose without the suit to train on 😛

[nocturne dove]

Yeah I can fix the issues. just thought it was funny to see 50% spider-man 50% spider-gwen combos the model came up with.

I guess base sdxl knows what "spider-woman" looks like pretty well too. Since, without prompting for "spider-woman" or training the lora with similar images, sdxl comes up with the mask that that character wears.

[oblique tartan]

How well does SDXL handle hands compared to previous stable diffusion?

[nocturne dove]

Somewhat better but not what you'd like it to do.

[violet totem]

For my setup it's pretty much a hit'n'miss still. Most of them don't look right to me so far.

[uncut steeple]

Better than ever, but not perfect

[boreal bough]

noice!
running your own flavour chain?

[boreal bough]

this should solve itself once mcmonkey gets to show off his new infinity grid generator

I really really hope its a parallel release with sdxl 1.0

[boreal bough]

run this image through Vit-H, see what it spits out.
Whatever pose word you get, is your path of least resistence when it comes to training that pose specifically.

You can retrain other words, but that's just painful, and can often cause unexpected issues

[oblique tartan]

i see thats rough. Also isnt They gonna do a presentation for SD XL. Hope they will touch upon that and whats their plan to improve it.

[west breach]

What's a good amount of loss to stop training a lora?

I just started training something today to learn how it all works. Spent ages captioning images. My hands are sore from typing all the captions 😆

[boreal bough]

Rip ❤️

I know the feeling xD

[peak dove]

Just delete VENV and start again

[west breach]

results so far from my 'grumpy style' 😆

[strange mist]

they're so... grumpy 👀

are you using a TI?

[west breach]

training a lora based on images i created with a 1.5 model

[strange mist]

ah, I see. looking good so far 🙂

[shy kelp]

I won the pantheon what does this mean? Also what is this even for? Picking the best fine tune of SDXL?

[strange mist]

congrats!!! 🥳 you've made it, yayy 🥳
well, that's all the fanfare you get, I'm afraid

[ionic dragon]

@sour obsidian how well the does the new controlnet work for qr codes?

[shy kelp]

lol

I used the same exact prompt that used to get front page in 2022 with Midjourney V3

Pretty sure it’s an overfit image or concept

As V3 and SDXL makes roughly the same image

Open source finally reached midjourney v3-4 levels

[strange mist]

that's not nice 😒
that's kinda cheating, I'm telling the mods on you 👀

[shy kelp]

Going to jail for Stealing prompts

[strange mist]

can you share outputs from both sdxl and midjourney in here? 🙂
I haven't used mj in months, would love to see how similar they are

[shy kelp]

Both V3

#🏅｜pantheon is my SDXL

So SDXL does beat it in terms of everything except “beauty” in my opinion

Like I think midjourney V3 has some secret sauce that makes all of its images beautiful

[craggy agate]

does anyone know how to use the refiner model for sdxl with a1111

[shy kelp]

I really wish they shared their secret sauce for V3 because V4 is just generic photographic model and V5 is something else entirely

But oh no our profits

I think SDXL beats V4 and is close to V5

It doesn’t beat V3 aesthetic though. Neither does V4 to be fair

[strange mist]

sdxl has a problem of stuff bleeding into other aspects of an image
so in mj, the universe is wholly contained within the bottle. in sdxl it has bled to the background 🤔

[shy kelp]

I deliberately picked the option that bled in. I thought it made it look better

There was another just like the MJ one

Wait new winner already?!

What is this 5 minute pantheon??

[strange mist]

oh? ok. but in general sdxl tends to do that.
well, you had your 5 mins of fame. let someone else shine now 🤩

[shy kelp]

I kinda wish there was a platform like Midjourney for stable diffusion where people actively made Upscaling and voting decisions which can be used for finetuning

Oh also SDXL beats V5 for text but I bet V6 will also have text

[primal nimbus]

bad planing. would have waaaay more ratings than they get now

[soft zealot]

same prompt, same seed, same sampler & scheduler settings etc the nonly difference is the number of steps.

On the left was 25 total , the right was 150

[heady vale]

if you are using an ancestral sampler it will always do that in any SD version

[soft zealot]

I am

[heady vale]

sometimes you can use 30 steps then 31 steps and bam, different image

[soft zealot]

nice to know (and I wasnt complaining lol)

mind you it is interesting what remains the same

and what chnages

[heady vale]

yep thats one reason many like it

[soft zealot]

although I only had the ancestral on the Refiner step

[indigo carbon]

SDXL is miles better than MJ, MJ lost this battle. I generate images that MJ would even dream to be able to do as good as SDXL

[ionic dragon]

but MJ supports many styles

and its way far away in terms of concept design

[indigo carbon]

so does XL..

[ionic dragon]

this is not concept

[indigo carbon]

just add ((whatever style))

[indigo carbon]

know what? give me any style to do this and I'll prove you wrong

[ionic dragon]

i amnot in a mood to think

just got back from work

im tired

SDXL doesn't create good abstract images too

but its way better than 1.5 or 2.1

[indigo carbon]

bruh, yes it does, hold up I'll show you

[ionic dragon]

we never know the situation after finetuning

[uncut steeple]

Whats your definition of abstract?

Stuff like this?

[uncut steeple]

I see

[ionic dragon]

something which repeats itself

like a pattern

[uncut steeple]

So more like this

[indigo carbon]

..

[ionic dragon]

i mean, something like waves, lines, geometrical figures, or miniature things

[soft zealot]

define abstract?

[indigo carbon]

sure, hold up

[uncut steeple]

Waves

[uncut steeple]

Repeating patterns

[indigo carbon]

yeah, SDXL destroys MJ

[uncut steeple]

This isnt a competition between those or any other, its more like a mutual evolution and each can learn from the other

[ionic dragon]

i dont know how to describe but like this

[ionic dragon]

the entire image should be the same

[indigo carbon]

@ionic dragon

[ionic dragon]

i am not saying SDXL is bad, im just saying that it lacks in few area, im not saying that MJ is good at abstract either,
i am just saying that SDXL has yet to improve along with MJ in temrs of abstract

[uncut steeple]

Also youre rating an unfinished product atm, thats what 0.9 really is, a preview

[ionic dragon]

like for example,
if I say someone is bad, it doesn't mean that the other person is good
i am just saying that the 1st person is bad, irrespective of 2nd person

[ionic dragon]

this is abstract

[ionic dragon]

yup

[indigo carbon]

even though SDXL can still do best =\

[ionic dragon]

i have many times said that SDXL is very good than MJ bcuz of the flexibility

[indigo carbon]

idk man, when I use SDXL on A1111 it reaches a level of detail I never saw other AIs get to

[uncut steeple]

I dont like it in A1111 atm, I prefer comfy since the workflow is more straight forward

[ionic dragon]

use comfy

sdxl is best in comfy

its shit in a1111

[indigo carbon]

this is shit?

[uncut steeple]

Thats more what you had in mind?

[indigo carbon]

it's better for me on A1111 than on comfy because I mastered my A1111 inference, so this isn't a fair comparison

[uncut steeple]

I still dont really know what you mean by that

[indigo carbon]

idk man. It is in fact faster for me on A1111 than on comfy. it's also way easier to prompt

I'm probably not the right person to ask because I'm very experienced with A1111

[uncut steeple]

Feel free to try the workflow im using

[uncut steeple]

So am I lol

[ionic dragon]

try comfy

you'll start to hate a1111

its pretty frustrating to use it at the start

[uncut steeple]

Nah A1111 still has a lot of uses

[west breach]

I still used A1111 for inpainting

[ionic dragon]

but trust me, after using it for a week, you'll love it

[ionic dragon]

yeah, controlnet, inpainiting are better on a1111

[indigo carbon]

oh, I did. I also have a lot of experience with nodes and I still like A1111 better

[uncut steeple]

Please dont ghost ping staff

[ionic dragon]

oh wait

i am trying to send something but i sent by mistake

i am trying to ask comfy

[soft zealot]

It could be worse, it could be like over on the civitai discord where everyone was given the permission to use "at everyone"

[ionic dragon]

@visual glade can we expect user friendly controlnet and inpainting in comfyui after the release of 1.0?

[soft zealot]

surely thats more generic than being SDXL related as controlnet is a much wider usage

also I think given that ComfyUI is only 4 or 5 months old that its quite amazing already

[indigo carbon]

yeah, I bet when optimized and setup correctly you can get the same results like I get with A1111. Its just that I really like an optimized interface

[indigo carbon]

also I bet that you can use SDXL's clip to interrogate photos in comfy

[soft zealot]

what you mean is that A1111 has a more use friendly default GUI than COmfy ,

[indigo carbon]

yeah, plus, it's capable of faster it/s when using 40xx cards optimization

both are amazing UIs, but it's just that one suits me better

[soft zealot]

zzzzzzzzzzzzzzzzzzzzzzzzzzzzz oh sorry it one of those mysetup is fater than yuors moments.

Its a bit like a tyre thread or an oil thread in a mortoring group ;o)

[indigo carbon]

did I just have a stroke

[indigo carbon]

oh, I understand now. I didn't try to flex that my setup is faster. I meant that A1111UI has optimizations for 40 series cards

[soft zealot]

probably can , I do know I can use it to interrogate an image to get the colour gradients used

[indigo carbon]

yeah, can we just get along? we both use SDXL here. we are just doing it in different ways. the last thing I want to do is to accidentally insult anyone.

[west breach]

using a lora makes things a lot slower, and extra 12 seconds for each gen

[smoky patrol]

Love these; what are you using for your prompt, btw?

[delicate grotto]

theres gotta be a way to remove the fog... idk why it denoises like that

[delicate grotto]

lora is an additional certain amount of dense layers right?

[west breach]

not sure, I thought it was like a multiplication on the original weights

[indigo carbon]

"candy land"

[delicate grotto]

i think it must add at least a few layers, just because you add a new word to the collection, and it also might affect the weights during the process as to override them

[smoky patrol]

I figured it was something like that.

[indigo carbon]

oh, what I'm using for my prompts? I actually made a template that work surprisingly well with SDXL

[delicate grotto]

so yeah, additional time makes a lot of sense because you use the same model with the new weights(adding them on top of the current ones) or something

[smoky patrol]

Yeah

[wary moth]

does anyone know if AMD gpu's are supported at launch? or do we have to wait for support?

tho i have an rx 580 so idk if it'll even work

[delicate grotto]

if you can run 0.9sdxl, then you should be fine

[wary moth]

@delicate grotto i haven't tried yet, i've been holding out

[delicate grotto]

im almost sure that you can run it on your cpu then
amd are quite slow with SD

[delicate grotto]

you can also use collab for that with au1111

colab*

[wary moth]

yeah it's quite painful lol, if i use controlnet it takes ~3 mins to generate. but i dont mind long waits if it means running SDXL locally

[indigo carbon]

this is the difference with and without the template

[delicate grotto]

should be ~1m for a 1024x1024 image, and you can pay ~12$ to get 100CU(im not sure how much is it in pictures)

[smoky patrol]

I love water in SDXL, or water in general

[delicate grotto]

that's a style or a lora?

[smoky patrol]

Very cool! 👍

[indigo carbon]

A1111 style. A1111 styles are templates for prompts

[delicate grotto]

mind sharing the prompt?

[indigo carbon]

this is the styles file that has it. idk how to use that on comfy though

[delicate grotto]

its just a prompt, shouldn't be a problem

[indigo carbon]

A1111 weights prompts differently from how it does styles. it might cause problems

[delicate grotto]

not really, you can press the paste button and it will paste your style

it usually places that after your original prompt

lets say you had
pos:
prompt1
neg:
prompt2

the style is prompt3

the output is:

pos:
prompt1,pos prompt3
neg:
prompt2,neg prompt3

[indigo carbon]

that's not how A1111 prompts work.

[delicate grotto]

there is a button to paste your current style

near the save image icon

[indigo carbon]

but sure, feel free to try it =]

[delicate grotto]

using the same seed you will get the same output

im just afraid to load a1111 since it started to BSOD sometimes when i load the models

dk it if is fixed, but i don't want my drive to die