✨｜sdxl

[gloomy lark]

or use the non-distilled 1.2 which isn't the new safetensors model file.

[meager canopy]

Not 1.2?

[gloomy lark]

1.1 distilled, or 1.2 non-distilled.

hopefully city96 will get his stuff updated and then we can do 1.2 distilled too

[meager canopy]

I can wait

Have you got it working?

...1.2 that is

[gloomy lark]

grabbing the non-distilled now

[gloomy lark]

yeah even non-distilled 1.2 isn't working with the city96 nodes. i guess he has to update stuff for both. so stick with 1.1 distilled for now

[meager canopy]

https://github.com/city96/ComfyUI_ExtraModels/issues/71

[smoky sentinel]

i ve try so many time too. Do you run windows by any chance ?

[meager canopy]

Yes, but it's not compatible with the node yet.

[smoky sentinel]

yup. F hope they fix it soon.

[gloomy lark]

Oh ok awesome. The city 96 guy has been really great with this stuff

[fierce heron]

anyone got a comfy workflow for this? https://civitai.com/articles/5834/silvi-v2-upscale-method-super-inteligence-large-vision-image

[meager canopy]

An upscaled variation...

[shy kelp]

I am out here...

[glass forge]

https://www.youtube.com/watch?v=KXQqo3Owjg0

is it worth to upgrade from the old controlnet models? anyone tried these?

[tawdry current]

Yes, they are fantastic. That new depth model is on par with marigold and is like 1000000x faster. Haven't tried the pose one because I don't make a lot of stuff with people in it

Mistoline and that new depth are the ones I use now days

[rotund surge]

It appears a 404 error U.U
Wait now it works, i´ll see it

[rotund surge]

Found it on reddit, I haven´t tested it yet

[copper kraken]

negative prompt: "simplified, low resolution, canvas frame, ((disfigured)), ((bad art)), ((deformed)),((extra limbs)), blurry, (((duplicate))), ((morbid)), ((mutilated)), [out of frame], extra fingers, mutated hands, ((poorly drawn hands)), ((poorly drawn face)), (((mutation))), (((deformed))), ((ugly)), blurry, ((bad anatomy)), (((bad proportions))), ((extra limbs)), cloned face, (((disfigured))), out of frame, ugly, extra limbs, (bad anatomy), gross proportions, (malformed limbs), ((missing arms)), ((missing legs)), (((extra arms))), (((extra legs))), mutated hands, (fused fingers), (too many fingers), (((long neck))), Photoshop, video game, ugly, tiling, poorly drawn hands, poorly drawn feet, poorly drawn face, out of frame, mutation, mutated, extra limbs, extra legs, extra arms, disfigured, deformed, cross-eye, body out of frame, blurry, bad art, bad anatomy, 3d render, digital art, bad art, (deformed iris, deformed pupils, semi-realistic, cgi, 3d, render, sketch, cartoon, drawing, anime:1.4), text, close up, cropped, out of frame, worst quality, low quality, jpeg artifacts, ugly, duplicate, morbid, mutilated, extra fingers, mutated hands, poorly drawn hands, poorly drawn face, mutation, deformed, blurry, dehydrated, bad anatomy, bad proportions, extra limbs, cloned face, disfigured, gross proportions, malformed limbs, missing arms, missing legs, extra arms, extra legs, fused fingers, too many fingers, long neck, 3d render, digital art, bad art, 3d render, digital art, bad art, 3d render, digital art, bad art"

🤣

[rotund surge]

That´s a big ahh negative prompt 😁

[copper kraken]

only 402 clip tokens

when the limit is 75 lol

yeah lol

[rotund surge]

The funniest thing is that if you use that in the positive prompt it probably would look better 😁

[copper kraken]

lol yeah

[blazing briar]

hello

[shy kelp]

that pink and yellow room is great

[copper kraken]

zoom in on the windows

[meager canopy]

I can't see any details inside the windows 🤷🏻‍♂️

[glass forge]

is that still a thing or its just ads more tokens if u go overboard?

[rustic garnet]

clip itself is limited to 75 tokens - no way around that.

you can only split the caption into parts of 77 tokens each and process each separately

how you then feed these tokens into SD depends on the implementation. You can just use all the tokens, but as sd is trained on max 77 tokens this will probably harm. You can also interpolate the tokens or down weight them

[glass forge]

i have a negative promt style wiht 6000 words

[zealous viper]

been working on a workflow for sdxl as I put it down for a while when 3 came out to play with it

[fierce heron]

link?

[meager canopy]

I have a good one on my Patreon. These are before/after images using the Silvi example image.

[waxen violet]

Pink sky with blue clouds and green sun

[shy kelp]

a green sun! preposterous!

[silk smelt]

almost, but the lady got in the way and her hat stole the green away from the sun

I guess the model knows clouds, but clouds are also part of sky. So when trying pink sky, it also projects pink to the cloud-part of the sky.

and sun, well that should always be yellow-ish or red-ish

tried to fool the model by replacing green sun by green sun-like star ... but it just wont get fooled

ok, enough. going to put (yellow, yellow sun:1.5) in the NEG prompt

[silk smelt]

We've hit a border. Something prevents us from having a green sun. Though this border protection worker seems familiar.

[gloomy lark]

really cool sharp lines on this one

ella

[fierce heron]

Thanks but I'm banned from Patreon 😅

[copper kraken]

why?

[fierce heron]

They didn't give an explaination

[copper kraken]

you should ask this octopus

[fierce heron]

I think nsfw content is against their tos tho

[vagrant valve]

@humble cape Apologies for the tag. I've been going through and love your images and the style. Would I be able to DM you some questions?

[humble cape]

Hello, I will be happy to show you how I generate my images

[meager canopy]

Nudes are allowed.

[meager canopy]

Looks like there's a sea wall on the horizon

Maybe a very long pier 😄

[copper kraken]

lol

maybe there is 😄

post climate change flooding

[meager canopy]

"I feel...different..."

[vast galleon]

managed to get all 3 out of the same model. I think it might be ready.

[meager canopy]

Drones battle it out over NYC

[meager canopy]

Spot the pope

https://tenor.com/view/sniper-gif-26102511

[meager canopy]

That small, overlayed image just makes it look like a badly denoised render.

[iron rover]

That’s my watermark 😂

[iron rover]

I keep forgetting to share the none watermark ones

[meager canopy]

It still doesn't look good. It's not clear enough to be worthwhile putting there, so you should either shrink it to stop obliterating the image, or remove it.

Just my thoughts, but it's up to you 🙂

[iron rover]

None watermark one

[iron rover]

I like em sorta big harder for ppl to strip and re sell because fb is filled with re sellers or artist theft

But I also make sorta small too

[meager canopy]

People are stupid enough to buy them?! 🤣
They don't belong to anyone anyway 🤷🏻‍♂️

[iron rover]

😂 it belongs to the original artist well to be fair I do add sfx etc to the photos so it’s not just SD /SDXL procreate does play a part

Before I drew in

[meager canopy]

Depends how much of a "significant" change you make from what AI created.

[iron rover]

After

[meager canopy]

That's so much better without the splatter! 😄

[iron rover]

Thanx but it needed the horror to it 😂 other than that it’s too girly for me 😂 jk

[meager canopy]

Just change the prompt to add tattoos

[iron rover]

True but I love drawing too

[broken nebula]

A man walks on Mars

[meager canopy]

Some variations of it...

[iron rover]

The first and last one are awesome second can def be fixed in procreate

[meager canopy]

No point when you can just squeeze out some more images 😄

[iron rover]

Yes that too 😂

See I thought the nurse outfit was perfect leaning more towards the Sony games

[broken nebula]

A cartoon girl in bed

[broken nebula]

A cartoon girl in bed

[broken nebula]

@iron rover A cartoon girl in bed

[iron rover]

Here’s the image u requested

[iron rover]

That’s def beautiful

[broken nebula]

@iron rover An astronaut looking up at the Earth

[copper kraken]

Here is the image you requested.

[broken nebula]

@meager canopy A cartoon girl in bed

[copper kraken]

Here is the image you requested.

[broken nebula]

@copper kraken A cartoon girl in bed

@copper kraken A cartoon girl in bed

[copper kraken]

Here is the image you requested.

[iron rover]

Here’s the image u requested

[copper kraken]

Here is the image you requested.

[broken nebula]

@iron rover

[copper kraken]

Here is the image you requested.

[iron rover]

Here’s the anonymousghost you requested

[copper kraken]

Here is the image you requested.

[iron rover]

Here’s the image you requested

[copper kraken]

Here is the image you requested.

[broken nebula]

@meager canopy A cartoon girl in bed

@iron rover A cartoon girl in bed

[iron rover]

Here’s the image you requested

[broken nebula]

@iron rover

[copper kraken]

Here is the image you requested.

[shy kelp]

Oh I want to play

Here is the image you requested.

[limber kindle]

The very particular/distinct look of some DMT trips : D

[fierce heron]

yeah, cause they use paypal or stripe and those processors dont allow explicit nudity

[iron rover]

Here’s the image u requested

[vast galleon]

Mangled Merge XL V3 has been released if anyone would like to play around with it. I'd love to see what images others can make with it. 🙂

https://civitai.com/models/447902

[gloomy lark]

did a few with it

[viral bone]

A drone swarm composed of three drones inspects a building in a realistic style, close to the real scene

#✍🏼｜rules-and-tos

[vast galleon]

Nice! They look awesome! What are you using to upscale? Need to set something like that up myself. 😊

[gloomy lark]

that one was just itself with a 1.5x upscale and 0.5 denoise. "upscale image by" node followed by another ksampler of it.

[fossil yew]

where can i generate images with sdxl please ?

[fathom flame]

https://civitai.com/models/558233?modelVersionId=621417

[vast galleon]

Got the upscaling process working in Comfy now. 🙂

[main glacier]

Does anyone have a good SDXL latent downscaler? I’d like to go from 128x128x4 to 64x64x4.

[half egret]

embrace the lord of frenzy !

[gloomy lark]

have i got the woman for him. 🙂

[spiral obsidian]

We have grapes!

[half egret]

gaze in to frenzy ~ xD

[gloomy lark]

hunyuan / realvis

[dim steeple]

hi! Wondering does anyone have ideas for how to get this style?

I can't find any loras for it

I need it for img2img

to make a image in this style

like style transfer

[vast galleon]

lite-brite style?

[shy kelp]

this is great, what sort of model and prompt are these action shots?

looks a bit like marvel movie but a bit more cartoony

[loud imp]

Is it possible to outpaint with SDXL models? I've tried inpaint controlnet models and none worked so far

(with A1111/forge)

[shy kelp]

yeah any diffusion model can inpaint or outpaint
because there is a way to do it by masking the latents

[loud imp]

in the img2img>inpaint tab?

[shy kelp]

I don't use A1111 so I don't know

I either use comfy, diffusers or direct pytorch workflow

[vital ermine]

https://civitai.com/models/560142?modelVersionId=623765

[iron rover]

The purge

Dollface from strangers

[gloomy lark]

This was a prompt I found on midjourney and rendered with hunyuan 1.2, refined with level 4 XL. prompt: trumpet player fights against a giant bloboid Cthulue-esque monstrosity by playing songs on a trumpet in a messy shady brick alley

[glass forge]

I just want a model that makes pictures that are 100% indistinguishable from the real world. Without Photoshop pitty work.

[iron rover]

One does exist her name is Photoshop

[glass forge]

I mean I can fix anything in PS but then we're back to square one aren't we. XD

[glass forge]

Right.....................

[gloomy lark]

I think this nailed what you're looking for: #✨｜sdxl message

[iron rover]

Maybe ps has advanced sense then after all its its first just has evolved sense then

I think ai has gotten lazy here lately and I see everyone not here but Bing users look like it’s the same prompt same art same color… if that makes sense

Fire fly is actually really cool if u want to evolve ur SD images into something unique

[glass forge]

I think AI kind apeaked. We have to accept that it will be just another (quite amazing) tool in our toolbox. but it's not a one click magic bullet.

[gloomy lark]

I think sd3 2b gets really close to that real photo look with the better vae. obviously it's not trained well enough hence the bad trumpet etc, but this is miles closer to a real photo than sdxl

[iron rover]

I agree Dodge Coin

[glass forge]

Yes that's why I feel bad for SD3 it is a better start foundation than SDXL but if no one trains it... :/

[gloomy lark]

well, supposedly better version soon

[iron rover]

Someone will train it just ppl don’t get the liscense

[glass forge]

IN 2 WEEKS! 😄

[gloomy lark]

finally got a reasonsblae trumpet

[iron rover]

Also Lykon said he will be making Dreamshaper / sd3 so there is that

Glad this room has zero drama in it and hope they don’t kill this channel like they did with Cascade

[glass forge]

I hope not! =0

[vital ermine]

when new version come out they archive the old channels as they did us with SD2

[gloomy lark]

sdxl will live on as long as we need it for output like this.... refinment

[vital ermine]

If you like cronenbergs.

[iron rover]

I noticed that what happened to sd XL?

That model came out at the same time as Cascade did

[vital ermine]

When I used to be here every single day a bunch of us were using the SD2.x channel. XL came out and they ripped it away from us. We asked for it to stay and they did but not for long as they archived it and no further posts allowed.

[iron rover]

I remember us asking for the please stay but I’m sure they can un archive so stupid they did that too

[vital ermine]

A few of us did not wish to move to XL so preferred 2.x. They decided to allow the channel to come back but it was short lived

I eventually left this discord to find happier grounds after I migrated to XL not long after.

[iron rover]

Everything short lives in this server it seems like

[green geode]

I found out how to make extremely good images with ssdxl-turbo
Just refine the image Generated with SDXL-turbo 3 times with the sdxl refiner.
The first image with SDXL-turbo and the second is the refined image.

[green geode]

did you try downsampling and then just color a other image based on downsampled source image. AI would just make artifacts

[clever cave]

not too bad but some might prefer first one since it has more detail.

[green geode]

Yes, sadly this approach can add some artifacts but it removes more artifacts than it adds. One example is the eyes. But sdxl-trubo surpassed sd3 with the same prompt.

[azure robin]

Лев

[green geode]

Можете ли вы, пожалуйста, использовать английский язык, чтобы минимизировать риск ошибки перевода?

[azure robin]

Lion

[green geode]

thats a fox

[vast galleon]

sad looking bird.

[dim steeple]

thats a good idea ty

[half gyro]

what causes broken artifacts when generating images?

cfg?

[shy kelp]

artifacts is a really general term

it can be VAE

[marble pivot]

Or a bad model

Could also be a lora issue, if you use that

[vast galleon]

is that SDXL tile?

[vast galleon]

2000 loras merged successfully

[copper kraken]

yeah you gotta specify what artifacts... best to provide a couple examples tbh

[half gyro]

the bits in her hair

I am using a lora a one I trained through civatai plus another texture lora called ReaLora

its not very common but sometimes its very lightly on the body etc

like its really visible on the stomach here

[copper kraken]

the stuff in the hair looks like a noise problem

that can be the result of a bad checkpoint/lora, or many other things with sampling

best thing to do is turn off all loras, and revert to some basic standard sampler settings that don't add noise... something like dpmpp_2m or euler, 35 steps, karras

and see if it goes away

that can help you narrow down the problem

[half gyro]

was using euler_a

alright

Ill test it out

[half gyro]

So its the lora

damn

do I retrain my lora?

wait

no

its still there

[copper kraken]

if it's still there and you're using a really generic WF... then it must be your checkpoint

or you got some stranger problem

[red minnow]

Batman Game

[tribal lantern]

Kolors, it's almost SDXL isn't it 😉

[red minnow]

Cats eating noodles is cute

[tribal lantern]

Those cats were such a nice surprise! (my ehm, creativity is basically "dear llm, i have these 2 prompts, create 10 more like them" and the cats were one of those 😉)

[shy kelp]

those were all cute

[tribal lantern]

Different kind of prompt (An atmospheric HDR 8k digital octane render....) It's pretty fun model, not the best (doesn't do various styles all that well, doesn't react to artists, prompt following is hit and miss for recent models), but it's very promptable for a lack of better word, once a prompt works it works for various subjects, its failure mode is still reasonable images and results are varied.

[vale eagle]

@gloomy lark I managed to merge Kolors with my sdxl ft. 1: original 2: merged. Using this for inference: https://github.com/kijai/ComfyUI-KwaiKolorsWrapper The original conversation is in SD3 channel but I think SDXL channel is more appropriate

[gloomy lark]

wow those are fantastic

[gloomy lark]

just as i was messing with lumina/cascade.. 🙂 (penguin image)

so I'm up to speed on using kijai's nodes, but how do we merge kolors with sdxl? I tried kind of but it just kept erroring out.

[wheat rose]

https://huggingface.co/spaces/gokaygokay/Kolors

[gloomy lark]

yeah... lumina on left, level4 sdxl upscale on upper right, kolors on bottom right

[wheat rose]

and they all get the too cute award

[gloomy lark]

hah well I'm highly jealous of what @tribal lantern posted earlier

[wheat rose]

take a look at the model page and tech page on the huggingface space i gave you the link to

[gloomy lark]

yeah i've been playing with it quite a bit since yesterday.

the biggest issue with upscaling stuff like lumina which is multi-subject, is upscaling with sdxl which isn't. but kolors is...

so now I just need xiaozhi's formula for sdxl merging

with it

[wheat rose]

i never use the AI i'm generating with to upscale. that's a recipe for disaster

[vast galleon]

Just now finding out that adding iterations to the Dare merge nodes is like merging on steroids. Super stoked about it. These are the results after smoothing the last run of merges.

[glass forge]

Michael Bay moment.

[vale eagle]

Need to skip a layer which has different shape between kolors and sdxl

and comfyui node could not recognize kolors as sdxl unet due to that layer. I write a script to merge it

[novel cargo]

create

a cute cartoon cat

[shy kelp]

anyone used the new controlnet union model ?

https://youtu.be/UBFEw1IUX_I

[hoary saddle]

my mememaker node is in the manager now, yay

[random badger]

#artisan-1 типичный завод в Пскове

[shy kelp]

mexican skeleton frog robot is amazing

[shy kelp]

everyone should leverage this, all in one controlnet https://huggingface.co/xinsir/controlnet-union-sdxl-1.0

[south frigate]

Are there any good models for SDXL controlnet anymore? I have a lot of models that perform poorly, once better, once worse, but certainly not the level as in SD 1.5. Are there any new sensible models for CN?

[shy kelp]

i dont know if a1111 supports that controlnet union but it works really well on comfyui as i've tested. and you don't need multiple separate CN models when using union which is about 2gb and cover all styles

[south frigate]

a, I haven't heard of something like "controlnet union" I'm already getting around to reading about it, thanks for the info
I hope it works in A1111 or Krita (ai addon) cuz really dont want to use comfy.

[shy kelp]

its pretty new, dropped a day ago, they will probably have support for it on a1111 in days to come, but so far its working good on comfyui

[south frigate]

i tryed, well, openpose in preview working good but in practice not working, at least in a1111 for now or I just set something wrong, but hmmm... it should be like this? (openpose_full)

[coarse kelp]

why does this still look ai generated?

[shy kelp]

dont think a1111 supported for it yet. Usually with controlnet you'd have to select a particular pre processor and load that respective model, but with sdxl union you select the model and select the preprocessor based on it

[tawdry current]

Are all these from the past few days from some newer version of your lora or a model merge? Been digging them a lot. Almost have a Berserk manga feel to them, but mixed with some high contrast color.

[copper kraken]

no new model... been fuckin with the RES sampler

most from this evening have been with the following strategy:

run a step with two seeds, then run two seeds for the next step (2x2), then two seeds from that next step for a total of 8 outputs over 3 steps

find the median output as a very rough estimate of "more probable" (thinking of this like a potential energy surface in chemsitry, except we can't take a single point energy here)

and then repeat

one of the more interesting things i'm noticing is it seems to be especilaly good at extracting styles etc from a lora

[shy kelp]

I've seen other workflows with contrast this good
but your workflow has by far the best colours I have ever seen
not sure exactly which part is doing that though

[tawdry current]

Ahh yeah that's a pretty interesting approach, it's like a seed average almost. You're kind of making a little bell curve for every pixel and then picking from the peak, kind of like a gaussian distribution function

Cool shit though man

After you have your little three step average, you then just ride it out to completion?

Kind of reminds me of the trick from a while back where you'd use 2-3 steps to prep a latent, but not return the noise, then feed that in as the base latent for the actual generation

[tawdry current]

so something like this? (im just adding them all and multiplying by 0.125 for an average, but i did the 2, lead into two, that lead into two each thing, they are just arranged for the screenshot)

i could probably write a simple node to pull this off to avoid summoning the flying spaghetti monster. Edit: looking at some of the previews, I can see it's not quite right, I think I need to change some of the add noise/return noise settings on them

[meager canopy]

https://tenor.com/view/joe-biden-presidential-debate-huh-confused-gif-16704157274113773062

[tawdry current]

bro im bored and trying to distract my brain with tedium

yeah there we go, just needed them all to add noise and to not return noise

[eager prairie]

https://i.ibb.co/q9f7q7X/Background.png

[rich crescent]

anyone here has information about fine-tuning? I have no idea how much more/less data it needs. the example I had from dreambooth dataset was like 10 images per object/class.

[languid ravine]

A coloring book page, white background, illustration of an pocket room Coloring Book page , Calm, Cozy, and Peaceful Spaces for Relaxation and Stress, ink drawing, line art, clipart, simple line illustration, Relaxing Corner, black and white,8K

[errant arrow]

Light gray and green landscape, pines and cypresses, temples, water ripples, cranes, mountains, water, Chinese style, gold foil silhouette, plan view side view, full length shot, photo realistic, soft shadows, no contrast, clear sharp focus, film texture , natural lighting, professional color grading, Leica lens

[vast narwhal]

This is allinone controlnet model. The middle is the original (random pic from reddit), Left is canny alone, right is openpose alone. The model is a pony merge

[copper kraken]

https://github.com/xinsir6/ControlNetPlus/blob/main/requirements.txt this i would watch out for

that requirements.txt might break stuff

[limber kindle]

258 lib reqs? Holy moly lol!

[vast narwhal]

No ideia what this repo is. I just downloaded the model

[copper kraken]

gotcha

yeah just wanna make sure anyone that decides to give that shot is first aware that they should give that requirements.txt some serious consideration

no < or > just == across the board

odds are it will uninstall and reinstall every single thing you got lol

[vast narwhal]

https://huggingface.co/xinsir/controlnet-union-sdxl-1.0

the current apply controlnet (advanced) node is all you need

[meager canopy]

Nice detail! Which workflow are you using?

[limber kindle]

Thnaks o// It's two samplers with an Ultimate SD upscale with relatively high denoise settings.

[hollow kraken]

guys, will a model understand me if i speak english or do i need to speak in danbooru tags

[vast narwhal]

Depends on model, just keep it simple and do experiments

[silk smelt]

realCrystalPonyXL generating some good looking group portraits. Since this model is a 40%pony/60%nsfwsdxl mix, I had to blur some parts manually.

[fierce tulip]

Why does this happen?

Every time I use a pony/XL model it creates this shit image and can't work out why!

[gloomy lark]

Kolors

[silk smelt]

looks very much like a vae error

[fierce tulip]

What vae do you recommend?

[silk smelt]

sdxl_vae

or, or, you might have set a weight in your prompt to very high levels

[fierce tulip]

What do you mean by that? Sorry haven't done it before

[silk smelt]

then its probably a vae error, but if you want share your prompt

[molten gull]

test

[hoary saddle]

pony is -2 clip i believe on built in vae

[slender coral]

Anyone know how to access / set the API in comfy UI?

[eager prairie]

Your prompt looks awesome

[vital ermine]

how does one use more than one CN? I used to use depth and canny (or lineart) together but in the new all in one I can't figure out how to do it.

[tawdry current]

Just put a second controller in line with the previous. Pipe your canny image into one and depth into the other and adjust their settings individually.

Oh and pipe the same cn model into both of them, just share the same node to prevent it from trying to double load the model

That being said, I am not 100% sure if the cn has an internal "automatically detect" mode for picking. I know in the guy's code, he pipes in an extra argument tensor like 0,0,1,0,0 or 0,1,0,0,0 and so on for picking

[vital ermine]

I saw no where to do that as before I would just daisy chain them.

[vital ermine]

are you talking about this node and just image out to image in of another of the same node?

[tawdry current]

like this if youre trying to use the same union cn for both depth and canny

[vital ermine]

that is the old way

[tawdry current]

no, it's the right way

[vital ermine]

just saying this is how the new one is done

the all in one for CN now

[tawdry current]

oh you can use AIO if you want, it does the same shit under the hood. also, there isn't actual comfyui support for union yet, like i said, there are technically extra arguments that have to go into the sampling. it happens to work for the most part, for now, but it's not completely correct. depth seems to work really well though, but that might be the default within the model

[vital ermine]

Hopefully he brings in a connection point to allow more so depth and canny/lineart can be combined.

[tawdry current]

well i just showed them working together

but anyways, from his code:

added_cond_kwargs = {"text_embeds": add_text_embeds, "time_ids": add_time_ids, \ "control_type":union_control_type.reshape(1, -1).to(device, dtype=prompt_embeds.dtype).repeat(batch_size * num_images_per_prompt * 2, 1)}

this isn't implemented into comfy yet through any plugins

for instance, canny would be torch.Tensor([0, 0, 0, 1, 0, 0])

[vital ermine]

text_embeds =, is that the one that throws a cli error as missing in my loras?

[tawdry current]

depth is torch.Tensor([0, 1, 0, 0, 0, 0])

`# 0 -- openpose

1 -- depth

2 -- hed/pidi/scribble/ted

3 -- canny/lineart/anime_lineart/mlsd

4 -- normal

5 -- segment`

there's the list of 0 0 0 0 0 options

and he technically supports multicontrol in his code as well

so you can do stuff like 0, 1, 0, 1, 0, 0

that would enable both depth and canny

[vital ermine]

iow, comfyui is the culprit?

[tawdry current]

no, the lack of implementing the special arguments is. it's not on comfyanon to implement every single proprietary thing for people, it's on those creators to make a plugin that meshes with comfyui

[vital ermine]

alright, thanks

[tawdry current]

here's the union creators git:
https://github.com/xinsir6/ControlNetPlus/

he has plans to support comfy, but its not done yet

looking through his code, it seems he takes that selection tensor value and "inject control type info to time embedding to distinguish different control conditions"

control_type = added_cond_kwargs.get('control_type') control_embeds = self.control_type_proj(control_type.flatten()) control_embeds = control_embeds.reshape((t_emb.shape[0], -1)) control_embeds = control_embeds.to(emb.dtype) control_emb = self.control_add_embedding(control_embeds) emb = emb + control_emb

so where ever stuff like that happens in comfy, is where he'd have to either find a way to inject it or to monkeypatch it in

[copper kraken]

requirements.txt could use some refining... lol

[meager canopy]

Looks like it is coming to ComfyUI though. https://github.com/comfyanonymous/ComfyUI/commit/faa57430b0ff882275b1afcf6610e8e9f8a5929b

This is only the model code itself, it currently defaults to an empty
embedding [0] * 6 which seems to work better than treating it like a
regular controlnet.

TODO: Add nodes to select the image type.

[tawdry current]

oooof, i don't think i've ever seen one that long

[tawdry current]

oh cool, figured it wouldn't take long for someone to throw together

but my point earlier is that it shouldn't be on comfyanon to implement all this himself

[copper kraken]

yeah, brutal, and everything is == with a specific version

dependency suicide

[tawdry current]

for real lol

[shy kelp]

thanks these are amazing

really unique ship designs

[limber kindle]

Breaking stuff intentionally

[next folio]

is it possible to have a pic of a character as reference to generate art about that character with one single image? wtihout training a lora, hope someone can help

[copper kraken]

look into the ipadapter plus nodes

[next folio]

how can i use that in a1111?

[copper kraken]

you can't, you should switch to comfyui or stableswarmui anyway though

they're faster, more efficient with vram, have more features and are better supported

[glass forge]

lmao the saviour we deserve

[glass forge]

is that CFG 1?

[vast galleon]

2200 loras merged successfully.

[tawdry current]

Merging 2200 loras would be like mixing every color in a painter's kit. It would just average everything out and you'd be left with something slightly better or worse than the base model. Unless the unet actually increased in size, you'd just end up with every lora stepping all over the same concepts they all share

[zealous viper]

Averaging out the concepts they all know wouldn’t be 100% bad if it gives you a more variety in what turns out to be good in one shot prompts.

I’d be willing to play with it at least to see if it turned out to be a good sparring point. Then specific Lora’s to get you to the end goal.

[zealous viper]

Is it something you’ve uploaded?

[tawdry current]

But that's just it, it wouldn't. You'd have some anime cartoon lora, some photorealism lora, some boring lora, some fine art lora, etc etc, all competing for defining the concept of say a human woman, her proportions/poses, her skin texture, her hair, and so on

That's why I said it would likely be marginally better or worse than the base model. It might help a tiny bit with anatomy, since people focus on making waifus with most major loras, but then it will just overrestrict composition and potential variety

[vast galleon]

Only up to 1800 https://civitai.com/models/447902?modelVersionId=619849

[zealous viper]

I’ll give it a try when I get off work.

I’m interested to see how well it does when I throw random things at it

[vast galleon]

Cool! Let me know your honest opinion.

@tawdry current it's not meant to be a waifu model. It's more of an experiment in model merging and to see how many I can merge in before it breaks beyond repair. But, there are plans to make specialized versions eventually. You're right however. But that goes for all models, by training or merging, no one is truly making the underlying architecture for sdxl any better. They are just moving weights around. But still, people do it because they can or to see if they can. So what.

[shy kelp]

there is a spectrum between underfitting and overfitting

and a second spectrum between acquiring new knowledge and catastrophic forgetting of old knowledge

very few models are at the optimal point on each spectrum

and if they are further to the left, then you can safely add knowledge or change the fit

this will move rightwards on the spectrums

but if you have some headroom then that is fine

if you do not have headroom then it will not work as well

my personal view is that the vast, vast majority of deep learning models are overfit

and that we would be better off with slightly less capable models that have better generalising ability

[tawdry current]

Right, but my point is that loras, in general, are very heavily biased toward making portraits of pretty women. Doesn't matter the art style either, just take a look at civitai and you'll see what 95% of the thumbnails are. No joke, I just scrolled through ~150-200 rows and counted 30 thumbnails that weren't females. There are five per row. That means that female related loras account for 96-97% of what I saw...

So the bulk of the loras you used in that 2200 are likely waifu oriented loras, or at least have a large amount of their trainings oriented on them

[glass forge]

Pony should be called SDXXXL

[tawdry current]

that being said, i did see an interesting lora that i want to try out now lol... https://civitai.com/models/521190/0682-hoarder-room

eww the base model is pony, nevermind

[glass forge]

looooool

That's an unreasonable number of furnitures of the same kind in one room.

[vast galleon]

I'm not putting loras of waifus in there because they are boring. I do put portraits in every now and then to help with faces. But yeah, no waifus, no specific people, no graphic sex acts, just loras I think are cool.

[tawdry current]

i've lived in places like that at times in my life, they really do be like dat

[glass forge]

I like how pretty much any lora or model you go to check it doesnt matter whatsoever what the thing specializes in based on example pics and description the community made pics below quickly devolve into nudes and orgies

[shy kelp]

for what its worth the model doesn't seem to be broken
I suspect you will take at least a subtle hit to image quality eventually and get some overfitted areas but that might be worth it
seems like a very efficient way to add more subject matter to the model

I think you may be on to a really good idea here

[limber kindle]

Nah this is is Kohyas Hires fix with sampling that kicks in too late : D

[vast galleon]

Thank you! I was thinking the same thing. It's kind of a form of RLHF training but through merging instead of fine tuning.

[shy kelp]

yeah it has the advantage of being drastically easier which might make it worth the potential issues

if you tend to use loras with looser weights that will help

[zealous viper]

still dialing in settings, there is 100% a bit of an over cook on cinematic bars on the top and bottom

still figuring out prompting for it might be some of that as well

[vast galleon]

I've noticed that as well. I usually put 'framed' in the negative. Sometimes it helps, other times not so much.

[zealous viper]

it also likes to forget about the background and make a subject on a black background

[glass forge]

What a madman. LAMO.

[zealous viper]

does seem to be able to upscale ok though

one thing i did notice a bit is that it would make table top models every now and then, thinking that maybe there might be one too many of those loras if there are any in there

would explain some of the behavior

[vast galleon]

Noticed that too. You kinda have to add it into the prompt or it gives a blank background often. Its done that since V1

[vast galleon]

Yeah there are a few lora based on miniatures in there. Once I get a better grasp on smoothing I will be starting over. The first few versions were done the old way, but I'm finding the DARE/Ties method to be a lot better.

[sharp scarab]

https://www.youtube.com/watch?v=6VVZk_J5vDk

[limber kindle]

Outpainting : D

[sweet abyss]

How can i create different versions of a image with a different colorpalette ? (i am using A1111)
Denoising works quite well to create "similar" images but the color palette is most likely identical.
I was playing around with controlnet but havent got a good result. May i could need some advice for settings / prompt.

[shy kelp]

really nice colours

looks a bit like film

[smoky shard]

@sweet abyss Something similar to this?

[sweet abyss]

yeah

[smoky shard]

30s film grain at work. 40s version too

[smoky shard]

Well, the easiest way is to use solid color as an input of img2img. In this case it was a simple light blue rectangle:

And of course control net to replicate the form of the original image

Oh, important part. High denoising strength (0.95+) with Exponential scheduler

[sweet abyss]

control net does not work well for me, i have top view perspective with small details which turn out like simple forms in control net.

[smoky shard]

What model and what settings do you use? Can you share your image?

[sweet abyss]

stable diffusion sdxl, dpm++2, sampling steps 20, 1368x2048

[glass forge]

controlnet is useless once u go beyond a thershold of weird angles

top of head, below weird side angles, yoga poses, forget it,

early days folks, emerging tech, we ar ebleedign at the edge - the days of rendering a perfect consistent 3d view of a thign form any angle are still some time in the future

but its coming.

[vapid flint]

how do I get pony to stop making every generation so fuzzy and 'painty'? I like it sometimes but I wish I could have something more defined

Using comfyui and euler ancestral, but it also shows up with dmpp

[glass forge]

i noticed that with pony too

images are very monochrome and washed out

try adding "vivid colors, contrast and saturated in the promt

[vapid flint]

It might actually just be a CFG issue

Reddit says that 7 is standard?

[glass forge]

yes go high if you want more defined

bake it

[vapid flint]

That seemed to help a lot actually, 4->7

[glass forge]

i guess pony likes high cfg

i didn't mess with it much coz it only renders a lady in a empty room no matter what I prompt. 😄

[vast galleon]

I'm gradually learning more about iterative dare/ties merging. Merged clips of Halcyon 1.7 with Mangled Merge XL V2 at 50% ratio, no drop, at 2 iterations and then smoothed it by merging it with the SDXL base clip at 1 iteration 50% ratio. Getting some nice outputs with it.

[gloomy lark]

really good stuff

[glass forge]

So what exactly happens when i merge to checkpoints?

[vast galleon]

Thanks!

[glass forge]

so if i merge 25 checkpoints and prune it does that mean i get a great checkpoint or a mess? merging is not the same as training i assume

[vast galleon]

Never tried it. I just merge everything already pruned. Worth looking into though. I'de be interested to see the outcome.

[glass forge]

thats the thing i tried mrging like 12 sd15 checkpoints and ...

idk

was the result better i am not sure

it was different yes

[vast galleon]

You gotta mess with the weights. The merging method also matters. I'm still trying to wrap my head around attention gradients and masks. Then factor in iterations which I'm convinced is merging on steroids ... but can easily break the model.

Not even to mention block weights which is still fairly foreign to me.

Then there are new methods that haven't been implemented into comfy yet like the DELLA method. Which looks promising.

https://arxiv.org/html/2406.11617v1 It's all gibberish to me, but I'm working on understanding it. lol

[meager canopy]

@copper kraken Have you seen Aura?

That was 1st try

https://huggingface.co/fal/AuraFlow/tree/main

[copper kraken]

yep it's pretty exctiing stuff imo

been playing with it

[smoky shard]

Pony models require different prompting setup:

[glass forge]

wasset?

[tawdry current]

Needs a ton more training, but it's definitely on the right track. They going to be very open with it and you'll likely see incremental updates to it pretty frequently. Think they stated that there's no reason for them to hide it behind closed doors when they want active community feedback on it

Wouldn't surprise me if you see monthly incremental updates to it

[copper kraken]

i just saw that... pure fire, that's why i'm excited

reminds me of the philosohpy and approach that led baulder's gate 3 to historic greatness

[tawdry current]

Well as long as people give them the right feedback and it doesn't turn into some waifu creator echo chamber, the model will continue to improve.

But it's off to a kickass start. I've seen some wild prompt comprehension examples people have posted and it nails them

Usually

[copper kraken]

absolutely

i was really surprised how well it understands complex prompts

on a lot of things it matched 8b

definitely better than 2b on many things

[meager canopy]

Seems limited on samplers that work with it

[copper kraken]

yeah, rectified flow

i'll admit it's gonna be hard to convince me to make a full switch away from SDE samplers

espec with what i've been doing more recently with my nodes

[meager canopy]

These are fast changing times 🤷🏻‍♂️

[shy kelp]

wow its a 6.8B parameter model

[meager canopy]

You've gotta go with the flow 😄

[shy kelp]

so its way bigger than the 2B

[copper kraken]

yeah, much bigger

[shy kelp]

so Kolors and AuraFlow are both apache license 2.0

I actually can't see how SD3 can compete when such strong models have true open license

I mean presumably Stability AI will now use their massive recent investments to train a much stronger SD4

but I can't see how SD3 can compete in the current generation

if you don't have open license then you have to have a much, much stronger model

such as midjourney

[meager canopy]

I tried adding the ollama node and use llama3 to enhance the prompt. It's flattened my 4090

[copper kraken]

😮

[meager canopy]

It wasn't worth the wait 😄

[copper kraken]

lol

[shy kelp]

ah yeah I tend to rent 4x RTX A6000 48GB when I do image gen stuff
its running big upscalers on big batches that is the biggest issue

[copper kraken]

this is one reason i like cascade

i can generate directly at 2880x1728 with 4gb vram lol

[shy kelp]

wow yeah that's really nice

have you used hidiffusion?

its the sota for native gen res

[copper kraken]

i remember lookin ginto it a while ago but never used it much iirc

[meager canopy]

Some strange eyebrow things going on here

[copper kraken]

oof

[shy kelp]

hidiffusion is best thing that came out in last 12 months in my opinion

[copper kraken]

looks like a failed face swap photoshop, that's odd

[meager canopy]

Maybe it's limited aspect ratios to 1024x1024...?

[shy kelp]

if you combine hidiffusion with canny+depth control net
with cherry picking 1 out of every 200 gens
then you can push models really far with resolution and aspect ratios

an easier version of hidiffusion is koyha deep shrink combined with PAG

[copper kraken]

most of my generations are keepers in my mind

i've found PAG messes up a lot of images tbh

[shy kelp]

PAG is subjective yeah

[copper kraken]

it fixes obvious problems, but creates really subtle ones that are hard to fix

creates noise patterns across the entire image

[shy kelp]

it can add a sense of sharpness

[copper kraken]

kinda low/mid frequency-ish ones

sometimes it looks like a vague pattern of dots, especially with faces/skin

[shy kelp]

yeah there are side effects

[copper kraken]

my goal lately has been to just get the sampling right on the first try

no controlnets, no ipa, no SAG, PAG, anything

no upscales, or refiners

[glass forge]

What is aura and why is it 16GB?

[meager canopy]

A model, because they made it that large.

[copper kraken]

lol

[shy kelp]

lol

upscaling is my main interest personally
I spend 99% time working on upscaling and 1% working on actual image gen LOL

[molten gull]

what can aura do, compared to the other models ?

[meager canopy]

More from Aura

[meager canopy]

#✨｜sdxl message

[molten gull]

does it require different usage ? or can i just pick it as model?

[glass forge]

ok so is it worth it? is just sdxl? or sd3?

[glass forge]

boobs are covered i see

i just installed kolors, like a ne wmodel weekly now 😄

[meager canopy]

I just want an image of some guns with some roses! 😄

[glass forge]

Gunns Ras Rosses.

Classic.

it sure is sus as the drum specifies. Self aware model

[meager canopy]

That's because llama3 is turning "Guns and Roses" into a full band prompt.

Bob would not approve

[molten gull]

hmm, looks that i cant just load the checkpoint from auraflow, does it need special nodes in comfyUI ?

[meager canopy]

Yes, you need to update. They were added in the past few hours

[molten gull]

what node do i need?

[meager canopy]

Just that auraflow one

[molten gull]

one of these ?

[meager canopy]

You don't need to install it, just update Comfy

It's native

[molten gull]

how do i update it ?

[meager canopy]

https://tenor.com/view/surprise-shock-shocked-dwaynejohnson-therock-gif-7451661064872989677

[molten gull]

run update_comfyui.bat ?

[meager canopy]

Yes 😄

[molten gull]

(it's a serious question with all the fragile code these days, ANYTHING could happen 🙂 )

[meager canopy]

It's portable, easily replaced if anything goes catastrophically wrong 😉

I update every day...sometimes more

I have a script that prompts me if I want to upgrade Comfy, and then if I want to upgrade custom nodes.

[molten gull]

i try to not touch all those fragile things if not necessary 🙂

[meager canopy]

Never had any real problem with it. Can fix it or rollback if required.

[glass forge]

come on.... is it good, is it worth the install?

[tawdry current]

It means the model didn't unload from the GPU. Set the alive time to zero to unload the model after it generates the prompt

It will stay in regular ram as long as you don't force windows to garbage collect by loading a ton more stuff. I use ollama and llama3 all the time with comfy on an 8gb gpu

[meager canopy]

After the first run, it's been quick 🤷🏻‍♂️

[tawdry current]

Well like I said, just set the alive time to zero. It will prevent that from happening again

[meager canopy]

What "alive time"?

[tawdry current]

that usually happens most often when using ancestral or SDE samplers, think it's even noted in the plugin git page

[tawdry current]

on the node there is a setting called "keep_alive"

[meager canopy]

Not on mine

[tawdry current]

oh thats why, youre using some other less commonly used one for it

https://github.com/stavsap/comfyui-ollama is the one im using

[meager canopy]

Failed gens? 😉 🤣

[molten gull]

art ! 🙂

[tawdry current]

bro those are sick, not fails

[molten gull]

i only get gibberish out of that aura thing though

those images are from further steps within the workflow

does it have to be a certain sampler / scheduler ?

[tawdry current]

yes

same as sd3 afaik

[meager canopy]

It's not so good at generating a prompt by default though, you need to coax it out

[molten gull]

which one would that be ?

[tawdry current]

the only difference might be whatever sysprompt they have in the code that isn't exposed, which you can manually change. also, in the pack i use, you can just use the advanced node and define your own sysprompt right there (could also be settings like temp and such)

[molten gull]

which sampler / scheduler ? @tawdry current

[tawdry current]

you gotta remember, the addon isn't the one doing the lifting, it's ollama that it makes calls to. so as long as they have identical settings for things like temp, context, seed, etc, they'll produce identical results

[tawdry current]

sd3 uses sgm_uniform for noise and euler/dpm++2m for samplers, well any sampler as long as it isn't ancestral or SDE

[molten gull]

thanks 🙂

i try this out

[shy kelp]

gonna try kolor on comfy

I should stop abusing free HuggingFace Spaces

they are very generous but I feel guilty using them too much

[tawdry current]

it works really well, im playing with kolors right now

[tawdry current]

https://github.com/kijai/ComfyUI-KwaiKolorsWrapper is the addon you use for it

just follow his guide and use the quantized encoders

[shy kelp]

this kolors gen is the best image I have ever made

I wish SD3 could do this

its literally midjourney only

[molten gull]

i m still only getting gibberish with aura model

trying euler + sgm_uniform now

[meager canopy]

Uni_pc / Simple

[molten gull]

i will try that, too

[meager canopy]

That works

Just don't try guns yet

[molten gull]

i get something like this

[shy kelp]

VAE

[tawdry current]

yeah probably the vae

[meager canopy]

Using the VAE in the model?

[molten gull]

no, what do i have to do and where do i get what ? 🙂

[meager canopy]

Nothing, it's in the model

[shy kelp]

there is a way of baking VAE

[shy kelp]

it was not common at first
but now most models bake VAE

[molten gull]

i m using this

with cfg between 3 and 5

(and start_step_at 0 )

[tawdry current]

yes, but show the red wire and where it's coming from

[meager canopy]

The vae decode

[tawdry current]

it should like like galaxy's

[meager canopy]

No need to use the advanced ksampler

[meager canopy]

Where is the start of the pipe?

Yep, that's it

[molten gull]

that's it, there's the problem, you mean ?

[meager canopy]

Not sure if Auto CFG will work with it

[molten gull]

its not in there

its directly from the auraflow thing to the tobasicpipe

[meager canopy]

Try bypassing your pipe, in case that's not working

The rest looks correct. Not sure if that ksampler advanced is messing with it, but shouldn't do.

[molten gull]

i try without the pipe

[meager canopy]

What's your empty latent image node?

[molten gull]

ha, now i know !

that one is taking a different vae

stupid me 🙂

hmm, but it shouldnt matter?

since i make an image out of it ?

[meager canopy]

Why?!

You're decoding the empty latent

You need that to create the image

[molten gull]

and later on encode it again

(with the correct VAE)

but i try remove the encode/decode part

[meager canopy]

Sounds like a waste of processing time

[molten gull]

yes, no doubt about that

waste of processing time, true

but cause of the error ?

[meager canopy]

Perhaps 🤷🏻‍♂️

[molten gull]

trying out now 🙂

[shy kelp]

wait

why run VAE on empty latent

what does that even look like

[molten gull]

i dont have to

[shy kelp]

fancy noise?

[molten gull]

i removed that part

yes, fancy noise

[shy kelp]

there is a one in a trillion trillion trillion chance
that the emtpty latent will randomly be a real picture
and so the VAE decode will work 👍

[molten gull]

that would be so cool ! 🙂

[meager canopy]

🤣

[molten gull]

still not solved

[meager canopy]

You didn't bypass the pipe, and try a basic ksampler, without the random additions