✨｜sdxl

[high skiff]

might be game changing for me

[noble shoal]

There are also Comfyui nodes for that.

[peak dove]

Victo Ngai, Henri Rousseau, Vladimir Kush, Remedios Varo, Davi Augusto, Rob Gonsalves, Georgy Kurasov, Yayoi Kusama, Lisa Frank, Marc Chagall, Joan Miro ...

Streamline Moderne, Zentangle, Vexel, Prismatic, Grand Guignol, Arabesque, Art Deco, Alebrijes, Greebles, Fibonacci...

[feral glade]

this is my mask and original image im usin stablediffusionxlinpaintpipeline

this is my output. i have color problem.

[nimble heart]

Looks like it started to work on the right there

wonder if the other edges are a gamma conversion thing, like using linear light RGB instead of SRGB or vice versa

when submitting to the pipeline

or ig retrieving since it's the outpaint

could try using RGB tensors directly instead of PIL images maybe

or vice versa

[feral glade]

can you check dm?

[icy brook]

https://tensor.art/models/657089241613905798

[west breach]

https://civitai.com/models/141592/pixelwave?modelVersionId=214780 🙂

[vale eagle]

Pixart-alpha offically release their weight

[vale eagle]

Pixart-alpha inference and decoding memory usage. diffuser version with enable_model_cpu_offload

without enable_model_cpu_offload

[fierce hollow]

not really much of an upgrade over sd(xl) for that much memory usage

probably more of a downgrade on the contrary, does it have some great text encoder comparable to dall-e or something at least?

[stray warren]

also your aether watercolor & ink 🙂

[icy brook]

"It's either THAT way or THAT way, idk!" 🤪

[mellow sand]

question: can i use comfyui img2img to slap an Art style onto a real picture? i want to make me and friends into GTA art style but i dont know if thats possible

and if possible is there a tutorial i have been tinkering on my own but got nowhere

[icy brook]

I'd use fooocus

so much easier to combine images with ipadapter

so straightforward

[mellow sand]

i'll look into this, thanks !!

[tepid heart]

Safe to say my first attempt at SDXL wasn't very succesful lmao

[crisp owl]

vae issue looks like

[tepid heart]

Ah might be, i didn't change my VAE off the one i had set, let me disable it

Set it to automatic now

[crisp owl]

If you use the 1.5 vae, it will not work

[tepid heart]

Yeah I was

[crisp owl]

well, it'll make an image, but terribly

lol

[tepid heart]

yeah the preview up to 98% looked fine, and then it deepfried it

seems sdxl really likes to take a long time for that last 2-3%

yay no more deepfried, it was the VAE

[crisp owl]

Yeah the vae decoding takes more time as it's much more of an image it's decoding compared to 1.5

[tepid heart]

Makes sense! Now I can experiment with it 🙂

[lusty moss]

Hey can you screen shot what you mean or explain? Using the Depth map trying to add to my workflow and believe I have the same problem.

[carmine gale]

What is the agreed difference between generating the exact same prompt and parameters in auto 1111 and comfyui? I missed this conversation? How are things different apples to apples, If everything is the same? Do samplers and see and all differ completely? From what I can see there is no comparison or any way to match or bring them closer? I was getting a really groove prompting in auto1111 but in comfyui it’s really hard to get to something similar

[crisp owl]

You won't without a custom node pack which forces the seed to be generated via the CPU I believe instead of the GPU.

seed or latent. Lemme look, unless someone else knows and sees right now

[icy brook]

The image is 305x388 px. Did you put it at that?

[lusty moss]

These transparent background images are awesome.

[tepid heart]

I just took a screenshot with a snipping tool so it's not exactly my image size

[crisp owl]

https://github.com/ltdrdata/ComfyUI-Inspire-Pack

Looks like it's in this pack

But it references this pack.

So, you'll need to read up on it. I don't use A1111 anymore, but these look like what you need

https://github.com/BlenderNeko/ComfyUI_ADV_CLIP_emb

[mellow sand]

cant really find anything regarding ipadapter

[nimble heart]

it uses the same one

isn't the encoder alone like 90% of the vram usage

[brazen inlet]

clean aaahhh discord with Discord+ and a wallpaper generated with cumfyUI

[fierce hollow]

the same one as... dall-e? as in some sort of t5? sounds promising if so

[steady grove]

wonder if i'll be able to use pixart with my 16

[carmine gale]

Thanks. Went through the documentation and unfortunately no in-depth talk about comfyui-Auto1111

[steady grove]

highlights i fiind with pixart is that it's a transformers model, it was trained in 10% of the time as sdxl, and it uses the t5 encoder

[steady grove]

it's very easy to import automatic settings into comfy, by dragging an image with a1111 meta data into comfy. that should generate the exact same image. so from there, you can engineer images that you can recreate in automatic just the same

the nodegraph that is used for automatic1111 metadata will give you insight to how it translates

[icy brook]

You just activate input image and advanced under the image field in fooocus. Then you can simply chose to imput several images at once and weight them to combine them. You can also steer with text prompt for what you want. It works well.

[dapper dragon]

angry monkeys because discord is broken

[steady grove]

actually... i been doing this myself . i can't get parity of generations between UI's. i drag a1 metadata in and it generates entirely new images.

[indigo carbon]

A1111 automatically uses CLiP_skip=1 and sampler ARG from GPU

CLiP skip is simply CLiP set last layer and the ARG does LITERALLY NOTHING, just different sets of seeds

[steady grove]

yeah it was also the workflow that comfy creates for the meta data doesn't do resolutions right or step counts, samplers, or seperate prompts for hires pass, and a lot of incompatibilities. the bridge was probably good a few versions ago but something is wonked now

there is a node in the workflow that sets the clip last layer

-1

[indigo carbon]

yes, that's the equivalent of CLiP skip 1

[steady grove]

i tweaked things around a bit and this is as close as i got things to the a1 image

[indigo carbon]

when it comes to the interpretation and aesthetics itself it should be completely identical; just the latent shapes would be entirely different- not for the better or for the worst

I can easily make images with ComfyUI A1111 probably can't

[steady grove]

oh yeah i wasn't suggesting one was more capable than the other. just that parity between UI's isn't going to be a good target. Go for aesthetics instead would be the best thing. this was just an exploration of the current state of it, since i suggested earlier that you can drag a1 images into comfy

i use a1 because i like node graphs they're good for some things ...

i like swarmui too but it doesn't play nice with custom nodes. seems i have to build a very specific workflow just like comfybox or things like that

[indigo carbon]

I just use my ComfyUI workflow with SSUI then it's a more simple GUI than A1111 while still using which ever workflow I set it to

[steady grove]

i know. people say that a lot. "its ismple it uses every workflow" but it does not. it doesn't even look like any of the photos on the giithub wiki at all

i go to websites with workflows, drag em in, and its diagnostics hell

[indigo carbon]

odd. I personally use it all the time and it's fine

[steady grove]

i know! i feel gas lit here.

yeah like, i just clikced "use this workflow" on the a1 node graph, since i loaded it into ssui's comfyui editor

[indigo carbon]

never did that personally, I just load the same workflow I use on vanilla ComfyUI

it works with AIT and all that

also somehow the preview shows the ENTIRE batch

[steady grove]

it don't work. it doesn't do anything other then the prompt and the one ksampler

previews in ssui are bees knees

and that upscale 2x button. how do i change thosoe parameters. they suck

[indigo carbon]

I never use that button. the workflow I use already does 4x res by itself

[steady grove]

negative prompt way up on the left liike thats different from the wiki

[steady grove]

i only want to use that button cause any work flow i import has nothing . only the pormpt and one ksampler settings. nothing else works. i have to go into the comfy graph and use it there

https://comfyworkflows.com/ sites like this are a chore to explore. so i grab what i can from there and wow, none of them ever work. i spent hours making sure all these custom nodes are loading in workflows and are showing that they loaded fine in the startup logs. whenever i import them into the ssui nothing works. i just installed it yesterday after getting pc back from shop

i would probably pick one to use it if any of them worked

[indigo carbon]

idk, I always just use my own workflow and it works in SSUI, I haven't updated SSUI in a while though; maybe they fucked up in the newer versions?

[steady grove]

dont know what magic people are doing. feels like those starcraft players who are like "oh its ez gg haha git gud" and don't help. there's a level of technical knowledge about what specific setups workflows must have for ssui to make them work. nothing is documented

the github readme doesn't reflect the current version hardly

i hear a lot of "it just works" so i try that and waste my time so shrug.. kind of discouraged from node graphs yet again.

https://youtu.be/M-C5eeDN7Ew detweiler helps a bit and gives some insight

i guess i'll try to do the most basic 2 stage node graph since the one i spent months building doesn't work

[indigo carbon]

that YouTube channel has given me cancer at least a few times

[steady grove]

he's the quality assurance guy at stability isn't he? he's the closest thing to official manual there is

i still feel like i'm swimming in nodes though. this iinformaiton probably would've been better presented here https://github.com/Stability-AI/StableSwarmUI/wiki

[indigo carbon]

this is how SSUI is working for me personally

it works with the AIT workflow I made

[steady grove]

you use primatives to make the ssui know your node graph. knowledge that is completely absent in the wiki and i would've never known had i not just watched that 5miin video

youtube tutorials suck because none of that is indexed by search engines if its not in a written document somewhere

discord chats have that problem too actually

[indigo carbon]

idk man, I never saw those ghetto YouTube videos, I just imported my normal ComfyUI workflow and it worked.

[steady grove]

comfyui is that lego model you get but never play withh ebcaues it was fun to put together and you dont wanna break it

[indigo carbon]

every time I'm about to change anything with my inference/workflow I always make a backup on my hard drive just in case

also with the environments, I just freeze the VENV and put it in my hard drive

[steady grove]

yeah. freezing versions so that you know exactly which ones work together and which don't. that only works if you never update anything

once you've updated, nothing facilitates discovering those very specifically compatible configurations. someone new can't come along and adopt it. it's very specific knowledge you have to make it work well

[nimble heart]

it's T5 XXL. The text encoding alone takes like 12 gigabytes of VRAM but the actual transformer inference only takes like 4 gigs

doesnt look half bad

its inference is actually slower than SDXL for me

by about 10-20%

[steady grove]

try some prompt coherence. "dwarf mining the inside of a collosal rose petal" or something

[nimble heart]

apparently it fucking dies with num_images_per_prompt > 1

[nimble heart]

-m "PixArt-alpha/PixArt-XL-2-1024-MS" "dwarf mining the inside of a collosal rose flower" --seed 0 -s 30 -b 4 -g 4

[steady grove]

thats the stuff!!

[nimble heart]

it uses DPM++ by default which is interesting

also the batch size thing might be my script lol

[steady grove]

i'd try the same prompt in sdxl but i'm training rn. oo theres bots though

[nimble heart]

already running it

./quickdif.nu -m (open models.json | get sdxl) `dwarf mining the inside of a collosal rose flower` --seed 0 -s 30 -b 4 -g -G

so ig SDXL doesnt know what to do

it picked up "colossal rose" and thats about it

[steady grove]

yeah looks like it. that's a gigantic rose haha

[nimble heart]

he's just having a lil sit down lol

[cyan crown]

😂

[steady grove]

love that pink rose color splashing off his armor

[nimble heart]

okay looks like it's not my seed batching code, the diffusers pixart pipeline is bugged for BS >1

also I cant seem to set the attention processor

any more prompt requests lmk

[half cedar]

Dalle3, prepending with muppet

[nimble heart]

reducing the cfg helps with the sorta oversaturation on Pixart. Still think XL based models do better at photos and whatnot
pixart CFG 2 vs Terminus XL CFG 4
high resolution dslr photograph of pink roses in the misty rain

[nimble heart]

pixart pipeline seems buggy RN. Can't change aspect ratio at all...

[nimble heart]

yea you're using super different params

parameters: cinematic photo by Luc Besson, high resolution dslr photograph of pink roses in the misty rain   <lora:xl_more_art-full_v1:0.8>, 35mm photograph, film, bokeh,  professional, 4k, highly detailed
Negative prompt: big hands, fake, fake hands, distorted, drawing, painting, crayon, sketch, impressionist 
Steps: 70, Sampler: DPM++ 2M Karras, CFG scale: 7, Seed: 2400344650, Size: 1344x768, Model hash: 31e35c80fc, Model: sd_xl_base_1.0, VAE hash: 235745af8d, VAE: sdxl_vaefp16.safetensors, Style Selector Enabled: True, Style Selector Randomize: False, Style Selector Style: ComfyUIPhoto, Lora hashes: "xl_more_art-full_v1: fe3b4816be83", Refiner: sd_xl_refiner_1.0 [7440042bbd], Refiner switch at: 0.8, Version: v1.6.0

[cyan crown]

yes

[nimble heart]

i dont really think its fair to compare a model that came out 2 hours ago to a full A1111 setup with loras and other junk

also you're using the wrong sampler

refiner needs to run on Euler

[lusty moss]

Best way to get good hands? They seem to be the biggest struggle right now

[nimble heart]

very carefully

[cyan crown]

With Automatic111 you can't choose specific sampler for Refiner. Also I found it works better with that one

[nimble heart]

both base and refiner need to be on Euler for the refiner to do its job properly

using the split method

if it's just img2img ing the reifner then ig it doesnt matter

[cyan crown]

are u sure? In my experiments I found DPM++ 2M working better

[sand pier]

i mean you can, there is an extention

comfy tho

[nimble heart]

I'm not suggesting. The refiner relies on the timesteps being close to Euler (DDIM works too I believe) so other schedulers will basically fight its training

[cyan crown]

so DPM++ "M for Base and Euler for refiner is better?

[nimble heart]

it needs to be the same on both

euler/euler

that's also the config SAI has XL Base/Refiner set to for Diffusers.

but all this is assuming you're using the sampling split where denoising ends early and the refiner resumes without adding new noise

if auto just treats the refiner like a high res pass it wont matter

[cyan crown]

Well at the moment I use Auto1111 standard

with switch AT 0.8

but i still don't like euler results

[nimble heart]

pixart might have some mild noise offset. images can get decently dark on their own

XL For the wendigo image is hilarious. he looks so non-threatening lmao.

its like you caught him changing

[indigo carbon]

I already did that experiment on something else. if T5 is loaded on >6bit memory usage is under 3gb

[nimble heart]

I dont think they quantize it at all

cause my mem hits > 11 GB just for T5

then it drops to 4 ish for the transformer

[indigo carbon]

they should, it doesn't hurt DeepFloyd if T5 is even at 4bit. I don't doubt that's similar to what DALL-E 3 does with it

[nimble heart]

why dont you tell them that lol

I'd say there's a 0% chance at best dall-e uses 4bit T5

the rest of the model is already fat AF why would a few gigs on a massive compute server matter

[indigo carbon]

I tested that on DeepFloyd, literally no reason not to quantize it

degradation is almost identical to how FP16 behaves on that text encoder (when it acts as a text encoder, if we're talking T5 on it's own I don't doubt it's noticeable.)

oh, also, I didn't notice PixArt was released; gonna experiment soon

[cyan crown]

This thing of Euler doesn't convince me

[nimble heart]

Pixart gritty analogue photograph of a demon and a child playing chess in a dark room

liking T5 so far. the transformer params are kinda fiddly though.

[indigo carbon]

how's the text ability and language understanding?

[nimble heart]

wdym

Its just running on my desktop so I can run whatever prompts you want

[indigo carbon]

it uses T5, so it's kinda expected to have a language understanding comparable to DE3

[nimble heart]

that's what this was kinda showing

got everything in there I asked for

XL models kinda struggle by comparison.
Terminus / XL Base

[indigo carbon]

try having text in images

T5 should also theoretically improve that as well

[nimble heart]

probably either A1111's euler or its use of the refiner in general is buggy/broken. so much of that UI already is anyways...

[cyan crown]

so with comnfy are u using euler?

[nimble heart]

aight got 2 prompts running for text

so for Pixart photograph of a dog sitting under a neon sign that reads "FARTBOX" and anime album artwork of hatsune miku with the caption "SING & DIE" the answer is big no.

[indigo carbon]

I personally use DPM++2M with karras, but they're all different

[nimble heart]

both those prompts work on XL base

[indigo carbon]

it completely ignored it.. peculiar

[nimble heart]

they likely put 0 training into text in the image

its a tiny tranformer model

like 0.6B params

ik it's not 1:1 comparable to a unet but its quite a bit less than XL

[indigo carbon]

isn't the UNET 0.6B?

[nimble heart]

its not a unet

[cyan crown]

and for refiner?

[nimble heart]

its a transformer

whatever that means idk tbh

but it uses the Transformer2DModel class instead of Unet2DConditionModel

for diffusers

[indigo carbon]

I've never encountered something like that. isn't that the architecture something like LLaMa would use? how would that work for diffusion

so it's entirely different from EVERYTHING basically; even DE3 supposedly uses UNET

[nimble heart]

yea it's the only thing that uses a transformer in my HF Hub cache

kandinsky uses a "priortransformer" class whatever that is

[tribal lantern]

pixart dissapoints me. I had some prompts SD faileed, Dalle-3 could do, but pixart is just as bad as SD. Wonder if the model is just too small, or Dalle-3 magic is more than just T5 and better captions

[indigo carbon]

If I remember correctly Kandinsky does have a priorSOMETHING and a unet, pixel diffusion

and uses ViT-14-something

[nimble heart]

that's all that's in the folder

no unet

[indigo carbon]

Kandinsky?

[nimble heart]

yea

2.2

[indigo carbon]

odd, I'll check what mine says

[nimble heart]

I use use_safetensors=True for everything so idk if that'd change it

tbh I dont trust the weird new models not to have dirty pickles

[indigo carbon]

..

[nimble heart]

the end pipeline doesnt use it it seems

[indigo carbon]

it's two seperate models, Kandinsky has a decoder (WITH UNET) and something else

[nimble heart]

oh the decoder does

the first stage doesnt

the prior

[indigo carbon]

yeah, the prior is meant to just make the shapes I think

they made it very stupidly

[nimble heart]

kandinsky gives absolutely terrible results in diffusers so idk

[indigo carbon]

I think SD3.0 will certainly use a different text encoder though; they said on the SDXL release something about SD3.0 being more coherent/instructive/whatever?

[nimble heart]

in fact i updated diffusers and now kandinsky doesnt even run lol

[indigo carbon]

idk where PixArt is going though; if they'll actually train it on a wider dataset that has text or whatever it might be a good model

[cyan crown]

maybe there's something with DE3 training

not only with encoder

[indigo carbon]

they claimed to have started working on SD3 wayyy before DE3 was a thing

they did say something about its language understanding though; I wish the showstage was archived so we can have a better idea of what SD3 will be- If I'm not mistaken they did talk about it on the original SDXL release

[cyan crown]

ok, but I mean that maybe for training DE3 OpenAI used somehow Chatgpt model

[nimble heart]

oh kandinsky works now ig it doesnt like the AttnProcessor

high resolution dslr photograph of pink roses in the misty rain kandinsky 2.2 default settings lmao

let me manually set CFG to like 2

yea better. idfk what kandinsky's diffusers pipeline defaults to but it's probably like 10 or some shit

[indigo carbon]

I did read that paper; they used ChatGPT with an image encoder to caption an entire dataset, then have the same ChatGPT process the text before it's sent to T5- making it understand stuff more easily.

[tribal lantern]

Dall-3 just did wht everyone thought would help: better captions of training data and a strong text encoder. It'd be weird stability didn't do (or try... maybe the results will be subpar) that for SD3

[nimble heart]

they did try it with dfif lol

[indigo carbon]

it's certainly possible. quantized T5 takes 3gb VRAM

so they could use it on a future model (possibly SD3)

or maybe something else, no idea

[nimble heart]

why even quant it down to 3gb in the first place

[tribal lantern]

t5 is the least intersting part to me, it feels overkill. but the better captions seem crucial

[nimble heart]

8bit is still < 8 gigs

[tribal lantern]

i hope we get something better / more specialized than t5, researchers pick t5 because it's available

[indigo carbon]

to fit on a 12gb GPU assuming the UNET will be like SDXL's

[nimble heart]

a 12GB gpu an probably run the full T5

you dont need the encoder to run at the same time as the unet

just use the encoder to generate the embeddings then move it to RAM

like what 99% of UIs do and Diffusers also does if you use the model offload

hurts inference speed like 2% just from the overhead of moving the weights

[indigo carbon]

2% isn't too bad, ComfyUI doesn't offload CLiP to CPU RAM when not used though

[nimble heart]

comfy doesnt even load clip into VRAM at all because its faster to just run it on the cpu than to copy it back and forth

unless he changed it recently

[cyan crown]

Time to go to bed

[indigo carbon]

well, it may be necessary to have the text encoder in VRAM if it would be T5

[nimble heart]

probably.

[indigo carbon]

or maybe SAI made an entirely new text encoder?

[nimble heart]

no

[indigo carbon]

yeah, so SD3 would certainly use some variation of T5/whatever other encoders are there

their goal is probably to still have the model run on normal GPUs though, so they'll either use a quantized version of whatever encoder or just a text encoder not as fat as T5

[nimble heart]

or just say "lol gotta swap it"

like pixart did

yea on Pixart with model offload I peak at exactly 12.0 gigabytes when T5 runs then only like 4.7 during inference

without offload I peak at almost 17GB

speed is the same, you just gain like a second of overhead from copying the model back and forth

[tulip kelp]

Do any of the SDXL GUIs come with an option to create output images that can tile in x/y ?

[bright valley]

There's ways to do it with comfyui and auto1111

[slender coral]

Can anyone see why this is happening to my images?

[nimble heart]

data lol

such a helpful and descriptive typing

[glad grove]

inb4 blocked 🌚

[nimble heart]

blocked?

is whoever had the 450 IQ decision to cram every output into an opaque 'data' type a moderator or something

[glad grove]

we are not worthy 😔 🙇‍♂️

[nimble heart]

have a neat cyberpunk thing PixArt spat out before i get banned then I guess

[slender coral]

Using Serge workflow, all output and input are "data"

[nimble heart]

gross

[thin nova]

anyone experience their trained LoRAs performing much better on finetuned SDXL models than on the actual SDXL 1.0? I train them using 1.0

I wonder if my LoRA model is just trash but the finetuned models make up for it

[vale eagle]

https://github.com/openai/consistencydecoder

[high skiff]

@bright valley The company I have the meeting with is extremely interested in multiple projects I'm working on. Turns out they use my workflow that was released when SDXL first came out, and they like it a lot.

Additionally, they were interested by the training I have been doing with realism LoRA, and they said that they would gladly sponsor my project going forward.

They informed me about three potential positions that they would be willing to hire me for, as well as telling me that the founder that I'm going to have a meeting with soon may have more.

Overall, I'd have to pay that that meeting went absolutely incredible, and I'm so excited at the prospects of possibly working with this company

Next step right now is waiting for the information on a meeting with One of the founders.

I'm not too sure when it's going to happen, but they'll get back to me when it's planned.

Until then, it's time to buckle down and grind some improvements to show off in that meeting

Oh, they also said that they might be interested in funding the entire research group that I'm a part of.

I will be keeping the company's name out of this for the time being, because I don't want to plaster all of their personal stuff all over the place. I'm just extremely excited and hopeful for this opportunity

[bright valley]

Cool let me know when you see a dollar

or if they want to fund the development of text generation in SD

[high skiff]

I think my chances were grealy improved because the person who I was talking to knows who I am and what I have done in the past. They had a good read on my character and passion for all of this.

I'm extremely excited to work with this company. They were my first pick from the beginning, and I like their product the most

[bright valley]

https://www.instagram.com/reel/CzTjy52NlqM/?igshid=MzRlODBiNWFlZA==

hellocivitai

Easy graphic logo design!? Say no more! Thanks to @the.harrowed, we now have an incredible LoRA, Harrlogos, for just that! Trained off of his own graphic design work, the possibilities for anyone to make their own incredible graphic logo truly are endless thanks to SDXL!

Link directly to the LoRA page in our bio!

#civitai #stablediffusion #...

Likes

125

Instagram

civitai put out a pretty great video today

[high skiff]

One thing is for sure. This company has a lot more passion and couth than NightCafe lmao. Really dodged a bullet with them

[vestal jay]

SDXL Experts.
Let me ask about the root cause of my issue. How can I get perfect hands on SDXL?

[spring fulcrum]

First off I want to say... Amazing work. It looks fantastic. I was wondering if we can use that lora you made to make words on pictures to sell?

[bright valley]

Not at this time, no, sorry.

[spring fulcrum]

Thats fine. I just figured it wouldn't hurt to ask.

All the same great work

[bright valley]

Absolutely, But hey thank you very much for the kind words.

Really appreciate it.

Thing is, it's trained on my own art, as I am always making logos, it's what most of my commissions are

So, trying not to shoot myself in the foot too bad by even releasing this 😂

[spring fulcrum]

That is a pandora's box and its hard to close. I have gone through a lot of the same issues

[bright valley]

[vale eagle]

1: Original Image 2: 1.5 decoder 3: OpenAI's Consistency Decoder

[spring fulcrum]

One thing I learned in business school that helped me not to worry so much about it was the fact that... Have you ever noticed you will find in most places a mcdonalds right near a burger king or wendys. It is actually done on purpose... There have been many studies that have shown that competition in the same close area with similare products actually causes all of their business to increase by alot.... Just something to keep in mind, it helped me not to worry about sharing what I have so much.

[bright valley]

oh, yeah sure, it's actually not even about competition YET, it's about me giving away the ability to create the stuff that I do, and in my style, by using the LoRA

Basically completely devalues my ability to create it, if everyone else can too

[spring fulcrum]

you have a good point. anyways im off to bed... 6am comes way too early. great work and have a good night.

[native knot]

Your link says it only works with SD1.5 latent mapping. This is #✨｜sdxl.

[vale eagle]

You are right

[native knot]

Just figured you'd want to hit the target audience. 🙂

[slender coral]

I know it's almost impossible to tell, but any ideas on this? ERROR diffusion_model.output_blocks.5.1.transformer_blocks.0.attn1.to_out.0.weight shape '[640, 640]' is invalid for input of size 1638400

[timid garnet]

Can you see the error ?! 👀

[slender coral]

Boobs not big enough

[kindred shoal]

Hand has 4 fingers?

[hoary saddle]

haha, that funny

think it hurts when she fist bumps?

how the hell did you even get it to do that?

[crisp owl]

tiled upscale with too much denoise

(guessing)

[timid garnet]

Nop, just a normal pic

[crisp owl]

haha that's funny then lol
I've had weird stuff like that with tiled upscaling, but haven't noticed in a normal gen lol

[timid garnet]

Here is a failed upscale

[crisp owl]

https://tenor.com/view/i-see-no-problem-here-crazy-eyes-gif-12788383

[hoary saddle]

everyone needs NippleKnuckles

[indigo carbon]

I did “man breaks through his wall with his bear hands”

that's surely what I got

[indigo carbon]

guys OpenAI just opensourced (partially, I think) DE3's VAE

I have no idea why, but it's just chilling on their GitHub

[vale eagle]

Consistency Decoder. I made a custom node for comfyui

But it only works with 1.5 latent

[indigo carbon]

why's that? as far as I know you can use different architecture VAEs on latents from different models

[vale eagle]

If you interested, here is the repo https://github.com/lrzjason/ConsistencyDecoderNode

[nimble heart]

Ayy that new model patcher node is badass. Just needs to work with the guidance rescale node to fix the zsnr oversaturation

[vale eagle]

Pixart-alpha: man breaks through his wall with his bear hands

Pixart-alpha: man breaks through his wall with his bear claws

[indigo carbon]

if you checkpoint save an already patched model, would the architecture be different?

[nimble heart]

?

wdym

ig "model patcher" isnt accurate. it just sets the sampling params that KSampler reads

the model itself is unchanged

[indigo carbon]

ohh, gotcha

[nimble heart]

so that model is already a velocity zero terminal SNR model

normally it wouldnt run in comfyui

[indigo carbon]

why not epsilon? that's what most models use

[nimble heart]

I didn't make the model lol

i know it's necessary for some of the things the model trains with

and zsnr i guess is an alternative to offset noise. lets you make some truly dark images

It's pretty fun when it shakes out right. These pics are with diffusers still to utilize guidance rescaling.

gets brighter too

model is https://huggingface.co/ptx0/terminus-xl-gamma-training if you wanna play with it

[indigo carbon]

why the graininess though?

[nimble heart]

its not done training

[indigo carbon]

this is what I get with that prompt

[nimble heart]

interesting checkpoint name

[indigo carbon]

what's wrong? It's a model I uploaded recently

it's a generalized model I made by block merging different types of models in calculated ratios

just having (dark photograph:1.2) in the prompt is enough probably

[nimble heart]

try dark photograph of a wolf in a black forest at night

[nimble heart]

30 samples 1256x840

[tulip kelp]

Thanks. I found a node for comfy but the auto1111 tile option only seems to work with SD1.5

[high skiff]

does anybody know what happens when you train on images that are not rounded by 8's?

[nimble heart]

kohya finds your IP address

[high skiff]

BEIN

lol

[indigo carbon]

this is what happens without dark photograph

wth? I added weight to dark photograph and it's now entirely different

[vale eagle]

dark photograph of a wolf in a black forest at night (50% training new version and jiemu beta)

50% training new version

[west breach]

bork in the dark

[indigo carbon]

adding some weight to dark photograph did do it imo

[icy brook]

I’ve been told by SAI that they’re not included.

[high skiff]

interesting

[icy brook]

I just made a script for that: to make sure all images are resized in relation to their recommended target sizes.

[indigo carbon]

or stretched to the closest 8px increments?

[icy brook]

From what I’ve heard that’s not what’s happening. But it might be different between trainers.

[high skiff]

I am training in kohya right now and none of my images are fit to 8 pixels, but they seem to be training fine, so I guess I will see how it works

my loss is decreasing, so who knows, maybe it will be good lol

[steady grove]

dark needs context because it's often describing colors too. so "dark night" helps to contextualize it into lighting, not fur tones

diffusion has full on adhd fixations about adjectives for colors

bright, dark, red, green, you know

use one of those and the whole entire scene is affected

[indigo carbon]

I used "dark night" here, it certainly did the trick imo

[steady grove]

night time photography is one i like too. you get some neat exposures

[high skiff]

@icy brookso, turns out it works fine in Kohya. My results look great

In fact, this training fixed an issue I have had for months

granted, I changed some other things around as well

[icy brook]

well good then!

[high skiff]

my Realism LoRA can do some pure magic, but I was finally able to fix the damn crust issues. Only issue is, I changed so many thing, IDK what actually did it, but hey, I have a process now lol

left was my previous huge hybridization, and right is my new layer I injected to fix the crust

you can see here these noise patterns and crusty bits

which are just gone now

its so much cleaner now

Granted the one on the right is higher res cause it covers more area, but you can see how it fixed the artifacts around the ears

here as well

[wet nacelle]

GUYS!!! CHECK THIS OUT!!!!!

[high skiff]

eh?

[wet nacelle]

oh https://civitai.com/models/185906/neil-breen

[wet nacelle]

ha

woops

I do that so dang often too

[crisp owl]

I love sdxl's creatures 😍

[cyan crown]

me too

[native knot]

Agreed.

[lusty wolf]

Who said bugs...

[shy kelp]

What is difference from A1111 and the discord bot? The same prompt in A1111 produces much less impressive results

[visual glade]

the discord bot is comfyui based and has some extra secret sauce

[cyan crown]

bugs from 80's

[indigo carbon]

they've confirmed that both the bot and Clipdrop use OneFlow, so it may not be ComfyUI

[visual glade]

the bot is comfyui but I'm not going to say which optimizations it's using in the background

[indigo carbon]

that means it uses some private version of ComfyUI then. they did say somewhere it's using OneFlow

[cyan crown]

Well with Auto1111 if u use the right prompt results are awesome

[sharp robin]

was up what is the latest and greatest with SD?

[crisp owl]

as in?
SDXL is the newest model

[sharp robin]

as in after its release,

i dont see too much new, i guess barrier of entry (gpu) is a bottleneck still

[crisp owl]

uh, lots of finetunes, bug fixes, additional features, etc.

ComfyUI is the best thing ti use for efficiency of GPU usage, but requires a bit more learning

Comfy puts out a blog giving some of the bigger updates
https://blog.comfyui.ca/

[shy kelp]

so how do I get nodes in comfyui to get good results?

[crisp owl]

Easiest thing is to just use someone elses simple workflow

[sharp robin]

oh I got my comfy worflows on point. Mikey was a big help

[sharp robin]

double click on the screen a search bar comes up

instal comfyui manager

[pure crystal]

since release, this mini era we understand the importance of fine tuned models and explicit styling (via prompting, upscaling workflows, LORAs) for maximum (amazing) quality of SDXL

[sharp robin]

oh can we train the clip encoder yet?

i think that was biggest issue

[pure crystal]

more capable language layers will help any txt2img pipeline

[crisp owl]

I don't do any training, but @indigo carbon @vital ermine and a couple others have discussions about it often. They may know more regarding

[sharp robin]

how about samplers i see they added more, any treasures there?

[cyan crown]

maybe training a model with better caption can help having better understanding of the prompt

[sharp robin]

thank you much appreciated

[cyan crown]

an idea can be let Chatgpt caption images and then use these captions to train a new model on sdxl

[sharp robin]

i think dalle does something like that

[cyan crown]

yes but they have also the other side

but this could be a nice try

[indigo carbon]

basically training CLiP in that way MAY help a little, but there's still a not high limit caused by CLiP's LM being 100M-ish params

so there will likely always be a limit in the coherency/understanding caused by CLiP. that's why newer models like DE3 use T5, which is just a fat text encoder

supposedly SD3 should have that issue resolved, but it's probably going to pop out of nowhere due to SAI being traumatized by when they were working on SDXL. so they're probably already training SD3 with a different encoder, they're just being very silent about it

[sharp robin]

this gets me excited, and wish is true

[indigo carbon]

T5 is certainly a possible text encoder to be used on SD3; when quantized to 6-8bits degradation is minimal and it takes ~3gb, which isn't too bad

they could pull off a UNET as big as SDXL's with this, just have the text encoder on CPU RAM and have it move to VRAM and back when being used, and the requirements could be lower/identical to SDXL

[cyan crown]

good idea for cpu ram. Do COmfyUI or Auto1111 use cpu ram to store models switching from base to refiner and vice versa?

[indigo carbon]

no, but they can easily do so.

well, idk about A1111, but Comfy has very advanced VRAM management I think

[cyan crown]

A1111 is :

Model loaded in 2.9s (create model: 0.3s, apply weights to model: 2.1s, load VAE: 0.1s, calculate empty prompt: 0.3s).

and after for refiner : Model loaded in 2.0s (create model: 0.1s, apply weights to model: 1.4s, load VAE: 0.1s, calculate empty prompt: 0.2s)

[indigo carbon]

A1111 is outdated. it's irrelevant for this atm. it's up to AUTOMATIC to catch up

[cyan crown]

that is good, but maybe using CPU RAM can go down to 1 sec

I don't remember COmfy times at the moment

[polar epoch]

I'm just waiting for comfy to support tensorRT, as i read on git that they ain't prioritizing it. So i'm staying with auto for the time being as tensorRT tripled/quadrupled my generation speed

[indigo carbon]

it has something faster called AITemplate

also more flexible

it's architecture specific, so either compile 1 engine for each model type or just use the precompiled engines

[cyan crown]

and what about loras?

[indigo carbon]

LoRAs are basically patching the model, not changing architecture, it'll work with any lora if you just hook it up to the AIT node

[polar epoch]

Can it use tensor cores to generate faster?

[indigo carbon]

literally what it does, and it will also support AMD soon

[fierce hollow]

I've been reading about this official AIT support in this channel for the past month or two but it's still nowhere to be found, how mysterious

[indigo carbon]

this node will be built into ComfyUI eventually: https://github.com/FizzleDorf/ComfyUI-AIT

[fierce hollow]

I've heard that before too, even back when it was still in the old, now archived repo

and yet 🤷

[polar epoch]

Hot damn, how low ago was that added?

[indigo carbon]

it started as a less official node more than a month ago, now it's up to Comfy to have it as a built in node

[cyan crown]

I read controlnet has problems with AIT. Is it true?

[indigo carbon]

yes, kinda. I was going to push some modules into the repo with the precompiled modules, but then the node was replaced with a new version which doesn't have controlnet compatibility yet

it wasn't problematic compiling the engines for the CNETs, but due to basically each CNET being different it wasn't as simple

[slender coral]

Anyone got a good resource on comfy ui local API?

The docs are pretty bad.

[nimble moss]

How to generate images

[cyan crown]

I'm trying but can't find AITemplateLoader in custom nodes in manager

[indigo carbon]

I'm not sure if that latest version of the node works atm due to issues on Comfy's end (I think). I'm personally using an old version of both ComfyUI and the original node for inference

[cyan crown]

ok it doesn't

[polar epoch]

what's the loader/node called? Just installed it.

[indigo carbon]

simply hooks up to the UNET/last LoRA chain. the old version of the node automatically selects the right engines for the right gen settings

[polar epoch]

Nothing here Added through comfy manager, and also pip installed requirements

[indigo carbon]

yeah, you likely installed that latest version of the node which isn't compatible yet. I just use an old version of ComfyUI and an old version of the node

[polar epoch]

Ah ok. which comfyui version, same for the a.i one?

[indigo carbon]

I can send here the commit I use, keep in mind that it's outdated

[fresh path]

u only need to do text encoder once vs like 60 times for the diffusion so it doesnt matter what size it is

[polar epoch]

No worries. As long as it can read SDXL, that will work for me Or just use SD 1.5 lol.

Unless it's a "hassle", could you pretty much just zip yours with no models? :P

[indigo carbon]

I use this ComfyUI commit: https://github.com/comfyanonymous/ComfyUI/tree/bc76b3829f5fbba7c5a439c7833d313a3ca87398

[slender coral]

Does anyone use the comfy UI API?

[indigo carbon]

with this version of the node: https://github.com/FizzleDorf/AIT/tree/acd1d80c52bc0713f8cc8e2f59fd50e1adb47ec0

[indigo carbon]

I probably can, but not sure how'll this server like people sending entire inferences

[zealous horizon]

Small prorotype for generating reliable text with ComfyUI as a backend and SDXL.
Wondering if I can somehow integrate this in Comfy as custom node 🙂

[polar epoch]

You can dm it to me

[indigo carbon]

yeah, alright

[polar epoch]

downloaded the comfy you recommended, downloaded latest of the AIT, but nope, still not there

[crisp owl]

What'd you do differently to get good text gens?

[fierce hollow]

if you look at the console it's probably failing at installing aitemplate and so not loading at all

[zealous horizon]

I sprinkle a bit of magic onto it.
Will release a case-study soon.
It's rather limited but good enough for many cases.

[crisp owl]

I like magic ✨

[polar epoch]

TDG sent me their thing, and now it loads as it should :) But AIT is only 43% faster than without. Which isn't bad i guess as it generates faster nonetheless. Just that automatic generates 3-4x faster with tensorRT :P

[fierce hollow]

trt sounds fun, I should install a1111 again

[wet nacelle]

Does anyone have or know of a generic ComfyUI workflow for image upscaling?

I want him but better and bigger

[polar epoch]

Downside though, you need almost one 2GB trt per sd 2.5 for each resolution variant you want. You can make a more variable trt for more than one res, but it's not as fast. But still more than 2x of AIT by the looks of it lol. I went from 7 it/s on 3090 to 22, then with new dll's for cudnn, 26-27 ish

[polar epoch]

Latent upscale.Here's my 4k upscale that looks weird and sorta ass, but look at details

[wet nacelle]

Without downloading it first is it just insert image and pick my local upscaler file?

[zealous horizon]

It can do some amazing things

[polar epoch]

Pretty much just this. Straight from first ksampler to latent upscale where you choose res, or upscale by, which multiplies by 1x, 1.5x, 2x etcc.

[wet nacelle]

How are you prompting it? DId you teach it to understand different ways of typing?

Ex: "words"

Ex 2: words

[polar epoch]

@wet nacelleJust keep in mind, vae decode will eat memory at 4k with SDXL

[wet nacelle]

Brotha thinks that I want 4k. Little does he know that I own a 3070Ti and a 1080p display.

[polar epoch]

Really wish vae decode could have a progress bar

[polar epoch]

But have 128GB ram, and 64 of those will be shared memory

[zealous horizon]

It's not trained. I just prompt it like you would prompt for text in SDXL. Woerks with every SDXL model

A refrigirator with letter magnets "A" "B" "C" "D" "E"

A spooky DVD cover called "Tales from the" "Prompt"

Different font? no problem 😉

[zealous horizon]

A man holding a sign that reads "I am home"

[zealous horizon]

A meme of a person with huge (nose:1.4) and giant glasses titled "Who nose" "what's up", caricature

[bright valley]

Very cool Pixel!

[glad grove]

(massive,giant,gargantuan,colossal nose:1.7)

[bright valley]

That is the same quest I've been on as well

[zealous horizon]

A meme of a person with (massive,giant,gargantuan,colossal nose:1.7) and giant glasses titled "Who nose" "what's up", caricature

[bright valley]

SD text is a fickle beast 😂

[bright valley]

What does your method entail? pls tell me no control nets or anything

[glad grove]

maybe latent manipulation ??

[zealous horizon]

we already tralked about this 😉

[bright valley]

did we? It must have slipped my mind i apologize

Was it here? I just tried to search for it and don't see a mention of it before today

[nimble heart]

he's got that quad exhaust

[bright valley]

Starring NOOEE NOEE?

[native knot]

That's what you get when you go low budget.

[bright valley]

I still think this one my buddy Dever did is one of the best takes on an existing logo

[native knot]

So many errors, but it's hilarious to me, so I had to post it anyways.

[heady vale]

thick crust crap 😆

[native knot]

Was dying when I saw on the left, "PLASE CRAP"

[bright valley]

Oh is that OpenAI's consistency VAE?

[native knot]

Who's post? Mine?

[bright valley]

Yours actually look more like Dalle3

[native knot]

Mine is SDXL using Juggernaut XL v6 model via ComfyUI with no real tricks...just prompt manipulation and good old retries with new seeds.

[wet nacelle]

https://tenor.com/view/jerma-jerma985-jerma-reaction-pog-poggers-gif-26453139

https://tenor.com/view/yass-yassified-jerma-jerma985-jerma-yassified-gif-24517005

[native knot]

ENGERGY

[zealous horizon]

Yeah, typo in my prompt. can't really do anything about that 😓

[slender coral]

any ideas lads?

Can't seem to figure it out.

[wet nacelle]

https://tenor.com/view/samuel-l-jackson-yes-yes-i-do-laughing-gif-14154876

[glad grove]

well he did appear on the shitty live action adaptation of Kite so i assume he watched the uncensored version 🤠

[wet nacelle]

He's a skippin and a hoppin

[native knot]

Supposedly, Shaq has the record for the largest normal purchase from a Walmart: $70k.

[wet nacelle]

THAT'S FUCKING TRUE!!!! holy shit man!

[sharp robin]

yo that was a sick lora u made, congrats. also any tips for more consistent prompting with it and lora strength?

sampler too if its not too much to ask for

really neat lora

[bright valley]

thanks I appreciate that a lot

Pretty much everything is spelled out on the model page

How to structure prompts for it

What seems to help most people is, don't over complicate it

[sharp robin]

thank you

[bright valley]

I've found the most luck with euler, but if you look through the gallery on the model page you can see all the parameters everybody is using to make theirs

[bright valley]

are these like.. photos of an arson or something? 😅

[wet nacelle]

house fire

nize house fire

[bright valley]

House fires are a type of arson

[glad grove]

walter white house/lab

[bright valley]

can be, I should say

[wet nacelle]

@glad groveyes. My type of fire. House.

[bright valley]

Ooooohhhhhh, that is very cool. But I think I hear my dog, calling me... for dinner.. thanks for the chat!

[wet nacelle]

https://tenor.com/view/jerma-incinerated-fire-devil-jerma-wrestle-gif-12587590786712530565

[icy brook]

Uploaded this to Civitai just now:

https://civitai.com/models/193615/amelie-lora-for-sdxl

[zinc cargo]

hi, is there a nice straight forward inpaint workflow for SDXL circulating?

[wet nacelle]

I often wonder what they do in there when muted.

[wet nacelle]

Don't worry guys. It's just airsoft.

[pure crystal]

we need to get this class of computing away from TSMC --> Nvidia. Especially for LLMs, the state of the art (OpenAI newest features) are barely real world usable. I am not anti-corporate, even this stuff running on Intel ( etc) would be a win for the ecosystem

1150 frames of GPT V (video transcription) is ~$30

[floral island]

this merge might be pretty decent 😮

[pure crystal]

dalle -> SDXL (dalle on right)

[viscid bronze]

Dalle3 is trash

[pure crystal]

The text encoder layer is very helpful for composition

[viscid bronze]

Ask it to draw you a group of german ww2 soldiers

[pure crystal]

I have no problem with a commercial service having content terms of use

[viscid bronze]

Who said anything about having/ not having content terms of use?

[wet nacelle]

You actually

[viscid bronze]

No, I didnt lol. Those are **your ** words. I didn't say anything about a TOS at all.

[wet nacelle]

You kinda did

[viscid bronze]

I kinda didnt.

[wet nacelle]

sure....

[viscid bronze]

Some group you got there lol

Also, good job following directions lol

[pure crystal]

ok no problem

[viscid bronze]

This is what a "group of german ww2 soldiers" should look like. btw.

[pure crystal]

if you want a freebie just ask

[wet nacelle]

Give a real response not an emoticon. @viscid bronze

[viscid bronze]

What is my response supposed to be? lol

[wet nacelle]

something with words that another person can gather information from.

[viscid bronze]

Try something with words that another person can gather information from.

[wet nacelle]

I'm just uploading images. This is the #✨｜sdxl channel after all.

[viscid bronze]

Give a real response.

[wet nacelle]

Are you autistic or 12?

You are just throwing back at me what I already said to you.

I'm just confused man.

What was your goal today?

I just realized something.

The first thing you said in here today was "Dalle3 is trash". You then proceeded to challenge another user on what Dalle3 can produce.

You came in here to be negative.

[sharp robin]

Guys let’s chill a bit.

[wet nacelle]

He blocked me (I think) so at this point it is over.

[sharp robin]

Do y’all have a prompt to use to test stuff out you default on?

[wet nacelle]

What does that mean?

[sharp robin]

Ehh saw that after. Damn phones. Prompt to judge a model/settings on. —-that you default on.

[wet nacelle]

I don't but if I did it would likely be something simple that I know works in other models.

I do keep a list of prompts and styles that I like. I test each one to see if I want to keep the model or not. The list itself is 120+ prompts and negatives as well.

[wet nacelle]

.

[sharp robin]

That’s too many to test. I want to build up maybe 5 prompts.

And usually do no negatives

If u need a whole dictionary in negatives. It’s kind of a bad model.

[wet nacelle]

I don't. I use different negatives for different models. I use a list of prompts that I like to see how they compare to my other models.

[pure crystal]

certain negatives for certain styles is a thing unfortunately. especially for 1.5 models

I would LOVE a community test suite for prompts and styles. it should output all the images and a lightweight html page.

[wet nacelle]

That would be wicked

[sharp robin]

Yes. Agreed

[pure crystal]

this is a natural thing from the devops/continuous integration world, pretty easy to script

[sharp robin]

Yeah for styles definitely but if every prompt for a variety of styles requires a long list of negs to look good. It’s not a good model imo.

[pure crystal]

yes

[pure crystal]

fighting early 1.5 models to do stuff got boring. controlnet helped a lot for making SD usable for "studio" work

[shy kelp]

is this chatgpt

[pure crystal]

yes dalle3

[shy kelp]

howd u manage to generate that

[pure crystal]

the prompt is there in the screenshot

[shy kelp]

it wont censor?

[pure crystal]

I find if you talk around the thing you want but dont go over their line it gets around the censor

[shy kelp]

can u try generating mickey mouse

I find it very hard to generate that

[pure crystal]

that's probably too specific

[shy kelp]

I try other ones like spiderman and it works

[wet nacelle]

Ask it to describe to you what you want to see in detail. Copy and paste that into the image generation part.

[shy kelp]

it works with others like spiderman but fails horribly with mickey mouse

[pure crystal]

there was a the 9/11 mickey image that went viral, they probably have a very nuanced moderation around cartoon mice

[wet nacelle]

okay. I don't care.

If doing what I told you to do doesn't work then it doesn't work. Although I feel that you didn't even try what I asked of you.

[shy kelp]

I tried before bro

[wet nacelle]

Can you share that with me?

[wet nacelle]

okay. Sorry that it doesn't work then.

[pure crystal]

https://futurism.com/disney-microsoft-ai-mickey-mouse

[wet nacelle]

That's wicked funny

[pure crystal]

he looks like his original thing which was like a monkey

[lusty wolf]

Oh...

[crisp owl]

well that's crappy

Man these chariots drawn by Thor's two goats are incredibly hard to get nice ones of, at least for me

Multiple goats, beast-like goats, goat headed thor, goats riding in the chariot, goatchariot, and occasionally one that looks decent

[indigo carbon]

Update on the text encoder talk: it's entirely possible that SD3.0 uses StableBeluga as a text encoder. one of the main reasons DE3 and Imagen use T5 is because either their LLM is too big (GPT-4) or that's just what they got (google with T5)

using LLaMa2 models as a text encoder would be genius; the LLaMa ecosystem is heavily optimized and accessible to figures like SAI. and SAI already made a few LLaMa 2 models; so it's fully possible that one of them will be the text encoder for SD3

[heavy bough]

Does anyone know if when using stability ai API, they automatically use a refiner when calling their SDXL 0.9 engine?

[modest pumice]

does sdxl use the same watermark library as previous versions?

the one that adds individual red pixels across the image

and a further question, would downsampling an image remove the watermark if it is present?

[nimble heart]

invisible watermark yea

but most libraries dont apply it by default

only exception is Diffusers and that's only if you explicity install the invisible watermark package

[modest pumice]

oh really

i used diffusers but didn't install any watermark stuff

[nimble heart]

you shouldnt have dots then

[heady vale]

use the 0.9 vae and the red dots wont appear

[modest pumice]

are there other types of watermark that aren't as obvious?

[nimble heart]

this is different

also in diffusers you can just
pipe.watermarker = None and it'll disable it

[modest pumice]

i feel for a so-called 'invisible watermark' it's pretty visible lol

[nimble heart]

while you're at it, pipe.safety_checker = None as well so you dont get random black images

[modest pumice]

ah nice tysm

[nimble heart]

I've got a short diffusers CLI script on github if you wanna look at it
https://github.com/Beinsezii/quickdif.git

doesnt use the refiner though

[modest pumice]

ah nice that looks cool

i'm currently doing my thesis on detecting ai generated images amongst real ones

[nimble heart]

yea 100% test without the watermarker since 99% of people dont use it

[modest pumice]

and got surprisingly good results so was slightly concerned there was a watermark present

but if i didn't install the library, there's no red dots, and i'm downsampling it anyway i think i should be fine

[nimble heart]

its very obvious when the dots are there

run pip freeze to make sure the library didnt get pulled by something else if you wanna be extra sure

[modest pumice]

yeah you can tell very quickly

[modest pumice]

i'll do that, cheers

[lusty moss]

Love Doom! What model or method you using for these?

[icy brook]

This is fine.

[nimble heart]

using another NN or traditional formulae? I know people have made nn's for that before but I was curious if you could look for noise patterns in the image since there's only so many ways to sample SDXL

like since most people sample with DPM++ I'm wondering if it's possible to recognize it with a pattern match against the final image

[modest pumice]

i'm creating a CNN in pytorch

[nimble heart]

also if it's "too good" it might be better to use lossy compression on the images since that's what most social media sites will use anyways. JPEG @ 85 quality might be enough to disrupt the noise patterns

or down/upsampling the image

I've seen both used to avoid AI image detectors before

[modest pumice]

yeah right now i downsample to 96*96

as i have 50k real images and 50k genned using sdxl, and i can't be bothered to have training take ages

https://arxiv.org/abs/2303.14126

my aim was to build off this

so use sdxl instead of an older version, and do more hi res images

[nimble heart]

also could maybe validate the model against some images in this very channel, since there's a large variety of methods used here

[modest pumice]

i hope to make the dataset public afterwards if all goes well

[modest pumice]

that's a rly good idea actually

i was gonna create some kind of inference web app once it's done

[nimble heart]

yea most of the images in this channel are with the auto UI or comfy, not diffusers

so might be harder

[modest pumice]

is there a major difference?

[nimble heart]

yes

mostly with the samplers

[modest pumice]

ah interesting, would you say diffusers has a lower performance?

[nimble heart]

performance like speed or performance like image quality

[modest pumice]

quality

[nimble heart]

kinda depends

[modest pumice]

or is that comparing apples and oranges

[nimble heart]

comfyui is the 'official' backend of SDXL so it's arguably the most accurate

but for divergent models diffusers will usually do better

[modest pumice]

apologies as i'm pretty new to stable diffusion as a whole, this discord has been offering great support though

[nimble heart]

a zero-terminal SNR model in diffusers and ComfyUI

[modest pumice]

ah interesting

i'm committed to diffusers for now as it takes a really long time to generate 50k images, but i'll def mention the differences at the end

[nimble heart]

for XL base its usually smaller

let me render an image

[desert sundial]

Hi, a while ago I downloaded the stable diffusion webui thing from github and used that with SD v2.1, is there a more up to date model I should download these days that can still run on my 6gb vram pc? Sorry but I don't know where else to ask this question

[nimble heart]

diffusers and comfyui on XL base

to replicate comfyui seeds in diffusers you need to set the scheduler timestep spacing to 'trailing' and use fp32 latent noise instead of fp16

inference can still be done at fp16 for speed

[desert sundial]

How does latent noise work? Wouldn't n dimensional noise be like impossible to generate and take up near infinite storage space

[nimble heart]

?

[desert sundial]

"latent noise"

[nimble heart]

literally just

latent += torch.randn(latents.shape[1:], generator=generator, dtype=torch.float32)

[desert sundial]

Wouldn't that be like billion dimensional noise

[nimble heart]

batch * 4 * (width / 8) * (height / 8)

so for a 1024 square that's only like 65k floats

[desert sundial]

Is SD 2.1 still the most recent publicly available version?

[nimble heart]

you're literally in a text channel called "SDXL"

for the new "SDXL" model

[desert sundial]

But i can't download that as a model

[heady vale]

2.1 is about a year old now

[nimble heart]

https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0

it's been out for half a year

maybe you're a time traveler and you just don't know it yet

[desert sundial]

Thanks! https://meta.wikimedia.org/wiki/Cunningham's_Law

[crisp owl]

also on on Civitai you can select for other community models

[desert sundial]

Nah i'm gonna run it locally

[crisp owl]

yeah

download the model

[desert sundial]

Will sdxl work with 6gb vram tho 🤔

[nimble heart]

not really

[crisp owl]

I run it with 8 fine

[nimble heart]

it's doable but it wont be a pleasant experience

I'd say 8 is the minimum for smooth sailing

[crisp owl]

Will probably have to set tiled vae nodes (assuming using COmfyUI)

[nimble heart]

more than that

the weights alone are larger than 6 gb

so you need sequential CPU offloading to the extreme

else the nvidia drivers will do it for you and you'll take 10 minutes per image

[crisp owl]

I have 8, I only do tiled vae for my 2x upscaled decode vae node.
I've got multiple VM's running for my daily work, plus editing software running, and I still don't oom, and I turned off that new feature Nvidia recently set the option for, so I'd redwall sooner, but haven't so far.

I'd be interested to see what someone with 6's usage looks like just purely running comfy with sdxl

[nimble heart]

the fp16 unet is 5.14 gigabytes on diffusers

so hypothetically if you're on nvidia with access to Xformers to reduce the memory from the inference shape and the UI of your choice can move the clips and vae to cpu when not being used it might just barely fit on 6gb

or you could buy one of the many gpus that have 12 or 16 gigs nowadays and just not worry about it

[desert sundial]

Waste of money

[nimble heart]

lol ok buddy

[desert sundial]

Hahaha

You bought one

Nah but i just don't want to generate images that often, and if i did i can just use a smaller model

[nimble heart]

i just use my gaming card

[desert sundial]

Yeah and you're lucky to be born into wealth

[crisp owl]

naaaaaaaaaah everyone has a chance at wealth. everyone

[nimble heart]

i was on food stamps in highschool lol

unless you're in brazil or something, computer parts arent that expensive compared to TVs and other electronics people buy

[ivory blaze]

I dunno why tthey parked on the sidewalk on the left of the image, but whatever

heres some SDXL horror since I been on that kick post halloween

[ivory blaze]

Thats a very american misunderstanding of the world, indeed.

[crisp owl]

I've traveled and lived in various parts of the world, including townships of Africa.

If you feel so, I'm sorry

[ivory blaze]

COOL STORY and I have been a medical volunteer in active conflict zones and post war. I have worked in numerous countries as a medical volunteer in impoverished areas. Seems like you need your head pressed a bit deeper under the water... imo

I don't feel so, I know so. Poverty exists because wealth exists, you cannot have one without the other a dichotomy is inherent, even 100 level economics TEACHES this in universities, even IN america

it is not a discussion of philosophy, this, it is a matter of semantic reality.

I do not have a problem with wealth, I do have a problem with the inadequacy and narrow view of what wealth is, you seem to display, however. Ill step off the soapbox now and get back to making silly images

here, have a dog

[glad grove]

anyone can be rich they just need a small loan of a million dollars

[ivory blaze]

lol

[slender coral]

Anyone know why in some of our images our tiles are messing up with upscaling?

[slender coral]

Almost everyone on this planet has a chance at wealth just like he said. This is why people immigrate, there's nothing stopping you from making money, other than your own self.

Chance != success, but Chance == opportunity.

[glad grove]

yea the kid who was born in angola in a town full of copper miners should just immigrate to europe to become millionare

🤡

[slender coral]

Only people who have given up complains about wealth, like you're doing.

Wealth doesn't mean you're a millionaire.

The chance of success is a copper miner buying a plot of land and starting a company, they have the chance, there's nothing stopping them other than excuses.

This is what he means by the chance of success.

[glad grove]

yea nothing stopping them except gangs and warlords who extort ppl and if u dont pay they kill u

[slender coral]

Then you have a chance to become a warlord. What exactly is your confusion about the opportunities here?

[glad grove]

well i agree with you about one thing he should become a warlord

[slender coral]

I literally left a shitty situation when I was growing up, I now operate at a good amount of money. I had to leave my family and everything behind to take a chance.

That opportunity was there, and I took it. Almost everyone has that same opportunity.

[glad grove]

https://tenor.com/view/drake-notebook-gif-20708336

[worthy orbit]

I was wondering the same, I use SD XL for inpainting and it generally works, but it struggle with some things (not much with others). I hate changing back to SD 1.5, and SD 1.5 quality is usually noticible over SD XL details, so I avoid it if I can (but it is faster and works better, with lower quality). There is an SD XL inpainting model but it doesn't work for Auto1111.

[zinc cargo]

exactly what i was feeling.

[worthy orbit]

yes I wonder how people actually fix things and the like, I have a real hard time with it

[karmic crypt]

These existential and sdxl topics are hitting me hard

[ivory blaze]

No you had that oppurtunity. This whole I am the main character and everyone's life is has same opportunities as me is ridiculous. You were lucky. Not everyone is. This a very large world with depths of poverty lack of resources, and zero way outs, you will be lucky to never ever have to see. But no, 80 billion people DO noot all that oppurtunity. If it makes you feel better to believe that, go ahead... it is nonehtless incorrect...

[slender coral]

Do you actually think you're smarter than others on Discord by thinking you're are the only one with experience? You're literally on discord spending your time arguing, your life is not that complex.

[karmic crypt]

I just got this 3090 rn

[slender coral]

People have opportunities, humanitarians give plenty of opportunities if you want to scrap the bottom of your barrel for excuses.

I literally donate to these funds myself.

[pure crystal]

The floor will drop out and pennywise will take you

[ivory blaze]

No, in fact it is that i know I am not, unlike you. You're make a very broad claim that is very WELL understood to be incorrect. It is not my own philosphy, it is that i have spent 20 years of my 45 working with poverty, low medical availability, and war/post war reintegration, and you got lucky cos you don't live in mom and dad's trailer anymore, and make a few extra 10ks a year than you did when you worked at burger king

[slender coral]

I own a company, I personally made 300k last year and dropped a ton in charity for tax write offs. Thank you for your sacrifice for others that never had any opportunities like you claim.

[ivory blaze]

Good for YOU!

[karmic crypt]

Noo...😭🙏

[slender coral]

Lmfao this lad thinks that a a good vs bad guy war is the reason people suffer. 😄

[ivory blaze]

I spent last year in ukraine with missles falling over my head as a medical voluneer, I'm glad you're successful! I care little abuot how much you make, I am only saying it is easy foor YoU to say YOOU had the opurtunity and that everyone does.... but they don't

[slender coral]

America sends billions of taxpayers money to Ukraine, then charges them extortion prices for arms, mate.

[ivory blaze]

I am saying even when it is over... those lives many, have NO chance to ever come much above the surface... you''re a lucky idiot who thinks everyone can make it just cos you did..

[slender coral]

Every Russian and Ukranian has the opportunity to leave.

[slender coral]

Yes they do.

[ivory blaze]

What!?

you are an idiot

NO they don't

not at all. If you think that you are laughable dumb

[slender coral]

Who do you think is funding Ukrains war mate? 😄

LMFAO

[ivory blaze]

I spen 60 hours every 4 days on a bus with refugees

no all of hem had that option

[slender coral]

Who do you think is funding Ukrains war mate? 😄

[ivory blaze]

not all COULD leave, we took who could

who the fuck cares/1

[half cedar]

Way off topic

[slender coral]

This is why you're stupid.

[ivory blaze]

it is not aboutt money

[slender coral]

https://tenor.com/view/hysterical-laughter-laughing-gif-25735842

Okay bud

[ivory blaze]

that's you, and this is me, grow up, stop acting like you know anything about this world

[slender coral]

You have a surface level understanding about the level at best.

[ivory blaze]

piece of shit

[pure crystal]

macro photo of a bazooka joe bubble gum and small comic on the wrapper, in the hand of a chubby boy, classic arcade machines glow in the background, log cabin clubhouse arcade, summer 1993, dappled lighting, fisheye lens

[glad grove]

now that sounds like a great name for a warlord "bazooka joe"

[crisp owl]

One person here is being demeaning

[fluid juniper]

does anyone know if stability plans to release a diffusion model beyond SDXL 1.0 soon?

[pure crystal]

I have a 6GB RTX 2060, think I can gen SDXL on this?

[glad grove]

yes but it will be slow,how much ram do u have

[pure crystal]

16 GB VAE NGMI

I can try that new pruned model

[glad grove]

yea it will be slower but it will work,just remember to use comfy cause auto sucks specially with 16gb ram and 6gb vram

[pure crystal]

it would be nice to just have it locally. I use all colab

[pure crystal]

nice, what model?

[lusty moss]

Little tipsy but I believe it is this model/Lora, Tinkle from Legend of Zelda and Fairy lol.

[lusty moss]

BTW my GF loves these

[shy kelp]

Oh man, this is awesome. I tried for a whole day on the weekend to get ComfyUI and InvokeAI to work in Colab. I managed to get them both installed, but I wanted to house the models in GD due to HD space reasons, but I couldn't get that part figured out. I have a couple of questions:

1. How do you copy and paste models into Fooocus MRE (Colab)
2. Where do Outputs save to?
3. If you want to save Outputs, then you have to save them before closing Colab, otherwise they are gone, right?

TIA.

[pure crystal]

beautiful spry Elf jester, jester being,, (astral plane, spirit quest, DMT art photography, detailed natural textures, mushroom spore pattern:0.8), high ceiling, award-winning, professional, highly detailed,

[lusty moss]

Love seeing other people prompts, not sure if it would work for my setup though. Will take the aspects into account.

[lusty moss]

https://tenor.com/view/bees-knees-jim-carrey-youre-the-bees-knees-gif-13909178

[lusty moss]

The third one definitely looks like a gypsy/carny . Supreme.

[lusty moss]

https://tenor.com/view/antz-oh-fucking-lord-oh-my-god-jesus-christ-in-heaven-oh-no-gif-25230956

[pure crystal]

dynavision?

[crisp owl]

haha yeah, best I could think of quickly to pair with a caricature type image

[pure crystal]

It's such a good model + unique model, I haven't tried the newest

[crisp owl]

Yeah socalguitarist knocked that one out of the park for sure. Very unique to that model

[shy kelp]

Damn, I was so excited, thinking I've finally found a way to not have to upgrade to a new PC, but getting these errors. Would paying for Colab help? i.e. better GPU's available vs the free version?

[lusty moss]

What Gpu You running?

[opal yacht]

I've answered the first part of your msg in #🍥｜anime message, will answer the rest soon.

[pale willow]

/

[lusty moss]

https://www.amazon.com/ASUS-Graphics-DisplayPort-Axial-tech-Technology/dp/B0985X2YR1/ref=sr_1_1?crid=2IDLZ4N3YCLYM&keywords=nvidia%2B3060%2B12gb%2Basus&qid=1699507757&sprefix=nvidia%2B3060%2B12gb%2Basus%2Caps%2C81&sr=8-1&th=1

My current runner.

[pale willow]

/dre

[glad grove]

/50

[lusty moss]

/wahttt?

Link?

[steady grove]

playing with this lora https://civitai.com/models/135634/doomguy-doom-marineslayer-doom-lora-xl and got this , if doom guy was a doom babe. adetailer on the visor worked so well

[lusty moss]

Master Doom Babe 💚

https://tenor.com/view/warrior-halo-master-chief-soldier-fighter-gif-16778644

[slender coral]

Now lets see with less armor

[glad grove]

here

[high skiff]

@visual glade I just had a severe bug with comfy when I was running some image gen tests for a LoRA I am working on

I am training at only 13.6GB VRAm, cause that is easily enough to run validation images in the background, but right now I just tried to gen like I always do, and comfy cause this to happen

it was fluctuating so severely that my PC was going unresponsive and freaking out every 5 or so seconds

it was sampling improperly too, my results were coming out really messed up

I have no idea what that was about but it took me a really long time to close comfy cause my PC was so unresponsive, I couldn't see what I was doing

[steady grove]

zero suit doom slayer

[steady grove]

this gif sounds like beavis ❤️

[slender coral]

Anyone provide some light on why my tiles are so bad some times on upscale?

[high skiff]

I restarted comfy and everything, and now it seems fine, but that bug/error was so severe I almost had to kill my training run with it

[sharp robin]

@high skiff fav sampler?

[high skiff]

I am indifferent

I always change