🆕｜sd3

[dull star]

Tekken 3 Loading screen with two fighters displayed. The image of the fighters are inside square frames. Between them there is a big "VS" text in metal and a massive lightning strike separating them. The fighter on the top left is Shrek with a confident face, and the fighter on the bottom right is Joe Biden with a scared face. Dark red background.
(OMOST)

(SD3)

neither are perfect, but SD3 got it better

[low stone]

Massive bionic squirrel with intricate machinery integrated into its body, towering over a crowd of spectators, amidst a backdrop of tangled wires and digital screens, in a stark, cool-toned light

unrefined.. did well with that one.

[dull star]

which model?

[low stone]

that's 0most from their page.

it doesn't say anywhere on their documentation page what diffusion model it's using on the backend.

obviously better than sd 1.5

i'm just wondering if it's somthing i can swap out

and after looking at it, it's really just super special regional prompting. which probably means it's not going to have any transformation or special concept ability over sdxl

[noble coyote]

diffusion.pytorch.model

[low stone]

ah, well that settles it. 🙂

[dull star]

well if the original resolution is this (without highresfix) then SDXL of course

yup SDXL

@low stone

[cunning lintel]

sdxl_name = 'SG161222/RealVisXL_V4.0' so ehm, that one?

[low stone]

ah ok

https://github.com/lllyasviel/Omost/discussions/3

glad I just asked this one in the discussion then.

ok that's promising.

when comfy node? 🙂

[cunning lintel]

in here https://github.com/lllyasviel/Omost/blob/main/gradio_app.py

[tropic aspen]

So what's the difference between this and RPGDiffusion?

[dull star]

uhh

[tropic aspen]

Or that other project that did the same thing

[dull star]

yeah its similar

um

[low stone]

yeah I took rpgdiffusion's instruction and was using it against Claude3 for my own regional prompting work. Amazing stuff, but regional prompting ultimately isn't going to cut it compared to true new LLM trained models.

[cunning lintel]

it's nice idea that repo, and i don't want to complain too much, but such a weird choice to output python code, why not json or xml, so any runtime can deal with it. AI ecosystem could use more old fashioned developers

[low stone]

it's way better than nothing, but the better concepts of multi-subject need a new model.

[dull star]

SD3 with regional prompting like this would be the best

[low stone]

I had a bunch of automation that would do a lot of the rpgdiffusion stuff, and I'd say that it would output amazing stuff about 30% of the time. real stunners. the rest of the time it was disjointed concept trying to merge into each other and failing.

because it's really just attention, whereas dall-e and sd3 and hunyuan actually understand multi-subject.

sdxl will try to make sense of what you are trying to do with regional prompting, but because it doesn't have the higher understanding of these new models, it often fails at the more complicated stuff.

[hallow lion]

Dagng the messunic era has dawned

[proven pecan]

about time 😁

[low stone]

hunyuan / ella / 0most.

[cunning lintel]

Those are 3 different prompts 😬

[low stone]

will smith with god powers with hundreds of children riding on top of will smith. God rays, laser beams, smoke, glowing yellow eyes. Perched on a mountain of spaghetti.

sometimes will is on a motorcycle, sometimes not. i think hunyuan is just interpreting "riding" for on a motorcycle

there's a lot of variation in what I get from those 2 as i go through the seeds.

[noble coyote]

I get to the GRadio interface but nothing happens!

[low stone]

next seed from hunyuan

[cunning lintel]

poor diffusion models dealing with such prompts :p

[severe phoenix]

when sd3 comes out where would you rather want to find out? here, reddit or twitter?

[low stone]

twitter.

[dull star]

reddit or here

[low stone]

reddit will just be "sd 1.5 just as gewd"

[dull star]

here because we get a ping

[low stone]

I saw lykon literally defending against the "sd 1.5 and sd 2.1 just as gewd" posts the other day on twitter.

[dull star]

no you see festivalman, portraits of big boob women is just as good

we don't need sd3 fr fr

[low stone]

hah yep. although this just reminded me to set my notifications to "all messages" on the announcements channel here.

[noble coyote]

RTX 2070 cannot deal with bitsandbytes 😦

[dull star]

bruh what

that sucks :(

what can't it do that bitsandbytes requires

[noble coyote]

Omost says use the Huggingface Space instead

[low stone]

my conda ain't want none when you ain't got vram son

[low stone]

yeah, but that space is getting overloaded already. lots of refused attempts because no available gpus.

[dull star]

aw

it sucks seeing all the 20xx series people being left behind

[noble coyote]

Yes, I got 5 generations and then gave up! 🙂

[noble coyote]

Keeping up with AI needs deep pockets!!! 😄

So Xmas 2024/Jan 2025 I 'might' invest in a "5090"

My 2018 RTX 2070 will suffice

SD3@ClipDrop

[neon wagon]

the reason nearly no body uses the artisan bot is it uses the sd3 2B api............. all images look like crap i get flashbacks to sd 2.1

[dull star]

can you show some images?

oh its in the channels

they are public

[cunning lintel]

Isn't it the same model as API, supposedly the big one 😕

[dull star]

honestly looks similar to the API

can't confirm of course though

[severe phoenix]

loool but honestly lykons sd3 post look like 1.5 checkpoints, i dont like that aesthetic and i'm worried sd3 will just look as bland as that. However, the images on here give me lotts of high hopes that the finerunes will look nothing like that. mayb its just a prompting and taste thing which is fine by me.

[dull star]

yeah I hope the images generated will have variety

[velvet ruin]

The concept creation still feels really great. With finetunes, we'll get plenty of variety

[sterile pendant]

My 2080 works with it, you just can't use bfloat formats. I use BNB all the time to put llms into Q4 mode

[dull star]

ahhh

[sterile pendant]

I should amend that in saying I use it for safetensor style llms that haven't or can't be converted into GGUF formats yet. Most of the time, the first releases of models are still in 16 or 32 bit formats, so they take a shitload of vram, but BNB will let you load them without having to actually GGUF and quantize them manually

[noble coyote]

So if you can change the LLM, my 2070 should work? What change (and what way to make that change?) 😄

[sterile pendant]

Times I've had issues, I had to manually edit the python scripts. I don't know what you're using or having problems with though

[verbal epoch]

SD3 next week? 🤔

[dull star]

nono, 2 weeks

[calm kayak]

for real?

[dull star]

its just an injoke

we don't know

[tranquil grove]

画一幅杭州西湖的油画

Draw an oil painting of the West Lake in Hangzhou

So Xmas 2024/Jan 2025 I 'might' invest in a "5090"

Keeping up with AI needs deep pockets!!! 😄

[tranquil grove]

ideogram+sdxl

[silver sluice]

hey guys I was trying to join this waitlist:

You can sign up to join the waitlist here.
http://stability.ai/stablediffusion3
But the link gives me a 404, is the waitlist closed or is there another way to join?

[low stone]

The waitlist is a lie. There's only the paid api or artisan channel here.

[silver sluice]

ok thanks i appreciate the truth over getting my hopes up for nothing lol

[low stone]

You probably don't have long to wait for the public release though.

[silver sluice]

do you know if the API is restricted/limited like does it throw errors if i try generating questionable content like 'gore'

[low stone]

It'll generate anything, but it'll blur if there's more than a small amount of skin etc. so generate away.

[silver sluice]

it'll blur? interesting I think i'll pass then. so when it's released is it just going to be a safetensors file that'll work with the existing comfyui workflows?

[low stone]

That's my understanding. You can play around with pixart or Ella or hunyuan projects while you wait. With refining sdxl stages in comfy, they offer similar abilities as sd3

[silver sluice]

thanks for the lead ill look into it

[dull star]

only if they release 2B first, which might be the case because comfy(??? maybe not) said that smaller models will release first and some other stability devs are hinting 2B

[low stone]

Yeah, lykon mentioned "waiting a little bit longer" today so probably soon. I assume it's 2b. That's fine though. I'm certainly used to upscaling 512 res at this point.

[dull star]

I wonder if now it'll make it faster

since the 16 channel vae is an improvement, upscaling from 512px to 1024px might look as good as SDXL highresfix images (1024px to like 1.5x or 2x)

this has to be with highresfix cause this looks immaculate

you zoom in and it still looks right

[low stone]

I've been upscaling Ella and pixart 512 res with sdxl. The benefit of sd3 at 512 is I can upscale with the same engine, so I don't have to lose multi subject so it should be great like you're pointing out.

[dull star]

yesss

[low stone]

Can't wait to upscale Ella with sd3. 🙂

[dull star]

I want to upscale with SD3 itself, not with sdxl or other models

[low stone]

Yeah exactly. Sdxl goes in the trash eventually.

[silver sluice]

can you gives me give me a point of reference, how many billion parameters are in t he usually 6.5GB safetensor files I usually play with?

[low stone]

I think sdxl is 3.5b according to people mentioning it on the internet

But it can't handle multisubject and text like sd3 can. So that goes a long way

And the better vae in sd3

[silver sluice]

https://yuanbao.tencent.com/bot/chat <- this was broken

https://github.com/TencentQQGYLab/ELLA <- this looks super interesting, so this is like a Comfy node that improves image generation?

[dull star]

it's SD1.5 and instead of CLIP for conditioning, it uses T5

ELLA

I think pixart is the best so far

for offline

haven't tried hunyuan but not even going to plan trying it

[hallow lion]

sigma is slow

very slow

[dull star]

yes

I expect similar speeds with 2B

ok wait its 512px, so no

[low stone]

Use the 512 model. It's fast and just as good after sdxl upscale

[dull star]

pixart also has 512px, but I think 1024px is the only one that looks decent

[low stone]

I use all 3 of these engines via comfy nodes

Made with pixart 512

[dull star]

huh

[low stone]

Yeah, and the render time is super fast. Upscaled through sdxl of course.

[dull star]

oh with sdxl 512px is probably good

I just want raw results

[hallow lion]

cool story cascad eis faster than 1024 sigma and makes just as good results

[dull star]

can it make just as complex images?

[low stone]

Yeah I just use all these things to give me composition and then sdxl does the rest

[hallow lion]

me too lol

probably ill sdxl refine even sd3 stuff for a while

[dull star]

makes sense

it may look bad on launch

[hallow lion]

yep

[dull star]

idk what lykon is using but I cannot believe that "he is just simply using 2B" 🤗

I wonder if he's just finetuning it in the background lol

[hallow lion]

so i see there is evena sigma 256

[low stone]

Seems complex enough. 🙂 I have automation does them all though for whatever prompt I want. Usually one of them has something good.

[hallow lion]

what about the T5 & SDXL-VAE sigma? 4.5B parameters

do i need a nasa computer for that baby?

[low stone]

The extra models city96 guy on GitHub has smaller versions of t5 that work great for hunyuan and sigma. I don't use the biggest versions and so far no issues with prompt following

[silver sluice]

Is sd3 at 2B better than sdxl at 3.5b?

[dull star]

yes

it seems to me

[dull star]

https://media.discordapp.net/attachments/1089974139927920741/1227267272058474607/whats-the-latest-update-about-sd3-v0-qlg8jt5z9xsc1.webp?ex=665b3337&is=6659e1b7&hm=1181b5f89b190fdc8cb8811f9a051a9ed9ac63c9e3776907b3a6bcf942496499&=&format=webp

[dull star]

Tekken 3 Loading screen with two fighters displayed. The image of the fighters are inside square frames. Between them there is a big "VS" text in metal and a massive lightning strike separating them. The fighter on the top left is Shrek with a confident face, and the fighter on the bottom right is Joe Biden with a scared face. Dark red background.

Cascade

Pixart Sigma 1024px

[low stone]

Text is the Achilles heel of all these other open source model. They can't even do text as well as sdxl.

[dull star]

yes its weird

for the complex prompts I'm after, cascade is not enough

but its still a good model though

[low stone]

Can't really do any of that stuff. Sd3 will beat them all at this complicated stuff.

[dull star]

the bottom left one is nice

its so stylized

[silver sluice]

No contest lol

So @low stone what makes sd3 so great is that it has the sdxl magic sauce plus all this Ella Pixar hunyun tech built into it? Also how big do we expect the 2B model to as far as file size?

[dull star]

V

[cunning lintel]

if it's pixart showcase day, here some more

[low stone]

Awesome stuff

[silver sluice]

So pixart is tech that’s available right now for public use right? Those images look great

[cunning lintel]

pixart is a bit weird, some things it does amazingly well, others totally not, i think it just "knows" less. But yes, available now

[dull star]

0.6B maybe? I wonder if that has to do with the lack of knowledge

idk how much it matters

[low stone]

Yes. Along with a couple of others

Sd3 will be better but it's fun to play with for now

[dull star]

there's still a little prompt blending sadly

but its the best offline in my testing

T5 at 4Bit and T5 at 8bit

[silver sluice]

For Comfy UI I just like to use a basic workflow like load checkpoint, clip text encode, ksampler, vae, save image

For sd3 will I be able to keep that workflow? In other words is sd3 going to be compatible with the Load Checkpoint node?

[dull star]

think so

but the "clip text encode" will be different for sure

at least like

same clip models as SDXL, but we'll need an additional T5 model

[silver sluice]

Oh interesting thanks for the insight so some nodes have to change

[dull star]

if you don't use T5, technically nothing might change

[silver sluice]

I don’t use t5 currently, I know it’s a type of transformer but I haven’t used it at all. So will SD3 be compatible with LORAs from any line of models? Like sdxl Loras will still work?

[dull star]

no they wont

SDXL Textual Embeddings might work

[silver sluice]

Awwww all my sweet sweet Lora’s 😢

[dull star]

but since we'll get 2B probably, its probably gonna be just as easy to train Loras

[silver sluice]

I’ve never trained one I just like using premade ones. So current workflow won’t be compatible with sd3, as well as loras and even embeds maybe do they old VAEs still work with sd3?

[dull star]

no, SD3 has a new 16 channel VAE

[silver sluice]

Like if I have an sdxl anime vae is it worth using it anymore or better off sticking to baked in vaes?

[dull star]

yeah builtin is gonna be good

16 channel instead of 4 channels for SDXL and SD1.5

[silver sluice]

Sounds fancy, how many Channels is sdxl?

Oh you answered before I could ask lol

So basically sd3 breaks everything again and all the community work to develop the sdxl line of models is gonna be trashed once sd3 is released got it lol

Oh another question, apart from the 2B model what other models do they plan to release and will higher parameter models work with an 8gb video card?

[hallow raptor]

Anyone having issues with image-to-image strengths?

It feels like anything under 0.7 hardly changes the image at all, and anything over 0.8 generates an entirely new image.

Today, I saw 0.74 barely change a few pixels and 0.75 (same seed, same prompt) generate almost an entirely new image

(the "Strength" setting is divided by 100 and string'd before putting it in the form data for the request)

am i doing something wrong somehow? i'm so confused

[sterile pendant]

The differences aren't really from 4bit vs 8bit. A simple analogy is that it's just differences in prompt "noise" like if you added a , , to a prompt to shift a seed or moved a tag before or after another. The t5 is mapping to roughly the same spots in the model in both images. You're just basically seeing a different "sub-seed" due to subtle differences in the precisions of the generated tokens. Ironically, the 4bit actually looks far better in this example.

[shut sable]

you can pass the 3-clip into the default one dw

[shut sable]

all TIs will work afaik (sd1.x, sd2.x, sdxl) just that one clip wont have that TI

[noble coyote]

SD3@ClipDrop

SD3@ClipDrop

[craggy wave]

I pray everyday for the release of SD3

[noble coyote]

For Omost I have Pytorch 2.0.0-cu118, when I need 2.3.0-cu121 😄 No wonder it will not work!!!

[turbid grotto]

little bit?

[noble coyote]

Desktop Omost quite slow, but at least I will not have to wait until a GPU becomes available...!

Omost Prompt = A punk squirrel meeting itself coming back thru space-time in an intergalactic wormhole

Very bright and clear pictures

That last one 'remembered' the tufts!!!

[white current]

SD3 Release on Monday !

I have figured out why it would have to be Monday:

First, June 26, 2023 - a Monday. This is no coincidence, but rather the manifestation of the "weekday function" W(x):
W(x) = (x + 3) mod 7
For x = date digit, 26 results in the value 1 - so Monday.
The perfect day to usher in a new era of generative AI systems with SD3.
But what about 06/03/2024? Here some obscure calculations come into play:

6/3/2024 is exactly 343 days after 6/26/2023 (SDXL Release)
343 is the 72nd prime number
72 corresponds to the quantization of the SDN model in diffusion iterations
6/3 is the 155th day of the year 2024
155 divided by the Euler number e ≈ 2.718 gives ≈ 57
57 is a happy prime number
And the new Stable Diffusion 3 will operate in 57 network dimensions!

Coincidence? I think not! The weekday was also chosen precisely:
6/3/2024 is again a Monday, since W(155) = (155 + 3) mod 7 = 1
So the same start day as for SDXL, implying immediate compatibility and extensibility of the two systems.
But the most significant factor is the supernatural number 343 itself:

3 + 4 + 3 = 10
10 is the prefix digit of the Unvischen
The Unvischen enables transcendence between realms of reality
On the 343rd day, the door to a new dimension of AI will be blown open!

Undoubtedly, the developers of Stable Diffusion must be in possession of these fundamental numerical secrets and have consciously chosen 06/03/2024 as the release date for the next stage of evolution.
A cosmic event that will change the world forever!
Coincidence? Clear signs that higher forces are at work here!
Monday, 6/3/2024, was not chosen arbitrarily - it is the only logical date for the great transformation.

[noble coyote]

77+26 = 103

So 103 days

[desert garnet]

bro cracked the code

[noble coyote]

two geau!

[hallow lion]

wtf lol

[dull star]

just 2 more weeks

[noble coyote]

I cannot wait for as long as a piece of string!

I want things tied-up before then!!!

😄

...currently, I am "OmOsting" ... it likes to take a little time!

[low stone]

FedEx truck loaded with computers making its way through a war zone filled with soldiers, zombies, pandas, and men with spears, all of whom are trying to stop the truck.

[noble coyote]

Gaza!!!

[noble coyote]

... may I put the panda/FedEx prompt thru Omost at all?

[low stone]

Ella and even just sdxl did better with it

[noble coyote]

I must try Ella - althought it's just as quick to use Jan.ai and then cut and paste the text?

[low stone]

Yeah I'm using llm enhanced prompts a lot with it

This is one of the expanded prompts for it: In a hyperrealistic style, a battered FedEx truck overloaded with gleaming computers navigates a chaotic war zone, surrounded by intense, smoky lighting, earnest soldiers, decayed zombies, frantic pandas, and fierce men wielding primitive spears, each attempting to halt its progress.

A few more sd3 renders. I have to assume the final release will look a lot better.

[noble coyote]

LLM's have cultured a deeper respect for 'natural language', especially when using T5

The results are richer and subtly changed - overall better.

[gusty gale]

I feel like better aesthetic is achievable with SDXL, but SD3 is just more coherent

[noble coyote]

Has SD3 been trained on "ethical datasets"? i.e. our favourite living artists have been excluded?

It might explain its shabbier aesthetic

[gusty gale]

SD3 is not a competition to currently open models, because it's not an open model

[low stone]

Is this sd3?

[gusty gale]

no

[hallow lion]

[hallow lion]

No. It's a fever dream.

[low stone]

Dall-e did a great job with it. I feel like sd3 api has the bones for a great image. Just needs lots of refinement. Feels like they're running it at 10 steps or something.

[hallow lion]

Tsk tsk its paid and still only 10 steps. Give at least 30 steps. Thats the good average

[noble coyote]

OmAst

No Pandas!!!

Omast cannot render text like Ella?

Again, no Pandas. What has Omast got against animals?! 😄

Oops, there's half a one bottom right!

[low stone]

That's pretty good

[hallow lion]

come one man

[noble coyote]

It is a lot flatter in colour than your examples?

[hallow lion]

cant even spell fedex

[low stone]

So this is just regular sdxl. It surprisingly got a lot of it right including the text

[hallow lion]

ideogram will wipe the floor with sd3

[low stone]

So this is ideogram. Got all the elements, but it always feels like it needs an upscale refinement to go from good to great.

[hallow lion]

yeah ideogram is ugly

but does everything

[noble coyote]

MidJourney (for comparison)

[white current]

SD3+Eye Detailer

[noble coyote]

Wombo Dream (for comparison)

A better one from Omost

[dull star]

yes SD3 opted out artists

peopel could sign up to have their art opted out

nothing stopping people from just training it back in like nothing happened 😬

[noble coyote]

So it'll be an "unbroken buck" to wrangle to a high, aesthetic standard?! Bring it on ...

Omost - Prompt = a vivid watercolor depiction of diverse Rococopunk Afrofuturist women with beautiful and bold head wraps walking toward a huge moon in the background, holding hands, walking away from the camera, collaborating in a creative and productive environment, women empowerment poster, photography, inspired by the styles of victo ngai and vladimir kush

[noble coyote]

Omost Prompt = Frail children of dust, dustbowl, okies, the great depression, howl, scream, anger, dust, windblown, desuetude, anxiety, American Gothic

Very acute pin-tack sharpness

Omost's limitations: same-iness

[severe phoenix]

annnd the winner is...lool mahn dalle is amazing i cant bliv it used to be laughing stock

[low stone]

Midjourney would be amazing if it didn't look like a splatter paint mess all the time because everything is always so stylized. Even if you turn the style to 0, it's still like that.

[severe phoenix]

ok hol up, i didnt know ideogram was this good, damn

[low stone]

It's always been this prompt following as of 1.0, but the visual quality has been slowly getting better since that launch around 4-6 months ago. I hope it's leading to being able to upscale to higher res

[severe phoenix]

if anyone of these platforms add img2img then they'd seriously be competing with sd. The only reason sd nd midjourney trumps everyone else to me is single handedly because of their core capability to do img2img.

[low stone]

Kind of shows the limitations of what sd3's engine can do. I asked for images streaming out of the computer like water. Ideogram understands what I mean, but sd3 doesn't. Tried a good number of attempts along with the magic prompt that ideogram generated.

[noble coyote]

SD3 sounds like an "admission that Stable Diffusion has reached the limits of its usefulness!"

[dull star]

wdym its perfect 😆

[livid wren]

A large and menacing gargoyle stands guard over an aged electric guitar in a dark and eerie alley. He is surrounded by fantastical disturbing creatures causing a scene of surrealism. Dead roses and rotted thorns adorn the guitar.

#artisan-1 A large and menacing gargoyle stands guard over an aged electric guitar in a dark and eerie alley. He is surrounded by fantastical disturbing creatures causing a scene of surrealism. Dead roses and rotted thorns adorn the guitar.

[noble coyote]

Done using i2i Searge SDXL w/flow

[noble coyote]

#🆕｜sd3 message

[noble coyote]

i2i Searge SDXL prompt = A wolf howling to the moon in the heart of a Transylvanian Forest. The moonlight casts a path thru the forest, and eyes glow of other animals hiding there

Searge has poor prompt coherence!

i2i Searge SDXL prompt = dark subtle impressionist chiaroscuro painting with deep forest green, deep maroon, and plum purple. The text "All Good Things Will End" is written at the top in a white font. A large Tyrannosaurus stands on a rocky ocean coast, barely visible in the darkness. Small shrubs on the ground. A bright comet cuts through the darkness, casting a haunting light on the scene. A wolf howling to the moon in the heart of a Transylvanian Forest.

[severe phoenix]

can you do this witthout searge so we see the difference?

[noble coyote]

OK, I will try ... but after the Champions League Final 😄

[severe phoenix]

loool yes bro! you got your priotitties str8! who you got in the finals

[noble coyote]

The non-Searge look is flatter, more like a cartoon. Mebbe because the method lacks Searge's secondary prompt ... which is "professional photography, 50mm, ultra detailed, intricate details, sharp focus, hasselblad, sony alpha7, rich colours, surrealism photography by Max Rive, Ryan Dyar, Ashraful Arefin, Elsa Bleda"

The primary prompt used is "dark subtle impressionist chiaroscuro painting with deep forest green, deep maroon, and plum purple. The text "All Good Things Will End" is written at the top in a white font. A large Tyrannosaurus stands on a rocky ocean coast, barely visible in the darkness. Small shrubs on the ground. A bright comet cuts through the darkness, casting a haunting light on the scene"

When I work out a way to add that secondary prompt ... but now, soccer 🙂 Real Madrid versus Borussia Dortmund

[severe phoenix]

ezkiss me, bayern never whom?

[dusty wren]

does anyone know approximate release date?

[rotund ibex]

2 weeks

[noble coyote]

OK, same prompt with secondary prompt in place/Searge w/flow

SD3@ClipDrop

SD3@ClipDrop

[lucid swift]

Wow this looks cool

[trail cape]

/deam

[noble coyote]

SD3@ClipDrop for the seed images into i2i Searge SDXL w/flow

[noble coyote]

SD3@ClipDrop for the seed images into i2i Searge SDXL w/flow

[grizzled edge]

#🏞｜general-with-images

[noble coyote]

[floral crow]

generate a samurai

[lucid swift]

dont post that lol people will steal your api key

you can generate here for fee https://glif.app/glifs/clv4ca3aq0004xklr050jusec

[crimson locust]

create a sign with the word: "Schäf"

[noble coyote]

Hey! Free SD3 - 9 seconds/generation!!! Weigh two geau!!!

[lucid swift]

in the chat is no ai bot go to the web page i linked

[lucid swift]

:9

[rain palm]

are we supposed to wait until July for the weights? that's about the same time it took for XL to release, 4 months.

[dull star]

for 8B, probably yeah to be honest

for 2B, it might be June

8B is just undercooked

[lucid swift]

probably. stage that people want to wait less. but they said it would relese soon so they are cinda the reson for that

The image showcases an opulent living space that seamlessly integrates nature with modern design. The room is dominated by a large, floor-to-ceiling window that frames a breathtaking view of a cascading waterfall surrounded by lush greenery. The waterfall appears to be located within a rocky cliff, and the verdant foliage cascades down the sides, creating a serene and tranquil atmosphere. The room itself is furnished with a sleek, dark-toned sofa that sits atop a light-colored area rug, providing a soft contrast to the polished dark marble flooring. In front of the sofa is a low, rectangular coffee table with a few decorative items on it, including a bowl and a book. To the left of the sofa, there's a stylish lounge chair with a unique design, featuring a wooden base and a cushioned seat. The room's walls are adorned with textured dark stone, adding a touch of rustic elegance. The space is illuminated by a modern, adjustable floor lamp that stands tall, casting a gentle glow over the

daim one day we will be able to finetune it on art this will be wild

[clear heath]

画图

[mystic vortex]

SD3 is still not available locally??

[noble coyote]

If you go to Glif - there is a one-picture-at-a-tme FREE generator

#🆕｜sd3 message

[rain palm]

and that's a legit SD3 model and not some "i trained a model on SD3 images"-model ?

[mystic vortex]

Hmm, thanks, but I am not interested in online generators which are not very permissive.

[rain palm]

i don't think it's a legit SD3 model anyway.
SDXL was hitting beta in April last year and it took about 4 months to release the weights. as SD3 hit beta in April this year i think it'll be released next month.

they gotta ruin the model a little bit first with filters and censorship.
another point of view is, they gotta give the community something to do until their next model.

[severe phoenix]

pl these are dope.

[clear heath]

1

[dull star]

also

2B is 1024px @low stone

just in case I haven't said

[low stone]

I'm getting so spoiled with hyper-sdxl, even with a 4090 i want ultra fast render times now.

[neon wagon]

PCM : Phased Consistency Model fucks LCM https://g-u-n.github.io/projects/pcm/

[low stone]

Wow neat stuff. I just figured out the hyper sd 1 step workflow with tcd and it's basically sdxl turbo speed (half a second render time) but it's full 1024x1024, not like 512 res sdxl turbo.

And it works in realtime with any sdxl model and the quality isn't terrible anymore.

So looking forward to this too.

[neon wagon]

i wish sd3 models the big ones would work with 12gb gpus

i dont want to use the 2B model version

[cunning lintel]

The world's (and beyond) weirdest stadiums 🤡

[low stone]

Whatever billion parameters a model is, the fp16 is often a little less than double that number in gigabytes. So an 8b is probably 14-16 gigs in model size for the fp16.

[dull star]

that's not so bad

[sterile pendant]

Sdxl still takes up like 4-6gb for the unet alone in fp16. Comparing to transformer based llms, since sd3 uses dit, an 8B model in Q4 will use around 5-6gb of vram and that's not including context.

Edit because I was distracted at the time and forgot to finish the train of thought: so even a Q8 version would take up ~10-12gb vram and the full 16bit would be more like 20-24gb vram. SAI has already brought this up about the 8B model and yes, it would require a minimum of 20-24gb vram(for the model alone, excluding prompt encoders and vae), assuming fp16 format since 8bit destroys the quality in image generation models.

[hallow lion]

make an image of a guy living in a sewer but being dressed rich

[low stone]

#1237460438229450772 message

[woven dock]

Yea that's cuz the unet itself is around 3b parameters

Squeeze that down to int8 and it's gonna be around 3gb for the unet

Q8 would most likely be 8gb as any transformer based model scales linearly per n-b of parameters when squeezed down to int8

[sterile pendant]

Right, it's around 2.5B. The point is that you can really compress them down below 16bit without a serious compromise in quality. I know people were talking about that 8bit experiment earlier, but if you look, there's a ton of loss in quality compared to the others. That being said, you could always do a workflow similar to pixart simga->sdxl where you resample the low quality but high prompt coherence image with a high quality sdxl model.

We'll see how well the sd3 dit format handles quantization, but I wouldn't expect much out of it. It's still going to eat a ton of vram

[woven dock]

As for training, L40s with 48gb vram can be had for less than a dollar per hour

I don't expect Lora training to exceed a day for sd3

[sterile pendant]

One other thing is that parameter to parameter comparisons aren't all that valid when the architectures are different

Unet vs dit is literally like trying to compare oranges to apples

[low stone]

A surreal scene of the Terminator, a cyborg from the future, attending an Alcoholics Anonymous meeting. He is seated among a diverse group of people, each with their own story and struggle. The Terminator's cold, metallic exterior contrasts with the warm, inviting atmosphere of the meeting room. A sign on the wall reads "Welcome to Alcoholics Anonymous, Where Recovery Begins". The overall ambiance of the image is a mix of sci-fi and emotional realism, with a touch of dark humor.

prompt: jean Claude van damme weightlifting a bison

[flint stream]

i guess that's ONE way to lift a bison

[azure ferry]

Generate an image of Sonic the hedgehog

[cobalt moon]

guy

holy fuck

[azure ferry]

What

[cobalt moon]

did you guy see that thread on r/StableDiffusion?

[azure ferry]

What happened

[cobalt moon]

https://www.reddit.com/r/StableDiffusion/comments/1d6szfg/sd3_medium_release_on_june_12th/

[azure ferry]

Why is dart's face from Thomas an emoji

[hallow lion]

in 2 weeks!

[outer cloak]

I hope this time it really is two weeks

[hallow lion]

looooool

[cinder junco]

No email for me (yet).

Whoops, just got it.

[tropic aspen]

SD3 Medium weights and code will be available for non-commercial use only

That... doesn't sound good

[cobalt moon]

r/StableDiffusion peeps gonna prepare to bomb it into oblivion

[tropic aspen]

I thought at least it would have that subscription thingy

[cobalt moon]

but hey at least it is open source

[hallow lion]

yea

[cobalt moon]

I do not care whether it is non-commercial or commercial

but seriously... I just met someone who wanted to commercialize every AI pics he generated on Civitai

lol

[void wyvern]

open source ftw

[tropic aspen]

Doesn't matter when ppl can't use it commercially

[hallow lion]

so get refining

and make comfy nodes and controlnets

[cinder junco]

Non-commercial will be problematic if you want to use images in a personal project (game, album cover, comic book, etc.)

[tropic aspen]

Which a lot of ppl in those areas are using SD for

[void wyvern]

the more awesome free stuff there is the better off we are
regardless of it being used for money or not

[cinder junco]

And might put people off fine-tuning.

[tropic aspen]

So it is a deal breaker

[tropic aspen]

It might mean PonyV7 for instance might not be SD3 based

[cinder junco]

It looks like they are offering commercial licenses, but not for free.

[tropic aspen]

He said here it's one of the reasons he didn't go with CosXL

[cinder junco]

CosXL doesn't have any commercial terms at all, I think. Looks like SD3 will be different.

[hallow lion]

ill refine it with sdxl then

:))

or in 3 years when gpus are storg i can refine it

lol pony

anything pony is lol

[desert garnet]

but pony v7 is sdxl

[tropic aspen]

We don't have V7 yet

[real terrace]

how can anybody know you made the images with this or that ai image creation platform?

[desert garnet]

but we have the owner of pony who talks here from time to time

[tropic aspen]

He said he would use XL as a last resort though

[cinder junco]

It's possible there may be some sort of watermarking function built in.

Refinement with another model would probably destroy it, though.

And, like with SDXL, it is probably easy to just remove the calls.

[real terrace]

remove the calls?

[cinder junco]

If SD3 is released with terms requiring a paid license for commercial work, and then someone fine tunes SD3 and releases it for free (would probably be considered non-commercial), and then someone wants to use the fine-tuned model commercially, who would they contact?

[desert garnet]

the calls

@jolly swan ponyv7 is it gonna be sdxl or sd3? 🤔

[hallow lion]

Stable Pony

These are not the stables you are looking for.

[real terrace]

As it is non commercial I guess they would have to contact stability ai first and then also the fine-tuned model creator

[crude yarrow]

Got an email just now. It's dropping June 12th.

[cinder junco]

The function calls that run the watermarking algorithm. Assuming that they haven't found a way of baking this into the model.

[hallow lion]

Now I can actually say in 2 weeks and its not even a meme anymore!

[desert garnet]

🐴

[cinder junco]

But it's also possible that the fine-tune would be strictly NC unless the creator first got their own license with Stability.

[crude yarrow]

I think the Stable Audio model is going to beat it still.

[cobalt moon]

i think Stable Audio is less competitive

because the existence of Suno and Udio

[hallow lion]

yeah

they took too long!

obsolete on drop

welp luckily image generation wise there's nothing to outshine SD3 anytime soon

[cobalt moon]

If you would like to discuss a self-hosting license for commercial use of Stable Diffusion 3 please complete the form below and our team will be in touch shortly.

well

yeah

[rose gate]

omg finally

[desert garnet]

2 weeks and we done

[wild remnant]

Man I still want video stuff

images are cool...but video is the next phase

[upper snow]

After 9 years in development...

that's the only thing the announcement makes me think of

[shut sable]

also no 8b model lol

2b is better

[upper snow]

I was hoping for the 1b. Hopefully it's not far behind. I need to do ablation tests FAST.

[shut sable]

the 800m should release, dont have a date but hopefully soon

[upper snow]

it's 1b now

[shut sable]

potato potato same thing

the main issue is the tencs though lol

[prisma cypress]

finally sd3 open release!

[dusky thistle]

wait which one is dropping? what's medium

[shut sable]

2b sized

[patent acorn]

no 4b?

[shut sable]

no(t) yet

it might get canned

[patent acorn]

2b is kinda mid to me

[shut sable]

its better than the 8b

[patent acorn]

ur sure?

[dusky thistle]

from 3 days ago

[shut sable]

yep

[shut sable]

its copium

[dusky thistle]

how is that "copium"

[shut sable]

or not copium

but

lykon is wrong...

[patent acorn]

even though they said that i FEEL LIKE they should release it too

[dusky thistle]

copium = you're trying to make yourself feel better after feeling like something fell short

[patent acorn]

for the 8b hosters n stuff

[hallow lion]

2B SD3 is still better than SDXL

[patent acorn]

but how do u train an 8b

[dusky thistle]

either way, i sure hope the bigger models are coming

[hallow lion]

so whast the problem

[shut sable]

"who in the community would be able to finetune [...]"
anyone with dual 3090s/ any gpu with 30+gb

[desert garnet]

oh they know a lot about falling short

[rain palm]

800M coming later?

[desert garnet]

https://tenor.com/view/two-weeks-gif-24950918

[shut sable]

i hope

[cobalt moon]

probably will come along with 2B

[dusky thistle]

rent time if you have to

it's not gonna kill ya

[rain palm]

TWOOOO WEEKS

[patent acorn]

2b will be under 16gb right

[low stone]

You guys see the announcement?

[shut sable]

hahahahahahaha

[cobalt moon]

it doesn't make sense if they pull 800M later than 2B

[patent acorn]

i mean 16gb vram

[shut sable]

oh vram

probably?

[patent acorn]

i hope

[shut sable]

i dont have the .safetensors but should fit

[low stone]

You guys see this? #📣｜announcements message

[rain palm]

nope

[wild remnant]

I just want video honestly...."Mr bean" voice

[dusky thistle]

yup

[patent acorn]

alreadyvsseing it on gmail

apparently.. i just lost to a match on mobile legends and got the announcement haha

OH hey astral

[prisma loom]

Sorry can some tell me what mean the medium of the sd3 medium?

[desert garnet]

medium

[cobalt moon]

( real-time announcement )

[shut sable]

2b / same as xl (in diffusion params)

[wild remnant]

Medium means psychic

[cobalt moon]

it came along with AMD's announcement of 9000x series CPU

[desert garnet]

i cant belive Lisa Su now works for SAI

[patent acorn]

the hate mob is gonna be over

[wild remnant]

They'll still cry

[void wyvern]

https://tenor.com/view/impress-based-american-psycho-very-based-impressive-very-based-gif-21665669

[patent acorn]

i forgot about that aw man

they gon say sd3 is worse than pony

[cobalt moon]

wasn't that obvious lol

[shut sable]

Clueless

[cobalt moon]

Pony is like manual tag + super large dataset ( 7 million image )

[patent acorn]

comparing base with finetune is a bad idea ig

[cobalt moon]

true

[patent acorn]

the problem for pony is the separate tags

[shut sable]

sd3 (at least what i goit to test) is better than pony

[patent acorn]

what would ponysd3 be, i would need without danbooru tags for comprehension

[desert garnet]

just gotta wait a couple of years and the finetunes will be better

[patent acorn]

also its not manual tagging

rhey trained a VLM

[cobalt moon]

oh

[hallow lion]

Now that people can't bitch about waiting, they'll bitch coz "it's just medium". XD As if most of us could run the biggest version...

[patent acorn]

astral is still typing long

i need their VLM 😭

[shut sable]

psst... it runs on a 3060....

[wild remnant]

@patent acorn

[patent acorn]

not bad try far cry gameplay

[desert garnet]

size does matter so i want the big one

[patent acorn]

that's what she said!!

[shut sable]

medium better than 8b fwiw

[patent acorn]

and the crowd laughs

[wild remnant]

Drum stick roll

[hallow lion]

what doe smedium mean anyway? 2B 4B 6B

how many Bs are there

[shut sable]

2b

[dusky thistle]

what's the API running?

[desert garnet]

sorry ma'am i dont settle for less

[patent acorn]

2b is medium kinda

2b-4b i assume

[hallow lion]

well thats still way above SDXL

[wild remnant]

A lot of Bees in the honey pot

[dusky thistle]

less

[patent acorn]

aint i hear sdxl is around 3b+

[shut sable]

xl is 2.8b ish

[hallow lion]

MY comfyui workflow is ready to bend over and recieve the SD3 nodes

[shut sable]

no custom nodes needed!

[cobalt moon]

ComfyUI is already said to be used by SAI staff for SD3 lol

[shut sable]

yeah comfy works for sai

[hallow lion]

i mean the controlnets and other stuffs

i know the bas emodel can be just loaded

yall better make those

[desert garnet]

soon (after sdxl tile)

[shut sable]

control lora maybe

[neat wigeon]

@dull star sd3 weights finally coming out 🔥 its not fake (2b only but still a w)

[void wyvern]

fine-tuning sd3 seems like it'll be a challenge

[patent acorn]

challenge?

its like

the same way u train with sd3

sdxl*

[void wyvern]

T5 XXL encoder

[patent acorn]

ohh

how do u get

the encoder

damn

cogvlm can do but idk

[primal summit]

2b is really all we need. 8b Most of us will not be able to use it in the first place except on external servers, and this destroys the purpose of training and running it locally.

[patent acorn]

how long will astral yapping be over?

[desert garnet]

blud about to drop the entire T5 captions

[tropic aspen]

You already basically need to train externally for XL

Unless you have a top of the line GPU

[wild remnant]

Are we happy everyone?

[shut sable]

3060 can run them..... (sure with offload but it can work)

[dusky thistle]

doesn't need to be anything crazy

[shut sable]

a lot of people overestimating MMDiT's reqs

[patent acorn]

true

[tropic aspen]

Seriously though, what is Astralite typing? It's been over 10 mins?

[patent acorn]

im gonna see DA YAPPING

[dusky thistle]

4060 TI = 350 bucks or so

that's 16gb vram

you can train with gradient checkpointing

[tropic aspen]

4060 ti*

[primal summit]

But developing the huge model will be an expensive and long process, in addition to the fact that it is larger than the SDXL
I very much doubt it would be popular even if it were released

[dusky thistle]

i'm sure it'd be popular

sdxl with multiple controlnets, loras, etc running is popular

[primal summit]

I doubt this is for Model 8b and only lora can be trained

[dusky thistle]

that requires more than a $200 card to do

[primal summit]

lol 3000 or more

[jolly swan]

I don't know. The plan was to base it on SD3 given that SAI has allowed commercial license for all previous SD version (for the Stability AI Membership participants), so obviously this is a very unpleasant development and we will have to see how this will play out.
Pony has pretty much killed XL and made a very huge dip in 1.5 use (at least in the extended Stable Diffusion community) but SAI has repeatedly ignored my attempts to have any dialog (even me sharing any learnings from Pony to help them) so my only assumption so far is that they do not care about anything except their internal API and its users.
If they do not allow commercial use for everybody or specifically to Pony (I did apply but I have zero hope to hear back) then V7 would be XL (aka v6.9), from that point a few things may happen. If the 2B model is great then some non commercial finetunes will come out but probably would get limited traction (as they will be limited to local users and no SaaS). Alternatively they will not be good and Pony will continue to dominate the community side of things, making the whole SD3 a big lol. We will see obviously, but I am excited even about XL based V7 as it will be packing a huge number of improvements and should stay competitive for a while. As for V8, maybe we will have a from scratch model, who knows 🙂
Anyway, I think this is sad and SAI is shooting themselves in the foot - they are significantly limiting model popularity. Perhaps I am wrong and they will have commercial deals with everyone but without strong community support they are pretty much only competing with top players like OAI and I don't thin they even can take on Midjourney tbh.

[patent acorn]

oh shit

its here

[shut sable]

its like so easy to train btw

[primal summit]

By the way. Is the pony model a separate model from SDXL or perhaps it is built with the SDXL architecture?

[shut sable]

they cooked for too long and basically uses a different "latent space"

(yes i know its not a different latent space but its hard to explain)

(the tencs are too far gone so that any model trained on base XL wont work on those and vice versa)

[primal summit]

This explains a lot why I can't combine them

[patent acorn]

oh yeah please scratch model

[jolly swan]

It's a high LR finetune, so somewhere in-between a from scratch using same arch and a regular finetune, Although I am not sure how close is it to from scratch, will have to run lots of experiments.

[patent acorn]

would want to see you guys recreate sd3

[jolly swan]

well, it's just me in my garage

[wild remnant]

Man ....i just want video lol

[patent acorn]

i see

[jolly swan]

But the fact that we are seriously talking about Pony vs SD3 makes me chuckle. 🙂

[ruby marlin]

Is the source code for sd3 available for commercial use? Just not the weights?

[patent acorn]

although clone of simo sd3 wasnt promising..

[primal steeple]

I don't think SD3 needs to beat the best SDXL checkpoints for people to create better SD3 checkpoints.

[sinful ore]

Do you think you'd train pony from scratch on sd3 architecture?

[cobalt moon]

it is a hobbyist project, what do you expected

lol

he only use 1% of SD1 budget

to train it

[patent acorn]

no wonder

and its undertrained yet

[shut sable]

ehhh, not that underestimated id say

[ruby marlin]

Oh I see, even the code is licensed

[shut sable]

sd1 was eons ago (like 4+ years)

[ruby marlin]

That is a big departure from the precedent they set with earlier mainline model released

[dusky thistle]

haven't read the deets - but it sounds like there isn't a commercial license for sd3?

[shut sable]

there is afaik

[cobalt moon]

you have to buy the license

from what we could see

[shut sable]

idk i dont mess with licenses, i just make a cirno

[sinful ore]

I imagine the current license on SD3 is just while the model hasn't been released to prevent leaks. Surely they can change it down the line

[cinder junco]

Probably. No confirmation, but implied.

[patent acorn]

im a noob at this but can astral just buy the license?

[cobalt moon]

it can probably be a Stable Audio subscription model

[tropic aspen]

He's pessimistic that he'll get a response given that SAI seems to be ignoring him

[patent acorn]

oh damn

[cobalt moon]

where enterprise or individual must pay a define amount of price to SAI

[void dock]

@jolly swan Just wanna say even though I hate ponies I've been using your model (the OG one, not one of the mixes)
for months. Gives great result and I consider the randomness of style to be a plus.

[cobalt moon]

RunDiffusion ( in #💬｜general-chat btw ) also say SAI ignored them

I... don't use PonyXL

[patent acorn]

pay 50k$ dollars for SAI to respond!!

/s

[cobalt moon]

probably not that much lmao. but we will see if SAI respond to both of them or not

about the licensing things

[patent acorn]

if they dont respond they will fall

😭

[tropic aspen]

Honestly, I'm not surprised they're trying to pivot from the core userbase considering the rumours that they are... not doing too well financially

[cobalt moon]

even the license aren't specifying where they cover. Whether it is the right to commercialize image generated using the model, or commercialize the models trained.

[wild remnant]

Can I run an API on this guys?

[jolly swan]

Glad you like it!

[cobalt moon]

but talking about it

nobody mention the 16-channel VAE

lmao

[patent acorn]

its one thing that makes the quality better imo

[rose garden]

Hey they are finally going to catch up with generating text

[wild remnant]

Yet they cant catch up to video

where's it more important imo

[patent acorn]

they already fell off financially yet dammit

[cobalt moon]

the task probably just gonna handled by Open SORA project

[wild remnant]

I guess Haiper is still top tier until Sora or Veo @cobalt moon

[patent acorn]

haiper?

[cobalt moon]

I definitely heard of Haiper

[rose garden]

I achieved text gen with SDXL back in October of last year (https://civitai.com/models/176555?modelVersionId=214296)
So it's about time

[wild remnant]

Yeah better than Pika and Runway

[patent acorn]

oh i thought haiper is open source

[cobalt moon]

oh wait

that remind me

Runway too

[wild remnant]

?

[noble coyote]

SD3 - 12th of June - named as "SD3 Medium"

[patent acorn]

https://www.youtube.com/watch?v=fJQbP34GoHQ

this looks decent

[wild remnant]

@patent acorn yeah you should stop by

[patent acorn]

yup

this aint sd3

[wild remnant]

i post frequently on discord

That's all I really care about now is video...images are cool but I like a challenge

[noble coyote]

#📣｜announcements message

Announced already - SD3 Weights drop on June 12

[wild remnant]

Yes we know

[noble coyote]

Hurray!

[ruby marlin]

We already knew it was going to be released, the news is that it's with a noncommercial license

[turbid grotto]

medium is 2b?

[ruby marlin]

Yeah

[cobalt moon]

it is not a strict non-commercial because

you can buy a commercial license for SD3

[turbid grotto]

is this possible to run 2b model and T5 together in 12gb vram? I had quite slow experience swapping sdxl and refiner

[cobalt moon]

just we do not know how much money was needed, or whether the 20$ subscription was included

[cobalt moon]

T5 can be run onto RAM

or CPU

separately

[turbid grotto]

oh nice

[twin tulip]

sorta, its month by month and terms can change

[ruby marlin]

This will effect the ecosystem though, for example they just called off the Pony finetune because of the license

[twin tulip]

its like paying for Netflix or Amazon Prime

[rose garden]

Can you expand on this?

Or point me to more info about it

[shut sable]

yep!

[ruby marlin]

It's just speculation but I imagine there will be less major finetunes and controlnets for SD3 because people aren't interested in a noncommercial licensed model

I wouldn't mind paying to use it for commercial purposes but I worry about the ecosystem

[cobalt moon]

I mean what's the point of commercializing it. People selling NSFW pics?

[twin tulip]

itd be fine if I could pay for 3 years with a gaurantee of no change in terms

[cobalt moon]

sure I knew commercial license can go as far to "the rights of the image"

[ruby marlin]

If there's no point then why the license, lol

[desert garnet]

wrong

[cobalt moon]

where it can affect uses

[cobalt moon]

people just find it a niche.

don't get me wrong, I didn't nullify your statement

it is a legitimate concern for sure

[desert garnet]

commercializing its usually ppl training their own models and hosting it on online gen sites like tensor.art

[rose garden]

Or like hartsy.ai

[cobalt moon]

but for individuals... that's just it.

[shut sable]

btw mcmonkey isnt in charge of anything release related

[cobalt moon]

is he a PR?

[shut sable]

he makes stable swarm ui

[cobalt moon]

oh

[low stone]

He was also fixing stuff on artisan when it launched

[lament glacier]

Hey guys, what is stable diffusion 3

I've heard it's coming soon. I just know SD 1.5 and SDXL

[cobalt moon]

a new text2image diffusion model after SDXL

[lament glacier]

Oh cool. Can I use it on Auto1111

[cobalt moon]

designed to lowering the gate of running text2image model with the quality of SDXL or better

[lament glacier]

What do you mean?

[cobalt moon]

basically hardware requirement

[lament glacier]

How does it differ than the previous models

[cobalt moon]

you need at least 4GB VRAM to run SDXL, and that on slow speed

[lament glacier]

I have 8GB

[shut sable]

VRAM or RAM

[cobalt moon]

VRAM of course

[lament glacier]

VRAM

[shut sable]

ok good

[lament glacier]

16 RAM

[cobalt moon]

RAM is irrelevant in this kind of AI field

of course

[bitter hearth]

i run t5 on comfyui using 6gb on bnb4bit, full model takes like 12 i last remember

[cobalt moon]

outside of LLM

[lament glacier]

So how does it differ than the previous models

[cobalt moon]

better prompt coherency

much more quality base to finetune from

[bitter hearth]

not always cause offloading to ram helps with low vram

[cobalt moon]

hm

[lament glacier]

Prompt coherency? How

[bitter hearth]

is the artisan bot already updated to sd3 medium?

[desert garnet]

the question is when and why

[lament glacier]

When and why?

[cobalt moon]

AI models always have difficulty on "the cube's left is cat, the cube's right is a dog, the cube's top is an owl"

it is just

confused

[shut sable]

think so

[desert garnet]

when your 3rd eye is open you will understand

[cobalt moon]

and will place the cat, dog, owl elsewhere other the prompt designated to do

[lament glacier]

3rd eye?

I didn't understand what you mean

[cobalt moon]

a metaphor

[lament glacier]

So SD3 won't give higher quality images?

[desert garnet]

idk lemme ask her

[cobalt moon]

just... the situation have been very complicated as fuck

for starters

or non-tech savvy

[cobalt moon]

we will see, but at least it should be on SDXL quality

based of what Lykon was able to generate

[lament glacier]

Ok so same quality as SDxl but better promoting

[cobalt moon]

( Lykon is a well known SD model trainer, Dreamshaper and AAM is his works )

[cobalt moon]

https://x.com/Lykon4072

you can check Lykon's X account for reference

(@) on X

Check out https://t.co/ZCNZgxsLwY, the videos show actual outputs from our API

Twitter

although most image there are not really challenging

and can easily do the same on SDXL

oh gonna clarify one thing

[hallow lion]

how much is a commercial licence anywayz?

[lament glacier]

I've seen him on civitai

[cobalt moon]

It is likely that only his recent posts are made with 2B, which is deemed to release on June 12

and the rest is 8B or something

[candid holly]

Someone should really come up with a site where you can just put in your GPU model, along with CPU model and RAM if relevant, and it just tells you the best models currently available that will run on your setup... 1b 2b 4b 8b, these various numbers are confusing to a lot of people who are just like "I have a 3060, what can run on my system?"

[cobalt moon]

that we need a theory

[shut sable]

more like people should optimize their code

[meager dock]

A GAME THEORY

sorry i had to

[shut sable]

(just put vram)

[cobalt moon]

I bet for hosting site, it can be 1000$?

each month?

[shut sable]

github pages....

[desert garnet]

lets ask Jensen Huang what gpu he recomends for AI

[shut sable]

b200

[cobalt moon]

H100

[primal steeple]

Well it kinda depends what you want, a 3060 can run most SD models, but you might prefer more quality over waiting 30 less seconds, or you might need lightning fast, or some middle ground.

[hallow lion]

so much theories... the onyl time well know how much better than sdxl it is and how well it runs is when you can actually try it

[candid holly]

he'll recommend you give him all your money

[cobalt moon]

or some mf who just want to have collections of AI porns

[sterile pendant]

Make sure everyone only compares base sd3 2b to the most fine tuned sdxl models that are overtrained on closeup portraits of women with the same five facial structures and that also suck at doing anything other than waifus... And then cry about how "bad" sd3 is...

[turbid grotto]

could finetuning improve sd3's general prompt comprehension even further, or it is already close to what we can get with this base because dataset already captioned by llm and not some random scrapped keywords?

[shut sable]

ye possibly

[jolly swan]

I have not called it off, but things are complicated now (and the change in license is bad, has not been communicated and does not follow the expectations set up before).

Maybe SAI is just salty 🙂

(here is some data scraped form Civit's public API, it's a bit limited because it's not real downloads but actually "downloads of the model release at week X", but it does represent a real trend)

[ruby marlin]

Sorry if I misinterpreted what you were saying, in any case I personally would be happy with another SDXL finetune if you think it's worthwhile

[desert garnet]

so this means new pony XL ?

[jolly swan]

I've spent 4+ months now working on new tech, the jump in quality will be amazing even for XL based model

[jolly drum]

Isnt pony trained on furry and anime images so you dont really need a lora for some characters?

[ruby marlin]

it's also nice that there are finally decent controlnets for XL

[jolly swan]

it's everything non photorealistic you can imagine, most characters are recognized well enough

V7 will not be so high LR so most tools should work from the start

• some real human

[jolly drum]

Hmm might give it a go tonight. Might need to take the ComfyUI pill sooner rather then later

Forge and 1111 is showing its limitations already

[desert garnet]

forge been abandoned for 3months already 😔

[cobalt moon]

I am still preparing my dataset and experiment for Pony-clone SD3 and SD1.5 though

like the idea of activation tag, quality tag, and artist tag

[jolly drum]

Still faster then auto1111 for sdxl mostly

[jolly swan]

There is a high chance I will be releasing my aesthetic classifier for V7

And some captioning data

To help with finetunes and loras

[turbid grotto]

is v7 still planed to be trained on sd3?

[desert garnet]

yea thats why i dont switch to a1111 🌞 its a slugfest

[jolly drum]

Depends, i use 1111 for sd1.5 as its still pretty solid

[cobalt moon]

I kinda did success on my recent SD++T2, which is based of AnyLORA SD1.5

but eh...

LORA compatibility is not that great and funny caption

[jolly swan]

Completely depends on what SAI tells me (if they ever do)

[viral plaza]

i do many things but yes i'm not in charge i'm just a dev helping relay info

[cobalt moon]

btw I originally plan to release the model but I figured that maybe it is better to just release T3 as a test model.

[viral plaza]

if you're hesitating from the noodles, Swarm gives you a user-friendly auto-like interface but with the comfy backend

[turbid grotto]

Okay thanks!

[shut sable]

aw, i was about to leak sd3

[viral plaza]

idk the details of the licensing decision making stuff (well above me) but you should definitely be fine one way or another to train finetunes on top of SD3 at least for public release

[jolly drum]

Bookmarked! Ill give it a go tonight

[viral plaza]

(commercial models idk the details but presumably memberships or something will apply and you can just do that)

[cobalt moon]

@calm zinc

sorry for ping but I think that might be useful for you

[jolly swan]

1. We run our commercial inference network, it's small but it's still a commercial project. Before that we were covered by the SAI membership program.
2. We partner with SaaS providers, if they can't use it, we lose strong incentive to base anything on SD3.
3. Any barriers make adoption slower/less likely, so that also destroys non monetary incentives.

[viral plaza]

ah

yeah it'd be very silly for the membership program to not include SD3 post-launch so hopefully that won't be an issue? lol

(nobody benefits from it being excluded)

[jolly swan]

If the license is actually - non commercial unless you are in the membership program than it's a nothingburger but communicated in the woooorst possible way.
If there are clear licensing options beyond that, then not communicating in advance is a bad look anyway (because SAI relies on community interest a lot).

[jolly swan]

It's either the license is bad or the comms are super bad. Or both! (Because SAI pretty much ignored me so far, lol)

[tranquil harbor]

img 2 video

[viral plaza]

comms are always wonky

hopefully will get cleared up by or soon after launch

[trim arrow]

How much resources is sd3 gonna take...

[viral plaza]

Medium takes slightly less than SDXL

[cobalt moon]

4GiB

( note that GiB is a little bit larger than GB, 1GiB = 1.07GB )

[dusky thistle]

will there be a 4b or 6b model?

[trim arrow]

Will a 3060ti be able to handle it?

[cobalt moon]

800M, 2B, 4B, 8B

this four models

[dusky thistle]

def looking forward to 4B/8B... have a 4090, won't hesitate to upgrade if i need more vram for the 8B

[viral plaza]

oh yeah no problem there

[trim arrow]

Oh awesome

[viral plaza]

if you got a modern nv card at all you're golden

(8B might be iffier when it comes out)

[viral plaza]

4090 can handle 8B fine

[trim arrow]

Yeah I don't expect 8gib to work on a 6gb card, lol

[viral plaza]

might upgrade to a 5090 when that's out just to be faster tho

[trim arrow]

Would a regular 3060 handle the 8gib?

That has 12gb

[cobalt moon]

Tight

Probably just inference

[trim arrow]

For inference?

[dusky thistle]

great to hear. hope the 8b is out this summer!

dyiiing to get my hands on the full size

another q: are sdxl sampling methods going to work at all with sd3?

i've been building a bunch of custom sampling nodes

getting ridic quality now by scheduling all the parameters for the RES sampler, etc

[viral plaza]

it's an 8B model, ie 8 Billion Parameters, not an 8GiB model -- but yes a 12GiB card should be able to run the 8B model happily with fp8 weights enabled

[viral plaza]

inference = generating images, as opposed to training

[trim arrow]

Oh. Yeah I don't plan on training!

[cobalt moon]

A machine learning term

[calm zinc]

Definitely good to know. Thanks for the ping.

[trim arrow]

I'll leave that to the professionals! 😆

[cobalt moon]

Probably gonna make a Reddit thread about this

[viral plaza]

Some things work the same others different. The Rectified Flow schedule means that for example Ancestral or Stochastic (SDE) samplers won't work properly, whereas normal samplers (Euler, DPM++, etc) are fine

[cobalt moon]

And questions

[trim arrow]

I'm assuming sdxl loras and such aren't going to work on sd3, right?

[dusky thistle]

gotcha... no chance they could be adapted for that? deterministic only?

[calm zinc]

Safe assumption

[cobalt moon]

I dont expected a X-Adapter-like tools at release lol

[trim arrow]

I feel like a dunce for asking that but things have surprised me before.

[viral plaza]

I won't say "no chance" because AI is magic and researchers invent "impossible" things all the time, but yeah at time of launch at least ancestral/stochastic is just fundamentally incompatible with rectified flow

[dusky thistle]

well shit. lol

[viral plaza]

Research team initially told us that Euler Discrete was the only valid sampler at all, but then comfy fixed so most other samplers worked

[dusky thistle]

here am i going bananas with this stuff

[trim arrow]

What in the hell are you doing

[viral plaza]

not at launch no, however i'd bet people will get working on that pretty quick

[cobalt moon]

Soon enoug, community :

"Here i am going to do porns with this stuff"

[dusky thistle]

getting the best quality i've seen out of sdxl

[viral plaza]

Very specifically text encoder LoRAs and TI files for SDXL will be transferrable, unet stuff will not be at all

[cobalt moon]

Actually

Will T5 become a bottleneck for low end device

[cobalt moon]

Like I know it could run on RAM, CPU and GPU, either one of them

[trim arrow]

You might have to explain the difference of that to me, I'm not tech savvy at all in that regard

[calm zinc]

Comfy give me anxiety

[viral plaza]

You. Scare me.
But yeah you'll probably have to start over on a lot of that for SD3, mucking with sigmas and schedulers and all ... that's all stuff that'll be quite different

[viral plaza]

this is what Swarm is for lol

gives you the power of the noodle soup without having to stare into its abyss

[cobalt moon]

But there are certainly devices that have awful low amount of RAM / weak CPU GPU

[viral plaza]

you can fully turn T5 off, use only clip, and get very similar results

[cobalt moon]

Wait you can?

[viral plaza]

so if it becomes an issue just skip it

[dusky thistle]

i mean, if it's deterministic only... then you're not really going to be able to influence the style with the sampling process, noise injection, etc... right?

[trim arrow]

I'll be getting 48 I think. I can't remember. 40-something

[viral plaza]

yep! SD3 was trained so all textencs are optional

you can also do T5 only, or CLIP G only, or...

[cobalt moon]

Both

[viral plaza]

(CLIP G + CLIP L, ie same as SDXL, is the best for not doing all 3)

[dusky thistle]

i'm under the impression that generally the euler/dpmpp_2m etc type stuff is basically various paths to the same end

[viral plaza]

Rectified Flow isn't deterministic per se, there's still a lot of sampling tricks you can pull, however generally the way to do them will be different and more limited

[trim arrow]

I'm excited for sd3. Once I started using sdxl I haven't been able to look back at 1.5 😆

[dusky thistle]

gotcha, thanks. dang

[cobalt moon]

I look back at SD1.5 because I run SDXL for 1 1024x1024 image for 40 minutes

On Comfy

[viral plaza]

Like at the most basic, multiple sequential sampling runs (a la refiners, hires fix, etc) is perfectly doable still, and all the tricks associated therefore are too

[cobalt moon]

( not CPU )

[viral plaza]

... dahek GPU you got?

[dusky thistle]

anticipate there being other technical approaches for influencing the style or structure of the image... not necessarily sampling, but stuff along those lines in spirit?

[cobalt moon]

MX130, literally a punny 2GB VRAM student laptop GPU

[trim arrow]

Ooohhhhh yeah that sucks. My laptop can only run 1.5, sdxl it just denies it

The fact you can even get sdxl at all is impressive

[cobalt moon]

Somehow much faster in A1111 than Comfy ( 20 min )

[viral plaza]

Definitely yeah. I'm most interested to see how long it takes before ControlNet/IPAdapter/etc are published for SD3. The fun part is controlnets are much more logically applicable to SD3's arch (as a new stream) vs SDXL (janked unet clone)

[trim arrow]

If I try to load an sdxl model, a1111 and comfy will crash no matter what

[viral plaza]

so might get better controlnet-style development happening

[viral plaza]

oh oof you poor soul yeah that's not great

[cobalt moon]

Is there a possibility for license changing

[dusky thistle]

btw, discovered a few days ago that setting the default dtype to float64 in comfyui results in a huge jump in quality for some samplers

[cobalt moon]

According to some hardcore open source people

[dusky thistle]

with minimal performance penalty

pretty weird, not sure where the key float is

[viral plaza]

SD3-Medium might be a bit better on that GPU than XL is, cause it's a bit smaller, no promises tho cause that's a pretty painful GPU

[torpid forge]

I hate asking questions like this because perhaps it's obvious if I look in tthe right place, but what is the token limit with the SD3 api? I'm seeing 77 token some places, but then I read it was 1000 characters in another place? I feel like it shouldn't be this hard to figure out

[cobalt moon]

Tokens does not equal to characters

[torpid forge]

I realize that

but can't find consistent information, so figured I'd ask

[viral plaza]

it's already gonna be free for noncommercial, presumably it'll get added to the commercial programs too (idk what the deal with that is).
Not Hardcore open source, but, like, ... close enough imo

[trim arrow]

Woooooo, free for us people who don't have a business lets go

AKA don't make money

[viral plaza]

free for personal usage is the big point for me, as long as that's true i'm happy. Commercial users i've heard are all happy with paying for commercial rights (if you're a commercial user, you're making money and can afford $20/month or whatever)

[viral plaza]

yeee

[cobalt moon]

So InvokeAI business model

[dusky thistle]

$20/mo is a steal

non-royalty based for anything is a steal for a biz

[cobalt moon]

Individuals definitely

[viral plaza]

That's. WEird and confusing. But interesting. iirc even fp32 vs fp16 makes almost zero difference

[frail tulip]

HI GUYS I AM NNEWW HERE

[viral plaza]

For that matter you can put weights in fp8 and as long as calculations are fp16 it's basically identical most of the time

[cobalt moon]

Hi

[dusky thistle]

this is the kinda stuff i'm getting in 14 seconds with all those sampling tricks... look forward to the next challenge with sd3 whatever that is

[dusky thistle]

yep. it really shouldn't matter.

[patent acorn]

so thats a good news? im hoping there will be major finetunes like sdxl has gotten

[frail tulip]

HI

[desert garnet]

whats the definition of commercial rights,is there a starting point like business who make 10k+ a month or 5k+ etc..

[dusky thistle]

i'll show ya the first example.

[frail tulip]

hii

[trim arrow]

Yes we see you hi

[frail tulip]

damn

[torpid forge]

ugh

[frail tulip]

hello

[viral plaza]

API doesn't have a hard limit it just has an abuse cutoff (ie you can't shove the entire transcript of the bee movie in or some shit, but as long as what you're giving is actually a prompt it should work).
Note this applies to modern (v2) API, the v1 API was weirder and that might be where the confusing info is from

[frail tulip]

i am akumitsu

[dusky thistle]

first is default. second is with default dtype for torch set to float64

[frail tulip]

u guys can call me aki

[dusky thistle]

i didn't believe it at first

[trim arrow]

Don't be giving out your name 😆

[viral plaza]

check https://stability.ai/membership

[patent acorn]

still concerned for astral, bro is not getting responses from SAI 😭

[torpid forge]

ahh! thank you very much

[cobalt moon]

1 million annual avenue

oh yeah

[viral plaza]

Is this only with your giant spooky workflow hacking sigmas and all? Possibly you're pushing the borderlines with some of that added logic?

[cobalt moon]

gonna make a Q&A thread on r/StableDiffusion with the question asked

[dusky thistle]

it's even with just dpmpp_2s_a and a simple workflow... the image changes considerably