🆕｜sd3

[wide pagoda]

What about 512x512, does it just generate cropped images?

[storm saffron]

Sigma is a bit of an oddball though, it uses very long captions (300 tokens) rather than the 77 people are used to, so to get the best from PixArt you need to type in a large prompt or expand it with an LLM

SD3 still has that 77 token limit though right?

[noble coyote]

Yes, my best PiXart-Sigma output is when I use Jan.ai to prepare a "natural language prompt" as input 🙂

T5 is better when natural language is input ...

[storm saffron]

I've been using zephyr7b in comfy

[noble coyote]

In fact T5 was conceived to use natural language

[storm saffron]

Probably shouldn't discuss pixart too much in the SD3 channel of SAI though. Feels weird.

[noble coyote]

But as too few people can get their hands on SD3 - we have to talk about something ... 😄

[dull star]

interesting, Pixart (DiT) had the entire image noisy/distorted when doing a higher res version on any model

at least, if I recall correctly

[noble coyote]

I'm @ClipDrop SD3 now ... anybody got an interesting prompt for me to try?! 🙂

OK, gone to Openai - Daily Theme for a prompt 😄

[dull star]

Photo of Criminal in a ski mask making a phone call in front of a store. There is caption on the bottom of the image: "It's time to Counter the Strike...". There is a red arrow pointing towards the caption. The red arrow is from a Red circle which has an image of Halo Master Chief in it.

ah im late

[noble coyote]

A moody and atmospheric fight scene between stick figures rendered in a realistic digital painting style. The scene is set in a dark alley, illuminated by a single streetlamp casting dramatic shadows. Two stick figures are engaged in a fierce battle, with one delivering a powerful punch knocking the other backwards. The realistic digital painting technique gives the stick figures texture and depth, making them appear almost human-like. The atmospheric lighting and detailed background enhance the emotive intensity of the scene

[noble coyote]

I've tried that one ... was OK

[dull star]

oh you were the one who tried it

I forgor lol

[storm saffron]

Pixart

[noble coyote]

A futuristic terrarium with a unique and captivating design. Inside the terrarium, miniature bioluminescent plants and flowers emit a gentle, glowing light, creating an ethereal and otherworldly atmosphere. The plants are arranged in a way that mimics a mystical, enchanted forest. Tiny, intricately detailed fairy houses and pathways are nestled among the foliage, adding a whimsical element. The terrarium's glass is etched with delicate patterns that reflect and enhance the inner glow, making the entire piece a captivating focal point

Create a square visual representation inspired by the DIY spirit and featuring prominent use of safety pins, styled as a monochrome street snap photograph from the punk era. The image should focus on a close-up of a single person's face dressed in typical punk fashion, caught in a candid moment as if walking through the streets and looking back over their shoulder. The photograph should emphasize sharp contrasts and capture the gritty, raw aesthetic of punk culture. The person should have a brightly colored mohawk or spiked hair, appearing self-cut and dyed to emphasize the DIY ethos, though rendered in black and white. They should be sticking out their tongue and raising all fingers towards the viewer, conveying a rebellious and defiant attitude. The person should wear dark eye makeup, with black eyeliner and shadow. Accessories such as safety pins used effectively but sparingly as earrings, body piercings, and prominent decorations on their clothing should be visible. The person's eyes should have a nihilistic, vacant expression, emphasizing a sense of emptiness or detachment. The background should feature a gritty urban scene with graffiti-covered walls, capturing elements of street art that align with the punk aesthetic. The monochrome style should highlight the rebellious spirit of punk with intense, sharp contrasts and a raw, edgy feel. The photograph should have a candid, spontaneous feel as if the person was unexpectedly captured while walking through the city and looking back. Add a grainy texture to the photograph to enhance the raw, unpolished aesthetic.

This punk prompt is a monster!!!

SD3@ClipDrop

SD3@ClipDrop prompt = A chubby Shiba Inu named XiaoShuai, standing on two legs with its belly exposed, not holding any items. XiaoShuai is wearing a cute human outfit that includes a brightly colored shirt and shorts. The Shiba Inu has a warm, tan coat typical of its breed, with expressive eyes and a playful expression. Scene in warm tones, realistic style, capturing the essence of a cozy home atmosphere

[viral plaza]

kinda. rn the SD3-Medium model mostly just makes it work at 512, idk if the release version will be the same or not

well not perfectly, but, like, relative to what you'd expect for borking the res entirely

[pine cedar]

Is this the 2b model of sd3？

[wide pagoda]

So it does closeups, but with "reasonable" framing

[low stone]

No, that's the 8b beta on the api

[pine cedar]

amazing picture

[noble coyote]

What SD3 does have is visual acuity - poor limbs, fingers and faces (at times) - yet with an enhanced visual acuity.

[dapper basalt]

If Stable Assistant sticks around and stuff, I'm gonna keep it I think. Its an awesome tool to finetune what you are trying to get. For example I had it optimize prompts. It'll be great once its fully featured. So o nce weights are dropped and it isn't necessary to burn through credits really quickly and to use it as a tool on the side, it'll be worth it.

[noble coyote]

I can never understand why Stable Assistant gives you 150 prompts/150 pictures a month (for $10); and ClipDrop gives you 300 prompts/1200 pictures a month (for the same $10)?

[dapper basalt]

SA needs the money and knows people will jump on SAI quicker. Cause of the new tech and brand name

[noble coyote]

Twice the number of prompts and eight times the number of pictures 😄

AFK

[dapper basalt]

Yeah true

[muted dove]

Boltning Hyper

[storm saffron]

@viral plaza do you know if it's just the standard T5 encoder that everything elses T5 uses or is this specific to SD3?

[viral plaza]

publicly available google/t5-xxl

[storm saffron]

Phew, I don't have to download it agaaaaaain.

[viral plaza]

https://huggingface.co/mcmonkey/google_t5-v1_1-xxl_encoderonly the encoderonly file loads more fasterer tho

[storm saffron]

🙂 I've already got one of those.

[viral plaza]

nice

[muted dove]

Also Pixart-Sigma

[storm saffron]

Pixart uses the same one at FP16

[storm saffron]

Nice!

Another one from pix

[muted dove]

Same 😄

[noble coyote]

Can this be used inside PiXart-Sigma at all?

[muted dove]

I'll let you know if it loads...after it downloads

[storm saffron]

Yes. I'm using an fp16 xxl t5 right now.

[muted dove]

Are you using the encoder only version though?

[storm saffron]

Yep

[muted dove]

How? I get AttributeError: 'NoneType' object has no attribute 'get'

Loading T5 from 'F:\ComfyUI_windows_portable\ComfyUI\models\t5\t5-xxl-encoder'
!!! Exception during processing!!! 'NoneType' object has no attribute 'get'

[storm saffron]

I'm actually using a BF16 one, oops,

[muted dove]

The clue was still there though, thanks! 🙂

Changed path_type from folder to file...now works

[storm saffron]

https://huggingface.co/city96/t5-v1_1-xxl-encoder-bf16 if you want the BF version (which may be more accurate than FP16)

[muted dove]

I seem to have all of them! 🤣

No wonder I'm running out of disk space

I have mt5-xl too

15GB

[storm saffron]

You can probably find a smaller xl as well.

[muted dove]

I have flan-t5-xl-encoder-only-bf16, which is 2.5GB

[storm saffron]

Does that work as well?

[muted dove]

Not tried

[storm saffron]

I wonder if the outputs are compatible.

[noble coyote]

Trying bf16 now ...

[storm saffron]

Nope, just tried, xl doesn't work, it's a different tensor size

xxl only

[noble coyote]

bf16 works - not made a noticeable difference - it might if I was doing photorealistic though (I will try that later!)

Trying XXL ... it's working ... again, no discernable difference?!

[sterile pendant]

Now you see why nobody uses llms at high precisions. You can look up perplexity vs quantization to see more about it.

[dull star]

I remember in the GPT-J days I wanted to run LLMs, but the best we could do is load-in-8bit lmao

[storm saffron]

BF16 and FP32 are very close in precision, FP16 loses precision in the sorts of ranges DiTs have.

[noble coyote]

I'm running a lot of art which does not need a great deal of artistic precision - do you mean precision in detail; or in prompt coherence?

[storm saffron]

Mathematical precision. 😄

[noble coyote]

I come to all this primarily as an artist ... glad to know that mathematical precision exists!! 😄

[low stone]

the city96 guy has a 3 gig version of the t5 for hunyuan, and I did a side by side comparison. Details on the face, lines on the clothing, various stuff like that are suddenly not as sharp or don't make sense. They're not major, but it's definitely noticeable.

[muted dove]

Isn't the T5 just an LLM that helps improve the prompt? I'm surprised it has any impact on the details in the image.

[low stone]

no that t5 is the thing actually doing the encoding for the image model.

it's converting the words to numbers.

[teal fossil]

Which is great. I tried it in TagGui and it's pretty neat. 🙂

[low stone]

hunyuan has a separate prompt expander model that they also include if we want to use it, but we have better stuff like gpt4 and llama3 for that.

[dull star]

OH SHIT STABLE AUDIO HERE?

https://stability.ai/news/introducing-stable-audio-open

non commercial license as expected

cant wait for comfyui implementation

[dusky thistle]

hope like hell we someday get the full weights for the full version

i'd go wild with that

[teal fossil]

@viral plaza On another note - could we theoretically load CogVLM instead of T5 XXL? At least in TagGui it gives me better results. Too bad it's quite VRAM hungry and CogVLM2 is Linux only...

[teal fossil]

Let's hope we get a good UI for that soon! (looks like audio-webui isn't actively maintained anymore)

[dull star]

honestly comfyui would be fine for me

[teal fossil]

Ooh... RIGHT! I didn't consider that... absolutely.

[dull star]

I bet we'll see some addon implementation a few days later

maybe around the time SD3 2B comes out

[raven fern]

is this the one we will need to download when sd3 releases?

[dull star]

no, you should download the bf16 one

smaller filesize, faster loadtime, less memory used up on CPU Ram

[raven fern]

ah nice, i like the bf16 version anyway :3

[dull star]

👍

[dull star]

wait a sec, the link to the model that Alex sent is the same filesize... maybe its fp16 as well

oh its from him, mcmonkey

[raven fern]

mcmonkey d luffy 🙂

[raven fern]

calm down Musk

[fair spruce]

Show Musk go on

they said to me 🙂

[raven fern]

good one

[rustic junco]

When using the api to generate images I get finish_reason: 'CONTENT_FILTERED',
seed: 2950283743
}

It blurs the image. Any way to say that it should not filter content ?

[low stone]

Nope. Have to wait for the 2b on June 12 for that.

[rustic junco]

Where did you get that information ?

[low stone]

Look here. #📣｜announcements message

[rustic junco]

Thank you very much! 🙂 So are we sure that we will be able to turn of that filter ?

[woeful spindle]

The diffusion gods have revealed their divine will unto us

[low stone]

The local version won't have filters. It'll just be capable of something or it won't. There won't be any blurred images.

[low stone]

Hah it couldn't quite do it.

[rustic junco]

Do you know what format the image-to-image mode has to be ?

I tried to send an url and tried to send the image as base64

both times i get bad requestb ack. I also changed the mode to image-to-image but doesnt seem to work

[raven fern]

what am i looking at lol

[raven fern]

kek

[fringe rain]

1dog

[raven fern]

that looks like a picture from an actual commercial LOL

[past flame]

"Inside of you, there is 2wolves"

[cedar gale]

I was bored. 😄

I think the coca cola drip is the best tho.

[rain current]

ideogram

[cedar gale]

Noice.

[rain current]

I am very impressed by ideogram. I took your image and used describe, then generated the resulting prompt... it has a lot of potential

Although sometimes it makes real crap

[cedar gale]

Its fun I like using SD for making outrages stuff. 😄

[raven fern]

now im thirsty

[rain current]

So then....

[cedar gale]

Yeah that kills em faster.

This might speed up things.

[raven fern]

lol

put a poison bottle

[cedar gale]

😦

ALOHOL.

Sad panda.

[rain current]

[rain current]

The patient may not survive...

[cedar gale]

They will be fine.

[turbid grotto]

some people saying text is gimmick but I have a hell of a fun with it and it is good

[dreamy sundial]

yeah it's pretty good for a base model

[turbid grotto]

I don't really understand how do glif.app host sd3. It takes 8s for gen while someone from SAI (or it was in paper) said it takes 40s or so on 4090 with 50steps. Do they have something more powerful? But why for free then, how do they profit?
At first I wasn't sure that it is real sd3 but it is not similar to any other services

[turbid grotto]

and it is yet undertrained

[raven fern]

i can fix her (in photoshop...yea...)

[cedar gale]

Yup she looks healthy.

[rain current]

with sdxl refiner

[rain current]

I haven't tried it, but could it be fake? Lately, I see a lot of https://huggingface.co/spaces/markmagic/Stable-Diffusion-3
// I see that it no longer works; it was working yesterday

[turbid grotto]

I also thought about that but what is then? It is definitely not Dalle3, Ideogram looks very different too, SDXL could never achieve such prompt following...

At the beginning I got fooled by one "free sd3" site. Decided to check metadata and found "Ideogram" there) But it is not seems to be case here

[low stone]

Well could just be using the sd3 api

[storm saffron]

Considering the source code was literally "exec(os.enivron("CODE"))" they were probably using the initial signup free credits on the SAI API

[rain current]

I think some people are using Ideogram. It's easy to make calls to Ideogram via the endpoint URL. I do it sometimes by inserting the bearer token, and it works well

[turbid grotto]

yea it could, but it just too generous, I didn't even hit limit once. Some of these sd3 glifs has about 30k plays while, in addition having gpt4\claude prompt rebuilding, or sdxl refining

[turbid grotto]

I made some comparisons but couldn't find obvious similarities between Ideogram from their website and glif

[hallow lion]

1 week.

😏

[hallow lion]

its like hl2 in russia came out before hl2 in america came out

😄

shady characters pushing SD3 in the backrooms and alleys

[low stone]

Sd3/sdxl refined

[cunning lintel]

first thing to try when weights drop

[dull star]

LMAO

I really wanna see the difference in prompt adherence

[low stone]

Sd3/hunyuan

[cunning lintel]

i just want to try dumb contradicting stuff like "luminescent Rose madder Bright lilac huge crooked (fur:1.9) (alien:0.2) (Titanoboa:0.5) (Meerkat:0.4) amethyst eyes. full body. Boreal Forest, puddle, centered, sunlight, focused, uniform light background, photo-realistic" and T5 is probably too smart for that :p (though it doesn't even look half bad)

[dull star]

I never got this actually. Can T5 also have prompt weighting?

[cunning lintel]

no idea, i'd think why not but it prob needs a new implementation

[low stone]

Refined, it's really neat

[low stone]

Wow these came out great

purple furry snakes in the shape of "SD3" are wrapped around the arm of will smith who is taming a lion

[cunning lintel]

grin, asking a bit much 🤣 one day, imagegens will happily oblige, i hope 😉

[low stone]

Other than the text portion, pixart did it more accurately.

[hallow lion]

sigma slow

[low stone]

this is with 512 sigma

[storm saffron]

If you think sigma is slow, wait till sd3, apparently it's also quite slow. I believe lykon said 30 seconds per image on a 4090? 50 steps.

[dull star]

isn't that 8B?

and also 50 steps too

In early, unoptimized inference tests on consumer hardware our largest SD3 model with 8B parameters fits into the 24GB VRAM of a RTX 4090 and takes 34 seconds to generate an image of resolution 1024x1024 when using 50 sampling steps.

[faint breach]

50 steps yeah. 30 seconds for that isn't so bad. thats 1.5 it/s at megapixel resolutions. not sure what the problem is

not even any optimizations done by the community yet

[dull star]

well for 8B that's pretty good, but for 2B, a 4090 doing 1.5it/s isn't that good imo
I hope MODE13 meant 8B

and yes, optimizations

[faint breach]

a lot of the time may come from t5 layer too. faster without it and many won't want to use it

the way a lot of civit users prompt, t5 won't be necessary

[dull star]

yeah lmao

[hallow lion]

if i can get under a minute 1024/1024 30 step ish i cna live

[dull star]

1girl, big boob, intricate won't need T5 for sure

[hallow lion]

now piling on loras controlnets ipa adapters will be a different story

[faint breach]

i've been using Omost which takes like a minute to make the prompt code in the first place lol

i'll be fine

[hallow lion]

what about "2 girls, one with A cup boobs and one with C-cup, intricate"?

[dull star]

im sure PonySD3 will get it right first try

[faint breach]

there's gonna be a lot of debate over the usefulness of t5. many will declare it to be another sdxl refiner layer since they won't see immediate gains

another prediction i have. controlnet won't work as well on larger sd3

[dull star]

how so

[faint breach]

same prblem with sdxl. more parameters means less influence

[dull star]

interesting

then again, wouldn't the MultiModal nature of it make controlnets just superior?

multiple input streams and such or whatever

rather than some hacky solution

[faint breach]

i'm not saying i have an educated prediction

[dull star]

and I don't evne know what input streams are

[low stone]

there was a thread on the sdxl channel today where andreac75 who makes a lot of top notch loras is just deleting all their stuff off civit because it's all just porn now. They didn't want to see all the heinous stuff people were making with their stuff anymore.

[faint breach]

and the porn side of the community have this notion where if you even slightly dont' want to see that, you're very obviously an fundamentalist christian oppressor

[patent acorn]

ermn error

[trim arrow]

sd3 hasn't even released yet...i think.

[patent acorn]

ik

[upper snow]

arguably the more important question: deep floyd stage 3 wen?

[hallow talon]

I wonder how long after SD3 2B releases will we be able to start training LORAs and finetuning?

[solid violet]

developer API has been out...
https://stability.ai/news/stable-diffusion-3-api

[astral grail]

if it's very easy, why is stability not officially doing that?

[cinder junco]

Do you want it to take even longer to release the weights?

[astral grail]

well if "very easy" means it would just take 1 day longer, then I think that would be worth delaying it 1 day for, yeah 😄

[cinder junco]

Of course. I think Stability would just do it if they thought it were THAT easy.

[low stone]

That's just going to be a comfy node update later. The weights don't need to wait for that

[cinder junco]

Easy probably means "you don't have to go to special gyrations to get the loss to decrease, but you still need to train with a lot of compute and data".

And rather than spending all that compute on training the model on other resolutions, I'd prefer that the issue with the positional encoding get fixed so that the existing model is more flexible.

[low stone]

The downside of tiled upscaling is that it takes many times longer than standard upscale image and denoise at 0.5.

It's probably why they're not doing it on the api and instead upscaling sd3 with sdxl turbo

[astral grail]

I'm quite sure @viral plaza in the message I replied to was saying that SD3 can easily be trained for higher resolutions, so no, that cannot just be a comfy node, it needs training

[low stone]

All of these models are 1024 res trained.

It would take a massively larger amount of time to train the base models of millions of images at higher res. It would take massively more money to buy that gpu time, which a near bankrupt company can't do.

So instead, they're delegating that to the community, which makes sense.

I believe that they have the best of intentions at this point. The reality though, is that we may never see 8b or even 4b if someone doesn't come along and fund them somehow.

So we should just get 2b going and do what we can with that until/if we get surprised with something better down the road.

[sterile pendant]

Nah, cnets were kind of a hack job to implement into unet based architectures. They should be exponentially better with DiT once people start rolling them out.

[hallow lion]

yes take 2B and run with it

dont do another Cascade!

[bitter hearth]

Stupid black bars

"action movie" in the prompt, really likes to add fire to things

[patent acorn]

looks more like a video game to me

[bitter hearth]

I have unreal engine in the prompt

What you said makes all the sense :p

[patent acorn]

ohh

[mighty pulsar]

/help

[bitter hearth]

The green reaper..

[patent acorn]

thats a pickaxe not a scythe..

i heard its pretty bad at scythes too

later when we get to see it finetuned

[bitter hearth]

Loooong scythe

[patent acorn]

thats a scythe now

[hallow lion]

at least Friday is close

[bitter hearth]

it's not bad at the scythe itself it seems, but same problem with guns, the holding part

[patent acorn]

yep

[bitter hearth]

Well

Fingers are always bad

[hallow lion]

Looks like even those 4Gb of VRAM are overheating...

[sterile pendant]

I wonder if we will be able to rig up a multigpu setup for sd3 in comfy with any form of ease. I'd use my second weaker GPU for t5 inference and my main one for sd3, that way I wouldn't have to swap models constantly or do t5 inference on the CPU...

I know that in the python code, it's usually as simple as saying cuda0, cuda1 or CPU, so it might be pretty easy to slap into a node

I just don't know if comfy would natively like it with the smart memory management stuff

[turbid grotto]

Is there alternative?

||not hf pls||

[dusky thistle]

What's wrong with hf

[turbid grotto]

idk it is just not that convenient if you don't know what exactly you need

could be skill issue

[weary snow]

What is likely going to be the minimum requirement for Stable Diffusion 3 Medium ?

[cobalt moon]

4GB

to be fair in 2024 almost everyone have at least 4GB

[viral plaza]

Technical things being easy doesn't mean business things are easy.
Like first -- Our technical team could release a thousand models... but how do we organize marketing around them all, how do we get safety testing on them all, how do we get legal signoff on them all, how do get licensing organized, etc. etc.
But also: if we decide to spend three days training 2B-2048, that's three days not spent training the 8B. (Opportunity cost: every action no matter how easy comes at the cost of something else you could've done with that same time/effort).

in short: something being technically easy doesn't mean a company like Stability can or should do it internally.
In fact, that's part of the point and value of open releases: Stability doesn't have to do everything, we just open release the models and what info we can, and there's ten thousand other people who will each obsess into any given subsection of work and make that

[viral plaza]

I'm pretty sure there's a way to software-patch the pos embeddings and make them friendly to other scales. Not 100% confident but similar things have been done before (see eg RoPE scaling for LLMs)

[viral plaza]

no just wait until launch day and we'll have clear proper ways to get any required files

[viral plaza]

oh yeah lol wasn't gonna upload an fp32 file in the year of our quantization 2024

imo we should be using a 4bit quant, but, need proper software support first

[cobalt moon]

4 bit quant?

as in LLM's Q4?

[viral plaza]

CogVLM is an LLM that takes image inputs, designed to generate text descriptions of those images
T5-XXL is a text encoder-decoder generic pairing, in which SD3 uses only the encoder, to convert prompts to inputs.

These models are not interchangeable, they're not even in the same category

[dull star]

I hope it gets figured out cause I don't know about how good it will upscale images (tiled) with depth of field in them

[viral plaza]

yeah

[cobalt moon]

oh you talk about T5

yeah that make sense

[viral plaza]

for example i use a 4bit T5 for https://github.com/mcmonkeyprojects/translate-tool and it works great

[cobalt moon]

there seem to be quite a lot of variation model for T5 though

there are fp16 version by theunlikely

[viral plaza]

(T5 is an architecture intended to be finetuned, there's tons of variants out there, it's kinda silly that we use a base model of T5 for sd3 tbh)

[viral plaza]

there's an fp16 encoderonly T5-XXL intended for SD3 here https://huggingface.co/mcmonkey/google_t5-v1_1-xxl_encoderonly/tree/main

(which is on HF because the SGM training code uses HF so i uploaded that to dodge the long slow load on the full fat fp32 pair file)

[dull star]

Loading fp32 T5, was always the slowest part when using deepfloyd and pixart lol

[cobalt moon]

just get oof with OOM when using pixart lol

I haven't try Tiled KSampler though

[noble coyote]

"... obsessing into subsections ... groan, if only!!!" 😄

[sterile pendant]

Pixart has a couple of smaller versions in fp16 and bfp16. Drops the size to around 8-10gb and speeds loading up a lot. You can find links to them on the extra models comfy plugin page

Though the absolute easiest way to deal with the slow loads is to just get a cheap $40 nvme in the 2-5 gb/s range and models load in seconds

[muted dove]

The one in the link is only 10GB

[patent acorn]

huh i dont understand, then how do yall caption sd3 datasets?

[viral plaza]

dunno, probably not

there's work on them but nothing release-ready last i looked

[patent acorn]

oh cogvlm i see

[viral plaza]

it's possible those get finished soon after and released or something, or community takes over, idk, we'll see

[cobalt moon]

it is pretty weird that some having memory about Stability will release ControlNet models at day one

[viral plaza]

CogVLM is an AI model that generates captions, not a dataset. The captions that don't come from CogVLM, are just whatever captions came with the source images

[patent acorn]

ight captioning with my own brain then

[viral plaza]

if you're doing your own dataset, yeah just write your own captions

[sterile pendant]

yeah, we were talking about the fp32 versions from before though. it was like 20gb.

[viral plaza]

if you're doing a megascale dataset (millions or billions of images or whatever), the collection tools to build datasets like that usually can copy out original source caption data

[patent acorn]

for instance im planning to finetune BTD6 Lora on SD3, well i have to make my own cogvlm finetune..

its like 400+ images iirc

[viral plaza]

I expect the HuggingFace team will publish finetuning info on day 1, followed later by the various community finetuning software vendors (kohya, onetrainer, etc)

[late compass]

i have one question

How to get membership or early access?

[storm saffron]

Do you know what the system prompt and prompt was that was used in CogVLM? I've had some good and some bad results from using it.

[teal fossil]

I got CogVLM2 running yesterday and my first prompt got responses that copied the whole question before answering. Made it absolutely useless for captioning. ^^
I rewrote the prompt and then it was way better.
Only persisting problem so far(which I can understand is unimaginably hard for an Ai to understand) is that right and left hand / arm are swapped when a person is facing the camera.
I'll have to manually check all captions because sometimes it gets it right, most times not. 😅

[sterile pendant]

Most vlms are good to go out of the box, since that's literally their sole purpose and what they were trained to do. But if you're trying to have a specific format, you have to do a little bit of sys prompting. I spent a few days a while back making one for llava 1.6 Mistral and it was around 1000 tokens long with examples in it. You can also use RAG as well, but the info still bloats up the context size. I know llava 1.6 ate up like ~2000 tokens for the image information alone, so I didn't use RAG for it when I only had a 4096 context size(no sliding window or other optimizations)

[storm saffron]

The most annoying thing is it starting every caption with "In this image" or"The image contains" or variations of those, which I then have to take out. Also I had one vlm that just kept on about handbags for some reason (it was a Microsoft one).

[lavish pier]

Anyone has suggestions for an image to text/prompt model? I am trying to get a prompt from an image to generate similar images

[dry wave]

an easy way to avoid that is by using a predefined ouput-prefix

question: Write a short caption for this image.
response: This image shows [autocomplete from here]

[sterile pendant]

The other trick you can use, granted it's double the time, is to use a second pass with llama3 instruct(or any good recent instruct model) to clean up the captions and format them better

[cedar gale]

Or use taggerui and discourage from caption:

[sterile pendant]

They are starting to make some llama3 based vlms now, so that should be awesome. Think there's a llava next llama3 someone is working on

[sullen moss]

1 week... 🤗

[sterile pendant]

But up until now, the best one I've used that was the most reliable was llava 1.6 Mistral. Granted, that's for consumer level hardware and easily runs on 8gb vram. Mistral was a beast until llama3 came out(again talking about smaller models you can run on a normal PC)

[storm saffron]

https://www.ollama.com/library/llava-llama3

[sterile pendant]

here's an example of a long description version i made a while back. it was part of a two pass system where this would be fed into a prompt generator llm. the system prompt for this is ~700 tokens

[viral plaza]

memberships program is @ https://stability.ai/membership
i don't think there's sd3 early access

[viral plaza]

i don't know the answer to that sorry

[late compass]

i have some matter if you can come to dm

please

[viral plaza]

should use CogVLM2 anyway instead of the original, would be better and prompting it will be different

[late compass]

[viral plaza]

i don't trust llava at all for architecture reasons -- it just has images fed into the context token space of the LLM, vs cogvlm has active transformer layers dedicated to the image

[sterile pendant]

have you tried llava 1.6 mistral? it's pretty decent

(for it's size i should add, being able to run on an 8gb and all)

obviously, for larger scope and better hardware, i'm sure there are better alternatives

[sterile pendant]

i'm playing around with a different version called llama3 llava next 7b right now and it's pretty good so far. the next versions of llava are based on 1.6. i think the one you linked is based on 1.5

[teal fossil]

That doesn't work as intended most of the time.

[teal fossil]

In my tests CogVLM always performed better than Llava1.6.

Btw @viral plaza is there a time on the 12th we should look forward to?
A Midnight release? Late evening in the West US?

[cobalt moon]

you can say midnight release or 12pm release in UK

... or just random drop

[viral plaza]

Thaaat's a good question, will ask internally

[radiant hound]

What image captioner did you get cogvlm2 to work with?

[teal fossil]

I went on a lengthy adventure yesterday (thanks with a lot of help from Llama-3 70B & an OT community member) to get the repo of TagGui running on WSL. I'm very very happy that it works now. The next time a repo asks for Triton or bust - I know what to do. ⚡

[noble coyote]

Omost asks for Triton - and when you install it (the only version available!) - it tells you it is the wrong one?!

[lucid swift]

RGB gaming sausage

[noble coyote]

On another note entirely - I've been trying Stability Audio (via Pinokio) - and it's quite some fun!!!

[lucid swift]

the local mode?

i wish i would know how to fine tune it xD

[noble coyote]

Pinokio puts it all into a local VENV yes

[lucid swift]

did oyu know that the model got leaked like a week bevore it relesed

[cobalt moon]

how much GPU does it need?

[lucid swift]

i think 7gb vram

[cobalt moon]

that ComfyUI node said 8GB VRAM

[noble coyote]

Here is my first groove ... an arpeggio of a C9 chord https://drive.google.com/file/d/1lG2fArbCe2AauplHH2BhEAsUaKF3mKih/view?usp=drive_link

[cobalt moon]

too

[noble coyote]

Pinokio uses GRadio

[lucid swift]

i dont have accses just post the aduio in discrod

[noble coyote]

I have an 8Gb RTX 2070 and it works OK, about 45 seconds (100 iterations) to get a 47 second mp3 file

[lucid swift]

sd3 "Spectrogram"

[cobalt moon]

that doesn't look like a spectrogram though

but close enough

[lucid swift]

RGB gaming catheter

[noble coyote]

Is my audio interesting enough for you to continue your interest in Stability Audio at all?! 😄

[lucid swift]

yes i want to finetune the model but i dknt know how

i alredy installed it

[noble coyote]

"Comms Room Ruby Portal!!!"

[dull star]

I'm sorry if this has been asked or answered already, but will we have to truncate prompts ourselves if we make our own Loras or Finetunes?

[patent acorn]

i hope sd3 is able to make this

[low stone]

No

[patent acorn]

awww that sucks

[low stone]

Sigh. Still no.

Maybe 2b on the 12th will be better

[dull star]

with 2B we can cherry pick easier too

can't wait to mess around with text encoders

[low stone]

This is just one failed attempt after another

[dull star]

poor calf is out of focus

[low stone]

Yeah when it's local I can just tell it to render 30 of them, with tons of various llm variations of it. Not gonna do that against a paid api.

[dull star]

fr

[low stone]

Hah

Can you get it with the cat snuggling the oxen though and taking a pic of both of them

[dull star]

damn, they don't want to cooperate

[dull star]

huh?

[dull star]

is this ideogram?

[low stone]

oxen is sitting next to a tabbie cat who is taking a selfie of both of them. Background is a farm.

Dalle

[dull star]

oh

looks pretty good

we could overfit Loras to make something like this

We'll see how 2B looks like

or fully trained 8B

Festivalman do you think that skipping 4B is a good idea?

It makes sense to me, you will have 2B, which is what the majority of people will be able to run, 800M for low end users, then 8B for enthusiasts or small businesses if the Enterprise membership isn't too expensive

4B might stick out like a sore thumb and would delay the other models

[low stone]

tabbie cat that is snuggling with an oxen while taking a selfie of both of them. Background is a farm.

Dalle/sd3

I think sd3 did a good job, it's technically the selfie pic itself

[low stone]

Every size of sd3 they release will segment the user base that much more. I'm on the side of releasing as few as possible.

[cunning lintel]

if anyone wants to challenge sd3, try something like this (Dalll) (saw the pic in reddit a long time ago, sadly sd3 really doesn't shine with abstract concepts like "mustache from her hair" 😢 )

thinking about it, that image is extra evil as it also has hands 😂 (this is sd3, closest i got, but that mustache, is so much more real :p)

[low stone]

Yeah I'm getting the same thing

Did a good job with "anime" though. It really looks good

Sharper than deepbluev4 that I've been using I think

[patent acorn]

ok the sd3 is well done

[dull star]

is this supposed to be mewtwo?

[desert garnet]

nah thats mewfour 💀

[sterile pendant]

That's fine though. I think people will see that fine tunes aren't going to be as needed and that loras will be more of the go-to. Eventually someone will make an xadapter to share the loras between editions.

[low stone]

Yeah but you can't share Lora against different base models. These different size models will literally have different training against different images.

[coral sable]

octopussy SD3

Often feels like SD 1.5, lots of trial and error. But when the high roll hits it's really good. 6 more days

[dull star]

yeah 2B will be probably more consistent, especially with Finetunes

[teal fossil]

Hmmm... currently curating all the CoGVLM2 captions... which are nice, but still utterly filled with "suggestions" and "seems to be"'s "possibly" "appears to be" and other nonsense...
--> and now I'm wondering if that's actually stupid cause those might still be in the SD3 captions as well.

[dull star]

if those appear at the end like I remember them being, then those are probably truncated anyway...

or they had a good system prompt for it anyway

I do hate those assumption parts of the captions though

[coral sable]

coping for the best

[dull star]

8B is heavily undertrained

but it is coping for sure

we can't expect images to suddenly look 100x better

[coral sable]

i'd sacrifice consistency for variety anyday. Variety is amazing on SD3

[dull star]

Oh I mean like anatomical consistency or whatever

it will have variety

they talked about the model overfitting, causing differing seeds to become useless

so I think we're going to be okay

maybe the Turbo model(s) will have that though

and finetunes

[coral sable]

it can have both, I'm just sayin what is more important to me. Had lot's of fun with API already

[dull star]

same

8B is really good at paintings, so I hope 2B Base will be good at it too

same dataset after all, no?

should be similar enough

[low stone]

Playground 2.5 is an example of almost no variation across seeds.

[dull star]

btw @coral sable idk if you have seen this, but this is a TRULY raw image from 2B, no upscaling (From Lykon)

this is how the face looks upclose

remember, no highresfix or other tricks

[coral sable]

bigger dataset, same tech. Wouldn't call that the same dataset

[dull star]

I see

[coral sable]

yea saw that, really good for a base model

[dull star]

I wonder if the 16 channel VAE is just that much better for smaller details such as eyes

what's unfortunate, is that like with Pixart, highresfix has an issue

resolutions higher will create very very noisy artifacts

#🆕｜sd3 message

I suppose by the time we get it, they'll fix the pos embed code, or the community will within a week

if Tiled upscaling is okay, then I'll just do that then for the time being

[coral sable]

highresfix/adetailer still needed, details drop significantly on anything that isn't closeup. But that's to be expected as consumer hardware is what SD3 is targeting

[dull star]

yeahh

[coral sable]

I wonder will there be launch event on 12th as it was with SDXL

[dull star]

what was the event again?

[coral sable]

I mean just discord call with bunch of people

[dull star]

ah

[coral sable]

Stability guys presenting new tech

[dull star]

oh so like the center stage type of call?

[coral sable]

yea

[dull star]

I can't wait to remake Tekken intros as Live Action movie stills

I wanna see how good img2img is with such an intelligent AI

oh heck I could have tried that with pixart already lmao

I keep forgetting that it can do img2img too

you guys have any plans on what to use SD3 for that you couldn't or wouldn't bother to make with previous models (too much controlnet or regional prompting involved)?

[hallow talon]

I'm hoping it's a lot easier to do images with multiple people in it without their features (hairstyle clothes etc) blending together with the other person's.

[trail frost]

Is there any new update on pixart models

[dull star]

only 2K model is the biggest model released so far

there's lora training code

its in diffusers

nothing much to be honest that I know of that is "new"

[viral plaza]

yeah the new VAE is majorly beneficial for small details

[dull star]

nice

[viral plaza]

I'm told there is not a specific time we can give in advance, the actual release might be a process over a few hours (model weights, code, etc.) rather than necessarily all go at once

[viral plaza]

no event planned

[teal fossil]

Thanks for asking. Usual routine then (while I'm probably all giddy). 😄

[sterile pendant]

Which is why I brought up: https://showlab.github.io/X-Adapter/
I'm sure someone will make something similar for the various sd3 models. Who knows, maybe by the end of the year you'll even see one that allows you to use sdxl loras with sd3.

They won't be perfect 1:1s though, but should be close enough if it's done right. Maybe like 90% accurate

[dull star]

I wonder if comfy and your UI (stableswarm) releases the update earliest

[dull star]

nice

[dull star]

yes nvidia I will keep buying your 24GB high end cards for 3 gagillion dollars

[raven fern]

🤣

[turbid grotto]

Is this refined?

[low stone]

yeah with 0.3 denoise

[turbid grotto]

I like it

[raven fern]

lol

[dim sinew]

lol - I have an 8gig 3070ti, it’s just never enough sometimes. (Not doing ai just yet)

[dull star]

T5 bf16 on CPU with SD3 2B fp16 will be good for you 🙏

[raven fern]

[dull star]

it takes about 10-15 seconds to T5 to process the prompt, but once you are done and you don't change the prompt
you will be able to change anything else (CFG, Sampler, resolution, etc) and get instantly to generating the picture without having to regenerate it

so the future is bright

[raven fern]

the future is now, thanks to science

[dull star]

or just don't use T5, we don't know how much worse it's going to be without T5

[raven fern]

according to the research paper, the difference is not that huge

[dull star]

yeah but they also claimed stuff like DALLE3 level prompt adherence and stuff so... 😬

[raven fern]

yea i guess will see :3

[dull star]

but I do believe that images without Text might be decent without T5

if its as smart as Pixart-Sigma without T5 then it'll be good

caues its still a massive improvement over SDXL

[dim sinew]

Can you break this down for a noob?

[low stone]

pixart has a 300 token context window. sd3 is limited by clip length

[dull star]

yeah its funny

[raven fern]

its kinda already broken down :3 long story short, you dont have to run everything on gpu

[low stone]

it's really the benefit of the t5 stuff over clip

[dull star]

SD3 2B, which is the model that will get released on june 12th, will run just fine on 8GB of vram

you can use SD3 2B with T5 (a text generation model, which can also be used as an encoder for image generators), which increases its text capabilities (and possibly accuracy to the prompt)

but its expensive, it takes up a lot of VRAM

[dim sinew]

I’m a hobby photographer, and I am just stepping into wanting to create AI generated landscapes for my fantasy world I’m starting to write.

[dull star]

ah

If you are looking for mostly portraits of people or subjects, or simple landscapes, current models are fine for that.

[pseudo stone]

wll we ever get sd3 large

[dull star]

if you want to make detailed scenes with meaning, then pixart and SD3 2B will be the perfect fit

[dull star]

yes

we will get all models

[pseudo stone]

when tho

[dull star]

800M, 2B, 8B

but for now, 2B is the most optimally trained one

[pseudo stone]

like a month after medium

[dull star]

if not two

8B is a VERY large model

[pseudo stone]

I hope

so

[dim sinew]

No no… like deep sci-fi/fantasy stuff. Really describe a scene and generate.

[raven fern]

2B will all the toys with it will keep you occupied enough until the other models :3

[dull star]

ah yeah, then SD3 2B or Pixart will let you describe your scenes in detail and give you the best result you can achieve offline.

[low stone]

sd3/pixart/hunyuan

[dull star]

Pixart is completely free, SD3 2B needs licensing for commercial use (we are awaiting more info about this, but if you are just making stuff for your own enjoyment, ipso facto, personal use, then this does not affect you).

[raven fern]

4D chess 😮

[low stone]

can you describe an example scene?

[dim sinew]

Well, probably eventually will need commercial usage.

[dim sinew]

I’m honestly creatively brain dead atm. Been a long day

[low stone]

sounds good, here ya go

[dim sinew]

It sure if that’s a building generating energy. Or a weapon that landed and started imbedding itself into the moon to destroy it…

[dull star]

I like SD3 for paintings too

[low stone]

Here is the image that you requested:

[dim sinew]

I’ve toyed with also training a model for a specific side project idea. But I have no idea if you start either full compositions, or individual objects by name.

[dim sinew]

I love this!!!

[dull star]

thanks

[dull star]

if all else fails, I'll still gladly use SD3 for paintings

[low stone]

basically just get a chat going with llama3, and keep changing elements until it does what you want.

[dim sinew]

I don’t even know what this is!

[dull star]

its a text generator like chatgpt

or rather, an AI chatbot

you can ask for suggestions for prompts and etc

[low stone]

yeah, it's llama3 language model, running locally on ollama, which has the open-webui front end so it looks like chatgpt.

[dim sinew]

I need to now where to go to get all this stuff 🙂

[low stone]

https://ollama.com/ https://github.com/open-webui/open-webui

[solid violet]

llama generates images now?

oh yea but not for commercial use*

[low stone]

so using those combination of tools and a comfyui backend, you can have the chat interface generate images based off llama3 responses just by clicking the picture icon on the response.

so you can have a chat back and forth, generating images along the way

add this, change this.

[solid violet]

you’ve just changed my whole world man. thank you this was really insightful. where did you learn all this?

• self portrait, “allegory of the cave” depicts @solid violet led to the light by @low stone

[low stone]

i do this stuff for work and spend way too many hours at home doing it.

[agile hornet]

Will SD3 be usable in Fooocus?

[low stone]

eventually

man this ultra on the api for sd3 is 8 cents per image now. I've thrown a good amount of money at it, but that's Dall-E money already. I think I'll wait for the 2b at this point

[patent acorn]

This is my fav so far

[patent acorn]

where 2 bees

[gusty trail]

In the imagination

[hallow lion]

is this lama thing worth it?

[raven fern]

https://tenor.com/view/2b-gif-2822797880514591387

[agile hornet]

According to Terrance Howard 1 Bee+1Bee=3 lol

[noble coyote]

There are many competitions on Discord - and there are some very good prompts available - I borrow many from the OpenAI Daily Theme ...

[noble coyote]

(One Bee + One Bee) or not (One Bee + One Bee) - that is the question?! 😄

[noble coyote]

Eric, the half-a-bee?

[noble coyote]

Using Llama 2 Chat 7B Q4 - this is its result - Against a dark, starry backdrop, a sleek, metallic structure rises from the lunar surface, its gleaming facade reflecting the pale light of a distant sun. The building's fluid lines blur the distinction between form and function, as if it's alive, pulsing with an otherworldly energy. As the moon's gravity pulls at its base, the structure's edges begin to distort, revealing a glowing core that seems to be the source of the energy.

[noble coyote]

... no glowing core - sadly!

[sterile heath]

I mean it depends how it’s implemented. You don’t even need to run both on the same device or at the same time. You could generate an embedding, save it, load the next model and pass the embedding to it if you wanted to be silly. You can also just prune, quantize etc.

[teal fossil]

https://tenor.com/view/dune-stilgar-lisanalgaib-fremen-lisan-al-gaib-gif-13633185611076219756

Me next Wednesday if SD3M turns out to be what is promised.

Every time I work on a Dataset or train a LoRA Atm... I can't wait to play around with that sweet Sweet 16-channel Vae.

[hallow lion]

😄

[muted dove]

+tax = $3500

[cedar gale]

😛

The RTXX 4099 costs the same tho.

[radiant ledge]

this one even has two fans

[modern rover]

how to create a image

who can tell me ?

[hoary pilot]

M

[muted dove]

#artisan-faq

[tardy crag]

Will the model released on June 12th have the possibility to input an image or will be just text to image?

[weary crystal]

Short question did you used the sd3 API or a local running comfyui with api exposed?

[compact forge]

You will be able to download and use it for anything you could with sd1.5 or sdxl

[cobalt moon]

are you even answering the question

yeah i think festivalman hook up the SD3 API onto Ollama / open sourced LLM webui

[compact forge]

Sorry for answering the question

[bitter hearth]

last 2

wait not last one, this is very cute

[dim panther]

A living room with a large window, hardwood floors, and a fireplace. In the center of the room, a stylish couple is examining different sofa options, including a modern gray sectional, a tufted leather chesterfield, and a mid-century inspired loveseat. The room is filled with natural light and the couple appears to be deep in discussion, considering the size, color, and style of each sofa to determine the perfect fit for their space.

[lucid swift]

cool stuff

[lucid swift]

yt thubnail generator xD

[low stone]

I use both. It all ends up at their api in the same way.

[low stone]

Specifically with Ollama/open-webui, that's going to their supported sdxl json which gets sent via api to a local comfy. As of this minute, it doesn't support sd3 api for the image functionality. It only supports dalle / a1111 / comfy, with their prebuilt workflows and then your prompt or the llm response

[noble coyote]

I am 'Omosting' ...

[cedar gale]

Sounds hot.

[noble coyote]

A beautiful water nymph, bejewelled

Using Omost LLM Setup

[cedar gale]

Aye it is pretty dope.

[noble coyote]

https://github.com/lllyasviel/Omost

[cedar gale]

Tested it a while, I made these:

[noble coyote]

Would they have been harder to make without Omost?

[cedar gale]

Yes.

On top, in front else to hard.

Also things like bleed between 2 objects.

Omost is regional prompting on steroids for lazy people. 😄

[noble coyote]

I'll try some 'positional' stuff

[outer charm]

So... You can download sd3 now?

[runic tusk]

No.

[outer charm]

Oof, thanks soul

[runic tusk]

Trust me, literally everyone will know when it's available. You won't miss it. It'll be posted everywhere.

[low stone]

Are these sd3 or omost? Impressive if omost

[cedar gale]

These are omost.

[noble coyote]

Omost - here is my 'positional' prompt - a giraffe is wearing a top hat. On the hat sits a green frog, eating a red apple and reading a blue book. A snail sits next to the frog. A butterfly sits on the book

Ultimately very very poor positional takeup at all.

Yet the Omost text does specify all of this prompt ... it just doesn't deliver?!?!?!?!?

[swift portal]

Hi

[cedar gale]

I think this is to much.

[noble coyote]

Keep it simple?

As I say, Omost picked up on all this and laid it out neatly section by section ...

[woeful spindle]

I dunno if we'll be able to do use sd3 instead of sdxl in omost but if we will, that's gonna be a huge improvement

does it say anything about changing the model on the github page?

[noble coyote]

If Omost is this disappointing though ...

[woeful spindle]

you cant really say "disappointing"

cus it's using sdxl

we all know how sdxl performs

we can see a clear improvement there

[noble coyote]

Nerdy Rodent did an Omost Video on YT - he opens the code and drills down to the model name ...

Someone said it was RealVis SDXL?

[cedar gale]

I think so but you can just swap the SD XL model.

You just need to convert it to diffusers and route it via config.

[noble coyote]

Tell me how? 🙂

When I'm Omosting, I swap-out Torch 2.0.0-cu118 (Xformers in ComfyUI) for 2.3.0-cu121

... and then back again when using ComfyUI

[cedar gale]

DM-ed you some stuff to avoid spam here. 🙂

[woeful spindle]

Could u send me too

[noble coyote]

Give me a prompt which will work in Omost, svp?

[lucid swift]

sd3

[dull star]

horse

watching a video

of a fox eating a hamburjer

SD3 is truly the best

🙏

[brisk cipher]

Where can I try sd3?

[noble coyote]

FABLAN@Glif

[trail frost]

Stability api

[noble coyote]

FABLAN@Glif is free

Free SD3

Omost

[brisk cipher]

Link

Can?

[noble coyote]

https://glif.app/@fab1an/glifs/clv4ca3aq0004xklr050jusec

[dull star]

@brisk cipher https://glif.app/@fab1an/glifs/clv55nhjy0003yr19n09wz0np if you want a completely raw one where you can do paintings and other artstyles, and even write negative prompts, I recommend this

gives you the most control

[noble coyote]

Omost - Peppa and Cthulu

[noble coyote]

From SD3@GlifyGlif

[past flame]

Omost is not remotely SD3

[rain current]

Absolutely, at the moment the only thing (in my opinion) that competes with understanding the prompt and with good quality is Ideogram

Example

[sick cedar]

Is that SD3? 👀

[rain current]

ideogram

[sick cedar]

It's not too far off then. SD3. I mean we are also comparing a undertrained SD3 model with a fully trained Ideogram. So i'd say that's pretty good.

In my opinion, i think SD3 did better with the Horse looking at the TV monitor prompt.
The burger on the screen looks more coherent on SD3.

[rain current]

I'm not saying it's better than SD3, especially considering that SD3 will have more training, and with controlnet, etc. It's just that, as of today, it's the only thing I see that's similar and free

[sick cedar]

Oh for sure yh.

[rain current]

But I also see that it is more obedient to the prompt "slightly plump". I'm not sure if 2B is capable of being that precise.

[rugged tartan]

I am seeing a significant difference between CORE and ULTRA in the API. the first image is comning from CORE the second is the result of ULTRA, same prompt and same seed. Ultra seems to ignore the details of the prompt. Does someone here is experiencing the same issues? In general everything done with SD3 in the API resembles the quality of SD-1.6

Prompt: Picture of a Anime-like real life woman with intricate face paint baroque style, Japanese facial features, realistic skin texture, photographic, photo-realistic

[rain current]

(ideogram + sdxl refiner)

[viral plaza]

if core fits your preferences, use core. Core is generally best stable setup and Ultra is best experimental/beta setup, rn Ultra is using experimental SD3 models and doesn't have the xl heavy finetuning that core has

[rugged nova]

If you don't mind, would you explain how to swap the model for Omost? I would appreciate it.

Nvm, someone make a fork to allow you to select a model https://github.com/runnitai/Omost/tree/main

[vapid radish]

SD3 VS My SDXL Merge
I know portraits are not going to be SD3's strengh until people finetune the art out of it.

I cannot wait to upscale image with SD3 locally though.

[dull star]

that's pretty good for a base model

and it doesn1t have that typical finetune look to it

[gray flicker]

girl

[lucid swift]

what propts did oyu use for the hyena

[dusky thistle]

Here is the image you requested.

[hallow lion]

make an image of the sd3 model training in the gym

coz it needs more training

[hallow lion]

is this what te fellow kids ar einto these days

[hallow lion]

elphant tho

:<

[cedar gale]

😦

[hallow lion]

text is the new hands

[hallow lion]

lol why a ghost

[bitter hearth]

You didn't specify

[hallow lion]

🤣

[bitter hearth]

runs away

[raven fern]

lol

[patent acorn]

sd3 request: man crying because there are no food in the fridge

[hallow lion]

Asking for a friend? 😄

[patent acorn]

not interested

[patent acorn]

2nd is better though he should face to the firdge

also

there are still FOODS!!

shouldve been an empty fridge i forgot

[real terrace]

touché, that's true, well it is not that smart

[patent acorn]

ough

[noble coyote]

Does Omost not have good prompt coherence?

[cedar gale]

Went for depressed instead of crying. 😄

[bitter hearth]

[noble coyote]

Just trying Anytest Controlnet ...

[noble coyote]

Originals made using Portrait Master. ComfyUI+Anytest Controlnet - prompt = beautiful leopard, sunny glade, hat, cinematic lighting. [NightVisionXL, dmpp_2s_ancestral, karras, 30 iterations]

Originals made using Portrait Master. ComfyUI+Anytest Controlnet - prompt = beautiful peacock, sunny glade, hat, cinematic lighting. [NightVisionXL, dmpp_2s_ancestral, karras, 30 iterations]

Originals made using Portrait Master. ComfyUI+Anytest Controlnet - prompt = beautiful fox, sunny glade, hat, cinematic lighting. [NightVisionXL, dmpp_2s_ancestral, karras, 30 iterations]

[pseudo vault]

What's the best generator for producing text?

[noble coyote]

Ideogram

Or try ComfyUI and the Harrlogos2 LoRA

[pseudo vault]

Any usable on a phone? (Android) Not near my computer anytime soon lol

[noble coyote]

Ideogram may be

Or free trial Adobe Firefly

Firefly can be fiddly on a phone

[hallow lion]

Nothing on the phone works.

[pseudo vault]

@noble coyote Thanks pal

[noble coyote]

Just trying on my Android phone ... Ideogram

[patent acorn]

nice

[noble coyote]

I logged in to Ideogram (free-to-use) via my Google account, and have generated these 3 images via my Galaxy A9 phone - SD3 will have to be very (very) good to get to this standard of text!

[dull star]

It may actually get these correct, it's just that the layout might look too 2D or pasted on. We'll see eventually.

[west haven]

A staff member in the office is wearing yellow clothes with three conspicuous letters: PPT.

[zenith talon]

ddf

[dim sinew]

If I gave you a 4031x2687 image of squirls to re-run with whatever prompt that was, could you convert it?

[dim sinew]

Yeah -- I'm deff looking to eventually use stuff for commercial purposes. Doing some research for a project right now. Need to learn how to do stuff in general, and then start working on building a model for the plans we have.

[rain current]

A close-up of a hyena's face. The hyena appears to be in a contemplative or playful pose, with its front paws placed near its chin, as if it's holding its face. The photograph is in black and white, emphasizing the texture of the hyena's fur and the intricate details of its facial features. The background is blurred, putting the focus entirely on the hyena.

[low stone]

One of the big benefits of ideogram is their magic prompter. It's crazy good at making great scenes with little input.

[dim sinew]

When I saw this I was like ... Oh look, its a decedent of Frog from Chrono Trigger.

[trail frost]

Im so excited for sd3 open release

Can we use that in our projects like sdxl 1.0?

[dull star]

if you use it and not make money, you can use it with no difference at all

it doesn't affect people like us

its only for the people who want to make money using the model who might have a bit of trouble

it'll get sorted out at release.

[urban arch]

The Goat needs 6 legs to be accurate.

[dreamy sundial]

what?? faces are clearly the strength of sd3, there's a ton of variety of faces, all sd 1,5 and sdxl faces look the same, when finetuned it will be mj6 quality for sure

[vapid radish]

I have never met anyone with eyes like this...

But an upscale with SD should help, just too expsense on the API right now.

[dreamy sundial]

this is a super undertrained base model, you're comparing it to an extremly finetuned model with a workflow and highres fix, of course is not gonna be as detailed

while all the people on sdxl look exactly the same

result of overtraining

[vapid radish]

No they where both 1024x1024 raw outputs.

[dreamy sundial]

and merging

[dreamy sundial]

but still you chose the worse result that sd3 could have gave you

[cedar gale]

Looks kinda hot tho. 😛

[vapid radish]

I'm not saying it is bad I really like it.

[dreamy sundial]

i've had far better results with better prompting, gpt4o really helps a lot in most cases

[vapid radish]

This is what SDXL looks like after my workflow (6K zoom in), I'm sure SD3 will look great as well.

[dreamy sundial]

it's crispier higher res yes, but the skin looks like plastic and eyes soulless, like doll, and i'm sure every image is almost the same, no variation, almost all sdxl models look like that

[dreamy sundial]

sd3 face variation is on par with mj6

raw SD3 outputs

[vapid radish]

I can do less smooth skin, a lot of people just prefer that look.

[dreamy sundial]

also the people are looking at different directions, sdxl people always look straight and the eyes are focused looking at the camera in a soulles way

[past flame]

It uses a chatbot to plan out where the given object will be in an image and then uses normal SDXL with some regional prompting extension

It has nothing to do with how SD3 architecture works

[noble coyote]

Often my SDXL 'subjects' give no clue that "there is a camera at all!" Until I use Face Detailer; then evey subject is looking straight-to-camera!

[sick cedar]

We also using a undertrained version of SD3 currently too!

[dull star]

I can't wait for 2B

I really really hope the paintings are good 🙏

just 4 days

[dreamy sundial]

me too man, imagine what things companies like leonardo.ai will do

[hallow talon]

I wonder how long after the release of 2B that it'll be supported in A1111? I can use Comfy but it's not ideal

[vapid radish]

Paintings seem pretty damn good to me.
https://glif.app/@Jib/runs/wy2hokyaknzijufrcujo3l4d

[dull star]

yeah 8B is good

I wanna see how good 2B is

[civic quail]

what are the requirements for sd3

[robust junco]

How about doing SDXL-> SD3 or SD3->SDXL->SD3?

[civic quail]

i have 8gb vram 32gb ram

[dull star]

refine with SD3?

🤨

[low stone]

I was fooling around with denoise yesterday. The biggest issue with sdxl is multi subject refining. It just wants to make everyone the same thing. I managed to upscale by having the various refining stages with various denoise levels, but it was super finicky and needed different denoise numbers per image, so in the end it wasn't useful long term. I'm really hoping that refining with sd3 will solve this because it understands multi subject.

For example.

Ella is really good at complex multi subject. But it just makes the sdxl upscale's job really hard

[dull star]

https://x.com/Lykon4072/status/1799377331003556059

Lykon (@Lykon4072) on X

"Ultra" now on Stable Image Services
#SD3

Twitter · Jun 8, 2024, 9:46 AM

probably posted here already

[robust junco]

yet have you tried the way mentioned? 🙂

[low stone]

That would be 16 cents per image at this point. This new api pricing is too expensive now. I can't see myself using it anymore.

I get 1000 images a month from ideogram for $20 and I can img2img with sdxl or sd3 2b when it comes out

[robust junco]

yes, I see,as well thought you would possibly have done it already

[low stone]

Sd3 is dalle price now, but we all know that for most things, it's not as good (the censored api sd3). Obviously local sd3 is a different story.

[mortal mesa]

which one is Ultra

[left parrot]

That should be enough to run the 2B model that will be released on wednesday, but not the Huge 8B mode that's coming eventually. 2B seems to be plenty good enough for most uses!

[dull star]

yeah considering Ultra is a whole SD3 workflow, it's less acceptable that it's DALLE price

[dreamy sundial]

i hope we could run at least the 4B model on 8gb vram

[dull star]

they might skip 4B

[left parrot]

yeah, the 4B doesn't seem like it has that much of a use case, stuck between the top quality 8B and the 2B that's balanced between performance and output quality

I suspect most finetunes and lora's will be based on either 2B or 8B

[turbid grotto]

what about upscaling with ella + kohya deep shrink?

[low stone]

I'm upscaling with ella for that first increase from native 1.5 res up to 1.5x that, but then going sdxl after that. the problem is that sd 1.5 checkpoints aren't very good compared to sdxl which are amazing at this point.

so it'll keep multi-subject until high res, but compared to sdxl refinement if it works, is night and day quality.

[turbid grotto]

hope there will be a way to quantize 8b , because I feel like 4b would be a perfect option for 12gb cards

[turbid grotto]

oh thanks

I found sdxl hyper a really good for upscaling with only 4-8 steps

[low stone]

that's what my ella workflow looks like.

first 2 on left are ella native, last 2 are sdxl.

so as you can see, the sd 1.5 checkpoints just don't have the content in them. they're very lacking in training outside porn.

yeah i'm using sdxl hyper now all over the place. it's amazing

[raven fern]

does it know Bob Ross style? :3

[dreamy sundial]

why?

[dreamy sundial]

or 8gb cards too

they promised to release all versions

2b is too low and 8b too high, i think 4b is what majority of people would use

[dull star]

Store at FP8, load in FP16, less space used and iirc, less VRAM too maybe

[dull star]

already too many choices to split the community

[vapid radish]

It will give you Boss Ross if you like (take that Dalle.3).

[turbid grotto]

yea sdxl is nice, can't imagine how good upscaling can be with 8b

[mortal mesa]

there is no 4B and community split talk never seems to effect anything

[raven fern]

i mean sure... :3

[turbid grotto]

hope it gonna work well

however, 2b could be really enough with controlnets and other stuff

[turbid grotto]

maybe with some optimizations? Even sdxl (2.6+something) takes about 8gb on my pc

[dreamy sundial]

it runs perfectly on my 8gb card, really fast also 12 seconds an image

[turbid grotto]

oh, I will run out of ram for 8b
even loading sdxl peaks roughly at 14gb

[turbid grotto]

it that standard model with 20 samples?

[dreamy sundial]

i think it was 15-20 steps i dont remember

[turbid grotto]

1024px, 20sp, sdxl = 19s on 3060

[dreamy sundial]

not bad

[turbid grotto]

then it is really nice

I didn't expect

[dreamy sundial]

yeah, on comfy tho

[low stone]

this is gonna be great when we have sd3 at home

[low stone]

do not taunt happy fun nordvpn

censorship dragon getting you down? use nordvpn.

[low stone]

Release the weights!

[compact forge]

is the artisan bot using sd3 release candidate already?

[hallow lion]

those hands are transcendental

she has 2 rings but ill let that slide

i guess that could happen

[bitter hearth]

Nothing wrong with 2 rings

[rich iron]

can it protect your chicken from Dokken?

[dull star]

very epic

[raven fern]

elden ring boss 😮

[raven fern]

very sharp details

[bitter hearth]

edge of technology

[raven fern]

cutting egde

[patent acorn]

edging

[patent acorn]

hello sd3 make me lego ninjago characters

[low stone]

a tornado of library books, weird supernatural ghosts flying around sliming the library

[raven fern]

slimy ghosts

[scenic shadow]

Has there been any word on if any finetuners got early access?

[silver sluice]

does anyone have a ComfyUI workflow that's compatible with SD3? I realize it's not realeased yet but I'm guessing for those who do have early access there's already a workflow developed for it? I just want to make sure I get the nodes installed and ready for launch day

[hallow talon]

I wonder how long after release of the model that we'll be able to train and finetune using Kohya? I'm definitely excited about the possibilities with SD3 when it comes to finetuning