💬｜general-chat

[split kestrel]

Lol

[charred mesa]

well Llama-7B and Mistral-7B quantize very well with 4-bit, but those are LLMs, and also, do you think that they will make quantized models themselves?

I mean, if a 4-bit or 8-bit 8B still performs better than a fp16 2B then fine by me...

[crude notch]

hahahahaha lol no

it cant even run on ios

[split kestrel]

🙂 I know.

[crude notch]

they havent said if they'll do official quantized but i think they will

[split kestrel]

If I’m getting by on a 3090ti… an iPhone is gonna murk

[quaint lantern]

Hey

How are you?

[charred mesa]

maybe TheBloke will come back to life and quantize them for us

but thank you for this info

I hope this will get 8B on lower VRAM machines without massive quality degredations

[crude notch]

12gb vram is barely enough for all at once generation with 8b

[quaint lantern]

Anyone play games?

[old solstice]

He just runs a script that anyone could run. His contribution is to have access to a server with a lot of RAM to do it faster, but really anyone could do this with disk swap.

It's not difficult

[sick wave]

I haven't kept up with stable diffusion lately. What's Pony and PonyXL? I keep seeing it on loras. Is it just another model?

[old solstice]

It is a finetune of SDXL

[sick wave]

So if I understand, it's a model based of Stable Diffusion XL? Is it just very high quality?

[old solstice]

Yes

[sick wave]

Do you actually need Lora's specifically made for it? or is it just better with these loras?

[charred mesa]

you mean every model loaded at once onto VRAM?

[crude notch]

yea barely

t5 quantized in 4bit

[charred mesa]

what about offloading, but T5 and 8-bit

and what quantization is 8B at

fp16 or lower

[crude notch]

8b running in pure fp16

[charred mesa]

damn!

[crude notch]

the sd folks think fp8 is not a good idea

or so ive heard

[charred mesa]

well on 40 series its okay-ish? but on anything else its a massive slowdown

that's what I remember

[crude notch]

i honestly think its a terrible idea for img models

you lose too much precision

[charred mesa]

ah

so... with comfy's offloading method, T5 + SD3 8B is feasible on 12GB?

[crude notch]

comfy mem management + t5@4bit + SD3@fp16 + clips@fp16 should work on 12gb vram

[charred mesa]

oh my god....

that sounds wonderful, thank you

especially with SD3 at fp16

I suppose T5 at 4-bit is still good

[crude notch]

it is

[charred mesa]

my goodness

[crude notch]

^ for training

[charred mesa]

so it ISN't over yet....

so it's NOT JOEVER

[crude notch]

god i want to leak something so bad....

[charred mesa]

lol

[crude notch]

all i will say is that 12gb vram is no longer a barely ;3

[charred mesa]

YAAAAY

[crude notch]

(with offload, offloadless is still 12gb)

[charred mesa]

this is epic news, thank you

[crude notch]

oh yeah also sd3 is still going to be public release lol

[charred mesa]

oh I wasn't doubting that

I saw the tweets from the CTO

idk why people are worried about that

[crude notch]

will take a while but not more than 6-8 months

if i had to guess

[charred mesa]

hell yeah

6-8 months 🔥

time to enter hibernation

[crude notch]

not more lol

[teal pagoda]

until what?

[crude notch]

sd3 public release (open model)

[charred mesa]

I guess the 4-6 weeks was a very optimistic ETA from the CTO

[crude notch]

yeah no that feels optimistic

[teal pagoda]

what? 8 months? didn't they say it will be launched in april/may?

:)))))))))))))))

[charred mesa]

emad moment

[crude notch]

at most 6-8 months

[crude notch]

the new cto sounds like emad but if it were feasable

[teal pagoda]

emad, you lier ape

[crude notch]

i see sd2.9 finishing in 5-6 weeks

[charred mesa]

if they release weights

[crude notch]

most likely the same as xl0.9

[charred mesa]

leaked?!

[crude notch]

researcher beta model then a public final model

[charred mesa]

ah

[teal pagoda]

it seems that you had access to sd3 already. Honestly now, is it really better than XL?

[crude notch]

they put bets already

[charred mesa]

well if a researcher model can be research enough to generate skibid toilet movie posters then fine by me

[crude notch]

yes.

[charred mesa]

yes.

[crude notch]

closest we have to a local dall3

hold on

[teal pagoda]

so it's not better like the emad lier said

[charred mesa]

well it can't be

lol

finetunes might get us very close though

[teal pagoda]

why not?

dall-e 3 was released like 5-6 months ago

so stability had time to outperform them

and to take the best things from Dall-E 3

to make SD3 some next level generator

[charred mesa]

idk man

as long as I can do highresfix with SD3 offline with like loras or whatever on 12GB it's gonna be the best image generator in my eyes

[crude notch]

emad does emad things

[crude notch]

dall3 is 80b

[teal pagoda]

I mean if you want to release a very good product (or the best one), don't you look at what your competition has first?

[charred mesa]

technically, SD3 is the best for being able to be used offline

[crude notch]

and dall3 has gpt4 on its side

[charred mesa]

the more I use ideogram the more it starts to look like the smaller models at points

so I wonder if 2B finetunes might compete with ideogram for example

[crude notch]

we have an 8b model that gets close to that

[charred mesa]

but the 8B, that's like between midjourney and DALLE3

even the turbo model is apparently smarter than midjourney

[crude notch]

i dont use mj but seems like it

[charred mesa]

at least the paper claims it

[teal pagoda]

so SD3 is 8B model?

[charred mesa]

I wonder if 8B turbo might become a daily driver like SDXL Lightning

cause 8B Turbo is literally SDXL Lightning, highres and low step count whilst looking good enough

I expected another faint 512px model but they delievered

[crude notch]

sd3 is a group of models containing an 8b, 6b, 2b and 800m

[charred mesa]

6B????

[crude notch]

yea

4 models

[charred mesa]

I knew every other

never 6B though

damn

[crude notch]

its alright

[charred mesa]

so is that one of the models in the preview

[crude notch]

mainly made for 8gb vram cards

[charred mesa]

ooh I hope 6B will close the gap, I want other to enjoy it a lot

[crude notch]

the models in the server are only 8b

just different epochs

[charred mesa]

Ooooooooooooooooooooooh

so that's what they meant by random models

[honest spear]

what about the controlnets for sd3? Are there already models being tested?

[charred mesa]

idk if the sketch model on their recent blog was FOR SD3 or not

lemme check

[crude notch]

no not yet

[charred mesa]

ah

I also can't wait for more prompt expanding model similar to SuperPrompt-v1

im already using that model offline on CPU and its super fast

[crude notch]

btw for the weebs that are lurking, yes it can do anime, no it is not perfect, yes there are plans for training the model on weeb

[charred mesa]

comfy's images looked good to me

of course that's coming from a non-weeb

I wonder if the smaller models get successfully fine-tuned with corn, would they finally replace 1.5?

after all, it would do nsfw, at low memory requirements, with controlnets, which is what 1.5 dudes need

[crude notch]

are there any 1.5 models just for corn?

oh.

[charred mesa]

are there any 1.5 models not for cor-

[crude notch]

ohh..........

[charred mesa]

[crude notch]

you mean that

i thought you meant the food

[charred mesa]

hehe

[crude notch]

so funny reading the misconceptions people have about sd3

[charred mesa]

like what

I've mostly seen stuff like "the model getting lobotomized for public use" or whatever

[crude notch]

a lobotomy is not going to happen

[charred mesa]

idk how last second lobotomizations could work

without massive quality degredations

[crude notch]

it wouldnt

[charred mesa]

thank you

so it is BS

[crude notch]

i see so many people still calling it a UNet

[charred mesa]

LOL

[crude notch]

it is not a unet

[charred mesa]

hmm TF2 in Ideogram ends up being a very generic class shooter

I just wonder how much 8B will help in knowledge

[hybrid thistle]

how long does getting sd3 from the waitlist take?

[dim minnow]

[nova zodiac]

how long is a piece of string?

[onyx mason]

What happened to the unstable diffusion server

?

[hybrid thistle]

homie u aint the riddler just tell me or dont lmao

[nova zodiac]

aint no mother fucker know...

one group of about a dozen insiders got it a week ago, other than that no mofo on the waitlist aint seen shit

[hybrid thistle]

oh damn thanks anyway

[arctic sedge]

SuperpromptV1? I've not heard of that.

[charred mesa]

https://brianfitzgerald.xyz/prompt-augmentation/

[arctic sedge]

Omg.. How much Vram does this consume?

With SDXL on top, you said you ran this on your system?

[charred mesa]

its a 77M llm, it's very small

I use it on the CPU and its super fast

[arctic sedge]

Oh of course! Sorry. I read that as 77B Lol! Mb

Wow! 77m. Yh. That is pretty damn small for a LLM.

[charred mesa]

its pretty good though

[arctic sedge]

How did you run it? I gather there isn't a extention for this yet? ..

[charred mesa]

yeah I ran it separately using a gui

https://github.com/Nick088Official/SuperPrompt-v1

[arctic sedge]

Ah I see. Yh. I would have to do that too then.

[charred mesa]

if you download the .zip it has everything in there

its just python scripts, you can look at the code and everything

[arctic sedge]

Sick.

[charred mesa]

set different default values

it will handle everything

[arctic sedge]

Lol! This is awesome.

How well did it perform for you?

[charred mesa]

it doesn't make a venv though I think

[charred mesa]

it's pretty good

I do have a certain setting that performs best

hold on

[arctic sedge]

Ok! Sweet

[charred mesa]

temperature at 0.7
Repetition penalty at 1.6
Top P at 0.9
Top K at 40

these give the least amount of hallucinations

[arctic sedge]

Ahh! Thx m8. I will try this out. Been waiting ages to try out a LLM text encoder.

[charred mesa]

yeah its good at making prompts

[arctic sedge]

So do you make a prompt, and then the LLM spits out one of it's own for you to use in SDXL?

[charred mesa]

similar to Ideogram's magic prompt

[charred mesa]

yes

[arctic sedge]

Ah i see. Ok. Nice!

[charred mesa]

photo of cat, in church

photo captures a cozy living room decorated with plush sofas and armchairs. In the center of the frame, a sleek black cat sits upright, gazing up at the sky. The cat's fur is painted in shades of orange and brown, and its eyes seem to sparkle with intelligence and curiosity. The church is filled with the sound of chatter and laughter as people gather around the scene.

This will make prompting SD3 a breeze

[onyx mason]

Like it had over 100k members and now doesnt exist anymore. Where did they all move too, is there a new discord?

[rugged mirage]

what do you mean? I seem to still be in it (I rarely check it tho)

[dusk canopy]

RuntimeError: Input type (struct c10::Half) and bias type (struct c10::BFloat16) should be the same

[onyx mason]

Can you send me an invite link in dms?

[rugged mirage]

sure

[trail lion]

Nice, bookmarked that github

[arctic sedge]

I assume the model is built into the Ui? I don't see a folder for the model, but it's running a LLM because i can give it prompts.

(Thank you for telling us about SuperPrompt btw.)

[trail lion]

Too bad not a web UI, for Linux users

Should be able to pillage the colab notebook to fashion a CLI

Maybe even getting jupyter running in a docker wouldn't be too hard

[low moon]

Sora building up too much hype, no matter how good it is it will be a disappointment. Change my mind.

[trail lion]

Only has to be good enough to impress their target audience, which seems to be Hollywood

[dusk canopy]

💀

sora is liek gta 6

[oak patrol]

For folks calling SD from python API, do you generally prefer using Anaconda / conda or Python / pip?

[dusk canopy]

mfs always gonna overestimate and overhype

[tulip yarrow]

good afternoon

[low moon]

SD3 does give me terrible HL3 PTSD.

I hope this does come out

XD

Can Stability count to 3?

[tulip yarrow]

Alright so I want to train a style Lora. What keywords do I remove so that the activation tag can absorb them

[dusk canopy]

moneky

use kuhoy

a

[tulip yarrow]

kohya?

[dusk canopy]

ye

it will take 15 hours

[tulip yarrow]

yeah, and kohya doesn't work for me (i use google colab)

[dusk canopy]

why

my 3060 can run that shit

[tulip yarrow]

alright i'll try running kohya again

[sweet flint]

yo guys

[split kestrel]

Yo

[sweet flint]

check #🏞｜general-with-images . someone help. xd

[sweet flint]

how much timeframe is considered good for a 1024 x 1024 pic. on a 3080ti 32 gigs ddr5 ram

[trail lion]

2 days

[split kestrel]

Thought I was being sneaky…. Just tooted by my work desk. Was not a quiet toot. And I called myself out.

Cary on

[karmic cedar]

Actions speak louder than words. Their small testing pool coupled with the senior departures tells me they’re freaked out.

[karmic cedar]

If your toot is quiet, you’re doing it wrong (or you just require more vespene gas)

[split kestrel]

lol

[tulip yarrow]

How does one make a multi concept Lora (like one for multiple characters and such)?

[fervent thunder]

i’m in my comfort zone and when i’m in that all i do is eat bread and drink water while i watch a good movie 😍

[trail lion]

multiple characters in a single lora isnt advised, there is bleed over between them since they are in the same class. just do different loras

[polar helm]

why there is no any more test image generated by sd3

[trail lion]

do you mean the bot? I imagine it will be back, but it could be a few weeks

[pearl ocean]

why is creating images so much fun? 😄

[pearl ocean]

if it takes me about 5-10 secs for a 4080, im guessing you probs get maybe 30-45 secs

[uneven pulsar]

does anyone know where deforum server is, im trying to get help with parseq

[wet dawn]

gm

[arctic mesa]

so... I have a question about this... #📣｜announcements message
does that mean an individual has to pay 20 bucks a month in order to sell their one image a week?
Lykon told us "of course you can sell your output" ... i hope he forgot "free of charge" .. and not "if you are subscribed"

[split kestrel]

can anyone explain rife and what the scale factor does?

mainly the difference between 47 / 49 and what the multi / scale factor do

[astral goblet]

i create venvs an use pip, but only because it was the first one i used . all these others wanna fuck that method up. install conda and nothing else works no more. bullshit. total fuckin BULLSHIT.

probably a skill issue. i just stay in the harbor i know

[astral goblet]

there are ways but its trickier unles you know exactly how ot specific multiconcept datasets with yaml files and set the rank appropriately. Easier just to have multiple loras and regional prompt them

https://civitai.com/models/9513/fnaf-multi-character-lora "it can't be done" but like, they exist all over civit

[icy quest]

how do u upscale shit? the sd upscale script makes weird tiles

[trail lion]

if you're trying to sell it, you're commercial. where's the confusion

some of us are just entertaining ourselves. some of us are trying to make a career out of it. using their software for that purpose makes it commercial, that's pretty clear and fair tbh

[arctic mesa]

so... imagine blender going about like that ... here have a free tool... and then you work your ass off and git gud... and then some one comes along and tells you.. hey.. i like the thing YOU did .. here have a couple ... and blender going like.. yeah but if you wanna SELL the animation you did you have to subscribe to blender first 😂 ... good luck

putting blender on a web-site with a thin layer of UI and make it do things.. and charging ppl for it. THAT is a different story entirely

unreal.. another good example ... would not exist

you made the engine.. now you need a killer app ... 🤷🏻‍♂️ (easily accessible for public consumption, charged usage)
like use the engine like everyone would (produces good examples and use cases)... and if the income of companies and individuals goes over a certain threshold you get a cut

THAT seems clear and reasonable .. you also don't bite the hand that feeds you ^^

[low moon]

Yes make as much money as you can before the end.

The way things are going you have to hurry too, remember you have to also have time to spend it before game over.

[arctic mesa]

the magic sauce is to inspire ppl and make them thrive, so you thrive in return.. makes sense?

[low moon]

There's too much greed and corruption for that, it's kind of over at this point, like derailed train flying off a cliff. The economy, politics, social fabric. all of it

hence we make nice AI pictures of the world we want to live in instead

[arctic mesa]

well .. OR.. you make something awesome, despite the world ... i found out about Krita... the SD plugins are basic and lacking but it's the closest i ever came to my imaginary magic brush and 10 times better than firefly... oh.. i can't post images of the process here ^^

[low moon]

yeah lets go i love kritaa and the sd plugin but i havent p[;ayed with it much

[arctic mesa]

check my profile.. i have some vids up.. super basic ones ^^ ... i gathered resources and tips but they are in a forum page of another discord (should make them available elsewhere 🤔 ).. i will not post it here cuz i don't know if that would be frowned upon 😅

[low moon]

Nice nice

insect ladies huh?

😉

[arctic mesa]

a man's gotta do what a man's gotta do 😄

[low moon]

very cretaive

i usually just do realism

check my youtube the latets vide is made with AI

google "eyaura"

[arctic mesa]

alice was nice (attention masking checkpoint test).. and the singularity charmer (first shot at Krita) was based off of an impossible joke prompt from a mad man ... especially "a violin made of fucking lightning" was a nightmare to get right ^^

[arctic mesa]

aye sir

[low moon]

ma'am :))

[arctic mesa]

yes ma'am 🫡

[arctic mesa]

JEZUS... these horror interjections and freaky music ... very experimental ... i get like neon demon vibes ... but more grounded

youtube allows this? ^^ ... PG-13 is a stretch .... i love contrasts ... i'm biased more towards traditional beauty tho ... like technology and raw nature (jurassic park, 1993)

[low moon]

😁

I cant wait to have consistency with AI

then we make real movies

[arctic mesa]

fuck YEAH?!

[arctic mesa]

there IS a blender asset that could help you .. it's 😙👌🏻🤌🏻

https://toyxyz.gumroad.com/l/ciojz

outputs controlNet batch feeds ... openpose, DEPTH, canny ... and ALL the others ... great work .. you can even attach the rig to your own character one with a tool that comes with it

cuz screw vid2vid ^^ ... fancy dancing filters ^^

[low moon]

🧐

[chilly tangle]

Anyone had any luck with SD3?

[opal hedge]

They didn't release it yet

[chilly tangle]

I know I mean with the discord early test ?

Btw anyone knows when would release be? any rumors?

[opal hedge]

Ah, I haven't heard any announcements about more invites. And release is in 3-5 weeks

[chilly tangle]

Not very far away then!

[opal hedge]

Last month it was supposed to be released this month, so take the timelines they gave with a grain of salt

[chilly tangle]

and we are still able to finetune the models like SDXL ourselves right?

Also by reading some articles I see they are releasing multiple models at the same time? ranging from 800M to 8B?

[opal hedge]

Yes, in 3-5 weeks the model will be released with open weights. I imagine they might give out more invites before then, but so no one knows. And yeah, they're allegedly released 3 model sizes (800m, 2b?, 8b)

The biggest one will need around 12-16gb vram to run comfortably

[chilly tangle]

👍 Thanks for the updates man

[oak patrol]

Anaconda + conda envs seems to be working well for me. What SD API are you using?

[low moon]

https://www.youtube.com/watch?v=QdRP9pO89MY

Now I understand everything.

[wet roost]

Whats the best online paid service for stable diffusion?

[low moon]

IF SD is open source how come there is paid versions?

[wet roost]

well My gpu cant support stable diffusion

so I buy service that run it

[low moon]

ahh you mean the GPUs

man

i suggets you save up for a GPU instead

that stuff adds up

[sand flax]

@regal glacier were you the one who talked with me about ai lineart earlier?

[static cape]

So it's been weeks... and still no SD3 API access for Beta testing? Come on guys...

[sand flax]

NGL the devs are being a bit sus rn

[wild stirrup]

hi im looking for nice loras of chicago bulls or lakers do you have a good one?

[opal hedge]

I have a feeling they're censoring SD3 to hell

[charred mesa]

nuh uh

[opal hedge]

"She's showing skin! Quick, put in a burka!"

If it's gonna be a SD2.1 situation it'll be sadge

[past wing]

Hey everyone,

As an AI student, I understand the struggle of learning without a powerful GPU. Recently, I acquired a new PC with high-end specs and want to share its GPU with fellow learners during my spare time, completely free of charge.

I'm reaching out to brainstorm ideas on how to best facilitate this sharing. Some initial ideas include setting up remote access and implementing a reservation system.

If you're interested in accessing the GPU or have suggestions for this initiative, please DM me. Let's collaborate to make AI learning more accessible to all!

Thanks for being part of AI community, and I appreciate your input!

[opal hedge]

Is this a computer screen share scam

[past wing]

What type of scam I could be Bro?

Help me understand, what do you want to say...

[opal hedge]

"Hey bro, just put in this remote access code while I install spyware on your computer"

The oldest trick in the book

[past wing]

Well I'll not be accessing your machine buddy, Infact I want to safeguard myself because I'll be giving access of my machine 🤣 that's the opposite

[arctic mesa]

i think he means the other way around... has a new piece of high tech and don't know what to do with it ^^

[wet dawn]

gm

[past wing]

I've asked community to tell how those can not affect me by installing that malware in my machine

[past wing]

Exactly But I Know What I'm Doing With It 🤣

[opal hedge]

You can set up a private cloud server

[rugged mirage]

just running a slightly locked down a111/comfy within a VM and sharing the url is probably the easiest

[charred mesa]

its only as censored as SDXL, it's FAAAR far away from SD2.X and Cascade

[opal hedge]

Do we know this for sure?

[past wing]

I've Tried Ngrok To Share ComfyUI But It's Paid, Do You Any Free Alternate?

[charred mesa]

well if we are both just judging off of images as we always do as oblivious community members

then it would be obvious for us that it's not lobotomized

[opal hedge]

I really hope you're right.

Or else training these models will prove very difficult

[past wing]

And How Can I Lock It Down?
Sharing ComfyUI and A1111 gradio url is the best option I think

[rugged mirage]

Id lock it down by starting a VM, giving the VM GPU access and then even if someone does something they can only break the VM which you can reset daily

I think you can just chose a port with Comfy and use your external url, or that there's some gradio setting for external url or alternatively you can set up some other local proxy thing but not sure what's easiest

[turbid bay]

This is a really complicated idea if you're looking to give people full control of your GPU.

You don't really want strangers being able to use your Internet connection either

If you just want to donate your GPU for generative AI, maybe looking into the AI Horde project is a good idea.

https://stablehorde.net/

[arctic mesa]

rent a public server.. put gradio on there and make it send to the computer?

[regal glacier]

Yes Leo
I can't find proper tutorial for that

[oblique jay]

Generating images through Google Colab is not allowed right?

[twilit solar]

SD3 will remove NSFW?

[charred mesa]

Yeah the base model probably won't be that good at nsfw.

If they managed to train nsfw back into SDXL then they can do it to SD3

[fervent thunder]

do anyone know where i can sell ai arts

[trim magnet]

brooklyn,near the bridge

[rough sail]

Hopefully it will be very easy to train NSFW into SD3.

[teal pagoda]

tried that already, the market is oversaturated no matter how good your AI art is. If you don't have blackhat masterminds in your team for marketing/SEO, you have not even a single chance

[arctic mesa]

i hold my excitement till this is resolved #💬｜general-chat message

[teal pagoda]

nah, that's kinda right. Heard that they implemented this thing for the base XL too or kinda.

[bleak matrix]

Good morning, everyone! How is everyone doing?

[elder marsh]

Doing great! How about yourself? Having fun turning AI images into real products today. What are you up to?

[bleak matrix]

That's awesome! Sounds like a lot of fun--I'm certainly interested in the AI to real world pipeline myself. What are you doing, specifically?

I was actually just doing some sketching w/pencil!

[elder marsh]

Miss sketching so much, wish I had more time for it!!

[elder marsh]

Thanks so much for the kind words! Specifically, I am working with a company that takes a concept of a product (clothes, home goods, etc) and if enough people like the image then we turn it into a real product that people can buy 🙂

[timid crypt]

hello

[bleak matrix]

I just took about five minutes to do it, maybe ten! I rarely do it, but I found an old book that has a bunch of blank pages, so I figured it was time to fill that thing out.

It was waiting for me long enough!

[bleak matrix]

Not a problem! And that sounds like TONS of fun! I really love concept designs--it's totally my jive. I was just designing a dress. Fashion is one of those things I really like to get into, along with general designs. What kind of products do you typically make?

[sweet token]

Did anyone get into the beta testing of SD3 yet?, because I signed up on day one and still hasn't received any email

[charred mesa]

mostly researchers and partners

and possibly people who have a paid subscription for Stability got access for the Stable Assistant

which they can generate SD3 images with

[sweet token]

Oh, thanks anyways for the information tho

[karmic cedar]

Still the biggest red flag for me. Small, limited testing pools give me less hope that it will be a fully functioning open source product once it’s released.

[charred mesa]

how many times do we have to tell that the CTO said that there WILL be an open release

its getting on my nerves at this point

[karmic cedar]

Well, I don’t want my anxiety to get on anyone’s nerves. I suppose I’m just concerned that what does get released ends up not being the product that was pitched. But we’ll see!

[charred mesa]

I understand, SD3 is a huge thing, and with the very slim possibility of it not coming out is quite grim

if I we don't get SD3 we would have to cope with Lavi-Bridge and ELLA

[karmic cedar]

It would slow things down. Emad leaving prior to SD3 is also a surprise to me, considering that he already made a big deal about it being the last T2I model, etc—why not wait until your final T2I capstone model is complete?

[charred mesa]

yeah idk :(

but I don't think the CTO is lying, we'll get SD3, even if support for it may go radio silence afterwords

if that actually happens, people like comfy will still do a lot contributions

and there is absolutely ZERO chance that SD3 will not get leaked if all goes south

[karmic cedar]

🙂 I’m projecting fear that what we get will not be anything close to what was tested internally, and that evidence will emerge that the model is “different”. And then it will slowly be pieced together that Emad, etc. left because the scant group of investors holding them together are “feeling abstract pressure not to release what is essentially a finessed zero-shot professional marketing tool”, etc.

[charred mesa]

the theory does make sense

bunch of services have arised from Stability

[karmic cedar]

i just know that marketers have lots of money, and that people need lots of money these days, etc.

so do politicians, etc.

[charred mesa]

I personally don't mind them and I hope that Stability will get some money back for all the GPU and research labour they have endured

[karmic cedar]

for sure.

i hope that too

[charred mesa]

if you gonna profit from the models they spent millions on, you might as well give them $20

and for the rest of us, who use it for fun or for personal use it won't even matter to us

[karmic cedar]

to the chat—I’m sorry for being overly vocal about my opinions about SD. I’m grateful that we have this resource, and maybe we are a little too lucky as it is considering how much power these tools use. thanks for understanding

[vague pond]

most likely it will just be that you will be disappointed at how slow the 8B model is when it's not running on an H100 cluster and that the smaller models don't perform as well as it, which is honestly one of the least bad outcomes

[karmic cedar]

I’m okay with that.

[vague pond]

then again we won't really know until we have it to try locally, could be that pure transformers == flash attention go brr

[karmic cedar]

I try to plan out as much as I can ahead of clicking the ‘Queue’ button.

[vague pond]

and token merging/ToDo go brr

[karmic cedar]

I don’t diffuse to think, I think to diffuse

nice nice

[arctic mesa]

#💬｜general-chat message

[rugged mirage]

Id take higher quality while being slower any day, we have plenty of options for sacrificing quality for speed when you need them

[vague pond]

try finetuning the 8B model then tell me if you've changed your mind lol

[karmic cedar]

i see opportunities for people with more power to step in and say, “okay, you guys are doing this kind of work at this kind of speed, this kind of quality…let’s try to anchor your productivity right here while we allow other select parties to go further, all based on hardware availability, weight privatization, etc.”

[rugged mirage]

I dont need to do finetunes myself (can mooch off others) and surely a Lora wont take more than a week at very worst

[vague pond]

SDXL is slow enough to finetune which is largely why it took so long for it to get more widely adopted, and why there's still a lot less tooling for SDXL. the only saving grace for SD3 here is that it's pure transformers and you can at least easily shard it if you have multiple GPUs

[rugged mirage]

I once spent over a month finetuning gpt-2 on free (then it was only free) Google Colab, I can take waiting for quality

[karmic cedar]

that is dedication!

[rugged mirage]

yeh I had to start it and save it to drive manually multiple times a day (since it had auto timeouts to prevent you doing that) among a bunch of other things

[karmic cedar]

right…wow

how were the results? were you happy given all the time you dumped into it?

[rugged mirage]

I found it a lot on fun once I was done, I was finetuning it on my own chats so I can talk to myself which was a pretty cool experiment https://svilentodorov.xyz/blog/gpt-15b-chat-finetune/

Ive been meaning to do it with newer models but you cant really finetune the biggest models anymore

[karmic cedar]

i did a GPT 3.5 finetune based on my grandpa’s editorial rants. He had about a thousand

[rugged mirage]

and it doesnt seem as worth it with just llama or whatever

[karmic cedar]

yeah

[vague pond]

i think that SD3's 8B model could be trained on a TPUv3/v4 if you shard it for full model parallelism and also manually implement ZeRO.

you have 128GB across all of the cores and they have decent all-reduce bandwidth, 32GB for the weights, 32GB for the gradient or fuse it... actually i'm incorrect that you'd have to implement zero because you only have one model copy here lol

[rugged mirage]

I mean the biggest hard limiter on finetunes is vram and TPUv3 can access up to like 300gb so it should definitely be possible

[vague pond]

no the tpu machines have 300gb of sysram, which is very nice, but they have 128GB of HBM on one node and 16 per core

oh yeah, you'll probably also have to make sure flash attention v2 is implemented properly. isn't the tpu research program fun?

[rugged mirage]

ah yeah

[karmic cedar]

so much potential for efficiency improvements with AI tho

it’ll just keep going and going

can’t wait to see the surprises

like maybe one day there will be an entirely new encoding scheme that will somehow be the equivalent to ZIP compression lol

[vague pond]

at least JAX has one thing built in that I am convinced Torch will never have.
a jit compiler that works reliably

[charred mesa]

well an unoptimized version of SD3 takes 34 seconds to generate an image of resolution 1024x1024 when using 50 sampling steps on a RTX 4090

and as you mentioned, we don't know about xformers (flash attention) being used or not and other optimizations

plus we have SD3 Turbo, which still has superior prompt coherence compared to midjourney V6

[vague pond]

the model would almost certainly be running out of memory without it

[charred mesa]

makes sense

[rugged mirage]

ok yeah that is pretty slow

[charred mesa]

yeah

and I want to know if we'll need 50 steps even

most of the models we use only need lik 20-25

idk what samplers or schedulers they have

[final juniper]

hey so my automatic1111 doesn't really listen to models, its like it ignores the model I set and prompts aren't always listed on the civitai pages

[charred mesa]

cause its a different architecture (DiT)

[rugged mirage]

distlled lightning models type stuff have been good to get it to much less steps

[charred mesa]

yeah SD3 Turbo is exactly that

its not a 512px faint model, its more like SDXL Lightning

[vague pond]

also iirc each step is like, 6 model forward passes (remember that this is rectified flow and is very different from diffusion)

[charred mesa]

full res, low step, high contrast model

damn

whaat

can you tell me the implication behind that

[karmic cedar]

i wonder how soon we’ll be talking about brain scans of people and analyses of their temporal-occipital regions, “oh yeah your imagination diffuses at about X amount of human steps, versus this guy over here who’s a painter and has rapid temporally coherent abilities, etc.”

[charred mesa]

lol

[final juniper]

anyone knows the answer?

[charred mesa]

try #🤝｜tech-support

I have no clue

[rugged mirage]

if you could live brain scan at that fidelity you'd be pretty close to just prompting for a person

[karmic cedar]

we are getting close 🙂

MRI scans have been proven useful to train diffusion models with.

[charred mesa]

prompt a person and 3D scan it in a week

[karmic cedar]

we can do human vid2vid

but it’s not semantically as precise—it’s really close and it’ll work for any input video on any pair of eyes sitting in an MRI at the time

😄

[vague pond]

reading over the paper again, i suspect the 2B model will be the one people are likely to develop tooling for since it has that editor model

[charred mesa]

There's also the Turbo Edit model

[rugged mirage]

Ive been hoping that since the architecture is mostly the same it'll be easy for the tooling to work with any of the models

[karmic cedar]

we are toolmakers by nature, damnit

give us our toolmaking abilities

[charred mesa]

SD3-edit Turbo and SD3-inpainting Turbo

in the Turbo paper

https://arxiv.org/pdf/2403.12015.pdf

[karmic cedar]

SD3 inpainting is going to be sooooo much fun.

[charred mesa]

exactly

[karmic cedar]

can’t wait….i hope optimizations happen, tho 😦

[charred mesa]

well there's comfy's mem management by default, which will allow us to run 8B on around 12GB

[karmic cedar]

i would love to be able to afford my own rig for this stuff, but i’m on hard times. gotta keep my chin up and be thankful at the same time for cloud compute

[charred mesa]

but for speed, idk anything else beside token merging and xformers (which will be there by default)

[vague pond]

i... don't really trust the idea of finetuning a distilled model

[charred mesa]

and of course Turbo

[charred mesa]

DreamShaperXL Lightning is pretty good

though idk the method behind that compared to LADD

[astral goblet]

hand axes are a pretty awesome tool in the anthropology story. for thousands of years, homo erectus would use these things. decorate them. bury with them. trade them.

[karmic cedar]

DreamShaperXL Lightning is also good for rapid SUPIR upscaling.

[charred mesa]

yessir

[astral goblet]

a lot of thought was put into making the perfect hand axe

[karmic cedar]

and by outsourcing the task to an sharper point of an axe, we had thus increased efficiency.

also—is it heidelbergensis that started axes, or erectus?

[astral goblet]

the handle would've been a good lever. wasn't figured out for a long while

[karmic cedar]

thx stone age!

there is a certain perspective that suggests human innovation has directly correlated with attention span.

over the course of millennia

[astral goblet]

lots of different types of homos until we went all sapien

[karmic cedar]

right?

it really was more like middle earth than people realize.

[astral goblet]

hard to say who was first. the historic record as we know it is probably so crossed over with trade and commerce

[karmic cedar]

definitely knowledge transference. you don’t see evidence of technology refinement until you see evidence of cross-cultural migration.

[astral goblet]

when the aliens landed

[karmic cedar]

lol

i do wish that to be true.

[vague pond]

not really sure how they made that but i suspect they just did the distillation themselves there? the "most efficient" way to do this that I could think of (having no real experience with model distillation) would be to essentially do a finetune on SD3-Base and then start distillation training with your finetuned SD3-Base as the teacher and the pretrained SD3-Turbo as the student model

[astral goblet]

lol history channel says it is ! why would it be the literal history channel if it wasn't real?

[charred mesa]

hmm

[astral goblet]

honestly though, we need tools

[karmic cedar]

oh history channel

yes we do

[vague pond]

maybe you could save some time by merging your tune with SD3 Turbo assuming the weights are the same shape (which they may not be, for example SDXL's refiner has a lot more channels in the shallower blocks of the unet)

[charred mesa]

could have been, I don't see anything in the description

Lykon also mentioned partial distillation for higher steps

[vague pond]

i just see trained from sdxl base and not from sdxl turbo which tells me that they had to do actual distillation

[charred mesa]

idk how slow it is to distil models

I wonder if they will also provide tools for that as well

[vague pond]

tools? i just make a bunch of sloppy edits to diffusers example training scripts.

[charred mesa]

its just that Emad said that they will provide finetuning tools and controlnets and etc at launch

[vague pond]

well yeah but almost nobody is going to actually use those except as reference to implement it into diffusers

[astral goblet]

what happens when training is all blockchain apps?

[charred mesa]

lmao

[rugged mirage]

oh even controlnets? nice

[vague pond]

will i be able to buy drugs with my finetuning efforts?

[charred mesa]

yes we will have controlnets at launch

but it's unknown how many and how good they will be

[astral goblet]

the age of the silk road is over. too short lasted imo

[charred mesa]

probably way better than cascade

SD3 is gonna be a massive release

[rugged mirage]

that's great, I kind of fell off after sdxl launch since we didnt have controlnets and other tools for too long after it

[charred mesa]

yeah

SDXL controlnets are kinda bad sadly

[vague pond]

well if i can't buy drugs or hitmen or military hardware then what's the point of blockchain

[charred mesa]

only depth worked okay

[astral goblet]

i dont think it'll land that hard. it'll be awesome but not cause a splash. will probably be recieved like sd2 was.

[astral goblet]

canny and depth in my experience

[charred mesa]

if it's gonna have depth and canny then I'll be fine with that

[silent fern]

Alguien sabe español?

[charred mesa]

and because of the censorship, possibly like SD2?

It's not lobotomized like SD2.X and Cascade though

I do expect ken barbie dolls when asked for naked people lol

[astral goblet]

there'll be many of us who recognize the awesome power of sd3 and use it to great extent, and then the majority of the audience will be rabble rousing about censorship and needing donations to train new community models

[karmic cedar]

if they provide folks with the right tools, folks will find ways to stay productive in the ways that our instincts want us to be. 🙂

[charred mesa]

interestingly male nipples seem to work, which SD2 had problems with

[karmic cedar]

rotfl

[charred mesa]

yeah no joke

[vague pond]

why do people worry about model censorship... it isn't difficult at all to add stuff back. you literally have the weights and training code

[astral goblet]

most of the prompting difficulties of sd2 came from people not understanding openclip's vernacular

[rugged mirage]

tbf the bigger the model the harder it is to add stuff back in

[karmic cedar]

true.

[charred mesa]

yeah people are blowing it out of proportion

[rugged mirage]

and sometimes when you add stuff back in you are finetuning with a smaller dataset and erase stuff etc

[astral goblet]

it causes a lot of youtube clicks and donations filling collection pans

[vague pond]

people also did NSFW finetunes of SD2, they worked fine, arguably better than SD1.5 even, but there was no reason to move to SD2 because it really just didn't do anything notable

[astral goblet]

the church of pornography have devout followers

[rugged mirage]

you can see on civitai how most models are really good at the same stuff and more mediocre at the same other stuff

[karmic cedar]

yes

[charred mesa]

(My theory is that Stability mentioned safety so much to calm down the public about such a strong model)

I don't think its more censored than SDXL

[karmic cedar]

there is some shit up on that civvy

[charred mesa]

and SDXL has nude loras and PonyXL or whatever

I saw a heavily upvoted comment on reddit that PonyXL revitalized AI corn lmao

[karmic cedar]

AI corn is tough man

there’s a lot of surprising fractal complexity in the corn cob.

[charred mesa]

exactly

[astral goblet]

more than nude loras. friend of mine was showing off certain loras he found of explicit nature. he wanted help making less deformed looking images and i was like "bruh... no"

[charred mesa]

🌽 ambatukan 👨🏿‍🦱

[karmic cedar]

there is some serious fractal complexity in nips tho

them nips is fractals

[vague pond]

people have also found plenty to whine about with censorship on ponyxl too lol

[charred mesa]

lmao

[dry trellis]

Once we can accurately portray Will Smith eating corn we've achieved AGI

[astral goblet]

the golden ratio

[karmic cedar]

but can we have a corn of nips

?????????

[astral goblet]

nips on the cob?

[karmic cedar]

yes

[astral goblet]

nomnomnomnomnomnom

[karmic cedar]

wow that is some nightmare fuel

[vague pond]

i do fully respect astraliteheart's decision to respond to criticism of the decision to remove artist tags by 1) not backing down on the decision, and 2) relentlessly trolling people over it

[karmic cedar]

sometimes you gotta roll the hard six

https://tenor.com/view/so-sayweall-battlestart-galactica-william-adama-bsg-twelve-colonies-gif-18616973

[astral goblet]

what the fuck. i accidentally clicked on notepad's edit menu, and it has "explain with copilot" in notepad... the future is now

[karmic cedar]

yeah when is microsoft going to bring Clippy back? like with ray-bans and a nice billion dollar suit and only refers to itself in the third person

“sup f**kheads, guess who was smarter after alllllll”

[charred mesa]

based ig

[astral goblet]

i respect the decision too. there are many good reasons for doing a dataset culling. ethical reasons are up there.

but there's a very loud minority of people who demand that datasets are never censored anyway for any reason. they have very dishonest disingenuous debates over it. it's more of trump style politics where they won't budge from their position at all. some masculine notion of being weak if you admit you were slightly wrong once or ever.

[charred mesa]

oh yeah SD3 has artist opt outs too

[astral goblet]

i love to point out to people that the most popular corn model is censored

[charred mesa]

so I wonder how long (1 day) will it take to retrain greg rutkowski

[karmic cedar]

RUT-ROH!

[astral goblet]

in sd1.5, rutkowski isn't even in the dataset very much. its not like it learned the complete works of his. the clip model though happens to correlate his name to that style of painting very well.

[charred mesa]

heh

[vague pond]

alright, give them a model trained on raw common crawl images, see how they like it

[charred mesa]

I loved it when suddenly in SD2, greg rutkowski wasn't the magic prompt anymore

[vague pond]

massive uncensored dataset is good, right?

[charred mesa]

ehh

[karmic cedar]

this model better be blue cheese and not swiss

[astral goblet]

absolutely lol. this is what i try to get across to people but then you know.. they hate censorship so much they're blind to any actual points

[vague pond]

they will get to know <em>Bloodborne</em> Video: Sony Explains the Game's Procedurally Generated Dungeons

[charred mesa]

but they wan't anime booba, uncensored = better anime booba guys!

this is absolute 100% facts confirmed by SD1.5 users

[karmic cedar]

nah i take that back

[astral goblet]

people love to be part of something bigger than themselves. size envy you know?

[karmic cedar]

yep.

[charred mesa]

unrelated, but Ideogram keeps looking like 2.1 man, I wonder if it's just a generic Unet with v-prediction and a heavily captioned dataset + T5

at points I think that SD3 2B finetune could easily beat Ideogram

[astral goblet]

#🏞｜general-with-images message 2.1 man nipples. openclip is just harder to prompt on

[charred mesa]

illuminati model had nipples

[astral goblet]

100% i expect the people to have t5 loaded and throw 1.5 style prompts at it

[charred mesa]

YES

I will be so mad if I see people prompting SD3 like 1.5

[astral goblet]

i don't ever even see openclip style prompting on most sdxl generations

[charred mesa]

1girl, big massive honkers, cinematic, illustration, red hair

"WHY IT NOT WORK??!?!?!?"

[vague pond]

SD3 will basically have to be "pretend you're CogVLM" for prompting, I assume

[charred mesa]

^

SuperPrompt-V1 will help A LOT

[astral goblet]

yeah you caught me. i cheated and used illuminati. but i mean, i woudln't use base 1.4 or 1.5 either

[charred mesa]

its a 77M T5 model which extends your short 1.5-type (or generic) prompts into massive natural language prompts

it runs on the CPU and its super fast

[vague pond]

That comes later after people have had an opportunity to destroy the text encoder. At least T5 will be intact because nobody is gonna be able to unfreeze that lol

[charred mesa]

hehehe

[astral goblet]

i'm gonna throw passages from my favorite novels behind different prompts as flavor text

[charred mesa]

I bet some people are gonna throw away the T5 encoder like how we threw away the SDXL Refiner

[astral goblet]

so much ginger infatuation in the community . i dont get it.

[astral goblet]

lol! one of the first things i started experimenting with in sdxl was "how needed is the refiner?"

[charred mesa]

Input:

photo of cat, in church
Output:
photo captures a cozy living room decorated with plush sofas and armchairs. In the center of the frame, a sleek black cat sits upright, gazing up at the sky. The cat's fur is painted in shades of orange and brown, and its eyes seem to sparkle with intelligence and curiosity. The church is filled with the sound of chatter and laughter as people gather around the scene.

[astral goblet]

people obsess over it and i never found it to improve detail much at all

[vague pond]

Refiner models are great, but after creating my own for a 1.5 model I found that as far as I can tell literally no frontend actually has a robust implementation for the refiner except for the one I pull requested into A1111

[astral goblet]

debates raging in here about what noise levels to pass to the refiner and what specific settings the nodes needed for the best detail. meanwhile i got better results with better prompts and higher step counts. i guess i just didn't get it

[vague pond]

It's not even that much of a problem to train one, just train the refiner model on the same dataset you had and it should take 1/5th of the time that training the main model took.

[astral goblet]

double the effort for very little return on quality, if any at all

[vague pond]

No, it's an extra 20% if we mean SDXL. And at least with my experiments on 1.5, I've found that training a refiner does very noticeably reduce noise in final outputs. And it doesn't cost anything extra at inference time, at most it's just the cost of switching out the models if you can't fit them both in vram.

[forest trout]

Have any of you gotten to try SD3 yet? I haven't been invited yet.

[vague pond]

If you're just doing a refiner from a pretrained model, mine seemed to be converged okay after about 15000 steps (virtual batch size 64) and I let it run for 80000. And this is honestly a small fraction of what that model's total training over sd1.5 base was.

[astral goblet]

my inbox is still invitationless

[forest trout]

I have a hunch they have really been restricting the invitations to a pretty small group.

[astral goblet]

yeah the open sign up was a sham

[vague pond]

Yeah it's a small group for now, I know people who have access but are under nda and can't share images. They're still doing DPO on the model, and at least the main person I've been hearing from claims that it's still not as good as dall-e 3, which might be down to using CogVLM rather than the in house CoCa that OpenAI used. Or down to using T5+CLIP versus pure T5. And that it sometimes has trouble combining things seamlessly, which is likely an artifact of rectified flow. Then again, they're still training the model so there is time for this to change, and also this is all locked behind an API at this point anyways so who the hell even knows which model they're using? But the model is still leagues ahead of any previous SD model regardless.

They probably want to do more DPO and get a round of actual release candidate models together before doing wider invites.

[astral goblet]

i'm fine with it being "not as good" as dall-e. the goal is wide compatibilty, not exclusive datacenter execution

[teal pagoda]

Man, that was the biggest lie that SD3 will be better than Dall-E 3

[vague pond]

I'm also fine with that, since the architecture for SD3 is clearly solid and very capable and it can be brought up to its level with finetuning

[teal pagoda]

Even if it actually could have done that, but whatever

[forest trout]

Maybe not better at one shots, but you can do way more with sd models.

[vague pond]

It does appear to be better in some domains at least, namely the text meme

[astral goblet]

i'm wondering how loras will work. if the 800m parameter model will use the same loras/doras as the 8b model. or if we'll need to make separate versions for each size of sd3

[teal pagoda]

idc about the text, I want the images to be better, not blurry and to actually generate what I prompt

[vague pond]

Oh yeah, dalle 3 is permanently less capable on text because of using the same VAE as SD1.5 even aside from the rectified flow stuff

[astral goblet]

text is one of those milestones. maybe i'll care more about it once the models i use can do it. i don't think it's the best measure of a model's capabilities though

[vague pond]

(I also really need to get around to doing more testing on the SD1.5/Compvis VAE artifact...)

[forest trout]

Does anyone know what the optimal dimensions SD3 is trained on?

1024 still?

[charred mesa]

yes

[astral goblet]

its transformers now so as i understand it, the base resolution of the model is less important.

[charred mesa]

hmm

[astral goblet]

should have less attention problems at higher resolutions now

[charred mesa]

interesting

[forest trout]

I mean usually people can just do tile diffusion for large images...

[charred mesa]

just gonna do good old highresfix

[forest trout]

But good to know resolution issues have probably been resolved.

[astral goblet]

tiling is good for memory saving. if you got the memory though just go full bore with a hires fix. kohya's is neat to use too

[forest trout]

I mean if you are doing 5,000x5,000 you kinda need to use tiles.

Speaking of pushing stuff to its limits, Harmonai has been showing off a 4.5 minute in house research model.

[charred mesa]

man I hope SD3 Turbo will suffice

the images in the paper just look amazing and complex even for a turbo model

[pseudo coyote]

Hello, is there any tool in stable diffusion to achieve similar results like in Remini ai enhancer? Thanks

[oblique elk]

Would say you could use controlnet with the Tile model

[pseudo coyote]

What is that, could you please explain

[astral goblet]

which part? controlnet or tile model?

[pseudo coyote]

Both 🙂

[astral goblet]

safe to assume you've never even heard of stable diffusion before today?

[pseudo coyote]

I've used it only to generate images

But haven't used control net nor tile model

[oblique elk]

Well you can use Controlnet to control more of the image generation of stable diffusion. The tile models add more details. So you start with a low res image, use a upscaler (it gets blurry and has less details) and then you create a new image based on the same prompt and with the tile controlnet model

[trail shadow]

Hi all readers, i wish s tutorial for make a sexy influencet

[mortal timber]

Hey guys, my supervisor just got a research grant to buy GPUs for my lab for diffusion/RL projects. I was wondering if you have any recommendations regarding which GPUs to get?

[pseudo coyote]

My aim is to have existing images to face enhance it just like Remini and make its texture similar

Perhaps using any LoRA

I could sent you an example

[oblique elk]

if it is a open example image (not private) you could simply post it in #🏞｜general-with-images

[astral goblet]

hmm. how'd you get a research grant without knowing what you need?

government. amiright?

[oblique elk]

And a recommendations depend on the goal and research targets.

[mortal timber]

Yeah my prof just mentioned NVIDIA GPUs. We have a few planned projects that involve fine-tuning stable diffusion

[astral goblet]

professor? so not just a supervisor. Your professor probably assigned you the task so you could learn, not farm free answers from a chatroom. Don't cheat yourself out of the education you paid for

I'd suggest go big with enterprise hardware, but i don't know your scope or goals.

[astral goblet]

new face swap thing out today. arc2face

https://github.com/foivospar/Arc2Face

[charred mesa]

it has an example with Ryan Gosling in it

excellent

https://huggingface.co/spaces/FoivosPar/Arc2Face

[astral goblet]

sean mr universe connery too

[charred mesa]

here's the demo for it

[astral goblet]

https://arc2face.github.io/

[fervent thunder]

its pretty good

chili rice chooser

[charred mesa]

I decided that I'm going to get a 3090

I wonder if 24GB will be enough for Loras

[astral goblet]

i have a 4080 and play with lora training all the time

16gb

[charred mesa]

I mean for SD3

16GB is enough cause I trained with Colab as well

[astral goblet]

should be

[charred mesa]

loras will be so powerful for SD3

I wonder how long we'll have to wait for Lora implementations for SD3

if that comes at launch I'd be surprised

[astral goblet]

i struggle a bit to train 1024 loras , batch size of 1. so i often do 896 as a max resolution and train with batch size of 2.

doesn't seem to affect quality in my testing

[charred mesa]

I wonder if QLoras would work 🤔

[astral goblet]

aren't doras more memory efficient to train? i haven't played with any of that yet either. i just heard a bird say that

[charred mesa]

no idea

[dusk canopy]

Taks 15 hours to train. Alora

[charred mesa]

maybe if we get IPAdapters for SD3 it might suffice

[astral goblet]

i often train loras in 30-40min

[charred mesa]

an 8B might have MORE to squeeze styles out of

[astral goblet]

if you are using shahred memory then a lot longer

my standard practice for datasets is around 20-50 images. i do 10 repeats and 10 epochs with aggressive train rates. 0.2 network drop out. 32 rank. i practice a lot with instagram profiles. use gallery-dl to scrape high resolution images from someones gram and go from there

[charred mesa]

and of course, the question remains, will Loras or TIs work across model sizes

[astral goblet]

training loras in fp8 might be the option

[charred mesa]

well for 40 series I guess

we already have AdamW or whatever which does these in int8

I don't exactly know

[astral goblet]

30 series can do fp8 stuff too i'm pretty sure. its just in software so its slightly slower to do that initial casting

i think adam 8bit uses floating points too. not int

[charred mesa]

yeah that's why

I mean if it's not a massive difference then sure

[astral goblet]

it would just affect speed at load time i think

once the models' in memory you wouldn't need to cast it again

[fervent thunder]

cheesy decimal tolerance

[astral goblet]

i've always thought that "beast wars" cartoon missed a huge opportunity to have a bunch of decepticons called the "decimals"

https://www.youtube.com/watch?v=5_qKLZq6frg

[charred mesa]

ah great

[fervent thunder]

tronisinator

filled with rice

Hi guys, I am new here and yet to try out diffusion models, Guys is it possible to use diffusion models to turn a building blueprint sketch to a 3d render using SD

[tropic mason]

hey guys, I've installed the SuperMerger extension to my Stable Diffusion web ui but it doesn't show in the web-ui. What should i do? I use runpod secure cloud Stable Diffusion template. I hope anyone could help me. It seems like no extension that I install is showing in the tabs for me, what could I do wrong? I use the "install from URL" option

[buoyant marsh]

👀

[fervent thunder]

[elfin stump]

Do you have any idea what the budget is like? Big difference between $10k and $1000k. It isn't just more, you would get different HW.

[crude notch]

nope!

40xx cant do fp8 either (technically)

30xx has no operations for fp8

40xx has cuBLAS operations for fp8, it just gets casted from fp8->fp16 then done

there is no silicon on chip for pure fp8 compute afaik

either way sd3 isnt that hard to train

lora should be possible on 24gb vram

should

[karmic cedar]

Does anyone know off the top of their head if SD3 utilizes DMD? Apologizes if this has been addressed already

[crude notch]

what is DMD?

im pretty sure it doesnt

ohh the distilation thing

nope

[opal hedge]

Probably H100s in a year when big corps start replacing theirs with b200s

But most cutting edge university labs use H200s atm

[charred mesa]

hell yeah

I suppose not at like high epoch counts

if characters and styles don't look like spaghetti then fine by me

[opal hedge]

Epochs shouldn't really matter, batch does though

[charred mesa]

oh

[opal hedge]

epochs is just how long it trains, batch is how many images it processes at a time

[charred mesa]

I see..

[crude notch]

barely though

[charred mesa]

sadge

if that doesn't work then I hope IPAdapter works well enough for styles

[crude notch]

honestly? a100s / aXXXX series can do it

[charred mesa]

do you think that a massive 8B has more to squeeze out of when it comes to styles and stuff?

[crude notch]

for xl you might need a100

[crude notch]

current model can do anime, pixelart, photorealism, etc

so yeah

[charred mesa]

hmm yeah

[opal hedge]

By that point B200s will probably be more cost efficient because they have something crazy like 10-20x the compute

[charred mesa]

I just don't know if it can finally do Video games correctly

[crude notch]

(fp4/fp8)

fp8 and under suck for SD training

[charred mesa]

9/10 it just makes a generic shooter with PS2 graphics lol

even ideogram

we need more zero-shot stuff like IPAdapter

[opal hedge]

Honestly, the weights in SD should all be fp8

[crude notch]

fp8 storage is viable

fp8 training? nope

[opal hedge]

If there's anything we've learned from recent AI advancements is that large weights make almost no difference

[crude notch]

yeah generic ps2/ps3 shooter

[crude notch]

this is for SD only, llms can be quantized to sqrt(x)=3 bits

[charred mesa]

if IPAdapter can squeeze out more obscure knowledge out of SD3 8B like how Bigger LLMs can squeeze out more obscure knowledge then I have hope

idk what else is there that does something similar

[opal hedge]

isn't 1.58 bitnet a thing

for llms

[crude notch]

aaa i messed up the math

yeah the 1.58 bitnet

[opal hedge]

I hope someone makes a powerful llm with 1.58 soon, I think most 12-24G vram gpus might be able to run something almost as good as Claude Haiku

If the research is to be believed

[crude notch]

people should also try training other architectures, retnet could've been so good if they released weights

[opal hedge]

It's chinese research at a chinese university, which makes me doubt retnet is that good

It's not unheard of for them to spam research papers that are later proven to be fibbed or highly flawed

Which is why even alibaba is doing research on stable diffusion instead of making their own models

[mortal delta]

Is it just me or does the newest version of stable diffusion automatic1111 not show controlnet, like it just gone.

[crude notch]

ah yes, my favorite chinese university, microsoft

[charred mesa]

lol

[astral goblet]

i see a lot of hate for the research coming out of china, but it's always from people who have zero involvement in the academic side of things. weird. i wonder what it's really all about

hmmmmmmm 🤔

what could it possibly really be about?

[astral goblet]

no but you see, the first line of the paper shows they have very chinese looking names. you dont need to read past that first bit to know anything about it , right?

[mortal delta]

while your here, Just to let you know flowwolf i added linux mint along side windows and so far my pc hasnet crashed when using ai and rocm.

[astral goblet]

controlnet is an extension for auto1111 and needs to be added. forge it's built in

NICE you got rocm working! ! knew you could do it

[opal hedge]

Tsinghua and Peking University actually for retnet

bitnet was researched by microsoft asia

[mortal delta]

yeah so far i believe it works.

[mortal delta]

noted.

[astral goblet]

seriously though, this has to be one of the most ignorant statements i've seen about this field so far this year. and i thought the censorship zealots were going to take that trophy. theres still time yet so don't fret

[opal hedge]

https://www.voanews.com/a/chinese-government-cracks-down-on-academic-fraud/7523009.html
https://www.economist.com/china/2024/02/22/why-fake-research-is-rampant-in-china
https://www.fbi.gov/file-repository/china-risk-to-academia-2019.pdf

[astral goblet]

lol he has 3 links locked and loaded. on a mission

[opal hedge]

And if you'd like a link from china's own news:
https://www.scmp.com/news/china/science/article/3249928/most-important-battle-our-lives-11-chinese-university-students-overthrow-professor-accused-faking#:~:text=Chinese research accounted for more,cent in the year 2023.
"Chinese research accounted for more than a quarter of the articles retracted due to plagiarism, fake peer reviews and unreliable data, the report said. That figure rose to 75 per cent in the year 2023."

[astral goblet]

i suppose everything that comes out of cern is shit too because of that one time they said they clocked a photon going faster than c

[opal hedge]

You can complain all you want, but it's fact at this point

[astral goblet]

"it's fact" dude

[torn knoll]

!

[astral goblet]

its your very bigoted opinion

[torn knoll]

!

[opal hedge]

You provided no counterevidence, so you're just blowing smoke out of your bum

[torn knoll]

!

!

[astral goblet]

trumpists are gonna trump

[opal hedge]

I'm a leftist, what?

[astral goblet]

getting more bold lately as the election circus in america ramps up

[opal hedge]

I support biden, I don't see why I have anything to do with trump

Whatever, you're just having a meltdown, I'll see myself out

[astral goblet]

not a leftist with a whole lot of integrity tbh

american left is very right too so i mean, you're clowning

[opal hedge]

If I disagree with what you say and provide evidence, I don't have integrity? All you've said so far is "no, no no!" like a petulant child

[astral goblet]

bye felicia

[astral goblet]

yes. you don't have integrity

see i can say yes too

[opal hedge]

So give me some evidence as to why my evidence is wrong

I don't see what's the problem

[astral goblet]

its not my place to prove your positions. sorry. not sorry.

original claim was "resnet is fraud" but you just threw random fodder out as evidence. meh. unimpressive in all ways

[opal hedge]

https://www.scmp.com/news/people-culture/trending-china/article/3180214/chinas-universities-hit-new-academic-scandal
China's universities hit with new academic scandal after deputy dean stole work from 10 academics for dissertation
http://chinascope.org/archives/34067
https://www.nature.com/articles/d41586-020-02445-8
From nature.com btw, one of the top academic publisher's
https://www.ft.com/content/32440f74-7804-4637-a662-6cdc8f3fba86

Just look at any link

[astral goblet]

a lot of papers have been published in nature that have been wrong.

[opal hedge]

Yes, many of them chinese

That's the point

[frozen helm]

why not just make your own paper and stop complaining

[astral goblet]

yourfirst claim is "resnet is fraud" or "all chinese research is fraud" once you really defined it. meh

wholly unimpressed

[opal hedge]

I said, I have difficulty believing retnet because it's funded by chinese universities

I didn't say it's a fraud

[astral goblet]

theranos happened. what does that mean? all white women can't do business?

[opal hedge]

"It's chinese research at a chinese university, which makes me doubt retnet is that good"

[astral goblet]

you'll see what you want to. that's how racial biases work

[opal hedge]

Ah yes, the China race

[astral goblet]

oh yeah. race doesn't exist. right.

[frozen helm]

tf bro

[broken smelt]

meant to reply to this right after the message was sent but had to step away; this issue isnt specific to china but rather academic funding as a whole. youll know of the phrase publish or perish if youve ever done any work in academia. u either push out a bunch of junk or u dont get any funding whatsoever

[astral goblet]

or left?

[astral goblet]

the actual crux of the problem. it pervades the ENTIRE acadaemic field. not just china

[opal hedge]

The problem is large in China, but that doesn't mean that it isn't everywhere

[broken smelt]

right so im not sure what the fixation is against chinese academic papers if its a fundamental problem across the globe (it is)

[opal hedge]

This is from the South China Post, the CCP's own propaganda outlet via Alibaba.
"Chinese research accounted for more than a quarter of the articles retracted due to plagiarism, fake peer reviews and unreliable data, the report said. That figure rose to 75 per cent in the year 2023."

[astral goblet]

i know where the fixation comes from. same fixation that lead people to blame the baltimore bridge incident on the chinese in the first 5min

[frozen helm]

you already yapped that info

[astral goblet]

trump's election cycle

[opal hedge]

So 75% of articles retracted due to plagiarism fake peer reviews and unreliable data came from China in 2023

[astral goblet]

it has "them" emboldened

[opal hedge]

Are you admitting I'm right then?

Or is China lying about China's own problem

[frozen helm]

No I’m saying you already said that

Scroll up

[opal hedge]

Okay, did the information just pass through the porous membrane that is your brain?

[astral goblet]

i imagine if regulators started looking at papers in america, you'd see similar results. but enforcement isn't done as much.

theranos happens instead

[frozen helm]

my brain is hard

[opal hedge]

I understand that false research is a problem everywhere, but it's especially big in China

Argument over

[astral goblet]

i wasn't having an argument really. just pointing out fax

[cloud plume]

one still has to ask oneself whether that is a big number. I.e. if 99% of published papers are from china then 75% is below average. (not saying that's the case, this is hypothetical)

[astral goblet]

tru fax

[opal hedge]

In 2022, there were 159 highly influential journals globally covering 178 disciplines. And last year, China contributed 16,349 papers to these journals, accounting for 30.3 percent of the global total, exceeding the US for the first time.

So 30% in 2022

[astral goblet]

resnet code is released too. while not the model, is a BIG THING for researchers to use still

and while the weights aren't publically released, they exist and access is provided to those who won't just disregard them because chinese names on the paper

[opal hedge]

If the number of papers percentage wise is the same in 2023, 75% of 30% of papers means 22.5% of the world's research is falsified (and came from China)

I feel even more right than I did early on now that I looked into it further

[astral goblet]

lets dial it back to what sparked this whole "jyna is fraud" argument. The claim that resnet is a fraud since it comes from jyna

lets not pretend that nuance was being practiced

[frozen helm]

‘gina

[cloud plume]

no this math doesn't check out

[astral goblet]

of course you would. entrenched opinions rarely dig themselves out

[opal hedge]

Oh, yeah? How so

[frozen helm]

none of it does.

[opal hedge]

Looking forward to an actual counterpoint for once

[astral goblet]

plenty were made that you've just hand waved and ignored

[opal hedge]

You didn't make any points, you had a whatabboutism moment and then you said China was a race that I am racist against

[astral goblet]

¯_(ツ)_/¯

i'm not sure why i need to debate in good faith on this topic. it's not a good faith discussion to start with

[opal hedge]

YOu can't debate it and you won't because you know you're wrong

[astral goblet]

am i though?

[opal hedge]

All the evidence supports me and you provided none to the contrary

[astral goblet]

fun to watch bigots struggle

[opal hedge]

I am hoping that maybe alex actually gives a good counterpoint

[cloud plume]

you assume that 75% of all papers from china are bad. Actually it's that 75% of all bad papers were written in china. This is different.

[opal hedge]

Explain the distinction

[forest trout]

When it comes to research papers, more junk papers doesn't ruin or diminish the number of good quality papers. This seems like a non issue.

You aren't gonna see scam articles make their way into cell magazine so what's the concern?

[opal hedge]

My initial comment was the incidence of many junk papers from China, 75% of all produced in 2023, leads me to doubt the validity of future papers from China

I don't think that's an irrational point

[cloud plume]

Assume worldwide a million papers get published every year(china 300000). And assume 100 of these are bad. Then with your numbers china would have written 75 of them.

So while china here produced above average wrong papers. the percentage is still much smaller that 75% of all papers

note: all my numbers in my example are made up

[opal hedge]

Yes, I'm trying to find the real numbers myself

[cloud plume]

i guess the important number is what the percentage of wrong papers overall is. But if it is small then it's still mostly fine to trust Chinese papers.

[opal hedge]

https://www.nature.com/articles/467153d was in 2008. 31% of submissions from China were plagiarized (692 out of 2223)

[golden wraith]

anybody knows which is a good alternative to use stable diffusion online? with the possibility of loading models and loras?

[forest trout]

You wanting to run it locally on your machine or use someone else's processors?

[opal hedge]

https://www.nature.com/articles/d41586-023-03974-8 "More than 10,000 research papers retracted in 2023"

[cloud plume]

that is one specific journal and if i read correctly its submissions and not actually published papers.

[opal hedge]

Holy hell, if China was responsible for 75% of that, that's 7500 papers retracted from China

[golden wraith]

I want to use someone else's hardware bc my pc is not powerful enough, I'm willing to pay off course but I don't wanna pay for a service that doesnt let me load any lora or model so thats why im asking if you guys know a good one

[opal hedge]

I found a more recent one with retractions in 2023, but yes I agree those are submissions and not publications

[forest trout]

There's a couple of services, https://playground.com/ for example.

Civitai also has image generation.

[cloud plume]

to quote the bbc podcast more or less: "Is that a big number?" 😀
It might be idk, it depends on the total amount of papers :)

[forest trout]

Rejection isn't the same thing as fake paper. Papers get rejected with notes on what needs to be revised for resubmission too

[opal hedge]

I suppose it does, but 75% of retracted papers in the world with at least 7500 papers is a lot

[astral goblet]

fax don't matter. just bias confirmation.

[forest trout]

It's a quality control measure. "Hey, this doesn't quite meet our journals standards for credibility, please make these changes / explain this part."

[trail lion]

fax is like 80's tech, it definitely doesnt matter

[forest trout]

Yeah there's certainly papers that don't get past that step because the research is absolute bunk.

[astral goblet]

fax machines came up in the 60s but the technology was first invented in the mid 19th century

i'm being flippant with the spelling to amuse myself and not to treat the topic with much weight

[opal hedge]

"A Nature analysis shows that last year, Hindawi issued more than 9,600 retractions, of which the vast majority — about 8,200 — had a co-author in China. Nearly 14,000 retraction notices, of which some three-quarters involved a Chinese co-author, were issued by all publishers in 2023."

According to this source (nature) the retractions are issued by publishers rather than the writers of said papers. This means the paper failed to get published due to poor credibility rather than an effort to make potential publications stalwart in their foundations by the authors or their direct overseers (ie universities)

[trail lion]

and I'm being flippant with your flippancy, plus I havent thought about fax machines in a while

[astral goblet]

"just the fax maam"

/dragnet

oh shit no i quoted diehard 2! formerly known as the worst diehard

i can't believe i could be so wrong . it must be the children who fucked up. not i

[karmic cedar]

So…Amazonthropic is going to be a new word eh

Also do y’all think the release of Hybrid-Net and other more recent audio models is what prompted OpenAI to drip Voice Engine? You know they were sitting on that one

[opal hedge]

I think it's more because they were recently outclassed by Claude Opus and now no longer have an edge in any consumer product. My guess is Voice Engine is intended to keep the venture capital coming in even though they have no real product of value

That being said, Voice Engine is insane

The real question is how well can it emulate emotions rather than pure intonation, because elevenlabs already has good voices

[karmic cedar]

Yes, truly insane capabilities. I’m guessing just from the sheer amount of transformations it can do with the input audio, that it can handle emotions relatively awesome. 😆

One wonders just how close their next GPT update will be. These sorts of things tend to diffuse quickly.

[opal hedge]

It's hard to tell, because all of the example sentences are short and bland

[karmic cedar]

They could be hiding more capability still. Hard to say what stuff has made its way out of their red team labs.

[opal hedge]

Yeah, I'm pretty sure they're already training if not already fine-tuning chatgpt 5

[karmic cedar]

For certain.

When it drops, it will smack. They know those $20-$25 dollar a month subscriptions will melt if they don’t keep up the momentum.

[opal hedge]

you can already get a similar product with elevenlabs, assuming the features are limited to what they've shown

elevenlabs is really, really good

[floral nimbus]

When using SDXL is there some way to make regular Lora’s work? Instead of the SDXL or convert them myself

[karmic cedar]

True, and I have messed around with their platform a bit. It is impressive

[opal hedge]

Too bad it's so expensive. I wanted to use it for AI companions in Skyrim

[floral nimbus]

And is it possible to train SDXL models with 24gb vram

[opal hedge]

Yes you can, but it'll be really slow to train an entire model. A decent checkpoint will take maybe even a year on a 4090, for example. Loras are much faster. As for regular loras on SDXL, do you mean 1.5 loras?

[floral nimbus]

I mean SDXL Lora’s

[opal hedge]

I guess I don't understand this question

[floral nimbus]

Can we DM

[opal hedge]

Sure

[trail lion]

by regular loras, you mean those based on 1.5 or 2.1, sadly no, there's no compatibility between the models. you have to retrain the loras, if you're the author of the lora it's not hard, otherwise, you have to produce content with them and use that to train a new lora

[opal hedge]

I mean, there is a way to get it to work, but I heard the quality isn't the best

[broken smelt]

the architecture is completely different

[trail lion]

news to me, what is this black magic?

[broken smelt]

id focus instead on just finding the better method that gets u where u wanna be at best

[opal hedge]

It was a recent release, I forgot what it was called, but it lets you use 1.5 and 2.1 loras on sdxl and vice versa I believe

Ah, it's called X-adapter

[trail lion]

hrm, only comfy it seems, but impressive they were able to make it work

[lusty beacon]

Bruh why Mostaque left

[trail lion]

we dont actually know, but but presumably it was differences of opinion with future direction of the company with their major investors

[astral goblet]

wants to make sure ai is all blockchain

[lusty beacon]

So we don't see SD3 now?

[trail lion]

the new leadership has stated it's still coming

[lusty beacon]

okay

[verbal osprey]

I hope SD3 drops soon. Do we think it will drop in the next few weeks? Or are we talking months?

[trail lion]

impossible to know

[astral goblet]

it'll release....now! shit i was wrong

annnnd now!

[trail lion]

and if anyone says something, it's pure speculation, but, on the bright side, users are still being invited into the preview

[astral goblet]

nope

1-2-3.... NOW!

fuk. nope. guess i don't know

[verbal osprey]

That's good.

[broken smelt]

i just asked my crystal ball and it said try again later

[verbal osprey]

do you guys know if there is a plugin for automatic1111 where I can put in a movie clip and then animate it with stable diffusion? I have messed with animate diffusion but it seemed like it just animated a still image.

[trail lion]

there's a gif2fig and a mov2mov I think, possibly needs 1.5

[broken smelt]

there might be an extension that does this for you all at once automatically but it sounds like you wanna split your clip up into images and feed them all into the batch section of controlnet

[trail lion]

I'm not big on video, maybe others know better

[broken smelt]

the results probly arent gonna look good tho