🆕｜sd3

[bitter hearth]

[desert garnet]

yea i heard theres not enough male fur content there 😏

[mortal mesa]

so ignore the license, it dont matter right?

[torn wharf]

until you make $1m a year

[torn wharf]

distinguished furs

[sage burrow]

While I've seen several people make money at it, I have yet to see anyone make as much as first world county min wage. Still Hella fun tho

[uncut river]

"Oh hi! Did you see my dog? I seemed to have lost him and thought he just ran behind that hill. Could you like help me plz?"

[sage burrow]

Oh believe me I try to rect8fy that daily 😉

[uncut river]

male fur content. Is the fur male? Is the subject male, or the content intended for males? 🤨

[scenic trail]

When you were invited to Stability AI to clean up the mess.

[sage burrow]

Yes.

[uncut river]

ah, cool!

[serene edge]

I need sd3 but not medium

[uncut river]

rare or well done?

and do you need any spicy loras on the side?

[gusty trail]

Just need another two weeks for the new release

[craggy crest]

@sage burrow they're calling your name here

[lavish osprey]

had the opportunity to meet Prem live at the beginning of the week, he's gonna do great

[sage burrow]

They put an ad out recently looking to hire more content monitoring staff. That's a good sign I think.

They never got back to me in particular though lol

[mortal mesa]

the trust and safety is neither trustworthy nor safe

[sage burrow]

I only have 8gb ram or I'd have about 20 made already 😉

[uncut river]

you make loras locally with only 8gb ram, or vram?

[torn wharf]

i was doing batches of 10 with 16gb. dit is more efficient i guess?

[sage burrow]

Technically I made a checkpoint with my 8gb gpu 🤣. It hasld an 18 image dataset. it sucked lolololol. The various programs want 12gb.

Probably the same for loras.

So no rofl

But there is gpu rental

I make sdxl and pony loras via civitai, but they don't have sd3 capabilities yet.

[uncut river]

same, but I have 12gb vram

im youhnr5 on civit, only made 2 fractal loras

and a merged sdxl checkpoint (locally)

[scenic trail]

With his portfolio, I don't even doubt it.

[sage burrow]

There's gpu rental, but I'm too concerned I'd pay for the hours trying to figure out the install.

Also I'm trying ro be patient and wait until sai releases all the needed info (better quality models)

[sage burrow]

I learned the merging thing the other day, fun! That can be done on 8gb gpu.

With 12gb vram I recommend everydreamtraimer 2. For checkpoints

[turbid grotto]

It is possible to finetune sdxl with 12gb vram

[uncut river]

yes, if you go the lora way

make loras, then merge them with the model

[sage burrow]

Hmmmmm

[sage burrow]

How do you merge loras with checkpoints?

[uncut river]

I think the first intended use for loras was tuning, not a add on module

[turbid grotto]

I mean no loras

[uncut river]

See 3th image on https://comfyanonymous.github.io/ComfyUI_examples/model_merging/

[sage burrow]

Lockup everydreamtrainer2 if I could make it do anything in my machine...lol

But even dreambooth can run on 12gb!

Also kohya

[uncut river]

Say LadyLalita, if you want, plz share you civit name

[turbid grotto]

First time seeing this, interesting. What difference between this and onetrainer?

[runic tusk]

3th

What about the 1nd or 2st one?

[uncut river]

thats normal model merge, without loras (but with vae included)

[sage burrow]

I hid all my loras on there, just did soke drow ones. BTW warning put on sfw mode when viewing the very few images I posted on there. My civitai username is oipteaapdoce

[runic tusk]

https://tenor.com/E1SI.gif

[uncut river]

you could have at least warned about the f#tas

lol

[sage burrow]

Less gpu requirements I think lol That was my primary concern when searching for training software lol

They are probably all the same essentially.

Kohya us the lora one though (as I understand it)

[turbid grotto]

thanks ❤️

[sage burrow]

I did rofl

[turbid grotto]

I'll try, but when it will be less warm where I live)

[uncut river]

i guess i did not understand all words

no harm done btw, i can live with it.

[sage burrow]

Also their discord help section really helped me with my dumb installation questions a lot

[turbid grotto]

okay, that is nice

[uncut river]

im going to do a series in sdxl/pony merge, to later img2img them with sd3

easier to get some poses in back view

[turbid grotto]

btw, is there already sd3 finetunes?

I have seen some tests

[sage burrow]

In theory yes, in practice, negligable differences I think. Though there could be more out now. There are a couple of neat loras aroun though

Where did all the art go!???

[turbid grotto]

oh they all are on huggingface?

anyways, I suppose civitai should allow sd3 after license change

[bitter hearth]

what does that name mean

[sage burrow]

Huggingface and also that sketchy site... shakker I think it is

[sage burrow]

They prob make more than a million though

[sage burrow]

The best lora ever is on huggingface, it's called friedeggs

[turbid grotto]

How are they profiting?

[sage burrow]

It costs money to make models on there. I think it costs money to render images on there now also (I'm not sure, I get my daily free buzz)

The real question is how is huggingface profiting??????!!!!!!!

[craggy crest]

you use their GPUs, you do wind up paying costs

[sage burrow]

Haven't had time to cherry pick yet, they'll get better

[gentle wolf]

our community’s high expectations yeah, if they weren't so high, everything would've been fine. /s

[bitter hearth]

its tea time

[late compass]

Which new version is coming?

[desert garnet]

balls version

[craggy crest]

HampMan?

[sage burrow]

😄

Glif/2b won that time!!

[dull star]

bet you guys already heard the news, but I'll still put here what might bring back home imo

• Continuous Improvement: SD3 Medium is still a work in progress. We aim to release a much improved version in the coming weeks.
• Free commercial use appropriate for individual use and small businesses (up to $1M per year)

[bitter hearth]

pfft 1 mil a year

i mean seriously, who doesnt make that

[sage burrow]

My dragons are greeble challenged today 😦

[tropic aspen]

https://www.reddit.com/r/StableDiffusion/comments/1dw6o4y/live_reaction_and_review_of_the_new_sd3_july/
So quite literally, they did not change the biggest issue ppl have with the model's licensing. Bravo

• Revocable license
• On termination of license (if revoked, or if you ever hit $1m in revenue, whether related to the model or not) must delete all derivative works.
• Can't train any other models on SD3 outputs.

From OML discord

[digital barn]

What bothers me is the fact they expect the community to fix the parts of the model they themselves lobotomized while also being hostile and condescending to said community

[fleet meteor]

Finally

[fleet meteor]

Yeah but now the community will be able to do it because of the license change

[tropic aspen]

No they won't because the licensing is still strict

And it still has that stupid deleting shit in it

[fleet meteor]

Emmm no

[tropic aspen]

Read the actual license

[digital barn]

If they do finetunes that will enable SAI to keep making trash models

[fleet meteor]

@tropic aspen

"WILL NEVER ASK YOU TO DELETE RESULTING IMAGES, FINETUNES OR DERIVED PRODUCTS"

[sage burrow]

Can't train other models? Did I read it wrong in my excitement? It seemed like one could. Because I plan to train a LOT

[tropic aspen]

That's images though, not models

[digital barn]

If they switched the license now does that mean they can change it at any given time?

Either way sd3 seems to be cursed

[tropic aspen]

f. Term And Termination. The term of this Agreement will commence upon Your acceptance of this Agreement or access to the Stability AI Materials and will continue in full force and effect until terminated in accordance with the terms and conditions herein. Stability AI may terminate this Agreement if You are in breach of any term or condition of this Agreement. Upon termination of this Agreement, You shall delete and cease use of any Stability AI Materials or Derivative Works. SectionAge IV(d), (e), and (g) shall survive the termination of this Agreement.

They say you won't need to delete the images, yet here if your contract is terminated (which can happen if you make more than $1 mil a year), you're fucked

"Derivative Work(s)” means (a) any derivative work of the Stability AI Materials as recognized by U.S. copyright laws and (b) any modifications to a Model, and any other model created which is based on or derived from the Model or the Model’s output, including“fine tune” and “low-rank adaptation” models derived from a Model or a Model’s output, but do not include the output of any Model.

[sage burrow]

Is that 1mill before or after expenses?

[vapid frost]

I guess that's not our case, isn't it?

[fleet meteor]

It is (unless you make more than 1M per year with it)

[tropic aspen]

It also is for the ppl that might think about fixing the model

[vapid frost]

Why?
As I get it, if the agreement is terminated, then you should delete all derivative works. It could be terminated only if you breach any term. The main term as I get it is the fact that you haven't paid if you reached 1M$. Or am I wrong?

[tropic aspen]

There could be other terms as well that we don't know about that could also get you terminated

The only one we know of is the $1mil one

[craggy crest]

read what stability posted, not what someone posted here

[odd basalt]

Liscense is fine

[odd basalt]

It’s cheaper in china for cloud fair

[vapid frost]

My lawyer (ChatGPT) checked the license and said that the main breach case is 1M$ and others are just about distribution without mentioning stability AI or training foundational AI models

[sage burrow]

My lawyer (Gemini Advanced) lied to me the other day about SD3+SDXL merges! 😭 lol

[fleet meteor]

🤣

I wish there were lora training guides on sd3, I would love to train them (or at least try 😁 )

[odd basalt]

SD3 the safest child play model that Chucky approves of 😂

[craggy crest]

search this channel for 'balls' - there's a guy that's been working on a lora who can probably give you a few hints

[dull horizon]

Thanks for the link!

[cunning lintel]

Seems some people need to come to terms with the fact that SAI's models are NOT open-source but their version of openweights. The license is made toxic so that commercially building on the work of SAI must share profit with SAI (no issue if there is no profit). Non revocable license are a rarity, so don't expect that, the deletion on termination is a logical consequence of termination. At the end you're at the mercy of SAI, it's not open-source, it's some weird amalgamation of a SAAS product that you run yourself.

That outputs of the model are derivatives is too much if you consider only the images outputs, but a distilled version would also be trained on outputs of the teacher, in which case it suddenly makes sense. I'd argue that using sd3 images as part of training doesn't make the new model a derivative, otherwise any imagemodel would be derivative from any single image i saw, which it clearly isn't as that's the whole reason image models are trainable (well except according to anti AI folks) so i'd say don't sweat it on that very strict interpretation.

Ultimately, unless you're heavily/big time commercializing your works on SD3, you should be fine, which should put fine-tuners and researchers at ease.

[lavish osprey]

went on reddit expecting people to roast the license, found out they're roasting @viral plaza instead

[frail shoal]

So civitai if they make 1 million a year, will have to delete everything sd3 related if they stop paying for the license. Still dead in the water if they don't manage to get civitai to their side.

[fleet meteor]

I doubt people would use sd3 output images as training and if they do..... WHY USING AI OUTPUT AS TRAINIIIIIING? bonk

[vapid frost]

I believe the distribution of the model is not the case

[lavish osprey]

Looks pretty reasonable: if you don't abide by the license you can't use the model.
That's literally on every single license agreement. What's the point of having a license if you can just ignore it?

Also the only ways to not abide by this license are 2:

• you make illegal stuff with it
• you make >1m$ and you didn't tell us

[dull horizon]

So seems like they didn't address the main concerns of the license, so it's still DOA That's too bad... hopefully they amend it again.

[frail shoal]

How do you know?

[cunning lintel]

you'd be surprised dalle, mj and ideogram datasets are available and finetunend with 😉

[lavish osprey]

also it's funny, last time I checked InvokeAI was a SAI competitor (that makes >1m$ revenue using 100% SAI products)

[fleet meteor]

Yeah I know but because its easier, not because its better 😁

[vapid frost]

Civit AI doesn't use the model, their just share it with users. As I get it, license says that they have to mention license and SAI and that's it.

[fleet meteor]

Nothing like real world examples for training (photos, drawings, paintings, well and everything else)

[bitter hearth]

why do all the UI developers have feuds with each other

[upper snow]

if the license were irrevocable, then how would any terms be enforceable?

[lavish osprey]

This is not the case but, in general, anyone not abiding with any license for any product can't use that product. That's literally the definition and purpose of a license.

[pale charm]

Invoke is our own product, fwiw.

[lavish osprey]

I might be wrong, just saying

[frail shoal]

Civitai makes money. If you make money and in some form use the model , that you have to pay the licence

[lavish osprey]

also how do you know Civitai is not already paying?

[vapid frost]

If they use it

[pale charm]

Because Civitai has banned SD3 from the site, and is participating in the OMI to train a new model that is actually openly licensed.

[lavish osprey]

they run inference service.

[desert garnet]

does shakker also pay or they just run it as is?

[frail shoal]

It says 1 million dollars you have to pay a licence and if you stop paying, you have to remove those models you hosted. Isn't that how it can be interpreted. If not, how should it be interpreted?

[lavish osprey]

The last license was kind of shit.

[pale charm]

It still is.

[low stone]

So what are the main concerns of the license that weren't addressed?

[frail shoal]

They do use it

[sage burrow]

Photorealism.

[tropic aspen]

@low stone

[vapid frost]

If they earn more than 1M on interference, than yeah

[dull horizon]

The reddit link above goes through it

Oh PcMacsterRace linked it, ty!

[uncut river]

to train on fake people

[fleet meteor]

That would work training on real photos 😁, which is hard nowadays because of the mf filters and oversaturated colors😡 🤬

[lavish osprey]

Explain why.
It's literally free for 99.9% of people and researchers and you only have to pay if you make >1m$.

Companies that make >1m$ on SAI products should definitely contribute to the community so that SAI can make more free models for everyone.

[viral plaza]

it's just one particular troll

[lavish osprey]

the only ways to infringe this license are to make illegal things and to make tons of money without contributing.

[viral plaza]

(at least as far as i can see)

[sage burrow]

<-- 0 sympathy for companies which make over 1m per year regarding having to pay for licensing!

[low stone]

I just don't see what the problem is. You use SD3 to make a derivative. You make more than $1M with that derivative but don't want to pay the license but want to keep using it. So you just want stuff for free and want to make money off that free thing with no limitations. ?

[dull horizon]

The video goes through why

[pale charm]

I'm all for SAI monetizing, but the team at Stability has not responded to any of my outreach regarding SD3 licenses. I'm very clearly well versed in what the community, as well as enterprises, are looking for in the license. It's amusing that SAI sees Invoke as a competitor when I've actively engaged with multiple executives at SAI to try to figure out a way to partner.

This license is not open source. It is not viable. And I'm really disappointed to have to come here to defend Invoke which has been building OSS since the early days and been trying to partner with SAI.

[frail shoal]

Yeah but they could. So they should be okay with the fact they will need to pay stability a licence fee for life. If they are okay with it then good for them. It's just a matter of will they be okay with it or not.

[lavish osprey]

it's always one particular vocal troll or minority. But damn 700+ upvotes. It's overshadowing the license.

[lavish osprey]

To be fair, most companies pay. Some claim "we trained our own model architecture from scratch"

[viral plaza]

700+ upvotes on dungeonmaster's post, not the troll

[frail shoal]

Even if you make 0 cent with sd3 you still have to pay the license. It says any source of income, not only sd3

[viral plaza]

dungeonmaster's post is complaining about the kling/luma spam and broadly I agree with the core point of it

[low stone]

That's not true. it specifically says you pay zero if you don't make over 1M.

the new license.

[lavish osprey]

As Alex said, I do not represent Stability as a whole, I am my own person.
Also who did you reach? I'm available to talk if you want.
I hope the new license addresses your concerns.

[frail shoal]

But civitai could

[low stone]

If they make over 1M, they have to pay. I don't see the problem here.

[frail shoal]

They will need to pay even if they did not make even 1 cent with sd3 but are winning over 1m in revenue a year

[pale charm]

It did not.

[lavish osprey]

that's why there is no fixed price and will be decided case by case

[lavish osprey]

DM me, let's see what I can do.

[frail shoal]

Otherwise they need to stop using those models, hence remove everything for the website

[upper snow]

but why can't they just make the products that they made with millions of dollars of R&D and compute costs available for other people to make money off of in exchange for absolutely nothing 😭 😭 😭 /s

[dull horizon]

The video also goes through why it did not address the licensing concerns, I'm not sure why that's being ignored

[low stone]

Are you familiar with eduacational vs. commercial licenses? Generally free for small businesses and educational institutions because they know people will get hooked on the product then go out into the world to bigger businesses and then start paying for real licensing. I don't understand why a company making more than $1M a year has a problem with paying licensing fees.

[pale charm]

here are the concerns --
https://www.youtube.com/watch?v=23tEjvCNZHA

You can DM me the response, or respond here.

[viral plaza]

Lykon S. Stabilityson, physical embodiment of Stability AI, is currently being piloted like a mech by new CEO Prem

[lavish osprey]

bringing this to DM

[pale charm]

Main issues:
Revocable license
On termination of license (if revoked, or if you ever hit $1m in revenue, whether related to the model or not) must delete all derivative works.
Can't train any non-SD3 models on SD3 outputs.\

[pale charm]

Sure.

[frail shoal]

My point is the fact that I'm able to bring those arguments means that the license it's still not clear and leaves room for interpretation. That means big companies will still need to think twice

[upper snow]

how do you expect a license to be enforceable without being revocable?

[lavish osprey]

this is pretty standard. I remember speaking to legal about this myself and they showed me it's on literally every license.

[uncut river]

that issue is no more

[lavish osprey]

had your same concern

[upper snow]

also the last provision is only even remotely enforceable if you are making the outputs yourself. ai generated images are public domain.

[sick yacht]

I don’t think it’s on an MIT or Apache license?

[bitter hearth]

its kinda the norm for entreprise licenses to have some negotiation / flexibility

[pale charm]

The revocability of a license creates almost zero incentive for any enterprise looking to develop the model as IP -- The terms can also be updated unilaterally.

As we've already seen. This isn't open source.

[cunning lintel]

There's some tricky bit to be fair. For example lumina is saying they'll use the SD3 VAE, they will license their model MIT, so everyone can do whatever with it. Now if big inference site uses it, or derivatives,that SAI license is attached to it -> problem

Of course, one could argue that researchers should just use properly licensed components (and in this case they definately should, it's a big group), but it's a slippery slope.

In a way it's fair SAI plays it safe, but there will always be a tension between how much is new and how much is based on based SAI derivatives, and at some point it won't be fair, as SAI especially puts their work out there just for researchers to use.

[frail shoal]

They should pay licence fees if they make money with Sd3, however the current way the licence is phrased does not require you to win money specifically with Sd3 in order to pay the licence.

[bitter hearth]

win money?

please explain!

[low stone]

Civit runs a site that serves models/images etc. They're making money off the fact that SD3 derivatives are on their site. People like me specifically go to their site because I'll see SD3 content. They're making money off it.

[bitter hearth]

and I pay them!

well not much

civtai is cracking down on adjectives however, gonna be hard to use it for generation probably

[upper snow]

"almost zero incentive" aside from, you know, the >$1M you would be making from it. Stability also doesn't particularly have any incentive to rug pull you, they lose their licensing fees if they update their license and terminate you as a result of the changes.

[bitter hearth]

we just need a new ai company imo

[frail shoal]

Then I agree they should pay the licence fee.

[desert garnet]

they can rugpull anyone if theres pressure from an outside actor

[tropic aspen]

OMI might be smth to look into

[low stone]

I feel like it's definitely a move in the right direction. Only time will tell how the community reacts, but I hope this means civit can at least start having sd3 content again.

[dull horizon]

I dunno why the fee part is even an argument the issues lay in the license

[bitter hearth]

I would pay a monthly fee to someone who is not so crazy censored

[low stone]

there's a service that's pretty much not censored, rhymes with instagram

[sage burrow]

It's generally our lawyers will call your lawyers, past the basic simple public for all licensing. For any product/service.

[fleet meteor]

Wait for finetunes

[upper snow]

unless your plan involves seizing the means of production in the name of the workers, you're going to find that literally any company producing foundation models needs a business model that involves someone other than investors giving them money, and will also need to take actions to avoid the wrath of regulators especially if they are high profile

[bitter hearth]

yea this might be a good idea

[sage burrow]

I figure that's what the contact us if you make over 1m button is for (on all software/services)

[bitter hearth]

I have no plan! I am too old! You young folks should do something tho!

[frail shoal]

Yeah if there is still debate about the licence this is not very good. I'm finding a lot of people that are very knowledgeable are still refusing to finetune it

[cunning lintel]

they said on Reddit that they can't cause they got no license and have asked for one since before the announcement of SD3 in feb. whether that can't get means SAI ignores us, or SAI and us can't agree on terms, is left to speculation 🤡

[upper snow]

i would expect that civitAI getting their license would come after the license is updated

[bitter hearth]

civtai is cool and all, but I really think they need some competition to get even better

nothing like money to make you git gud

[sage burrow]

AI dreams: to need one of those fancy SAI licenses! 😄

[low stone]

yeah well, a lot has happened since then. New CEO, new investors, it's basically a new company.

[dull horizon]

If the license isn't viable for that individual, then it makes sense for people to refuse to finetune on it.

[cunning lintel]

Either way, i'd really hope everyone would shut up about that license now, normal non big companies can use it all free of charge. i'd think that'sabout as good as it can get for products that are this expensive to train.

[frail shoal]

The revocable license could be also revoked for nsfw finetunes. And fine-tuning cost a lot of money so...

[dull horizon]

It's purely at SAI's discretion, right?

[pastel latch]

any one fancy stress testing the license?

[frail shoal]

Yeah so if they like they can revoke it for whatever reason

[dull horizon]

Yeah, that's an issue. I don't blame people for not finetuning with the way the current license is

[sage burrow]

lol

[uncut river]

I wish there was a img2img method to inject stuff into the model, like first make the normal image but with an additional button "merge into model"

[dull horizon]

There's stuff like IP Adapter that can have a similar effect

[cunning lintel]

I'm much more interested in how much of the new SAI models will be open-weights and whether they'll heavily differentiate between open releases and API only offerings, like that shiny 8b model in the API 😉

[dull horizon]

but not merging it into the model, obviously, but yeah, not aware of non-training method of doing that

[upper snow]

their current AUP only prohibits using their model for creating NSFW content that is already explicitly illegal, they also already know that a sizeable portion of their market share involves NSFW models and inference services that use them and are quite unlikely to change that since that would mean less money. I'm not concerned about it at all, and if anyone gets dinged for making models for creating deepfake porn of celebrities then I will laugh at their misfortune.

[dull horizon]

No, the revocable part is something seperate

[desert garnet]

this just in,new president bans pics of feet and sai gets forced to revoke license of all ppl making feet pics

i made it simple for you

[upper snow]

if that ever happened, I would imagine that most people would have more to be concerned about over the law than they would about the license.

in fact there is absolutely nothing that could be different about the license that would help in that situation

[desert garnet]

https://tenor.com/view/frankt-gif-23053264

[uncut river]

https://edition.cnn.com/2024/07/01/politics/presidents-immunity-supreme-court-what-matters/index.html

[desert garnet]

luckily my new president is not part of a big group of powerful politicians who wants to ban all porn

[uncut river]

It's not only for trump, but all us presidents

[low stone]

how is this relevant to sd3?

[desert garnet]

yea theres a few dems on that group

[uncut river]

not directly, just to follow up the statement on president banning feet pics

[frail shoal]

Revocable part of the license is at their discretion, so it can be any reason

[sage burrow]

I feel that all presents are equally viable as zombie and/or gay romantic dinner prompts, regardless of affiliation 😉

[uncut river]

doesn't all customer serving companies reserve the right to stop their service?

[dull horizon]

When someone has the ability to take away everything on a whim, it makes sense for them to evaluate whether or not it's worth it

[upper snow]

f. Term And Termination. The term of this Agreement will commence upon Your acceptance of this Agreement or access to the Stability AI Materials and will continue in full force and effect until terminated in accordance with the terms and conditions herein. Stability AI may terminate this Agreement if You are in breach of any term or condition of this Agreement.

Here is the part of the license that directly contradicts what you just said.

[dull horizon]

That's something different drhead

[desert garnet]

depends if its a justifiable legal reason

[upper snow]

Name the section where it says that the license can be revoked at any time, for any reason.

[dull horizon]

https://www.youtube.com/watch?v=23tEjvCNZHA

Watch the video, he goes through it I'm surprised you didn't, to be honest, since it's been core to the conversation for awhile

[dull horizon]

Let me know when you're done watching the video, I have to brb for awhile

[upper snow]

That is a video. Name the section of the license.

I am not going to watch a 20 minute video when 30 seconds of you hitting ctrl+F on https://stability.ai/community-license-agreement would be sufficient.

zoomers...

[desert garnet]

skill issue

[upper snow]

Thing is, it's not my job to find supporting evidence for someone else's argument.

[frail shoal]

As long as there is debate, the new license won't fix anything

[desert garnet]

its not your job to do anything this is a discord server not your professional reddit cv

[dull horizon]

Why are you being such an asshole?

II. RESEARCH & NON-COMMERCIAL USE LICENSE

Subject to the terms of this Agreement, Stability AI grants You a non-exclusive, worldwide, non-transferable, non-sublicensable, **revocable **and royalty-free limited license under Stability AI’s intellectual property or other rights owned by Stability AI embodied in the Stability AI Materials to use, reproduce, distribute, and create Derivative Works of, and make modifications to, the Stability AI Materials for any Research or Non-Commercial Purpose. “Research Purpose” means academic or scientific advancement, and in each case, is not primarily intended for commercial advantage or monetary compensation to You or others. “Non-Commercial Purpose” means any purpose other than a Research Purpose that is not primarily intended for commercial advantage or monetary compensation to You or others, such as personal use (i.e., hobbyist) or evaluation and testing.

Anyways, I'm outty, there's your proof

[upper snow]

It says that the license is revocable, but does not say under what conditions it can be revoked.

[low stone]

"Subject to the terms of this Agreement, "

[frail shoal]

Exactly

[low stone]

so those terms are why it could be revoked.

not "any reason"

[upper snow]

And this is where it says where it can be revoked:

f. Term And Termination. The term of this Agreement will commence upon Your acceptance of this Agreement or access to the Stability AI Materials and will continue in full force and effect until terminated in accordance with the terms and conditions herein. Stability AI may terminate this Agreement if You are in breach of any term or condition of this Agreement.

[frail shoal]

Still vague , what are the terms of agreement, the whole license? Do I need to get a lawyer to understand it ?

[uncut river]

at least they promise to only terminate the agreement if you're in breach of it

[sage burrow]

I'm glad they didn't ask for my CIvita userid in that form lol

[uncut river]

no, it's not long or hard to read

[dull horizon]

You do you, my dude.

[upper snow]

They're right here, and frankly most of it is quite comprehensible: https://stability.ai/community-license-agreement

[cunning lintel]

prob should get a lawyer to look at it and not a youtuber 😉 But it all starts with expectations, SAI models are not open-source and not free as in free beer. Whether that's fair, guess it depends where you come from. I get the feeling people want too much.

[upper snow]

It's also not a promise, it's a binding part of the contract. They cannot terminate the agreement unilaterally as long as you follow its terms.

[uncut river]

a "written binding part of a contract" is just a fancy promise

[bitter hearth]

@sage burrow how long will people talk about the new liscene now

[upper snow]

the kind of fancy promise that you can sue over breach of contract for (wow there's actually no binding arbitration clause in this)

[desert garnet]

til someone releases a sex finetune on civit then everyone will shut up

[lavish osprey]

I am talking to hipsterusername right now fyi.
Please refer to the blog post to get the "easy to read" version and the intentions behind this.
For the license agreement itself, unless you're a lawyer (and I am not), it's more complicated

[uncut river]

im quite happy with the new license

should take away some fear to create derivatives, like loras. hope it will be on civit on site gen soon

[dull horizon]

I don't finetune models or use them commercially, it doesn't affect me. It affects the finetuners and sites that host SD3, so I'll leave that to them

[sage burrow]

I hope civit makes less than 1m per year 😄

[dull horizon]

If people deem the license viable to them, then they can use it. If they don't? Then they won't. It's really that simple.

[uncut river]

the financial people must be working hard @ civit

[dull horizon]

I think peoples concerns are completely valid

[desert garnet]

breaking news,civit creates several shell companies/sites with online generations that make 500k each to avoid paying license fees

[sage burrow]

Huggingface is even more awesome though

[sage burrow]

One would hope not. I pay for my biz software.

[frail shoal]

Hugging face is a great project with a very shitty UI/UX

[bitter hearth]

VERY

[frail shoal]

And that's why civitai ai exists

[bitter hearth]

and its another very bad ui site

that search

[cunning lintel]

civit and huggingface is difference between developer and end-user platform 🤷‍♂️

[upper snow]

i feel like we must not be using the same site? civitai's interface is slow and pretty much changes every time that I have a new model to upload, HF is simple and fairly fast and I can just upload stuff from the command line

[bitter hearth]

How do you make your money back on open source however? Licensing will do that?

[frail shoal]

It's way easier to find stuff in civitai

[hallow lion]

hooray youll fix stuff! XD

[frail shoal]

By selling your services to big companies

[hallow lion]

hugginggace feels like the comfyui version of civitai

its not user friend;ly but the good shit is hidden there

[frail shoal]

Invokeai has a great inference interface and they make money with selling their services to companies. Stability can sell their model to companies and services if they new to make some.

[sage burrow]

Comfy is my fave! That explains things lol

[hallow lion]

i hated comfy and now it is my fav too... i think most users go through this cycle - denial(nah A1111 is fine) anger(i cant do this and that here wtf... maybe i shld use comfy) acceptance(ill use comfy)

[frail shoal]

Yep comfyui in my opinion is the greatest project that needs protected at all costs. Even revered

[viral plaza]

if you want comfy's power but don't love comfy's noodleinterface, Swarm has a friendlier frontend main tab for that

[upper snow]

what stage am I on if I'm jury rigging triton kernels in to the point where it's faster than comfyui

[hallow lion]

Raptured.

[sage burrow]

Comfy was the first ones I tried. I liked it because I manged to get decent images first try. I've seen uses for 111 that look helpful, but fortunately there's comfy workflows for them

[hallow lion]

yay they fixed the licence. Also -->Continuous Improvement: SD3 Medium is still a work in progress. We aim to release a much improved version in the coming weeks. <--- IN 2 WEEKS!

[frail shoal]

Yes swarmui is the other side of the coin. I see a bright future for these two projects. I only hope they will expand their inference code to llm's with llama.cpp, langchain and llamaindex and they could completely dominate all the open source userbase.

[karmic silo]

What's Swarm?

[upper snow]

I will probably abandon A1111 instantly the moment that someone creates a frontend that easily supports every feature that A1111 does, is extensible, and also has tablet pen pressure support and image editing that is otherwise at least on par with gradio 4's image editor

[hallow lion]

Swarm is for noodlephobiacs

if you want the benefits of noodles and the look of A1111 swarm is for u

[upper snow]

comfyui puts the spaghetti in the frontend, A1111 puts the spaghetti in the backend implementations

[hallow lion]

idk it becomes kind of intriguing and addictibve to hook the noodles up - kind of like an old moog synth

its not that scary after a while

[sage burrow]

What can 1111 do that comfy can't, even with the fancy nodes?

[upper snow]

i see people constantly complain about interfaces for inpainting and regional prompting

[cunning lintel]

i still think there is room for some nice polished UI (invoke is closest, but always a bit behind) swarm on top of comfy feels like you had one problem, now you have two, as i inevitable ended up using custom nodes in comfy, but swarm is super easy to get started and set up.

[viral plaza]

oo pen pressure is one I haven't tried to get working yet - I do think I have the tools to make that happen tho

[silver sluice]

@jolly swan Hello good sir, do you have any official word or update on what you think of the new license update?
#📣｜announcements message

• Does it answer all the questions you had?
• Will you consider training an SD3 pony model after you release 6.9 with this new licensing update?
• Side question: Do you have an estimate for when you think SDXL Pony 6.9 will be released?

[sage burrow]

Whichever version glif is running knows who Goodra is, but Taesd and Flash via HF both do not. They do know what lululemon is though.

[upper snow]

tbh what I really need is a good krita plugin, the one that exists just doesn't have as much flexibility as I felt like I had from the last decent krita plugin, and the old one doesn't adequately support things I do now. But having pen pressure on a gradio like inpainting interface would be a step up

[sage burrow]

That sounds like SD on a phone or tablet? 🙂 🙂 🙂

[compact niche]

cool

[compact niche]

awesome

[simple thistle]

the mask editor in comfyui actually has pen pressure

[low stone]

.

[jolly swan]

Some of them but not everything.
I will wait for legal review just in case.
The model is still meh so I need to understand what is the priority between OMI/XL/SD3
Well, I am trying to start training it but the data dungeon is not letting me go (i.e. I am dealing with captions quality now).

[compact niche]

i try a weird workflow with sdxl refiner

[silver sluice]

• What's still missing they didn't answer?
• When you say wait for legal review maybe you could just wait for civit to turn SD3 back on and consider that 'good enough'?
• What's OMI?
• So you're still in captioning stage good stuff, is it just a matter of waiting for the process to finish or is it a more interactive hands on process that you have to like babysit if you will?

[jolly swan]

I am still not sure the license is written in a way that actually reflects the announcement so I will wait for Civit to confirm
https://www.reddit.com/r/StableDiffusion/comments/1dp2as9/update_and_faq_on_the_open_model_initiative_your/
I am doing round of finetuning model on curated captions, then running it on more images, fixing captions by hand and then gong back to finetuning on larger dataset

[craggy crest]

usually you sell services

[viral plaza]

that was pretty easily actually - swarm's image editor now has initial pen pressure support

currently just for brush size dynamics, but now t hat i have the foundation there it can easily be expanded to do other things

[silver sluice]

thanks for the answers, much appreciate your work again, if you don't mind me asking, what is about the quality of the captions generated that requires further refinement by hand?

[kindred mica]

Whoa you’re orange

[viral plaza]

whoa you're green

[jolly swan]

A lot of things, and it gets worse when you have multiple characters + nsfw. There are all kind of failure cases, not getting the right character names, misnaming things, missing OCR, etc...
The quality is already better than anything publicly available by a mile (aka things like Cog), but I want to make sure I squeezed everything I can.

[silver sluice]

got it so the vision model kind of hallucinates sometimes or mixes up elements, makes sense, thanks for the insight and good luck man

[lavish osprey]

the points InvokeAI is making are essentially 2 (and only apply to companies making over 1m revenue, but in reality they apply to nobody):

1. I am a company that just hit 1m$ revenue and I'm waiting for the Enterprise license. The license agreement says I have to delete everything.

• No. we do not expect you to delete anything while you wait for the Enterprise license. Nobody expects you to do that. That's phrased the way it is because there is no way to say otherwise in legal terms, we can't put any grace period in the license, but still nobody wants you to delete anything as soon as you hit 1m. Also no company hits 1m overnight. Usually you know in advance when that's about to happen.

2. I am a creator who made a lora with character X. I can't use outputs of character X to train another lora?

• That point in the license refers to "foundational models". A lora, finetune, retrain, retrain from scratch, etc, is not a foundational model. It basically means that you can't use sd3 models or outputs to make a """new""" model architecture and claim it's your own just to bypass the license.

Both are pretty standard legal terms and are in multiple licenses. Of course, being something made by SAI there should be an outburst over the simplest things

[dull horizon]

And when legal teams review them, I'm sure that'll all be sorted After all, it's up to the finetuners and sites like civitai on whom they want to host or use

Though I do apperciate the response

[lavish osprey]

I'll make sure they're addressed in a faq. There is no way legally to put them differently in the license document

so if you decide to be scared by that, your choice I suppose.

[dense forge]

love the progress on the license!

[shut sable]

i just see "if you make less than 1m in all revenue, you can use sd3 with no paid license"

[lavish osprey]

of course if all you do with SD is cool avatars for your employees, you probably won't pay shit lol.
That's why there is no price on that tier.

[shut sable]

and if i make cirno?

[lavish osprey]

depends on how much money you make with cirno images

(plus this is all self-reporting, come on lol)

[shut sable]

powe r bill says i lose 22$ a day

[lavish osprey]

then we give you 1 $ per image

[dense forge]

When are we going to see 8b in the wild?

[shut sable]

real

[lavish osprey]

(by the way that's net margin, not revenue)

(if you make 21$ but you lose 22$, you still have 21$ of revenue even if your net is -1$)

[shut sable]

whenever the s.ai higherups say so, some devs Acutally Want a good model to be published

[lavish osprey]

yoinked powers strike again

(but I personally don't think 8b is ready for release)

[shut sable]

the ddpo server shows Extremely Good Promise

[lavish osprey]

(there is 100% gonna be a "X doing Y" prompt breaking the model)

[shut sable]

someone will find a prompt that breaks any model

something like "cirno from Touhou making a nuclear weapon"

[dense forge]

I think also nobody can tune sd3 atm even though they are trying

[shut sable]

no-one is trying hard enough*

[dense forge]

kohya is trying pretty hard lol

[lavish osprey]

I hope we can give them help and resources soon enough

[dull horizon]

thank you!

[lavish osprey]

most of the answers to "why SAI doesn't do X" are usually "we want but we need time and resources"

[shut sable]

why doesnt sai just release 8b as 8b beta, like how they released XL as 0.9

(safety, its always safety)

[cunning lintel]

well, there was a time i thought this #🆕｜sd3 message was bad (from 8b), but times have changed, that one is perfection :p

[lavish osprey]

most models are kind of bad at sideways and upside down anatomy and faces

[shut sable]

humans are also bad at that (thatcher effect)

[upper snow]

well that was kind of the plan for 2B but we know how that worked out. SAI is eternally cursed to have an early release that splits support for any mainline model they release

[dense forge]

I have hundreds of bad gens for every model I train

[lavish osprey]

||that was supposed to be 2b release||

[shut sable]

why doesnt sai just release a model correctly and not rushed

[upper snow]

Just use 2B to figure out what you want to do with 8B, if you're done with testing stuff before 8B releases you aren't doing enough ablation testing.

[shut sable]

e.g. XL, audio-open (although that isnt actually s.ai but actually just harmonai), svd

[shut sable]

hi comfy

[upper snow]

like I cannot imagine trying to do any major project on SD3-8B without doing basically the whole training run on SD3-2B first. You don't want to piss away that many cloud compute credits on ablation testing

[uncut river]

I hope in 2 weeks time they also improve the 2b version

[heady wolf]

What is ablation testing?

[upper snow]

you change things about your training regimen in an organized manner to see what works better

[lavish osprey]

likely a better candidate than what's being proposed now

[simple thistle]

That's the 2B that was released

[lavish osprey]

too many fennec anime girls in the dataset, for sure

but I meant as a President candidate

[viral plaza]

like doing image gen grids, except for training if you did a grid it would cost 50 million dollars so you do it more carefully lol

[simple thistle]

not enough, it really likes putting human ears lol

[lavish osprey]

you know we actually had to put "that prompt" in the training grid, sigh...

[lavish osprey]

not real kanjis, 0/10, sd3 sucks

[cunning lintel]

Can't you make a perfect case that public testing in discord is needed again so that all popular failure modes are detected ahead of time 🤡

[lavish osprey]

you have no idea how many times I suggested that lol

[sage burrow]

Oh boy would you need a moderator if that happened LOL

[lavish osprey]

we did that already with SDXL

(well, the leak happened that time, so that's possibly why they didn't want to do that this time)

[craggy crest]

vett the testers then and require NDAs

[sage burrow]

sdxl testing on discord, and the server didn't get shut down by discord?! lolololol

[cunning lintel]

in SDXL times it went really amazingly well (filter helped)

most weird thing was a few people creating the same prompt constantly for weekson end 😂

[uncut river]

(with a bit of img2img)

[sage burrow]

that sounds extremely boring!

[sullen moss]

Api 8B

DALEE + SDXL

[sullen moss]

Unfortunately, the understanding of anatomy in dynamic poses is still very lacking. I hope that by the release of 8B, this will be improved at least a little.

[cobalt moon]

I mean

the PR team have say that the SAI will release an improved version of 2B ( SD3.1 I will say )

I will still looking over the entire situation though

by the way PR team pretty much complete their job in the term of clarification ( without looking at legal team lmao )

[sullen moss]

There can't be a proper model without proper datasets. I still don't understand the point of putting in so much effort only to lobotomize everything afterward

[cunning lintel]

Surely 2b can get a whole lot better, SAI has data SDXL, cascade and 8b show that data is there. And even if not it's crazy that pixart with its tiny dataset is more dynamic than 2b.

[cobalt moon]

3.1 is successful or not I think it will prob held back by hardline open sourcers.

[cobalt moon]

like what, open source is our mission and shall NOT impose non-commercial license

[low stone]

trying to have her doing something cool, but I think we know what the problem is. 🙂

[cobalt moon]

woah I did not say OMI is a bad thing. I mean sure they was created and formed in the midst of "closure" of open SAI.

[simple thistle]

yeah SD3 falls apart pretty quickly with complex stuff

[cunning lintel]

meh, they'll realize the reality, that AI models are not open-source, maybe a few will choose to wait forever on "promising new initiatives", but if the model is better than alternatives and training works, most people will start using it, especially when third party tooling/extensions trickle in like control net/ipadpter style and such

[low stone]

8b to the rescue

[lavish osprey]

not enough attention on data, the only attention block is shared with text.
This is (also) why the model behaves better when you use long and descriptive prompts.

and why 8b has the edge when it comes to coherency (sheer number of params compensates the lack of attention)

[cobalt moon]

Hm. I mean there is no such things as a free lunch

one must pay for the cost of open sourcing model

[lavish osprey]

looks like Ultra to me, am I wrong?

[low stone]

I wrote a quick gpt instruction for runwayml gen 3 to make video prompts that include camera angles etc. I'm finding it makes really neat stuff with images as well. here's the prompt for this one: anime art, Camera sweeps through a neon-lit city street at night, quickly zooming in on a blonde-haired girl with fox ears in a pink sweater and jeans. She suddenly leaps into the air as a massive robotic creature smashes into the ground. The girl gracefully dodges laser beams, flipping and twirling in mid-air. She lands on a building's edge, summoning a glowing energy blade. She charges at the robot, slashing through its armor in a dazzling display of agility and power. Camera ends with a wide shot of the city's skyline as the robot explodes in a burst of light.

the model is bound to so something with all of that. 🙂

[lavish osprey]

nice shoes

[cobalt moon]

hm maybe I should make Comfy Classic Fennec Girl as well

[lavish osprey]

change her to fox, just to piss him off

[slate salmon]

@lavish osprey Hi lykon, are you going to release a new version of sd3, because my team has done some fine-tuning work on the existing sd3medium and improved some body structure, I'm not sure if we need to continue】

[cobalt moon]

he is just a small dev though

he doesn't worked for real training of SD3

[lavish osprey]

if you were successful, please share the results, might steer our research

[low stone]

more with that prompt style

[lavish osprey]

definitely Ultra and not 8b

[low stone]

indeed.

is ultra not 8b at its base?

[lavish osprey]

I can't confirm, nor deny

[low stone]

ah

[upper snow]

it's the secret 16B model

[lavish osprey]

definitely not lol

[sage burrow]

Good except hands 😦 via glif

[lavish osprey]

what I can say it that it's not really fair to compare Ultra to an open model, like it's not really fair to compare Dalle to an open model

because they're not models

[cobalt moon]

16b is too large lmao, to the point you need A100 to run it

[cobalt moon]

so what's it then?

[lavish osprey]

it's 16gb without TEs, so it can run on 3090 and 4090 too

[lavish osprey]

I'll tell you when OpenAI reveals what's in Dalle

[cobalt moon]

though I'm wondering how will the model came out if we maximized the requirement to run it / inference

[sullen moss]

By the way, DALL-E in Bing and DALL-E through the API generate the final images quite differently. Today, I asked ChatGPT to do the same, but it didn't work out. 😄

[low stone]

with dalle on api you can do 1792x1024 in "hd". can't do that in bing.

[lavish osprey]

probably different prompt enhancements

[low stone]

bing doesn't do the "hd" mode.

[lavish osprey]

and size apparently

[low stone]

the hd mode fixes hands and fingers/limbs etc to more exacting degree

[lavish osprey]

(which might even use different ckpts entirely)

[low stone]

well you can do widescreen in bing, but you won't get the hd mode.

[sullen moss]

Bing

[low stone]

hd mode takes longer too

[sullen moss]

Api

[slate salmon]

@lavish osprey We just fine-tuned the 1.6w body structure, which contains a small number of nude images with an epoch of about 5, and the results so far are satisfactory，

[lavish osprey]

can't you publish the finetune on hf/civitai/shakker/etc?

[slate salmon]

don't know if the sd3-2b has a problem with insufficient pre-training

[cobalt moon]

I wanted to train a SD3 model with my anime dataset

[lavish osprey]

well, guess not on civitai, but till..

[cobalt moon]

... with that funky CogVLM captions.

[lavish osprey]

pretty sure you'll get a lot of instant karma being the ones who "fixed sd3 medium"

go for it

[cobalt moon]

yeah

[lavish osprey]

I prefer MiniCPM. Can also be instructed to output style and arrange the output in a json

[cobalt moon]

never heard of that

[lavish osprey]

and style alignment is pretty good

[cobalt moon]

I am also going to train it on Chinese caption too as I understand Chinese and Chinese myself

[lavish osprey]

also recognizes text (in Chinese and Japanese too)

[fossil nest]

@cobalt moon could u check dms

[slate salmon]

This is a simple demonstration

[lavish osprey]

t5 will fight you on chinese captions by the way

[lavish osprey]

I see a watermark text there

[cobalt moon]

T5 cannot do Chinese?

[lavish osprey]

anyway, current 2b can also do "standing" pretty well

main issues are with weird poses, continuity in general and hands

(my guess is due to the lack of attention)

[sage burrow]

glif

[slate salmon]

@lavish osprey If you are ready to release a fixed version with a better base model, we will not continue to fine-tune the existing model and look forward to your release of a new good human structure model

[lavish osprey]

there is a chance you can diffmerge anything you do on the next iteration

worst case it's a good experiment

and you'll just have to repeat it

(or you'll release before us and you'll be regarded as the saviour of the world)

(which would be a ok by me)

[sage burrow]

I need to prompt gloves!

[slate salmon]

Thank you for your reply.

[fleet meteor]

Anyone knows if connecting this nodes like this is the way its supposed to be connected?

[craggy crest]

yes

closeup of a clear glass bottle full of a forest encampment, wigwam, longhouse, tepee, grass house, wattle and daub house, chickee, forest, cliff, canyon, river,standing upright on a boulder in the sonoran desert.

[fleet meteor]

Idk why but somehow the sd3+t5fp8+clips all together in 1 file is way faster than they separated

It runs at almost the same speed as sd3 withouth t5

[sage burrow]

hmm, getting better, needs a little editing

[lavish osprey]

fp8 t5 is half the size of fp16 t5.
If you're having speed issues with fp16 t5 it means you're going past your total vram, meaning you're offloading and reloading all the time

[upper snow]

It's not enough to give it a default horny bias, right? (I usually prefer to avoid that if possible)

[sage burrow]

There was a sale a the shoe store, two left feet sale...

[fleet meteor]

Nono, the strange thing is that I used fp8T5 + sd3 + clips = 40 seconds
And I use integratedfp58+sd3+clips in a single .safetensors = 20 seconds

And using up to 7gb of vram only , im confused because they´re supposed to be the same thing

[sage burrow]

What, wait?! How, with what? (in laymans terms? :D)

[fleet meteor]

This instead of that :3

[lavish osprey]

did you use the safetensors with no clip?

(the 4gb one)

[fleet meteor]

I use a single safetensors that said it had the t5 + clips, its 10.1 gb

[slate salmon]

Yes, but we have observed that the human body does have better integrity, and we are trying to use more training sets for the next round of fine-tuning, and if it is successful we will open it up to the community

[lavish osprey]

uhm

looks like a report to mr comfy then

unless you're keeping one of those files on a slower drive

[slate salmon]

We also observed some interesting features of mmdit that differ from unet

[sage burrow]

Thank you! 🙂 And it's in comfy form!!! 😄

[fleet meteor]

https://tenor.com/view/cat-cats-cat-love-cat-kiss-kiss-gif-24690536

[sage burrow]

My GPU lol

[coral sable]

So there is no subscription anymore, we can do youtube videos with SD3 and sell art under 1M$/year?

[craggy crest]

that's what it says, yes

[coral sable]

perfect, small people should be happy then

I wish we have ETA for 8b, I'm in love with it

[bitter hearth]

https://tenor.com/view/pat-pat-gif-24957682

smol people

[torn wharf]

within a year

estimate

[coral sable]

this is us

[coral sable]

source?

[brittle nexus]

SD3 has an incredible training on Halloween pumpkins and zero about human feet

[torn wharf]

me. i estimated.

ETA

[coral sable]

api gave me censored feet, but it can do🤭

[brittle nexus]

Yep. Feet was promoted to sexual organs now

[desert garnet]

only the slim ones

[coral sable]

🦶

ok artistic version works well IMO

[craggy crest]

SD3 medium

[bitter hearth]

free it

[coral sable]

on todays post is says 2B fixed version will be released in following weeks, so latest next month I would say

[craggy crest]

don't set dates

[torn wharf]

in b4 next week people are mad because they pwomised

[brittle nexus]

https://tenor.com/view/horny-jail-go-to-horny-jail-bonk-doge-cheems-gif-17582752

[coral sable]

bruh if t hey say weeks, it would be weird to come in 4 months 😄

[cobalt moon]

and then they mad because it is not finetune quality

r/StableDiffusion logic

[coral sable]

I don't even look at it anymore

[cobalt moon]

it become r/Kling at this point

( can't say much because welp r/StableDiffusion is if not, one of the largest AI subreddit )

[craggy crest]

don't set dates anyway, that's want causes rumors and then trouble

[cobalt moon]

I don't think it will taken months, like 6 months for them to release 3.1

again I am not a ML researcher

[coral sable]

yea it's like AI spam bots are spreading ads and self upvoting the posts, there is huge bias pushed up on majority

[cobalt moon]

but lobotomizinator tended to be a post-training process

it is a fix nevertheless.

Unlike SD2 I think SD3 is much, much... much more complicated

[coral sable]

Will to address the issue is most important

[bitter hearth]

[cobalt moon]

we already got some controlnet mode in SD3 though

Canny and Normal if I remember correctly

[torn wharf]

controlnets showed up like a day after release. they were really easy to train

[sacred jewel]

SD3 never disappoints

Is it my skillz?

[cobalt moon]

yeah

everyone got skill issue like Lykon said

[sage burrow]

i can't hold it in any longer... 2 WEEEKS!
lol

My bodies were going to well, the limbs were the correct number, and going in the right directions... until I made her a fox lady

darn tails!

[craggy crest]

probably not, but still. someone says a date on here, and suddenly it's "set in stone" all over the net - and then people are mad when it doesn't happen. And the company never said it would happen by that date.

[sacred jewel]

There is "in two weeks" and then there's the Trumpian "in a couple of weeks"

SAI never specified which model they used 😛

[brittle nexus]

Laara Birohks

[craggy crest]

well - "two weeks" isn't what the annoucement, or the devs, said...

[torn wharf]

sd3 is many different archs. like 2b, 4b, 8b, and a lighter one maybe too. it'll have to be like sd3 2b.1 or something

[cobalt moon]

2B is parameter...

it is make more sense for SAI to do SD3.1 2B than SD3 2B.1, because it is non-sensical to put 2 billion parameter with .1 after it

[torn wharf]

yeah each parameter size will have it's own versions

[craggy crest]

it'll probably be sd3.1 i.e. 2b

[torn wharf]

sd3 2b rev 1

owls are trained really well and i hav to really push the rng and weights to make my lora work here. i like this result tho

[sterile pendant]

Just train a separate img2img model that can un-gaussian blur an image. Should only take a few months on a 1k cluster of a100s 🤣

[torn wharf]

it gets more foamy lookin as i crank up the weight. it is cool how it caught detail. thats the new vae more than anything i did

[torn wharf]

i fixed it my lora fixes sd3 . problem solved.

[sacred jewel]

my LoRA turns most things to absolute crapolis... but I will get there one day LOLOL

Some times, it does not melt the subject... but still 😢

[torn wharf]

all is good until lawyers demand i destroy my balls

[fleet meteor]

Lmao

[torn wharf]

the day stability executes order 66. destroy all models

bears don't make good clowns. dont hire them for parties

PSA I REPEAT BEAARS ARE NOT GOOD PARTY CLOWNS

[torn wharf]

ancient terracotta balls

little known fact

in the future the ball army rises again

none of it would've been possible without the pioneering spriit of the frontier balls

[bitter hearth]

did your lora go from balls balls to balls looking things

or can you still prompt for balls balls

[torn wharf]

i'm experimenting with not saying balls in the captions. it was hard

but if you prompt balls the model knows balls

[bitter hearth]

balls indeed!

[torn wharf]

with and without the ball word. nothing about balls in the captioning but the network figured out the ball part anyways then generalized the rest

[bitter hearth]

https://tenor.com/view/cat-blind-flashbang-light-mode-gif-21281004

[torn wharf]

in a galaxy far far away, a long ass time ago , there were balls

[torn wharf]

pew pewpewpepew pew pew pew

[torn wharf]

the death balls

[verbal epoch]

https://stability.ai/news/license-update

[lone copper]

is it still comfy-ui only?

[torn wharf]

some of the other uis support. sd.next is one

[lone copper]

Im really curious how come a checkpoint doesnt work on some UIs but I suppose its quite complicated and probably outside my competence field

[craggy crest]

SD3 requires unleaded gas and some of the UIs can't burn unleaded gas yet

[lone copper]

Thanks! As expected, I have no idea what that means lmaooo Im gonna do some research

[craggy crest]

the various UIs are like cars. some of them require stuff they don't have yet in order to run SD3

[torn wharf]

so sd1.5 is leaded gas? that explains a lot

[craggy crest]

heh

[noble coyote]

Licensed proper!!!

[scenic shadow]

Excited to see how much more attention 3 gets now that finetuners won’t be worried about having a patreon/ donation income license conflict to pay for training.

Even more so that we are officially getting a revised model to work from when they finish the retraining.

[frail shoal]

not many, but lykon would do good to bring his dreamshaper brand to it, to encourage others to do it

[cosmic mantle]

https://tenor.com/view/bear-dance-dancing-lit-get-it-gif-14380729

[uncut river]

morning

[keen palm]

They fixed what was wrong with the updated license from yesterday?

[cobalt moon]

removed Creator License

though the ability to revoke license and ban on SD3 output as dataset is still there

[keen palm]

Okay, but the questions from Invoke has not been answered?

[cobalt moon]

what's the question?

[keen palm]

https://www.youtube.com/watch?v=23tEjvCNZHA&ab_channel=Invoke

[cobalt moon]

you expected me to watch the entire 18 min video?

[keen palm]

yes?

The licenses are revokable

Just saying that there's no way that we will see finetunes and such on civitai any time soon. Even if someone figures out how to finetune the model

[cobalt moon]

someone just did and said they finetuned it earlier

and ping Lykon of whether they are going to release a 3.1 or not

sure I guess we are going to back to the era of pre-Civitai. But you wouldn't say it get no traction at all

[cobalt moon]

( for anyone who wondering about this )

[keen palm]

Civitai isn't banning SD3 anymore? or why is it being asked by Lykon to release the finetune on civitai?

[lavish osprey]

you can read my reply to the video

[cobalt moon]

the finetuners have release one generated image using his finetune.

If I comprehend correctly I think Lykon thought the finetune is already done or complete

[lavish osprey]

I didn't assume. There is an "if" in front of the sentence 😉

[sage burrow]

So, Kohya or Onetrainer for sd3 loras?

@torn wharf ?

[cobalt moon]

yeah

[lavish osprey]

too much comfyposting, let's begin a Miku wave

[sage burrow]

Context? Or does sd3 know miku? Goigle thinks it's a local restaurant lol

I negative prompted anime and everything 😦

[sage burrow]

I looked up her age, sd3 needs to put more clothing in her...

Better.

[coral sable]

a bit of a plot twist

[bitter hearth]

^ when you inpaint hedgehog on someone's head lmao

is this SD3? and the miku?
anatomy seems good

[sage burrow]

Everything is sd3

[bitter hearth]

wow

is this a workflow or just prompt?

like is there a refiner pass with SDXL or something?

[sage burrow]

https://glif.app/@FireCreeper21/glifs/clvsa1w1x0001m1lykzwx6e98

[bitter hearth]

thanks

[sage burrow]

The cartoon foxes were sd3 ultra

[hallow lion]

SD3.5 in two weeks.

[bitter hearth]

ah I checked the glif its running SDXL over the output

that's maybe why the anatomy is better

[hallow lion]

hey thats cheating

[desert garnet]

lets hope they didnt use one of the improper impure models 🤬

[hallow lion]

probably they did, they told me to chill out or ill get banned and i didnt promt for ponies...

[desert garnet]

you can prompt for ponies (as long as they carry their id to check their age 😉 )

[bitter hearth]

prompts for Steven Seagal as a pony

[desert garnet]

https://tenor.com/view/huh-pepe-gif-8145044083223085060

[hallow lion]

[bitter hearth]

it uses a cinematic model

its a bit of a funny workflow because it upscales 4x with Siax but then downscales 2x

[bitter hearth]

We need

More balls

[hallow lion]

To train your SD3 model you first must take her to the gym.

For more AI related life hacks follow me on Myspace.

[lavish osprey]

cool, evil migu. Prompt?

[hallow lion]

Hot tip, for real this time.... Don't try tackling anything bigger than 1024/1024. I tried refining with SD3 (coz it has neat details dooohhhhhh) some sdxl/sd15 concoctions at 2048 and everything around the edges was a horrible mess. 😄

[lavish osprey]

webui just released SD3 support, even if just with Euler sampler

[coral sable]

High quality. A beautiful devil Miku with long red hair and red eyes stands gracefully under a starry night sky. She has large white wings and a glowing halo above her head. She wears an elegant white gown with golden accents. The stars twinkle around her, casting a soft glow on her serene expression.

Not very refined prompt, but did the job 😄

[lavish osprey]

added "anime art" and sent to Ultra.

[low stone]

@lavish osprey cascade does anime art amazingly.

[sage burrow]

Sd3 ultra also works.
Though here I thought 2b was doing better than it does!

[lavish osprey]

yep, Cascade is a cool architecture and very well finetuned.

Unfortunately it got released in the Sora period when everyone was drooling over DiT (which is at the very cutting edge of this tech and still being researched).

anyway I personally prefer SDXL over Cascade from an usage/architecture perspective.
I admit that base XL is less pleasing than base Cascade out of the box, but the base release state is not that important once you get finetunes

and XL is easier to finetune and to use (it's not by chance that the most models are still using SDXL as their base, see Kolors)

[low stone]

It's definitely more complicated to use. It does fine lines like nothing else though.

[lavish osprey]

it inherently performs refining

[low stone]

Yeah unfortunately that Kolors thing seems Linux only for now because of its dependencies.

[lavish osprey]

gonna be supported in Comfy in no time, since it's mostly XL.

but it's non-commercial, so...

[low stone]

Awesome. Hah yeah I'm not commercial so I just get to have fun with it all.

[lavish osprey]

true, but some people get angry at non-commercial models and decide it's not worth finetuning them

[sage burrow]

Via glif

[frail shoal]

are there any good sd3 finetunes yet ?

[urban arch]

The eyes remind me of Achmed, The Dead Terrorist.

[craggy crest]

whatever it was you were drinking at 4:30 am, don't drink it any more

[low stone]

Some output from Kolors without refinement with other models.

[lavish osprey]

pretty much what you'd expect of SD3 finetuned with MJ outputs

similar to Mann-E

[sage burrow]

Well this is what my GPU and glif-bot were up to last night, I'll have a word with them!

[craggy crest]

when the user's away, the bot will play

[low stone]

I would say that the output is very good, but it's not sd3 levels of prompt adherence. Even the other open source models are better at what I asked for in these examples concerning multi subject.

[craggy crest]

those are cool, though i'm not so sure abou the vomiting giant bug

[low stone]

Prompt for that one is : massive anthropomorphic gelatinous blob spraying koolaid onto enthusiastic patrons at a medieval tavern

In the other models, it comes out as more cutesy than it did here

[cunning lintel]

How do you run it, just diffusers and the repo? I used that and it seems only the default euler scheduler gives ok results, wondering if I made mistakes

[low stone]

https://huggingface.co/spaces/gokaygokay/Kolors

I was using this

[uncut river]

it's a bit unstable, but almost laying on the grass ! 😄

(im sure she'll explode if going any lower)

[low stone]

With these settings.

[gusty trail]

You could merge Kolors with SDXL checkpoint

[low stone]

Maybe without the clip part.

We'll have to see once comfy nodes are released.

[gusty trail]

You could load the checkpoint with unetloader

[cunning lintel]

Ah thanks, I'll see how they use it, prob the defaults

[gusty trail]

a test

but using clip and ksampler instead of the official glm encoder

[low stone]

Sure, but the magic is the glm encoder. That's what elevates it above what we already have

[gusty trail]

Sure

[cunning lintel]

It doesn't know groot when I tried 😂

[low stone]

But you bring up a great point. If it's just an sdxl checkpoint, this could potentially let us use existing sdxl checkpoints with it

[low stone]

I don't have Linux so I can't run it locally. Can you swap in a regular sdxl checkpoint and see if it still works?

[cunning lintel]

It's trained from scratch I suppose, totally different weights I think, but can try

[gusty trail]

The right image is 50/50 merge with my sdxl ft

[low stone]

I think your result shows it could work. Using existing checkpoints with this

[gusty trail]

I think it means the glm encoder could work with other sdxl checkpoint? or with a certain % merge with Kolors

I will test the merge to run with the official script tomorrow

[edgy kelp]

Unlike what you'd get PRETRAINING SD3 with MJ outputs, that way you'd get a mangled model that needs community fine-tunes to fix the model

[lavish osprey]

that's the difference between Unet and DiT.
DiT is "language oriented" so has usually better prompt understanding capabilities. Also DiT scale better at high number of params.

Low param DiT is pretty hard, as you might have noticed. Weorkin on it.

[cunning lintel]

: Model expects an added time embedding vector of length 2816, but a vector of 5632 was created. The model has an incorrect config. Please check unet.config.time_embedding_typeandtext_encoder_2.config.projection_dim. not exact the same unet 😉

[lavish osprey]

I didn't take part in the previous pretraining, but from what I could see from the logs there was no MJ dataset used there.

[craggy crest]

what you do not want to see when you open your fridge

[cunning lintel]

Does that mean (also saw your comment yesterday about attention) that the updated 2b model might have a slightly different architecture?

[fleet meteor]

👀 is Kolors a sd3/sdxl finetune or is it a pixart/other AIs?
PD: I saw the link you posted, nevermind 🤗

[jade mulch]

So hey,how do i use stable diffusion?🤔

[low stone]

sad trombone. thanks for trying it.

but XiaoZhi 50/50 merge being successful is highly encouraging for future possibilities.

[lavish osprey]

Kolors is sdxl with a different noise scheduler and GLM model instead of clip

trained on MJ generated dataset

• chinese typography

[low stone]

comfy has a workflow out there for merging cosxl with existing sdxl models. I wonder if that workflow carrying over the clip of the original entirely, would work. but setting 0.01 on output blocks 0-4 so the most is kept from the model you're bringing in.

or don't touch clip at all, leave that out entirely.

[frail shoal]

Explain pixart sigma that is a dit with 600m parameters. And there are already finetunes that fix anatomy. I'm guessing something went wrong during Sd3 2b training, or maybe did not train on enough data.

[low stone]

the first thing they did was make a 900m parameter version. 🙂

[lavish osprey]

correct me if I'm wrong, it's 512px, right?

[frail shoal]

No it's 256-2k

[low stone]

pixart is 512/1024/2k. but the finetuners immediately ran into low parameter count issues and someone released a base model version with 900m

[frail shoal]

That was a member of the community

[low stone]

yeah exactly

very interested in what datavoid releases for that 900m.

[frail shoal]

The guys that made the 900 version are also making a 1.2b but I'm not sure if they will release it

[frail shoal]

Not really, they just tested the 900m and found out it learned details easier

[mortal mesa]

TBF i dont see any of what Lykon is saying in their paper https://github.com/Kwai-Kolors/Kolors/blob/master/imgs/Kolors_paper.pdf

[low stone]

it was a while back but they cited anatomy issues and various other things that were problematic with 600m.

it was a set of comments on discord.

[frail shoal]

Again, not really, anatomy issues in terms of hands only. But fine-tuning fixed that. However the 900m has better anatomy if finetuned the same way.

900m just learns more efficiently

[lavish osprey]

look at the code.

[cunning lintel]

have to wait for comfy, but i just can't imagine it works, would be fun though :p

[lavish osprey]

GLM rather than T5 is interesting

[frail shoal]

For sure, I'm curious to see the results

Glm 9b vision from the cogvlm creators also it's a great llm

I'm wondering what size glm are they using

[gusty trail]

6B

[frail shoal]

Mhm, 8bit could be still very large then in terms of gb

Do you have any idea how much ?

[lavish osprey]

if I had to guess it's probably this

they started from low param count and are going up (after a lot of optimazation on that architecture size)

while the research team in SAI started from 8b (which worked decently) and went down.
but, again, I wasn't involved in previous sd3 research. I started after the previous team left, so it's just guesswork on my end

[frail shoal]

Ah that could be the key to the success then. But also could be careful dataset curation. They only trained with 10m images and have great results with only that. I'm guessing also using only the T5 might do something but I'm not sure.

[mortal mesa]

junk in junk out

[fleet meteor]

Im hungry now 😋

[frail shoal]

I'm assuming Sd3 was still trained with laion ? If that's the case it's not at all a good dataset. Captions for that are not great

[bitter hearth]

there are side effects to having a smaller dataset in the way models like Pixart do
they have less subject knowledge

[fleet meteor]

EXACTLY!

[bitter hearth]

or I should say knowledge of less subjects

[lavish osprey]

They only trained with 10m images
That's orders of magnitude lower than SD3 8b (and new 2b wip) and even inferior to previous 2b.

One of the issues of the "woman lying on grass" prompt (ONE of the issues) is that the model is trained on positions from any angle, and fails miserably when doing upside down bodies or sideways (like most models, especially if undertrained). Upright bodies are mostly fine.

One thing a finetuner could do is overfit "woman lying on grass" on upright framing. Boom, easy fix.

[mortal mesa]

im gonna become a dataset dealer, psst hey you, i got some uncorrupted pristine data here

[frail shoal]

They are undertrained for sure. But still do a large range of things with only 10m

[craggy crest]

there's a market for that

[lavish osprey]

ngl knowledge training is the most expensive part.

[frail shoal]

I thought you guys were in the billions in terms of images. My bad than. Considering that, Sd3 does pretty great than

[mortal mesa]

lol ya i was half serious, know some old photo bugs

[lavish osprey]

we are, that's what I said.

even the finetuning dataset is ~1m high quality images now.

[lavish osprey]

knowledge training is by far the biggest cost, with even tagging requiring multiple gpus for weeks (basically as expensive as training)

[odd basalt]

There really is and services for it on the darkweb 😂

[mortal mesa]

human tagging!

[frail shoal]

I'm guessing there are still other problems with Sd3 2b. I could not in paint with it when I can with pixart sigma. (It produces mushed faces etc) Also img2img has problems. The denoising slider works weirdly in Sd3. Could these be just undertrained problems that could be fixed with more data ?

[gusty trail]

" any angle, and fails miserably when doing upside down bodies or sideways" if it is true, it won't only happen in woman lying on grass. It is a lack of caption on angle?

[odd basalt]

Best solution I found is procreate or the best model of them all photoshop …

[lavish osprey]

denoising is very different for sd3 2b. I noticed 0.15 denoise is roughly equivalent to 0.35 of sdxl.

[fleet meteor]

That´s what I´ve been saying for months, Get 10k people, pay them the minimum wage and let them tag for a month, 6-8h/day

[frail shoal]

Ah sorry, understood the contrary.

[lavish osprey]

multiple reasons. IMO the biggest one is small attention size shared between text and data.

[low stone]

The one armed man did it!

[frail shoal]

I could not denoise well with it. I needed more like 0.5 to feel something changing. Also when it does it destroys the picture.

[fleet meteor]

AI captioning is not that good, it can describe the image but not which person/object is in the image

[dull star]

did you try turning down the model shift?

[bitter hearth]

human labelling in general is way worse than people think
there have been a lot of studies on this
the humans often lose to something simple like BERT

[dull star]

for me, turning model shift back to 1.00 just for the highres fix part helped it to not morph objects

[fleet meteor]

Oof, maybe that's with people who have English as a second language

[frail shoal]

I tried even removing it.

[dull star]

interesting

[craggy crest]

humans suffer from ruts - they tend to only be able to think of one or two words for any given thing, and you want a lot more than that for a label. it's not an apple, it's a green apple, granny smith apple, round fruit (etc)

[frail shoal]

I just feel that Sd3 behaves very differently from other models. That's why people are having a hard time to fix it.

[dull star]

well it is SGM scheduling, for me, sgm scheduling used to do the most amount of distortions in highresfix (~0.35 denoising or lower?) compared to stuff like exponential

(sdxl)

[mortal mesa]

mmm pumpkin bread

[craggy crest]

different neural net. they're having a hard time because they're refusing to learn to talk to it

[frail shoal]

It is a dit. It is very different from other dits

[dull star]

MMDiT, but the difference being is FLOW MATCHING

[frail shoal]

It's not like this is exact science. This is still very new

[dull star]

so it may be similar to Lumina compared to Pixart

[craggy crest]

yes, and it doesn't think the same way as other models. you start at the ground, learn how it thinks, learn how to talk to it, then you're fine.

[dull star]

Pixart is compatible with SDE samplers and all current schedulers, SD3 needs ODE(?) samplers and a specific scheduler with optional model shift

[craggy crest]

swan made of clouds

[frail shoal]

Not a skill issue sorry. The model has problems with it's training and people have to admit it. Otherwise it is never going to be fixed.

[dull star]

SD3 2B does have its shortcomings compared to 8B

[odd basalt]

Sd3 trained with photoshop

Photoshop is best with SD3 XD

[torn wharf]

It can be both

[craggy crest]

definatley a skill issue. sure the model has issues. ALL the AI image generator models out there, stable to dalle to midjourny, have issues. you learn how to talk to the AI, you can avoid those issues

[desert garnet]

you have to learn how to respect the model

[mortal mesa]

i dont want to become friends with my generative ai

[craggy crest]

now it's sad

[fleet meteor]

Yes, but the model needs more data, there are things that it doesn´t know

[frail shoal]

Pixart is compatible with all samplers and schedulers including along yours steps. And Sd3 is compatible with only one. This is why I'm saying something must have went wrong during training with Sd3. Pixart authors did not design their model to be compatible with anything out there. It just happened

[torn wharf]

Learning how to manipulate someone isn't the same as being their friends. Just have a Machiavellian relationship with the your model

[dull star]

it has to be the flow matching, training shouldn't mess up compatibility with samplers 🤔

but I could be wrong!

[bitter hearth]

sampler is quite a broad term really

[craggy crest]

sure. that doesn't mean it's not a user skill issue, however. i've worked with it enough now i can get it to give me what I want without much work

[frail shoal]

I don't think so but I might be wrong. We will see if any experienced finetuners are able to fix it.

[torn wharf]

I feel you. It seems more like an architectural problem then one with fine tuning

[bitter hearth]

sampler is not really a neat category of one type of method
there are samplers doing completely different things

[dull star]

there definitely were problems with the dataset/training (anatomy problems and weird random knowledge gaps for example), but samplers should be about the arch

[desert garnet]

if they cant its also a skill issue (they didnt know how the model brain thinks or respect it)

[craggy crest]

one thing that works really well is to feed a different prompt into each of the encoders. use their strengths.

[torn wharf]

Yeah. Recognizing where one problem lies isn't disregarding other problems

[frail shoal]

Did you test any open source model out there and compare it with Sd3? If not then you are not capable of seeing the problems since you have nothing to compare to

[odd basalt]

U have to train ur generator feed it information learn prompt engineering …. Advance ur skills let AI teach u

[dull star]

but damn is the 16ch VAE more noticeable on 2B more than on 8B

hope its just cause 8B is undertrained

[desert garnet]

you have to caress the model,lick it