🆕｜sd3

[craggy crest]

that guy's for 4 lora's out for 3.5

https://huggingface.co/ariG23498/sd-3.5-merged

[bitter hearth]

merge between models large and turbo?

oh damn

[craggy crest]

and there's this https://huggingface.co/argmaxinc/mlx-stable-diffusion-3.5-large

that's a finetune

[bitter hearth]

is it me or the turbo 4 steps looks crispier than merge?

[craggy crest]

i'm biased

[bitter hearth]

i really have no issue with turbo as is

speed, quality

[craggy crest]

me either

i know how much work went into it

[bitter hearth]

good effort 🙂

[craggy crest]

yes

[bitter hearth]

btw the higher than modelsampling node value is the smoother image looks .. high values would even make image look rather too smooth

[craggy crest]

i will probalby never use that node, actually

[bitter hearth]

you should

it gives you fine tuned artistry

and value would range from 1-5

i should check how it looks below 1

[craggy crest]

;) i can get that out of base with just the right prompting

and adjusting shift and cfg if needed

[bitter hearth]

0.6

[craggy crest]

ostrich egg

[bitter hearth]

i think its safe to say value should be between 1-4

[craggy crest]

deis and beta for sampler/scheduler

here's a prompt to play with: photo of <SUBJECT> riding a majestic eagle through the skies above a vast forest, with the wind rushing past him and the trees stretching out far below. He wears a leather aviator jacket with a fur collar, goggles strapped over his eyes, thick gloves to protect against the cold air, and a scarf that flaps dramatically in the wind. His expression is one of exhilaration and freedom, capturing the thrill and majesty of flight.

[turbid grotto]

didn't touch it yet

[turbid grotto]

btw, are you using sd35lturbo with cfg 1 or 2?

[bitter hearth]

turbo with cfg 1

thats' schnell

trying out the modelsampling

[craggy crest]

SD3.5 large, steps: 16, cfg: 3, shift: 2, sampler: deis, scheduler: beta

with the photorealistic lora i posted the link to earlier

[bitter hearth]

so what im seeing from the image i render, trying to control character's pose via prompt is not always great

[craggy crest]

Thats the reason for open pose. Ipadapter with controlnet for 3.5 is being worked on.

[bitter hearth]

you know that wasn't a issue with sd1.5

or sdxl even

[craggy crest]

Sure it is. Thats why ipadapter and open pose was created in the first place

[bitter hearth]

if i leave body poses completely upto the model it can create decent images like this one, but any indication to instruct it for specific pose can cause anomaly

[craggy crest]

Im on my phone, so this'll be slow, but prompt? Settings?

[bitter hearth]

a woman in casual white shirt and shorts.

default settings on turbo

if you add a woman in casual white shirt and shorts, relaxing indoors you'l start to see some strange outputs

[craggy crest]

What, specifically, does "relaxing indoors" mean?

[bitter hearth]

what do you mean

she is chilling in her home

its just broad specification about indoors - outdoors

[craggy crest]

You are talking to an artist that has lived all its life in a box, cant read your mind, and only knows about the world through images it was shown. Thats a generic phrase that could mean a lot of things, and is confusing. Be more descriptive

Or itll guess, and you wont like the results

[bitter hearth]

think of it a broad generalization to let the Ai run its possible outputs

[craggy crest]

Re-read what i said. Thats exactly why you are getting results you dont like

Its a computer, not a human

[bitter hearth]

you cant strictly specify every tiny detail

that would be too rigid for ai to run its scopes

and not a good approach for artistic styles

for example you can specify a blonde woman in red dress or a woman in red dress and the Ai would have the flexibility to give you diverse hair colors

[craggy crest]

You could, but you dont need to. However, you will get strange if your prompt is generic. Use open pose if you want to be generic and use refrence pose images

[bitter hearth]

this is not too odd to ask a woman in casual white shirt and shorts, relaxing in her livingroom is it?

but what i found out is that any indication of body poses can render strange result

[craggy crest]

Prompt: a blond woman leaning against a tree trunk, one hand brushing hair from her eyes

[turbid grotto]

because standing is easy to generate but other poses needs more training or aligning because the are more rare in training data

[craggy crest]

Tell me what you think relaxing in her living room looks like. Remember, im a machine without any udea of what that us as ive never done that

[bitter hearth]

yeah im not compalining about it per se, im just pointing out that the common issue with poses is still prevalent

[craggy crest]

Because the promot is too general and generic

[bitter hearth]

thts' the thing about prompt understanding, it could have been better trained

[craggy crest]

You did not soecify a pose, though, thats the point

[bitter hearth]

relaxing hint towards wide arrays of body pose

[craggy crest]

Use a good prompt, itll have no problem

[turbid grotto]

even with specifying "lying on the bed" anatomy is bad

[bitter hearth]

lol i know but this is not a complaint

[turbid grotto]

we need more tuning and a bit overfitting

[craggy crest]

No it does not. Not even to a lot of humans. Especially not to a computer

[bitter hearth]

this is my expectation for something better

[craggy crest]

No, we just need to use clear, specific prompts

[craggy crest]

You wont get what you expect, if you do not clearly communuicate what you want

[bitter hearth]

you can't fixate language in specific order, thats not organic, also why LLM are actively worked on to improve human interactions

[turbid grotto]

can you show example of such prompt for "lying on the bed"?

[bitter hearth]

its more about understanding complexity rather than making it complicated

[craggy crest]

Even a human artistvwiukd have a hard figuring out what you were actually after with thst pharse, snd humans have an imgination. Computers do not

[bitter hearth]

no he won't .. its a broad artistic freedom

i would be ok with any pose or any styles, but the deformity

[craggy crest]

Sorry, but please understand that you are actually programming that machine with your prompt. Make it clear, specufic, and concise

[bitter hearth]

thats where you have to push the limit if you want to expect a functioning AGI

[craggy crest]

Computers dont function that way

[bitter hearth]

you are avoiding the objective of Ai

what i think of Ai is far more potent than human collective brain

that is expected, when i dunno so im just pointing things along that line

[craggy crest]

That isnt what AGI is, and you favorite image gen is in no stretch if the imagination AGI

[bitter hearth]

this is about AI

you need to work towares AGi through Ai

[craggy crest]

This is about getting a correct result out of a tool

[bitter hearth]

i admire you technical understandings but sometimes you push to defend flaws why?

[bitter hearth]

if you are saying this is the limits of sd3.5 i acknowledge that fine, but thats not good enough, and that's what im suggesting .. something better maybe something to work on

[craggy crest]

AGI= artifucal general intelkegence. I E. An ai that can do stuff it wasnt specifically trained for. Not an image generator specifically trained to create images.

Youre not going to magically make AGI out of stable diffusiin by being unclear with your prompts

[bitter hearth]

technically AGI would involve all modality in humanoid form but Ai can be digital depiction of our mind interactions

[craggy crest]

Nope

[bitter hearth]

thats cause we haven't reached a phase to call any AI module developed enough to function on it's own

[hardy surge]

AGI is the endgame of AI and is more an ethical and philosophical construct than an actual usecase.

[bitter hearth]

please dont paste any wiki texts

[bitter hearth]

humans need to work on their social system and ethics by principles before they can dictate Ai which would have far greater intelligence capacity than collective human brain

[turbid grotto]

don't get me wrong, I like sd35l and see potential, but it has obvious anatomy problems which was not so prominent for sdxl, relatively to it's size

[bitter hearth]

people are klling each other over religion, economy, and politics, please dont train AI with all that nonsense ethics

[hardy surge]

Yeah. Agi will literally make the human race irrelevant in terms of cognitive progress.

[craggy crest]

Why would i paste wiki texts?

[bitter hearth]

just saying

[craggy crest]

No it wont

[bitter hearth]

it would

why are you saying it wont?

human organic brains dont have the bandwidth to match Ai

[hardy surge]

That’s the whole point of a fully developed AGI and also the point why it’s unlikely to ever fully happen 🙂

But we will get close. And that’s a problem for many.

Specially in a hyper capitalist world.

[bitter hearth]

how Ai will evolve past human reach is that Ai will reach a point of functioning on it's own well enough to train itself

[craggy crest]

A cat is NGI =natural general intellegence. Agi is just an artificial cat. A baby is ngi. Agi is just an AI that can do stuff it wasnt specifically trained to do. Not "super smart" it could very easilly be very dumb

[hardy surge]

It’s like my parents told me: you can be whatever you want to be. 😂

But as an AI.

[bitter hearth]

you are hard focused on the limitations, which i understand you have to take into account but you are not doing that to push past those limations, you seem to settle with those limitations

[dusky thistle]

the problem with the agi vs human intelligence debate is the part about human intelligence

[hardy surge]

Think of the matrix. Best example for AGI.

[craggy crest]

Learn to prompt correctly, or use tools like open posr

[hardy surge]

That is for sure debatable 😂

[bitter hearth]

i hope you live well and i hope to pick up this convo with you in 2030

[hardy surge]

An agi will for you 😂

[dusky thistle]

i wanna be a bot when i grow up

[hardy surge]

In times when the internet became obsolete

[hardy surge]

Aahahahaa this

Guys any useful controlnets for sd3.5? For depthz, cannyedge, normals?

[dusky thistle]

it's been like 3.5 days

[craggy crest]

Some people need to stop listening to the media and do some research

[craggy crest]

Its being developed

[hardy surge]

Lately things move fast

[bitter hearth]

i totally dont 🙂 but i do read some articles that put forth some interesting concept on possibilities

[hardy surge]

Cool! You got any leads?

[craggy crest]

It has to train

[bitter hearth]

and i would chime in to say avoid hollywood nonsense

[dusky thistle]

any cnet that drops 3 days after the base model is bound to be trash that someone rushed out just to be "first" and get PR for their startup etc

[craggy crest]

Yes. And no i cant tell you more than that

[craggy crest]

Yes you should

[hardy surge]

A business partner studies ai ethics in Cambridge about this topic, they do some interesting research about agi and human adaptation towards this topic. We are already at a point now where the possibilities of ai need 10 years of human development to unlock all tools we can gain from them to boost our performance. So anything from now on is already future tech.

[turbid grotto]

[bitter hearth]

yeah .. we are already in it.. and things are moving fast

but we dont have any AI module that can function independent of human inputs yet

[craggy crest]

And obsolete by the time it hits the media

[hardy surge]

Agents are the first move towards this. Just needs the learning curves and adaptability to think rather intuitive by combining streams.

It’s all in the beginning of it. ChatGPT remembering complex conversations, Claude using human user interface, agents for crm.

[craggy crest]

We do too. Yohe has created several

[bitter hearth]

there are also guardrails in place that prevents Ai to learn things on its own exponential pace

[craggy crest]

Nonsense

[dusky thistle]

it'll happen one way or another eventually

multicellular life never gave up

AGI has billions of years left before our planet is toast

[hardy surge]

Sounds like the Blackwall from cyberpunk phantom liberty.

[bitter hearth]

no we dont have Ai that can function w/o human inputs

if you mean turning the switch to let Ai do its things thats not what i mean

we dont have systemic grid to allow for ai to function on its own

its kinda like conceptualizing bullet trains on a muddy farm land

[craggy crest]

a blonde woman sitting on the wooden dock, dangling her feet in the fish pond. She is holding a rose in both hands

[bitter hearth]

i have had sexier pics with sd1.5 with those kinda prompt

[craggy crest]

Yeah, we do. Yohe has created several automomous agents

[craggy crest]

Ok?

[dusky thistle]

and even if we didn't have it yet... it would happen eventually

[bitter hearth]

such as what.. im curious

[dusky thistle]

AGI has billions of years left to evolve

[craggy crest]

You can go look him up and research

[dusky thistle]

only way it doesn't is we nuke ourselves

[hardy surge]

Would you guys say with the last 3 days that you prefer the outcome of sd3.5 over flux1 from a lookdev & aesthetics point of view?

[dusky thistle]

the default aesthetic yes

[craggy crest]

Nano-light years?

[dusky thistle]

i'm really not a fan of the default aesthetic of flux

[mortal mesa]

mwashed out

[dusky thistle]

it's DPO'd to hell

[craggy crest]

Greatly prefer 3.5

[hardy surge]

Always looks like high cfg

[hardy surge]

Okay nice

[dusky thistle]

much more diverse outputs with 3.5

[hardy surge]

The license model for enterprise with stabilityai is also a lot easier to handle.

[mortal mesa]

AI is gonna make new medications to "help" us

[hardy surge]

That’s great to hear from all of you. Thanks.

[dusky thistle]

and it's a real base so we'll see real finetunes

[craggy crest]

You dont want a 12 billion parameter lora?

[dusky thistle]

i've tried finetuning flux, it's a nightmare tbh

[hardy surge]

Actually funny

[craggy crest]

Theres already one on huggingface

[bitter hearth]

just for clarity the possible outcomes of an Ai that is developed enough to function w/o human inputs would also be surpassing the collective human intelligence, and for that to happen you will need entirely different system grid but if you are talking about something like Ai beating humans in a game that's not what i mean, those are limited areas

[craggy crest]

Thats what flux is

[craggy crest]

Wrong

[bitter hearth]

im beginning to think your definition of Ai is quite different from mine

[dusky thistle]

these are all sd35 base

[hardy surge]

Amnesty international isn’t it

[dusky thistle]

it's just got a better sense of style

[hardy surge]

That’s clean

[dusky thistle]

a lot of issues with coherence can be cleaned up easily with SDE sampling too

[bitter hearth]

have you looked into modelsampling node, that can influence image tones a lot

[craggy crest]

You realize hes a programmer?

[bitter hearth]

sure, but im asking if he looked into it

[mortal mesa]

ya lol im giggling, if he only knew what his workflows and nodes were like

[craggy crest]

Right?

Like telling matteo about some minor node

[bitter hearth]

well that wasn't intended as you are making it look like

[mortal mesa]

i just thought it was funny, that guy is a pretty advanced user by far, i know you didnt know

[bitter hearth]

i wanted to know your definition of Ai and you posted that image, that doesn't explain much to me

[dusky thistle]

got my own version of it that allows you to pick between SD35 timestep scaling and flux timestep scaling

[bitter hearth]

nice

[craggy crest]

I told you to go look up yohe

[bitter hearth]

i dont have that node yet

[dusky thistle]

threw it up on my res4lyf repo tonight

[bitter hearth]

ok!

[craggy crest]

We're top of the leader board, or at least we were earlier, on huggingface

[dusky thistle]

what is on top?

sd35L?

[craggy crest]

https://x.com/StabilityAI/status/1849956236445483317

Stability AI (@StabilityAI) on X

We're so back 🤗

Twitter · Oct 25, 2024, 11:28 PM

[dusky thistle]

awesome

glad there's as much interest as there is

SAI deserves a lot of credit for this one imo

[craggy crest]

Yes.

[dusky thistle]

2b was a total lemon but this is pretty great

[craggy crest]

Definately

[dusky thistle]

it was fun training/sampling my way to convincing photos with flux, but just beyond that and the prompt adherence, just not a fan of the aesthetics

[craggy crest]

https://huggingface.co/models?other=base_model:adapter:stabilityai/stable-diffusion-3.5-large

[dusky thistle]

this however... loving it

[craggy crest]

Loras, a fine tune, a merge... already

[dusky thistle]

Nero's got something cooking on onetrainer too that will allow finetuning SD35L on a 24gb card without the big slowdowns of block swaps

[craggy crest]

Really happy with the community response

[dusky thistle]

don't know anything beyond that it's apparently going to happen soon-ish

[craggy crest]

Thatll be cool

[dusky thistle]

for sure, gets some multigpu support going with something like that... get 8x4090s and you're in fn business

[craggy crest]

Matteo's got stuff cooking too

[dusky thistle]

matteo's always cookin'

[craggy crest]

You working on anything special?

[turbid grotto]

sd35l has the best look compared to any other models form y opinion

[craggy crest]

Agreed

[dusky thistle]

not yet on the training side

waiting for the 5090 atm

[turbid grotto]

all we need is additional anatomy tunning and it is dream model

[dusky thistle]

at a min i'll get a pair and use one for the conditioning and one for the unconditioning to double CFG speed lol

i'm sure ppl will like that... oh, yeah, just grabbed a spare 5090 here to handle my negative prompts

[craggy crest]

Theres a really nice photorealistic lora on that huggingface link for that now

[dusky thistle]

what i'd like to do then is get some training code going that shuttles the latents back and forth between staggered blocks of the model loaded alternately on different gpus

[dusky thistle]

theoretically i think you could do that and suffer almost no performance hit

just run a batc hsize equal to the number of cards

[craggy crest]

Assuming you have more than one gpu

[dusky thistle]

yeah, that's the plan

just not crazy enough to load up on cards right befor the next gen drops

i've got a 25 amp circuit in the basement doing nothing

perfectly positioned to cut a hole in the wall and connect into the central air to exhaust the heat to heat the house in the winter

[craggy crest]

Made you a new avatar @dusky thistle

[turbid grotto]

that is cool but I really want to train something myself, waiting for tools and methods to mature a bit

[craggy crest]

Yeah, hasnt even been a week yet

[turbid grotto]

i been told that at nf4 and 512, lora can be trained with under 8gb vram already

[noble coyote]

3.5L Turbo LLM (Meta's Llama3.2 - about 30% slower than Llama3.1)

Don't know yet if it's better than 3.1?

[turbid grotto]

3.2 is the same model but with vision?

[noble coyote]

I'm using 3B - vision models are 11B and 90B

I might try the 11B soon ...

[signal shuttle]

How is 3B slower then Llama 3.1 8b?

[bitter hearth]

they have quantized version of 3.2 which should be faster

[noble coyote]

On my PC, Llama3.1 is 37 seconds/iteration. Llama3.2 is 49 seconds/iteration.

In 5 step Turbo

[noble coyote]

Send me link? 🙂

[signal shuttle]

Are you using quants?, since llama 3.1 q_8 on my computer is 80t/s

[bitter hearth]

https://ollama.com/library/llama3.2/tags

[noble coyote]

I am using Ollama.com Q4_K_M - so it is quantized

[signal shuttle]

What is your computer gpu+cpu?

[bitter hearth]

ok, but just wondering why did you choose q4 when you could use q8?

[noble coyote]

At Ollama.com I don't see a Q8

[bitter hearth]

there is, if you expand the list

[noble coyote]

Found it!

... downloading ...

[bitter hearth]

nice, and i was thinking maybe q4 had some advantage that i could replace q8 with and save up space and memory

[noble coyote]

Good - 25% faster!!! 45 seconds/iteration instead of 60

[bitter hearth]

aweome, i was using the 1b q8 model and i noticed some speed increase too

[noble coyote]

3.5L Turbo LLM (Llama3.1)

[noble coyote]

3.5L Turbo LLM (Llama3.2:3b_instruct_Q8_0)

[turbid grotto]

will we be able to make turbo model from finetuned one's in the future?

[signal shuttle]

(SD 3.5L)

(SD 3.5L)

(SD 3.5L)

[icy drift]

There is no way to get this running locally other than Pinokio. Comfy doesn't support it. The diffusers version is broken. Local install fails. Only Pinokio works (at least as of yesterday).

[icy drift]

Omnigen altering a person's age much younger:

Slightly older:

Much older:

Omnigen can transform the material composition of an object. (I had to read the original paper in order to figure out how to prompt the model. I tried many prompts that failed before understanding how to do this.)

It understands material deformation physics (it can conform a generated material to geometry that it infers from the input).

(I am using a guidance scale of 3 and 20 inference steps. This model might give more photorealistic results with other settings. I will test that later, but you can always refine these images with Flux.)

[bitter hearth]

looks good

[icy drift]

It can integrate object / person inputs.
This is actually the most impressive use of inputs I have ever seen out of an AI model. This blows IP Adapter out of the water. Also, check out that perfect hand and grip??!!
All that said, it did not follow the prompt perfectly, because she is not taking a drink.
I will try rerolling it.

Results just as good, but still not drinking. I will try rewording the prompt.

So far, Omnigen seems great at extracting features from a person, but if you want it to extract features from an object, you need to tell it.
These results are shockingly more impressive than before. No joke, this thing understands physics and solid objects WAY better than Flux.
Still no drinking though. Moving on.

[bitter hearth]

hands are right yeah

[icy drift]

And in all honesty this photo quality is top-tier. Not necessarily the textures or aesthetics (although I haven't experimented with the settings yet), but just the solidity and anatomical details. (The eyes are messed up though.)

[icy drift]

I don't think I've seen any model get hands of this quality, and it does this effortlessly. (Even though that's not its main selling point.)

[bitter hearth]

could you try a few sci fi or fantasy prompts like a dragon or a spaceship

not sure if it only knows photos of people

[icy drift]

Added to list.

First try. The hands are definitely not perfect this time, but better than Flux. And again, this is better than Loras and IP adapter.

I mean seriously... Those are the same people. Reposed and relit following my prompt. Can we do manga now?

[turbid grotto]

all that done by 3b model???

[turbid grotto]

can it generate something with that spongebob?

[icy drift]

This is OmniGen, which is a 3.8B parameter model, yes.

[turbid grotto]

awesome but it seems to be suuuper heavy

slow to be precise

[icy drift]

I'm not sure what you mean by heavy. Right now, the only way to run it is Pinokio, and it is completely unoptimized. If Comfy supports it, this will be blazingly fast.

Effortless style from text.

[turbid grotto]

and we need quants

[icy drift]

Yes, FP8 should take up less than 4GB VRAM.

[turbid grotto]

wait, then I can run full precision

the model in repo is 16gb, is it in fp32?

[icy drift]

Full 32-bit precision is 15.2GB, but the Pinokio code is putting a bunch of encoders and context into the VRAM.

[turbid grotto]

can it completely change form, for example, transforming into voxel style?

[icy drift]

Added to testing list.

[icy drift]

These generation speeds are so inconsistent...

Failed extreme watermark removal (but a small one works fine).

@turbid grotto Yes, It can infer geometry and then simplify forms. (I did not expect this.)

It can handle extreme relighting effortlessly.

[rapid pivot]

try something very weird

that doesnt even make sense

[icy drift]

I am happy to add things to my test list, but your recommendation is a little too vague.

This model is definitely not magic. It takes careful prompting to get what you want.

[rapid pivot]

Hmm
First thing that came to mind is to make them metal sheets with glows of angel halos

[icy drift]

If I don't understand the prompt, I can't know if it failed though? (What does it mean to make them metal sheets.) And it fails very easily.
For that relighting test, first I tried "change the background to an underground volcano", and it just copy-pasted them as if it had cut them out in photoshop, without changing the lighting at all. You could even see the cut-out lines around the guy's hair.

Minimalist logo redrawing.

[rapid pivot]

Did you try ?

[icy drift]

No, it's very slow to generate one image on my PC. But I'm 100% sure it could change their texture to metal and add halos. It has already passed similar tests. (You can scroll up to see what I've done so far.)

[rapid pivot]

[icy drift]

It lacks aesthetic quality, but it can generate a fantasy subject.

[icy drift]

Urgh. Here's why I'm definitely not going to try random nonsense. This model fails so easily. Zero change here. So much wasted time. (It doesn't fail randomly, it's just very sensitive to how you word the prompt.)

[rapid pivot]

is it a very slow model to run or is your pc like mine

[noble coyote]

Just d/loading Pinokio (again!) OmniGen looks fantastic. It can take two photographs with a prompt like "take the middle character from photo1, and the character on the left in photo2 and make a new photo with them together!"

[icy drift]

I do not think it can repose a dragon. I suspect it could for an android. I will try with a specific cat next.

[icy drift]

Yes, it actually aced a similar test like you won't believe. Better than loras.

It can reposition a cat while preserving features not specified in the prompt, but it is less accurate than with people.

[bitter hearth]

this one is amazing

didn't even realise that would be possible this year

[icy drift]

Same here. I still can't believe it. And that was a first try. No cherry picking.

The quality is its biggest weakness. I will experiment with settings next.

[turbid grotto]

it is more low poly than voxels but very good

[icy drift]

Passes across the board...

[icy drift]

Yes, that's what I prompted for. I expected it to fail, which is why I threw so many terms in there.

[flat oracle]

understand, thanks!

[icy drift]

An amateur, realistic photo of a woman taking a selfie on the porch of her house. She has freckles, and is smiling slightly. Her hair is a little messy.
20 steps, locked seed (123456789), Guidance at: 1, 3, 7, 10
For photorealism, guidance increases contrast, clarity, quantity of small-feature details, and prompt adherence.

No combination of guidance and steps seems to produce good anime results. This model would need fine-tuning for anime.

It can do ethnicity changes fine. (Although the model will prefer asian faces by default.)

On my PC, without optimization, 2048*2048 looks like it might take over 15 minutes per generation. (But I killed the generation after 1 step, so I don't know.)

[rain current]

(SD3.5L TURBO Q8)

[icy drift]

OmniGen's current code does not support 8-bit floats or quants. Someone much more knowledgeable than me would have to modify it.

Gotta go. I will try leaving a 2048*2048 generation running, at least to see what kind of quality it comes up with.

[noble coyote]

Still d/loading 15.5 Gb checkpoints

[icy drift]

Nevermind it only took 3 minutes. There are definitely artifacts here, although different from the normal diffuser artifacts I'm used to seeing. 2048x2048. In that case, I will try a 4096 if it will let me? Hmm... Nah, no point. It already can't do 2048.

I will try telling it to upscale.

[turbid grotto]

can it "fix hands"?

probably too specific

my little test of first sd35l finetune!

[split bramble]

comfy released the version of 3.5 large with the encoders built-in ... does anyone know if they did the same with Turbo?

[turbid grotto]

sorry, I don't know but what a point of it?

[split bramble]

takes more disk space, but keeps things a little cleaner and simpler to use.

[turbid grotto]

oh yea, one node for all things

[split bramble]

Yeah. So trying to see if there's similar for Turbo. Doesn't look like they released one.

[noble coyote]

Opened a ticket with pinokio as OmniGen errors out each time

[turbid grotto]

another comparison\test

[noble coyote]

SD3.5L Turbo LLM (Llama3.2)

[oblique parcel]

featherlion

[icy drift]

Aw, too bad. Is there a Comfy issue requesting support yet?

[noble coyote]

Pinokio is the only working OmniGen - I updated SD Next - but apart from having a snazzy new GUI - its fallen short of supporting OmniGen. I'm sure Comfy.org are on the OmniGen trail as we speak ... 😄

[noble coyote]

When you start pinokio - does it d/load 10 new files from Cuda each time (like mine does!)? 🙃

[dull star]

yoooo

[oblique parcel]

up to your imagination if hes skydiving without gear or if hes a giant

[dull star]

impressionist oil painting by Claude Monet of A man in a forest whilst sitting by a tree. The man is shirtless and is barefeet. He looks frail and cold. There is a lot of mud and in front of the man there is a pond. The weather is overcast and the pond has ripples on it. The scene is dramatic and depressing. The man is looking down in sadness.
@cunning lintel Flux - Pixelwave finetuned model

the arm and knee is melting together though but aside that this one is nice

I wonder if I just got a lucky seed haha

⚠️ close to nudity but doesn't expose anything

16 steps with dpm++2m and beta scheduler

huh...

also there is a fix for euler_ancestral that makes it work with rectified flow models

[noble coyote]

SD3.5L Turbo LLM (Llama3.2)

[dull star]

you're not supposed to ignite the tip of it!

but this isn't very painterly like, the guy has perfect hands or it isn't hidden in his pocket

[dull star]

nice

[pseudo owl]

prompt is kinda dumb but somehow mochi made a nice video still(not mine)
prompt: make an ai video about me as a future marine engineering and talking about my success

[dull star]

yeah wow this is nice

for an offline model, man

did it take like more than 10 minutes

I heard it took 30 minutes on a 4090 for someone

[pseudo owl]

Yeah I can't wait 1hr for one vid lol(my gpu isn't that great), I got it from the genmo website which uses the same open model + an upscaler. It only takes like 3-4 min there and they also show the video each step.

[dusky thistle]

SD3.5L is fantastic.............

[dull star]

if it were to take 5-6 minutes on a 3090 I'd generate some vids myself, especially if there was vid2vid

[dusky thistle]

looking through a bunch of overnight generations right now

huge hats off to the folks at SAI

so much more diversity of style than flux

cant wait to train this thing... gonna start this ewekend

and the watercolors are really great

[pseudo owl]

Yep same, either I use fal(which also uses same model but less steps and no upscale but takes 1 min) or their offical website which uses more steps but 3-4 min. There is a lot of optimizations left and the employees also said vid2vid, img2vid should come soon.

right now I think with lots of experimental optimizations that Kijai implemented, it takes like 5 min on a 4090.

[dull star]

I want vid2vid to make video games look like live action, like those GEN-3 videos, but open source

cogvideo would have been fine but I don't know how good the comfyui plugin implementation is

[pseudo owl]

CogVideo is actually pretty great too, mochi is clearly much better in text-to video but CogVideo supports so much more(trajectories, img2vid, vid2vid, and more)

[gentle mango]

im in need of some assitance, all of my images come out blurry regardless of what i do

[noble coyote]

Not enough cfg or not enough steps?

[gentle mango]

cfg is 7 and steps i use between 20 and 40

[noble coyote]

what is your workflow? post a png here

[craggy crest]

turn that cfg down to 3.5 or 4 and steps to around 32. and if you're using any flavor of sd3, set your shift to 2

@winged seal list of loras already on huggingface https://huggingface.co/models?other=base_model:adapter:stabilityai/stable-diffusion-3.5-large and one finetune so far under the finetunes tab

[winged seal]

I wonder how much work training out the dream shaper aesthetic will be in 3.5

I haven't had the time to get AIT fully set up, and I'm not even sure what dataset to use to try and do that 😅

[winged seal]

Oh wow, you pinged me right on time lmao

[craggy crest]

i saw you typing and knew you'd get a chance to see it before it got buried

[winged seal]

Nice lol

[craggy crest]

it went from 28 yesterday to 34 today

so watch it frequently

[winged seal]

I do have goals to try it soon, just been dealing with more emotional/personal problems

[craggy crest]

it's a dynamic list, just bookmark the URL

[winged seal]

@craggy crest and by the way, my boss went through the chat and didn't find me saying anything bad. I just hope we don't try and pull him back into things in the future 😅

[craggy crest]

anything we can do to help?

[winged seal]

Oh no no no, it's not anything people here are responsible for

[craggy crest]

he's on my friends list. don't act out, i won't have to tell him to read the chat

i'd prefer you and I were friends and could work together, quite frankly

[winged seal]

What I'm saying is he agrees I didn't do anything except provide insight as a well seasoned trainer, and relay the information of my research partners and their findings. There wasn't really any act out

I just don't want any of my future results to be lumped in with the accusation that I am financially incentivized for flux to do better than SD3.5, cause I'm not by any means 😅

[craggy crest]

he and i talked.... we'll leave it at that.

[winged seal]

The insinuation that my boss would lie to me is not exactly appreciated, but that's not that important. Anyways

[craggy crest]

not insinuating anything, just letting you know i already talked to him. let's consider the incident closed and move forward?

[winged seal]

I like the aesthetic of the first two. Now my general wonder is how does one fix the really plastic/fake same face and still allow for that aesthetic/style underlying? That's gonna be a real challenge

[noble coyote]

Prompting

The PNGs hold the w/f

[winged seal]

Do you have an example where it's fixed?

I'd love to see one, if anybody does, cause that's a pretty huge hold up on 3.5 for me

[noble coyote]

How do you mean fixed? Same face/different surroundings?

[craggy crest]

on that lora's page, there's a really nice photorealistic lora that works well

[winged seal]

Ah alright, thanks. I don't currently have it running at this moment but I'll save them for later

[winged seal]

That's not what I was asking for or wanting tho

[craggy crest]

i know, but you might try it as it helps with what you were asking about

[winged seal]

Photorealistic implies it's a baked in style, no?

[winged seal]

Same face/plastic face specifically. The goal is to still have the style of the image (like the overall aesthetic) but with faces that look less synthetic and manufactured

[bitter hearth]

refine with SD 1.5 is what I do :shrug:

[winged seal]

Does that fix the facial structure tho?

[bitter hearth]

if you use a noisy enough sampler yeah

[winged seal]

Fixing facial structures is easy, like with flux, but that always inherently changes the style to be more photographic

[bitter hearth]

very minute changes in sigma matter a lot for img-to-img
sometimes you need to start on a very specific sigma

[winged seal]

Ok, so it looks like 3.5 doesn't like multi res images, that's a monumental bummer

[bitter hearth]

sorry what's a multi res image?

[winged seal]

Training on multiple resolutions of the same image or a dataset of multiple resolution groups

[craggy crest]

i used it with 3d pixar style prompts, and it had a nice effect

[noble coyote]

ddim/ddim_uniform don't work!!!

[bitter hearth]

oh I see you mean training

[noble coyote]

with SD3.5L

[winged seal]

For example, I train flux on 256, 512, 768, and 1024, which makes the results much more robust and take a lot less time

[bitter hearth]

yeah most models in the initial training phase spend most of their time at 256, this is best technique currently probably

[winged seal]

Compared to just 1024x, it takes about 35% less time, and gets less stuck in the same composition

But it seems as though SD3.5L is too stable in its current state to take proper advantage of multi res images. Maybe later once it's patched up it will be more viable

[low stone]

@bitter hearth hey there, did you ever successfully get flux dev fp16 to tensor on rented hardware and get it back to a local 4090 machine where it worked? I got it working with that flux-lite-alpha thing because it was 16 gigs, but the visual difference is rather significant.

[bitter hearth]

no I forgot to do that, sorry
I would have to look up how to save the tensorrt model cos I don't actually know that

[noble coyote]

ddim/ddim_uniform worked the second time in 3.5L Turbo

[bitter hearth]

oh the stock nodes have save load, its easy
I've been using torch.compile a lot lately instead personally

[noble coyote]

3rd output ddim/ddim_uniform - top-half is poor

[bitter hearth]

would not recommend ddim

[noble coyote]

I'm learnin' 🥳

[winged seal]

For anybody using flux, euler makes flux look really bad too

Ipndm-v is ideal for flux in my findings. Much bette colors, contrast, details and color temp

[oblique parcel]

album cover or something

[noble coyote]

My Flux looks OK using Euler - so I will change up and see what gives

[bitter hearth]

some people have got stochastic samplers working with Flux

[noble coyote]

I use Flux Super - mebbe 3 x KSampler compensates for Euler?

[reef jacinth]

Are all loras for models older than 3 incompatible with 3.5?

[bitter hearth]

3 ksamplers will help a lot yeah

cos each ksampler adds some noise

its a bit like noise injection

[noble coyote]

"I hate stochasm!!!"

[bitter hearth]

lol

[winged seal]

I would assume to some extent, but I would recommend against the, just because both models are unstable in their own ways

issues that needed to be overcome in 3.0 will be overcome in 3.5, which means the fixes may over-express and give some very weird issues

@craggy crest did the last little push and I am trying my first run with 3.5 now

[low stone]

what kind of speed increase that does that give you?

tensorRT is literally double speed.

[winged seal]

Downloading the model

TensorRT speed increases are different from cars to card, and model architecture

[low stone]

on a 4090 it's double

[winged seal]

My 3060ti always got a good bit bigger of a boost than my 3090

[noble coyote]

Is there a TensorRT for 3.5L/3.5L Turbo?

[low stone]

it doesn't work yet. just outputs noise or black boxes.

[winged seal]

I doubt it this early on

[low stone]

i've opened an issue on comfy's repo for tensor about it.

works with sd3m, just not the new sd3L

[winged seal]

And I can only imagine how long the compiling will take 😅

[bitter hearth]

I haven't benchmarked it, its significant but I am not sure if it matches or comes close to tensorRT

[low stone]

not that bad. takes about 12 minutes on other 16 gig models like auraflow / flux-lite-alpha which are the same size.

[bitter hearth]

the latest pytorch added a speedup to torch.compile apparently

[winged seal]

12 minutes per resolution per batch size 😅

Did they at least fix it to where adding LoRA's in doesn't mean you have to fully re-compile?

[low stone]

yeah, i'm always only doing 1 image at 1344x768, so it works out for me

[winged seal]

That's fair then honestly

[low stone]

lora doesn't work still, have to merge those in if you want it THEN do the tensorRT treatment

[winged seal]

I see the benefits

[winged seal]

Yeah, that makes it basically useless for me then. That's why I never focused on it working before

[bitter hearth]

if you don't use tensorRT, could still use torch.compile its much faster compile time but for a smaller speed boost

I think its mostly good to use one or the other

[low stone]

well, at least with flux I've got 3-4 loras that I usually merge together in with the base model. I have to do that anyway because with full flux dev fp16, it won't fit with all those loras in 24 gigs anyway. so I have merge so it'll fit. so TensorRT'ing that thing for double speed would definitely be worth it for me.

[winged seal]

Ok, so it looks like training SD3.5L is about the same speed as Flux. I was worried it would be slower

[low stone]

is there an example page for torch.compile?

[bitter hearth]

this is all I saw https://github.com/comfyanonymous/ComfyUI/commit/d0b7ab88ba0f1cb4ab16e0425f5229e60c934536
I browse the new commits to find new stuff

[winged seal]

It seems to be a smidge faster than training flux

Maybe about 10% faster?

[bitter hearth]

which training tool are you using?

[winged seal]

Which is an improvement

AIT

[bitter hearth]

ah ok

its a shame its not 3/2 faster, given that it's 2/3 of the param

[winged seal]

It should be noted, flux has a huge one up here, which is holding back SD3.5

If SD3.5 reaches a point where it can handle multi res inputs, it will likely train way faster

[bitter hearth]

yeah cos you could drop down to 256

[opal bay]

oh @noble coyote theres never really a good time to use ddim, i dont think its taken into any consideration by anyone when making models or UIs, its just a legacy thing tbqh

[low stone]

thanks. so you just do flux diffusion model loader, this torch compile model node, then to ksampler?

[bitter hearth]

yeah exactly that

[winged seal]

Yeah, if I could do 4x multi res, I'd expect SD3.5 to be maybe 2x as fast as flux to train

[bitter hearth]

and try to get up to date CUDA to get the FP8 boost on Ada or Hopper

[winged seal]

But I assume that will only be the case on consumer GPU's. I don't see that happening on workstation cards

[bitter hearth]

in inference, SD 3.5 seems to not go beyond 1024 without funny squares appearing
not sure what is going on

[opal bay]

am I wrong btw, re:ddim? I cannot think off my head of a single use for it demonstrated where its the best tool in the box

[bitter hearth]

you're right, I don't recommend ddim

[winged seal]

That's a shame... Flux was an astronomical leap forward with high res generations. I think that will be SD3.5's biggest weakness for sure

[opal bay]

I'm quite excited about gguf versions of 3.5

[bitter hearth]

in the I-Max paper they trained a 2048 checkpoint of Lumina which looks amazing but they did not release it

hopefully they will soon

[opal bay]

just gguf being used in text2img in general

[winged seal]

I think a lot of people down play flux's native capability to do more than 2048x resolution when SDXL and SD3.5 can't even do 1280x

[bitter hearth]

oh Flux can do 4096 now, with I-Max

its this: https://arxiv.org/abs/2410.07536

[winged seal]

I believe it. It does phenomenal 1440 and 1440 Ultra wide gens

[bitter hearth]

but the Lumina checkpoint can do 8k

[low stone]

well, 1 megapixel is still ideal for flux otherwise you start getting stripes at random. that said I've got a really nice and simple upscaler for flux which makes those 1 mp images look great. lots of detail etc.

[winged seal]

I have tested this claim, and I have found that if you're able to put any LoRA That's even moderately well trained on top of it, it completely fixes that issue

And, if you have any LoRA's That make the issue significantly worse, they are likely either improperly trained, or trained in Kohya.

Since moving to AIT, basically all of my trainings have greatly improved flux's ultra high res generations

Like here

[bitter hearth]

apparently Flux works with DiffuseHigh, that might also take it to 4k, by a different method to I-Max

[winged seal]

Native 3440x1440 I got with a realism LoRA I made in about 2 hours

[bitter hearth]

whoah

[winged seal]

No upscale, no second pass, just raw diffusion

[low stone]

so the answer is that 99% of lora are trained at 512/768/1024. To be able to render at higher resolutions and not get the stripes, they have to be trained at 1344 and higher. The problem, is that this is where 24 gig cards hit their limit. To do that you'd need to rent something like on runpod etc.

[bitter hearth]

native 3440x1440 flux with one ksampler?
without using library like I-max or DiffuseHigh?
that's amazing

[winged seal]

Where did I say that? I train at 1024 and below as well. The issue is just that Kohya specifically does not train properly

Here's another

[bitter hearth]

this image is so amazing on 4k monitor

[winged seal]

In fact, the one that I trained that allowed me to do this high resolution generating was actually trained at 960x

[bitter hearth]

what do you think about simpletuner?

versus AIT or koyha

[winged seal]

Absolutely not. I refuse to use it because of the creator

[low stone]

hmm... looks like i need triton installed for this torch compile thing.

[bitter hearth]

oh yeah its linux only sorry

[low stone]

https://tenor.com/view/so-pissed-im-work-mad-gif-23416426

[signal shuttle]

There is a triton for windows if am not mistaken

[bitter hearth]

there was some very recent thing about triton in windows

[signal shuttle]

https://github.com/woct0rdho/triton-windows

[bitter hearth]

maybe this https://old.reddit.com/r/StableDiffusion/comments/1g45n6n/triton_3_wheels_published_for_windows_and_working/

not sure I don't really do windows

for this stuff

[winged seal]

@bitter hearth not sure if you saw my response, but I am vehemently against simple tuner and everything to do with it. I will not use it, condone it, or support it in any way

[low stone]

ok thanks. looks like I'll need to spend some time getting all that together. I'll have to try that later tonight.

[bitter hearth]

why is that? I'm not aware

I've been making a fine tuning script in Pytorch and one in Jax personally

[winged seal]

I used to be very close friends with the creator of it for over a year. We were part of the terminus research group, where I and many other very talented individuals would compile together our knowledge and share it with one another.

The creator of simple tuner is an exceptionally horrible person, one of the worst I've ever met. Exploiting individuals, abusing individuals, fleecing nudes off of individuals under the guise of training data, making a heinous accusations about me and other people in front of our employers because he was jealous that we got positions he didn't, stealing training code, stealing data sets, you name it

I defended him for a long time, but after months and months of him abusing me and other people that I enjoyed in the scene, I was finally able to get away from him, at which point he took some of my methods, my data set source is, and the resources that various of my research partners spent months and months compiling together, and he turned them into a bastardized repackaged version that's not as good as any of our individual projects

[bitter hearth]

ok thanks for letting me know, wasn't aware of things

[winged seal]

Of course, I don't expect anybody to be majorly aware, it's stuff that is quite personal. However with that said, I'm always open to sharing my experiences, because I don't believe that he deserves any form of platform built off of all of the horrible things he's done and all of the information he's blatantly stolen

I'd go into more detail if I didn't feel like I'd get banned from the server for the sheer depravity of things he's done

[winged seal]

One of which was confiding in me about personal problems and depraved fantasies that he had that he was trying to get better with, and I shared similar experiences with him about things that I wish to get better with, and then he weaponized those things that I told him, turned them into 100 times worse accusations than what they actually were, and purposely blasted claims of genuinely illegal and deplorable nature in front of two of my potential employers

But anyways, I don't want to trauma dump in here. All I will say is that I have dealt with a lot of truly terrible people in my life, and I think he might single-handedly sucker punch every other person I have interacted with out of the way in order to claim first place as the worst person I've ever interacted with. And that hurts to say, because I defended his actions for over a year before I realized I was a victim too

[bitter hearth]

okay I wasn't aware of this, seems like there was a lot of history

[craggy crest]

:( ouch. sorry you had to go through that

[winged seal]

I assure you this is like the 1% I can say lmao. He is the most deplorable person I have ever met. Luckily, he's such a jackass that his bad attitude and missdeeds keep getting him fired over and over again so people aren't stuck being around him

Maybe some day he will learn to be a less insufferable person, but the half a dozen job firings in the last year don't bode well

[noble coyote]

(I've just been to the Chip Shop!)

[noble coyote]

You seem talented enough to walk out - and immediately into 6 new jobs!

[dull star]

@cunning lintel

[dull star]

Van Gogh version

[cunning lintel]

Nice! What's the secret to suddenly get paintings? Does it work for any scene

[dull star]

The secret is to use PixelWave Flux

https://huggingface.co/mikeyandfriends/PixelWave_FLUX.1-dev_03/tree/main

[cunning lintel]

that's cheating :p

[dull star]

its a dreambooth-like finetune

no lora

if SD3.5L gets a finetune that improves paintings I might use that instead

[cunning lintel]

I'll have to download it 🙂 looks good

[dull star]

except that SD3.5 sucks with anatomy but we'll see

[dull star]

also I haven't tested it too much

there are Claude Monet-isms when I prompt for it, like using random colours in places

Van Gogh as you just saw there

Caravaggio didn't work iirc

[craggy crest]

have you tried the photorealistic lora on the huggingface space?

[dull star]

have not

[cunning lintel]

i always liked to use Peter Mork Monsted / jacob van Ruisdael

[dull star]

I might try it later

well I'm not sure if those will work

but I'll try

expect a 1% chance of it working

[icy drift]

Running OmniGen through Pinokio's setup, I don't have Triton. Not needed.

[dull star]

Zdzislaw doesn't work for example

[craggy crest]

they work jsut fine with 3.5. it's flux you can't train, not 3.5

[dull star]

oh god peter mork monsted makes photoreal paintings

I wonder if regular Flux Dev with a low guidance would get you that already

painting by jacob van Ruisdael [... rest of the prompt] @cunning lintel

🤷‍♂️

I mean close enough

[cunning lintel]

It's plenty close, esp compared to other models

[dull star]

I use ddim_uniform scheduling with euler_ancestral sampling (update comfyui if euler_ancestral is messing it up for you)

16 steps I think

for no lora this model is impressive

[cunning lintel]

I update comfy much too much, i have it 🙂

[bitter hearth]

SD3.5 Turbo ...
a lookalike between Elon Musk and Donald Trump with long hippy hair and 70s baggy outfit.

when people were using SDXL they had to run additional extensions to cross mix characters, now you can do that easily with sd3.5 vanilla model

[silver sluice]

pretty amazin stuff what you guys can create with just prompting, i think personally loras has turned me into a lazy prompter, sometimes I don't even cite a certain style and just add the lora to get that efect

[bitter hearth]

loras are certainly helpful for very specific tasks with great results if trained properly, but i also enjoy playing with regular prompts to bring out ideas into images

raw prompts reflect on the model capabilities, so far 3.5 gives pretty good outputs

while chatbots like gemini and the likes censor any real life character depictions

[silver sluice]

all valid points

[dim geyser]

why does it look like steven tyler 😆

[bitter hearth]

a lookalike cross between Elon Musk and Donald Trump wearing 60s hippy outfit.

didnt come out right this time... i think i need to edit the prompt

[silver sluice]

do a cross like Steven Tyler and Elon Musk let's see what it gives us

[bitter hearth]

ha...

btw im also leveraging on modelsampling node

[silver sluice]

the glasses ruin it, iits more fun when the glasses dont cover the face to see a better idea of the cross. im not getting any elon from this latest take

maybe the Elon goatee?

[bitter hearth]

yeah

the area of mouth too

[silver sluice]

same prompt using sd3.5L on my system

a lookalike cross between Elon Musk and Donald Trump wearing 60s hippy outfit.

[bitter hearth]

oh, im using turbo

4 steps

[silver sluice]

sd3.5_large_fp8...aled | 🌱 2854914518 | 🦶 26 | 🦮 6.0 | 🎤 ddim | 🗓 10/26, 4:39 PM | ⏱️ 108s
maybe I should do more than 26 steps or lower the cfg?

[bitter hearth]

30 steps is ideal for Large

[silver sluice]

your results look way better it almost looks overexposed for me

[bitter hearth]

cfg maybe between 3-5.5

but with turbo i have cfg set to 1

[silver sluice]

wow interesting cfg at 1 with turbo

[bitter hearth]

and for realism aspect of it i utilize modelsampling node .. its very efficient within a value of 1-3

[silver sluice]

sd3.5_large_fp8...aled | 🌱 948269089 | 🦶 20 | 🦮 7.0 | 🎤 euler | 🗓 10/26, 4:41 PM | ⏱️ 47s
supposed cross between trump and elon, looks terrible

[bitter hearth]

heck yes and within 10 seconds for each image

[silver sluice]

47 seconds for that image isn't bad at all for me, id say thats a new record actually

[bitter hearth]

btw when you use euler you will get softer and smoother look

[silver sluice]

sd3.5_large_fp8...aled | 🌱 2413612102 | 🦶 29 | 🦮 5.0 | 🎤 euler | 🗓 10/26, 4:43 PM | ⏱️ 78s
i changed it to portrait, increased steps to 30, and reduced cfg to 5 as suggested and reuslts idneed look much better now

[bitter hearth]

but honestly i prefer dpmpp_2m and control sharpness with modelsampler 🙂

[silver sluice]

i don't like to mess with custom nodes I like sticking to classic workflows

i dont even mess with shift

[bitter hearth]

not custom really

its part of the comfyui tools

no need to download anything special

[dim geyser]

Elon approves of the new representative

[silver sluice]

yeah i know i'm just saying I don't like to go outside of the 'Default' workflow where it's just load checkpoing > clip text encode > ksampler

i guess it's a result of i don't actually use comfyui but rather I use it through my website and i don't have support for 'custom' or basically nodes outside of the classic setup

[silver sluice]

that looks like steven tyler attended a Tesla event

[dim geyser]

"steven tyler presenting on stage at a Tesla keynote, wearing a black shirt and pants. Elon Musk looks on from the side of the stage, pleased."

[bitter hearth]

ahh ok, yeah i like to keep things simple and organized

snapshot of my general workflow

[silver sluice]

so where's this "modelsampler" node you were talkinig about in that screenshot?

[bitter hearth]

i have ollama on the far left too, i switch between that and manual prompting often

on the loders

[silver sluice]

that's what id consider the classic setup, i'd say you're using all the standard things I am too, nothing special like you're not even the node to expose the shift

[bitter hearth]

its more coherent for me in that way, i could add other stuff but not when i dont need them for specific purpose

[silver sluice]

where's the modelsampler though?

you were talking about controlling sharpness with it

[bitter hearth]

the light blue and pink loaders

[silver sluice]

where is that in your screenshot?

[bitter hearth]

at the 2nd row in those loaders

[silver sluice]

oh i see, i missed that the first time, thats what I call the "shift node" so you are using it, so you're saying you like to use dpmpp_2m and mess with the shift to control sharpness?

[bitter hearth]

yeah it has big influence on the image tone

[silver sluice]

sd3.5_large_fp8...aled | 🌱 3775773810 | 🦶 32 | 🦮 4.0 | 🎤 euler | 🗓 10/26, 4:49 PM | ⏱️ 137s

[bitter hearth]

and you dont need to change the value too much, you can have big results between 1 - 5

[silver sluice]

that looks more liike a video game jacket thhan a hippie jacket

yeah crystalmancer was showasing the shift parameter the other day

[bitter hearth]

nice

i also noticd when using sd3.5 with modelsampler, a value of 1 makes for a nice realistic image

[silver sluice]

this is what my form looks like, I've just been too lazy to add the shift field in the form, considering both Flux and SD3 use it I think it's about time I get around to it

[bitter hearth]

is that a custom UI?

[silver sluice]

yeah it's 100% custom using ComfyUI for the backend

i even use the vae_decode preview from ksampler to dynamically show the rendering inline

[bitter hearth]

neat .. looks interesting to be using your own UI

[silver sluice]

i just spent like 20-30 hours rewriting how I handle the generating of workflows for ComfyUI, right now I have it set up so iti's a workflow for all stable diffusion models and one for flux models but it's really messy and it makes it hard to add nodes like the shift node or model sampling node as it's called

[bitter hearth]

i can imagine

[silver sluice]

basically im generating a custom workflow per image based on what options I use

this way also avoids overhead like loadingi stuff I dont need to use

i have almost 9K loras installed taking up 1.6TB of space i think

[bitter hearth]

i would say my system is doing fairly ok using comfyui

and these turbo and schnell models are nicely optimized

speically with quantized versions

i use q8 for both those models

[silver sluice]

that's another thing

[bitter hearth]

yeah

it adds up at the end

the benefits i mean

[dim geyser]

is Q8 just fp8 or is there more to it?

[silver sluice]

id like to use gguf for sd3.5 but i cant because im bound to the limitations of these two workflows I have set up so with this new system im building ill be able to use gguf for sd3.5 similarly id be able to use safetensors for flux images

[bitter hearth]

better precision and accuracy than fp8

[bitter hearth]

and we have about 2 days now, till 3.5 medium drops

[silver sluice]

there's a little share button on each image and when I click it I get this little panel at the bottom so i can then just click Post Next and it takes me to the CivitAI website where I can share it, https://civitai.com/posts/8388007

internally I'm using BLIP2 and JoyTag to generate a post title and the tags for it

[silver sluice]

i think Flux has taught me the importance of patience for quality, sd3.5 medium is kinda meh, when im gettingi 47 seconds on the large model that's "good enough" for me to not want to mess with something lower quality

[bitter hearth]

well yes but also turbo model is not really lacking much .. you can have great results much faster w/o any bad compromise

[silver sluice]

yeah that seems to be the sentiment across the community, ive heard that multiple times, it's hard to believe really that's why i haven't even tried it yet

[bitter hearth]

there is definitely use for large model but for professional reasons for those who are making images commercially but for people who are just fiddling with it for casual use it wont matter too much

i dont make money out of making these images

these are mostly for fun and play

[silver sluice]

me too

im just generating images for fun, i have a remix pipeline and then i share them on civit, my motivation for generating high quality is so that i can share high quality content on civit to farm reactions

feels good to be 'validated' for a nice generated image lol

[bitter hearth]

2-8 steps is fine in my opinion I do that for most models where possible (hyper, turbo etc)

[bitter hearth]

im guessing you have goal with developing UI or at least in the process of creating something that you can polish and market later?

[silver sluice]

this is the size of my image generation queue, 155K thousand images, $73 in costs if I were to use RunPod to generate them all, and 101 days of processing to make them all locally

[silver sluice]

nope im just developing the UI for myslef, for fun, as a hobbyist programmer, its a mix o programming and generating images for fun, they go hand in hand

[bitter hearth]

ahh ok

[silver sluice]

so far I've genearted 550K images locally with my system

top Pony and Flux models I have

[bitter hearth]

i moved away from sdxl genre

now that i have sd3.5 and flux

[silver sluice]

i think Pony is still king of creativity, Flux is king of realistic and SD3.5 is the king of illustrated images

[bitter hearth]

thats sd3.5

comparable to flux

[silver sluice]

that's really good for realsitic, speaking of which right? lol

[bitter hearth]

yeah cause of modelsampling at 1

i couldn't do that before

[silver sluice]

it is comparable, i mean it takes a big man (SAI) to admit they've been bested by Flux when they posted that chart showing Flux still bats SD3.5 in aesthetics, there's something to be said for that

[bitter hearth]

yeah but i think fine tuning is necessary in some areas of the anatomy

[silver sluice]

for flux or sd3 or both?

[bitter hearth]

if that's done sd3.5 will rock

sd

[silver sluice]

i think flux has weak anatomy but I think its ability to generate high quality realistic + high quality anatomy loras makes for a better model overall

[bitter hearth]

flux may not be as easily trainable as sd3.5

[silver sluice]

that is true but as of right now theres like 100x more loras for flux than there is for sd3.5

plus you can even train flux on civitai for like $5

[bitter hearth]

yes but most of the loras are buzz farm

i came across handful of loras that create bad hands than the base flux model

[silver sluice]

true

[bitter hearth]

it could be partly due to how black labs designed their model that are hard to train

[silver sluice]

yeah thats the main reason a lot of people already prefer sd3.5 in the state it's in over flux despite higher quality for flux

trainability goes a long way in the long term

@bitter hearth hey i igot a prompt for you if you can try it on turbo for me real quick

[bitter hearth]

modelsampling left to right.. 0.5 - 1.00 - 2.00 - 3.00 - 4.00

[silver sluice]

its very subtle but def a change

[bitter hearth]

use the arrow key on your keyboard to swap image views

[silver sluice]

ok here's the prompt
a painting of a pink Jester giving a high five to a panda while in a cycling competition. The bikes are made of cheese and the ground is very muddy. They are driving in a foggy Forest. The panda is angry.

[bitter hearth]

each of them are distinctly different

[silver sluice]

yeah i think it's more noticable when i toggle through them with the arrow keys, especially the eyes

my preference is 0.5 for the realistic look

[bitter hearth]

no cherry picking ..

i had a fixed seed btw

let me try another

[silver sluice]

using turbo on glif https://glif.app/@rennurtsfx/glifs/cm2l63ksi0000pp15p3us3usn

[bitter hearth]

cherry picked one

[bitter hearth]

i like the image contrast in it

[silver sluice]

pink jester ✅
high five ✅
cyling competition ❌
bikes made of cheese ❌
muddy ground ✅
foggy forest ✅
panda angry ❌

missed 3, i can get dalle3 to nail iti 100% not miss a single thing, using that prompt, and i can get flux destill to miss just the cycling competition aspect

i consider the cycling competition aspect to add some distant bikers competing with them

[signal shuttle]

Glif only uses T5XXL and Clip_L and doesn't use Clip_G so quality may be effected

[bitter hearth]

Dalle is from openai that MS later bought?

[silver sluice]

dalle3 is openai and openai still owns it

[bitter hearth]

and OmniGen is a MS product i think

[silver sluice]

openai gave microsoft access to use in their little website tool thing but it's sitll there's and available through chatgpt

[bitter hearth]

ahh so not sold to MS

im looking forward to when omnigen drops

[silver sluice]

it's kinda not fair bc internally dalle3 rewrites the lazy prompt to something that nails it

[signal shuttle]

Omnigen is out

[bitter hearth]

not usable on comfyui yet

but yeah its already out, i tried a few render on their webui .. i think the quality falls short of sd3.5

but the omnigen perk is more about integrating controlnet and bunch of other stuff

[silver sluice]

dalle3 feels like it regressed, it used to get this prompt really well

[bitter hearth]

you could just prompt w/o needing any extra tools

[silver sluice]

i ran it using glif + i turned on the claude rewrite feature and this is what i got

[dim geyser]

MS is a big investor in OpenAI

could be where some of the confusion comes from

[silver sluice]

i like dalle3's concept of bicycles made out of cheese the best

yeah ms owns like 49% of openai or something ike that

[bitter hearth]

btw modelsampling does more than influencing image tone, it has impact on content too such as the character

[craggy crest]

they own all the technology until such time that openAI achieves AGI - that was part of their 'exclusive partnership" deal

[silver sluice]

yeah exactly i knew that

[bitter hearth]

problem is by some older definitions we already have AGI

[sacred jewel]

https://civitai.com/models/891111?modelVersionId=997177

[silver sluice]

yeah the AGI term can be broadly interpreted and misused imo, I lean on the side of the fence that we're not there yet, we've just made really fancy machines that can memorize a lot of stuff and recall it, true AGI is being able to reason in a different way

[pseudo owl]

Yeah it’s obviously much worse in quality then sd3.5 and even worse then even sdxl sometimes.

But the main thing about Omni gen is that it can do everything ip adapter, controlnets, pulid, do without any extra model and better sometimes.

[bitter hearth]

if its possible to fine tune we can imagine where that would lead to

[pseudo owl]

Yeah it’s possible to finetune and adding new modalities as well is easier then training separate controlnets too.

[bitter hearth]

i hope so, and we can have model that's dynamic and easy to use

[pseudo owl]

Yep, it’s a bit slow though but just because it’s unoptimized right now.
It’s a lot smaller then full sd3.5, flux and even sdxl too(around 12b, 16b, 4b).

OmniGen is just 3.8b parameters.

[bitter hearth]

sounds super cool

i wanted to try it out but not supported on comfyui, any idea when they will release a working model?

[craggy crest]

castle carved from fruit, lofty rose spires, dripping water, cotton candy clouds, surreal landscape, symetrical fractal fantasy

[craggy crest]

AGI is Artificial General Intelligence = regardless of the fantasy twists that social media has put on it the last 2 years, it ONLY means an AI that can do things it wasn't specifically trained to do. doesn't mean super-intelligent or anything else. a two-year old child is NGI - natural General Intelligence. it can do all sorts of things that 1. aren't instinctive and 2. it wasn't superficially taught how to do. an AI could be no smarter than a 2 year old child and if it could do things you didn't train it to do, it would be AGI. Social media is made up of people that have no clue and that love to grab onto a concept, weave it into all sorts of wierd fantasy, mix in science fiction, and then publish it as fact. doesn't make it fact.

SD3.5 large

[bitter hearth]

i have to say, that's philosophically and technologically deficient understanding of Ai and AGI, you may be sharing a practical outlook of the current infrastructure which has a long way to go, but that's not the definition of objective goal of AI

[sacred jewel]

Polaroids LoRA

[bitter hearth]

if you can picture this concept that a developed AI or AGI however you want to spin the intelligence would be the last frontier of man's invention that will give unprecedented results

[craggy crest]

that's the defintion, from the scientific community, that has always been what AGI is

AGI is something that we've been working toward for years

not something that just came up in the last 2 years

[bitter hearth]

they might be afraid of the fact AI will surpass their intelligence

[craggy crest]

that is ASI - Artificial Super Intellegence - and no, they're not

AGI != ASI

[bitter hearth]

lets not keep labeling each cateogry

that side tracks the point

im talking about intelligence in general

[craggy crest]

sigh here - https://arxiv.org/abs/2303.12712 an actual paper for you to start with

[bitter hearth]

thanks i will read it, but not atm

bookmarking

[craggy crest]

there are quite a few others on there if you feel like digging through them

[bitter hearth]

ok

i will read the article when im less distracted but ty for sharing

[craggy crest]

warning, its a deep rabbit hole with a lot of branches

[bitter hearth]

thats fine

i want to understand their concept of ai

[craggy crest]

certainly - just block out time to read and dont' try to do it all in one chunk

[bitter hearth]

Tuesday, their latest model.. Intelligence.. pffft 😄

it is a statistical machine.. still

[craggy crest]

that has nothing to do with intellegence. you'll find plent of people that are mensa level that cant' do math

and that's grammer anyway

[noble coyote]

Using Huggingface OmniGen

https://huggingface.co/spaces/Shitao/OmniGen

[craggy crest]

the question isnt' asking it to do math, it's asking for the correct way to write the sentence

[bitter hearth]

I knoow, people would say this:

It is statistical

in any way

[craggy crest]

shrug. this is a discussion for #💬｜general-chat

[bitter hearth]

Here is an image

[bitter hearth]

watson was considered intelligent bot too back in those days, but all these chatbots are not the standards of intelligence that's in progress, we havent achieved that bar yet

[craggy crest]

https://www.ibm.com/watson watson has been updated

[bitter hearth]

the litmus test is that we dont have artifical intelligence that can be incorporated to its' own autonomy if your life depended on it

[craggy crest]

still havn't looked up yohe and his autonomous agents he's made?

[bitter hearth]

no, i havent, couldnt find links for it, if you can share that would be nice

[craggy crest]

you can start with his profile here https://x.com/yoheinakajima

Yohei (@yoheinakajima) on X

VC by day, builder by night: @untappedvc, @babyagi_, @pixelbeastsnft. Build-in-public log: https://t.co/UdHHGbZba5

Twitter

[bitter hearth]

ty

bookmarked but will go through them later

[craggy crest]

you might also look at the autonomous stock trading agents that've been in use for several years

[bitter hearth]

what you are suggesting is fragmented modules for isolated tasks

[craggy crest]

nope

[bitter hearth]

thats not the broad scale integration

[craggy crest]

but i'm not going to continue down this path with you. you have stuff to read and research

[bitter hearth]

i know what you are saying but i will read the articles later to see what makes you think what you think

[craggy crest]

Prompt: four color illustration from a children's book about a puppy and a basketball

[bitter hearth]

you realize what they are presenting at https://www.yohei.me/ are specific modules

[craggy crest]

i thought yoiu were goign to read and research later - that you had stuff you were in the middle of? i'm gonna stop giving you rabbit holes when you say you're busy

[bitter hearth]

im looking at that site casually

[craggy crest]

you know that leads to 'oh hey, i've been sitting here for the last 9 hours'

[bitter hearth]

but do you understand the point im making ... if we had global scale AI we wouldn't be stuggling over energy crisis, we wouldn't be struggling over cancers and so on

those are some of the objective from materialistic pov

we dont have that yet

i think we have differences in our expectations rather that where ai is and where it needs to go

just like sd3 and sd3.5

i also acknowledge your practical views, we can't have an overly big leap, we need stepping stones regardless how far the reach is

[craggy crest]

i'm not giving you any more rabbit holes right now ;)

[bitter hearth]

i went through all the holes there is already

and im not bragging about it, there is nothing to brag

its a reality how i see

we are in self destructive mode and AI will have our last chance in steering away

[craggy crest]

SD3. 5 large: prompt: four color illustration from a children's book about a puppy and a basketball. The puppy is standing up its hind legs, bouncing the ball on its nose

[bitter hearth]

those are isolated modules btw, not largely integrated to human systems

but promising never the less and necessary

[craggy crest]

you're gonna be sitting here at midnight at this rate

[bitter hearth]

also understand you can develop Ai to build itself but that's not the same as implying whether that Ai has access to all the possible information archive or lacking, this is also one of the pressing issue of censoring Ai, you need to feed Ai all the information, including human lust just for the sake of practicality of life

im interested to go through them, good content and i have time

this is bit off from our actual topic but when you baby sit an Ai you will make a lame ai

ai needs to see everything and compile them into meaningful solutions to where mess happens and how to achieve utopia

[cunning lintel]

that stuff is soooooo cool, one prompt 3 totally distinct images, i missed that, it's sooooo much simple fun to use

[winged seal]

@craggy crestOk, I forgot to get back, but I trained SD3.5

I have only looked at the generated sample images from AIT, but I seriously hope there is something wrong with the way it inferences, cause the results look bad lmao

now I have to deal with downloading and running SD3.5 locally

[bitter hearth]

did you have a range of resolutions

cos that's currently the big unknown thing with SD 3.5

whether it can go much outside of 1024, and if so, how to do it

[craggy crest]

sigh. maybe the issue is, i dunno, the way you trained it? i think, really, the better question is does your boss like what you did or not - as he's training it and quite happy

[winged seal]

I mean, there is no real information on how to train it, so its just the best I can try lol

[craggy crest]

talk to @calm zinc and see what he's doing then

[bitter hearth]

I'm hoping we still get a paper for Flux and SD 3.5 at some point

in the absence of that, what I have been thinking is we can piece together information from third party papers
currently there are two on Flux https://arxiv.org/abs/2410.07536 https://arxiv.org/abs/2410.10792

[winged seal]

I doubt it, seems like no companies really care about giving proper resources for others to spend their time and money realizing their projects for them lmao

[bitter hearth]

almost all major models have papers though including previous ones by this same team

essentially the original SD3 paper applies to Flux and SD3.5 anyway, to a decent extent

since the two papers on Flux that I listed essentially say that it is a similar design to Peebles and Xie 2022

which is what SD3 is

[winged seal]

I am really hoping the bad results are just AIT's sampling, cause it looks ass from the first sample as well, so I am gonna assume that

[bitter hearth]

what resolutions did you use?

in the process

[winged seal]

1024

[bitter hearth]

ah ok so its not the positional embed issue

[winged seal]

even the non trained one looks abysmal, so I assume its just the modified trainer not having proper sampling settings

I guess now I have to do all the BS to get 3.5 running locally lol

I hope it will work with my GGUF text encoder loader

[craggy crest]

the SD3 paper is already out. 3.5 is just a modification of sd3. you'll never get a paper on flux

[winged seal]

alright, so I need the FP8 SD3.5 model, and hopefully I should be able to use my other workflow and the gguf text encoders

[craggy crest]

did you try what @gusty trail told you he was doing?'

[bitter hearth]

I agree that 3.5 is also Peebles and Xie 2022 structure

[winged seal]

I didn't see anything, sorry

[craggy crest]

https://civitai.com/models/879701/stable-diffusion-35-fp8-models-sd35

[craggy crest]

search this discord for from: XiaoZhi and read his posts. he was talking to you

[winged seal]

are we having any progress in having less shitty websites to host models on? Or is Civit still just kinda all thats really relevant>?

[craggy crest]

sorry, but that struck me as incredibly funny the way you said that

[craggy crest]

you could use the GGUFS https://huggingface.co/city96/stable-diffusion-3.5-large-gguf/tree/main

[bitter hearth]

LOL yeah
also when you say we will never get a paper on flux
was this something someone there said or is this an inference?

[craggy crest]

you will never get a paper on flux. i'll be happy to DM you and we can discuss thsi if you like

[bitter hearth]

okay sure

[winged seal]

oh cool, so they have the ggufs already, nice! I am more interested already haha

[craggy crest]

i gave you all the links yesterday. deep breath and start not doing 20 things at the same time

[craggy crest]

dm sent

[winged seal]

Flux was big enough to make ViT's actually usable. Kinda a smart idea for SAI to have waited for all of the flux improvements to be slapped on SD3.5 to make it much more viable at launch'

[craggy crest]

none of the flux so-called improvements were 'slapped on' or used on at all, on 3.5

tehre are no 'improvements' in flux