✨｜sdxl

[shy kelp]

i'm following someones guide and he uses this nodes, there is no way to change the denoising

😢

[plush drift]

why you are using this if I may?

[shy kelp]

dude in the video used it

[plush drift]

i mean what is the advantage? over the stock nod

[shy kelp]

idk, never used comfy before in my life until today

[native knot]

Depends. Right now I'm using the refiner extension for A1111 and I typically have it set to somewhere between 20-40% of the steps. If I was using Comfy, I'd tune the .cfg to no greater than 4...(probably 2), and would put my steps at about 8 out of 21 total. But it very much depends on what I'm going for. Often-times, I'll run a plot script to find the look I'm going for and tune from there.

[shy kelp]

thanks

my gpu is too shit for a plot so imma just use those values lol

[native knot]

I'm doing all of mine on a 3080.

[shy kelp]

i got a 1080 ti

[late marsh]

Job done

[dense chasm]

Hi folks,i've just finished my character lora training based on sdxl1.0 in huggingface hosted inference API as below:
https://huggingface.co/frank-chieng/michelleyeoh
you can follow the READ.me and have download the lora with your own choise to make a refine img2img,huggingface is more interactive and mature than civitai

[buoyant flame]

what should i do to fix this?

[native knot]

Inpainting.

[buoyant flame]

how do i do that

[native knot]

Depends...what are you using?

[shy kelp]

this is what i did without any refiner at all on a finetuned model

tested upscaling tooo

[buoyant flame]

stable diffusion checkpoint

[native knot]

No...as in, what tool?

[buoyant flame]

text2img

[native knot]

Automatic1111, ComfyUI?

So Automatic1111.

[high skiff]

I am gonna try making my dataset 4x bigger with some refined captions, and much more aggressive training, cause damn this is already amazing

[buoyant flame]

THIS

[native knot]

Should have a button for send to inpaint.

[plush drift]

yeah, doing img2img without refiner yeild amazing results too

[buoyant flame]

ok, now what?

[plush drift]

???

https://cdn.discordapp.com/emojis/1004939086546604102.gif?size=64&quality=lossless

[native knot]

Once it's in inpaint, use the tool in the image screen to mark the area you want re-done. Choose "only masked", adjust settings, click generate.

[plush drift]

in ComfyUI

[shy kelp]

the refiner is absolutely destroying anime faces

idk if i'm doing it wrong

[plush drift]

not just that! but everything

i think they need to rename to the definer

[boreal bough]

"base with no mods only refiner"

[plush drift]

or disfiner

[shy kelp]

so do people just use the finetuned versions without any refiner at all ?

[plush drift]

I use a CUI img2img wf

[plush drift]

i use vanilla SDXL

and now my MODEL!!

jaja

[native knot]

lolwtf

[plush drift]

why?

[unreal plover]

why not ? Chocolate is good

[plush drift]

https://tenor.com/view/skeptical-futurama-fry-hmmm-i-got-my-eyes-on-you-gif-17101711

[shy kelp]

more like chocolate pudding

[plush drift]

chocolate you say!!

[plush drift]

I love it

[native knot]

That's a winner

[boreal bough]

@plush drift I love you! ❤️
I never even considered running 100 steps XD yeah. samples are working properly now

just tried it

[late marsh]

Great! 😂

[plush drift]

I am daying over here!! someone stop me please hahahahah I cant stop making these amazing funny things

[plush drift]

I am happy that it was a helpful tip, cuz honestly, I will never understand why someone will spend 40 hours on training a lora

and then... oops,

it was poop

[boreal bough]

:"D you just described 50% of my loras

but tbf, I'm running various configs to see how it works out

[plush drift]

you are running it locally? or GC?

[boreal bough]

locally. rtx4090 - so I can get through my tests relatively quickly

[plush drift]

yes!

[boreal bough]

and my big 20~30h loras - I already know will work. But I'm just checking the progress of my database expansion

[plush drift]

this is the way

[buoyant flame]

no matter how hard i try, this hole in her suit will not go awayu

[boreal bough]

at 4k images it takes a full day. dont even wanna think about the final form of 30~50k images

might as well do a finetune at that point XD and see how close loras get to full tunes

[native knot]

adjust the prompt to be more focused on the suit, tune up the cfg a touch

[plush drift]

I do not want to be an ass, as I have my personal recipes! but it's waaaaaaaaaaaaaaaaaay below that

trust me, more steps and longer time doesn't mean a better lora

[shy kelp]

let me guess, its all about the right dataset

[boreal bough]

goal is to stay super flexible - and yeah, not like 10 epochs + 1 repeat is "a lot" of steps

but I'm training multiple hundreds of concepts

[plush drift]

in one lora?

[boreal bough]

its why I keep making loras for every 1k expansion, making sure my weights stay evenly distributed

[boreal bough]

yep. my current iteration has a bit over 100 concepts full learned, and fully flexible

not training via overfitting

so poses stay true to sdxl base

[plush drift]

a mix of both, but sure a the right dataset is a major player

[boreal bough]

training via overfitting now takes me exactly 14minutes, for a lora that produces more than 80% good images

[neon heath]

Are models that fined tuned by sdxl are better than sd1.5 ? Or they feel the same?

[shy kelp]

god i wish i had a 4090 to do that

[vocal rapids]

finally can run controlnet for SDXL in comfyui 😆

[plush drift]

pitstop! this this so fucking cute!!! must share!! ♥️

[shy kelp]

i trained a lora on my 1080ti once and it took 3 hours

so horrible

[boreal bough]

16gb is all you need. and you only lose like 4 min

[neon heath]

Well use colab?

[boreal bough]

so I dont think anyone complains about waiting a whole 20 min

[shy kelp]

2200 steps

1 epoch with 100 repeats

😂

i have no idea what i'm doing

[plush drift]

can you elaborate more on this?

[shy kelp]

the base sdxl model produces better looking anime images than the finetune, who would've thought

[plush drift]

lets be real, I was using 3060 till last month and I got a 4090~! let's be real please hahahaha it's a beast

[shy kelp]

how will i loose 4 minutes

it takes hours on my setup

nowhere near 20 minute lora training

i dont have room to experiment with lora params because it will take a day to just train 1-2

no money for collab or renting gpus either

[plush drift]

https://tenor.com/view/heres-a-dollar-saturday-night-live-weekend-update-heres-some-money-have-my-money-gif-21609356

[hoary saddle]

any chance there's a hidden gem comfyui node out there that would overlay a text string onto the image? ex. (add a caption to the image at x_y coords using python/pillow?

[neon heath]

Wait you guys pay for google colab?

[boreal bough]

caucasian/asian/african/indian/etc.. <- so images of "woman" aren't biased towards any ethnicity
age groups are now properly working: 'girl' now produces images from ages 16~25. 'woman' produces 30~40.
I'm using mroe than 50% real photographs of normal people, so no bias towards studio like setups, or model photoshoots
outdoor/indoor tagging, so the concepts stay away from any other words

basically this lora is intended to remove the current existing biases on short prompts, and reinforce prompt control on longer prompts (so words aren't just casually ignored)

other than that, 'side view', 'rear view' and many posing words are trained via 500+ images, so they function really well, without being a control net clone

[plush drift]

oh! so you are a nerd! i love it

[shy kelp]

dont forget the .png

linux user spotted

[plush drift]

ok now I get it, this is really some hard work cant wait to try your lora once it's ready

[buoyant flame]

what should i say to get it to make more of this?

[plush drift]

but make sure to nerdout on writing a proper bio or a guide so we can use it properly

[shy kelp]

use danbooru interrogate

[boreal bough]

its why I had to enable clip training 🥲 so that normal words are useable - else it genuinely would have needed a wiki XD

[plush drift]

would you please stop spamming these children photos

wait! CLIP? I thought it's not wrking with SDXL

[shy kelp]

dude why does the refiner always nuke faces

[boreal bough]

yep. a whole guide on lora training will drop eventually ^^ for simple till ultra complex training

#🔧｜finetune message

its why stuff like this gets posted occasionally XD

[plush drift]

@shy kelp @buoyant flame kindly stop spamming children creepy photos

[shy kelp]

https://media.discordapp.net/attachments/855828060577464370/964097579564564480/CatHuh.gif

[plush drift]

thank you

[shy kelp]

the face detailer strikes

[neon heath]

Wtf

[autumn forum]

Don’t use the refiner problem solved lol. Refiner is only good at realistic stuff

[plush drift]

ok, seems like the chat is pooped out for now,

@boreal bough let's catch up soon,

[hoary saddle]

is there a py template for making comfy nodes?

nm, think i just found it

[shy kelp]

💀

[hardy cipher]

https://github.com/comfyanonymous/ComfyUI/blob/master/custom_nodes/example_node.py.example

I'm assuming you found the same thing?

[hoary saddle]

close

[candid walrus]

last i read you were having issues where you couldn't get your loss to go down, bullet list what have you changed sense then?

[high skiff]

it seems as tho the issue was CUDNN

its either that, or a code error I had with my captions

regardless, I seem to have fixed it

my LoRA's were legit empty

[candid walrus]

nice, so you reinstalled/built cudnn?

haha

[high skiff]

like, learned nothing through over an hour of training

indistinguishable from base SDXL

[high skiff]

skipped CUDNN as a whole

[candid walrus]

yeah that wild

[high skiff]

I did have an error saying it couldn't find my captions, but without captions, it should have learned way more destructively, not less... so IDK

I did like 3 dozen runs

so far the very early tests for this realism LoRA are showing insane promise

[high skiff]

like this one

Identical prompt, seed, everything, just with the LoRA enabled on the second

[candid walrus]

so skipping CUDNN, was that a setting in kohya or like a flag?

[high skiff]

You have to do a whole separate process now to do it

[candid walrus]

(not that i have the same issue, just interested in the workings of it)

[hardy cipher]

second one less cinematic to the max

[high skiff]

so I just skipped that process

[high skiff]

the point is the second one looks like a real person, not some ransom plastic guy

reminds me of

[hardy cipher]

yes

[vale eagle]

You might missed this setting for your caption. The default wasn't txt as I remember

[candid walrus]

are you using kohya?

[high skiff]

this guy from spongebob lol

[high skiff]

the problem was I used a friends caption script, which was a little bugged

so it output images as .jpg.txt or .webp.txt

Which Kohya in turn said it couldn't read

[candid walrus]

ah

you got a local windows intall of kohya i'm guessing, so you've tweaked out the cudnn

[high skiff]

no, I just didn't install it

installing it is an after step

it doesn't just come in kohya

[candid walrus]

gotcha

[high skiff]

its a whole separate process you have to opt into

[true fjord]

What command to use for generate images?

[high skiff]

it even says now next to it as an option "Only do it if you have to, otherwise avoid"

Which I am not sure why it says that now... But it appears as tho I was infact having issues cause of it

[candid walrus]

i'm currently just using the runpod with the setup .sh script, cudnn is already installed on the docker when i get there, the script then does tweak the install to match requirements but yeah, its all self contained there. I don't have enough vram to run local like id want it to

[high skiff]

oh lord lol

[candid walrus]

are those bottom teeth or tic-tacs

[candid walrus]

less teeth = less tic-tacs

[high skiff]

even on subjects, scenes, and clothes I didn't train on, with a prompt optimized for SDXL base realism, I still think my very early LoRA test is providing awesome results in terms of realism improvement

My name is Markiplier

ah damn, he deleted it lmao

the grain/scratchiness in my LoRA images is from under training, not the dataset images, just so you all know

the whole LoRA is very undertrained just to see if it would indeed do something

and it did!

[buoyant flame]

how would i get the AI to render on of these

[candid walrus]

yeah it's got a nice vibe going

we're being trolled

[buoyant flame]

me?

NO

im trying to get help,

AN TOSE ARE NOT A MANS GENTIALS

[candid walrus]

i'm just messing it does look mighty suggestive tho

[high skiff]

So far the dataset is 60 images, trained for only 50 steps total per image

I wanna get that up to 200 images, 150 for humans, 50 for others

And I wanna restructure the tags as a whole, cause I know how to better trigger the LoRA's now

[buoyant flame]

its like one of those clockwork doll keys

[high skiff]

all images are at/around/above 4k res

[candid walrus]

mabye 'two letter Ps back to back' jk now i'm trolling lol

[high skiff]

back to unsplash I go!

[candid walrus]

so with 50 steps total per image what does that put your epoch/repeats numbers

[inner stratus]

Hello every body.
I tryng ti use SDXL base 1.0 in web UI A1111 and ist dont works.
when I select the model in the checkpoint box it generates the image error and goes back to the last checkpoint I had selected.
I use "git pull" to keep A1111 updated.

Tnk for any help.

[high skiff]

10 epochs, 5 repeats

[candid walrus]

best i can think of is try to use some sort of controlnet

[high skiff]

like I said, very under trained haha

but god damn, is it already showing huge improvements

[unreal plover]

i mixed voxel lora and pixel lora

[high skiff]

that looks fucking sick

damn that looks good man

[unreal plover]

yes i like the syle too

[candid walrus]

yeah i've done a disposable flash photo lora, on sdxl it started to change the faces, so my dataset isn't 'diverse' enough, though i then did try it on a 1.5 model and while it did change the faces less, i like how it did it at least. But yeah maybe this weekend i'll spend some time messing with the data set to get it more diverse

[unreal plover]

voxel syle, lora:VoxelXL_v1:0.9, a small village with people talking in front of the door, at night, backlight, miniature, tilt-shift, film, bokeh, professional, 4k, highly detailed, lora:pixel-art-xl:0.75

[candid walrus]

that'd be a dope video game

[high skiff]

god damn that is a bright jacket

jesus lmao

[candid walrus]

red was like 'there can be no shadows'

[native knot]

https://giphy.com/gifs/vampire-horror-dracula-ZlSvoOqJLlbMc

[high skiff]

Shadown't

[shy kelp]

yoink ?

[high skiff]

yoink what?

[shy kelp]

is that a yoink from a stock

[high skiff]

oh, just a screenshot

its from unsplash

free high res/high quality stock photos

that one is a paid image tho

[shy kelp]

what if

[high skiff]

was just admiring the BRIGHT

[shy kelp]

using ultralytics text detection + automatic inpainting

from interrogated prompts

[shy kelp]

🧠

no more captions anywhere

[lilac raven]

Was more ControlNet released for SDXL?

[high skiff]

lmao

[vocal rapids]

only canny now.....

[trim orbit]

https://tenor.com/view/zulupif-pif-gif-18916841

[late marsh]

BONK

[soft zealot]

I've now just linked those fields to my main width & height selectors

[high skiff]

I would do the same, but for whatever reason comfy doesn't let me pair the value going from the width and height into the latent

so I have to have 2 different primatives for width and height

[soft zealot]

ahhh Im not using Primitives becasue of that, Im using an Int Node from Impact Packs (I think it is)

Yes it is. Experimental ImpactInt

[soft zealot]

sorry I misunderstood , you're on about splitting the output of empty latent image arent you?

You could do that with a custom node that reads the size from latent but irs easier to have 2 inputs to it either from a primitive or other node

[high skiff]

oh no, I am just trying to use a basic primative to control the side of the latent and actual image widtch at one

its not that big of a deal

[trim orbit]

what is a big deal though is a new undocumented comfyui hotkey

[soft zealot]

ah whic is what I do .

I have 2 ImpactInt nodes as the width and height inputs , They go to 2 reroutes in my busbar and then from there they split to the latent space and the text conditioners

[gloomy barn]

I tried loading that cube image in comfy, it even claims bad models, although I select models I have installed

[trim orbit]

ctrl+b to make a node act in bypass mode

i love it

[high skiff]

wait, like it disables it?

[trim orbit]

yeah like a lora, it just passes through instead

[high skiff]

oh thats hot

[trim orbit]

SO HOT

[high skiff]

I need to use that

thank you

[trim orbit]

ctrl+b

for bazongas

[high skiff]

that will make my data collection for my LoRA comparisons really easy

[trim orbit]

yeah lora wiring was a pita lol

[gloomy barn]

what is this? AITemplateLoader

It says that node is missing

[trim orbit]

AIT are a whole new framework for image generation

models recompiled as AIT

[gloomy barn]

ah, so I cant run that workflow

[strong field]

oo thats hot

[ionic dragon]

used face detailer in winston's workflow
did the lora get overtrained?

[gloomy barn]

is there a way to install that?

[gloomy barn]

If you said "in bed", then the lower left image worked

[strong field]

https://github.com/FizzleDorf/AIT

if you using SDXL make sure you download SDXL branch zip

[ionic dragon]

i just want to check the face, but i dont think its like Selena Gomez

[trim orbit]

its not for the faint of heart who can't get things working on the first try and give up forever

[gloomy barn]

I cant even figure out what dir to run that in, much less downloand a branc zip

[ionic dragon]

@boreal bough

[uncut fiber]

i installed it throught Manager, but got error not valid win32 apps.

[autumn forum]

As much as I want to try this out I feel the best way to run this is to wait for comfy to implement it. I’m too scared it’ll break something😅

[strong field]

its a hack job right now

[gloomy barn]

I now git got it into my custom nodes folder...SDXL branch.zip Where is that found?

[strong field]

but when it works it works

when it doesnt, well you get this

[autumn forum]

Lmao

[gloomy barn]

Copy your comfy folder. Work on the copy.

[autumn forum]

I’m good lmao. My comfy folder probably like 30 gigs or something

[gloomy barn]

that's not big

a lot of it is models

make a copy of all but the models

[soft zealot]

@clever verge OK so tying this upgrade again and same thing as where I eneded up.

For refence I deleted the entire original install and fi a 100% fresh clean install

It all works as adverised providing I don't select sai-base in any of the 3 Primitives i use as remote controls

If I convert the Input to widgets and select sai-base same thing,

Ahhh thats interesting, in the new nodes its "base" not "sai-base" so maybe I do need to either replace the nodes or edit the jsons , hmmm

[gloomy barn]

then if you need to revert to the copy because something broke, copy the models there

[indigo carbon]

not needed anymore. we pushed it to master branch

[trim orbit]

maybe two issues going on here. your lora is little underfit and the facedetailer has settings that are putting the face a little off kilter

i feel like these detailer passes benefit from controlnet so much and today i want to try to get the canny model wired up to the facedetailer

[gloomy barn]

OK, so I have git got the thing. Someone said the sdxl zip isn 't needed, so I will try running this

[late marsh]

Nightmare fuel

[soft zealot]

@clever verge renaming th e"base" in the json to "sai-base" seenms to have fixed it although to be sure everything aligned going to completely replace all the nodes and primitives and use your default naming

[ionic dragon]

i didnt get you

[trim orbit]

underfit lora meaning you didn't train long enough or had bad settings

[strong field]

thats great news

[trim orbit]

off kilter faces meaning when the facedetailer pass hits, it shifts the face a little to the left. like it looks pasted on

[indigo carbon]

why do you need face detailer? are faces bad with whatever you are using?

[trim orbit]

sometimes. i find it helps. i don't "need" it. but sometimes with portraits it does so well.

it'll really drive eyes home

[ionic dragon]

yeah true, even polished it

[gloomy barn]

That's what it generated from your cube image

[unreal plover]

try heun sampler to see

[candid walrus]

honestly selena i think is a hard lora to make, her look has changed a lot through the years, so the data would have to be consistent to the look aimed for

[indigo carbon]

nice! by how much did this boost your performance? I might work on a table showcasing it

[gloomy barn]

No idea. I haven't been running comfy. It took a long time to generate the image

[late marsh]

RUN!

[strong field]

@indigo carbon Have you tried the AIT VAE decode?

[indigo carbon]

weird, that workflow should be a few seconds. what GPU do you have?

[indigo carbon]

works only on FP16 VAEs

[trim orbit]

you can usually count on them to wing instructions, do one key thing wrong, and declare with certainty how things work

[ionic dragon]

didnt use face-detailer

[strong field]

lol should have asked earlier instead of troubleshooting black screens

[trim orbit]

almsot better. you might have a bad prompt or settings on your facedetailer pass then

[trim orbit]

woah what is that thing?

oops i did that wrong

woah what is it?!

[ionic dragon]

didnt use facedetailer

[ionic dragon]

i trained it using 16000 steps

[trim orbit]

thats what i mean. the ones with facedetailer might've been set up bad. those are better

doesn't seem like an underfit lora

[ionic dragon]

ok

did it overfit?

[trim orbit]

if it's over fit then usually you'd see the subject in one pose, one expression, one outfit, one location

mode collapse

[ionic dragon]

oh ok

[indigo carbon]

you can use the AIT workflow I made and add load_VAE select SDXL_VAE_FP16, then use that on AITVAE decode/encode

[trim orbit]

i think you got a good lora there but that is a lot of steps for it. experiment with it at lower strengths to see what you get.

as for facedetailer, i find prompts like "an extreme closeup of subject, adjective eyes, adjective lips, expression, other detailing tokens" are good prompts

[strong field]

have you noticed improvements using that node?

[ionic dragon]

how can i improve the lora?\

[strong field]

not sure if worth the trouble was more experiments

[trim orbit]

peircings in the face detailer pass help a lot too for that goal

[vale eagle]

AITVae is available now?

[indigo carbon]

don't think so. I'm generating images in ~10-12 seconds anyway

[indigo carbon]

so, whatever

[ionic dragon]

i dont know how face detailers work, so i used winstons

[trim orbit]

in a few cases i have. it's just another tool and isn't one i use on every generation. the workflow i got for it has a two reroute nodes that i discnnect or connect when i want it

[trim orbit]

#✨｜sdxl message this is what i set up

might be out of date

[trim orbit]

i'm still waiting for the day when the JWST catches a glimpse of one of these and it sees us see it

[clever verge]

Hmm, sai-base is renamed to only base. Should be enough to just select base again instead of sai-base? I can test it to see if I get the same problem.

[indigo carbon]

only on the FP-16-VAEs

[autumn forum]

I’m curious would this work with custom node samplers? Like impact pack and stuff that uses pipes

[indigo carbon]

did you try it on my AIT workflow? I'm considering adding it

[strong field]

dont see why not

setup the model prepipe and done

[autumn forum]

If so I may end up trying it.

[strong field]

i think ive tried a similar thing but i dont have that workflow atm

[soft zealot]

im suspecting some odd caching going on. If I went in and renamed base to sai-base all was happy but if I put a new SDXLpromptstyle node in(after another clean isntall) it simply wouldnt accept an n existing Primitive input so I ended up with all new SDXLprompt style nodes & all new primitive nodes

[hardy cipher]

they're out there waiting

[autumn forum]

Also do I just install the custom nodes from the GitHub or is there anything else involved with setting ait up?

[strong field]

i think clone the git, and load a workflow

takes some getting used to on the prompting and styling

[indigo carbon]

just master branch, it will work. for live previews (with my workflow atleast) you need the Comfy_manager live preview.

[gloomy barn]

When one of those realizes I can see it...it will be terrified

[autumn forum]

Gotcha

[strong field]

Prompt work needs to improve on my end for sure

[gloomy barn]

Why in an image of a cube do you have things like bad hands in negative?

[clever verge]

Thinking about changing the handling of no matching style selected to revert to "base" style. Not sure if it fixes all your issue though.

[soft zealot]

in orrder to find out I would need to undo what Ive just done, delete the upgarde , redo the upgarde............................

errrm

🙂

[clever verge]

Did you get this error in the console? "No template found with name '{template_name}'."

[ionic dragon]

@trim orbit

[soft zealot]

no

[trim orbit]

they do look a little plasticy. have you tried generating with the lora at a lower strength?

[soft zealot]

@twri it literally just turned the style input red and went to prompt finished in like 0.5 seconds

[ionic dragon]

0.7?

[trim orbit]

have not. i haven't given ait a shot yet since i've been playing with canny on sdxl a lot

[gloomy barn]

on that cube image, if I make it a different size, it makes a stretched image. I must be missing someone on the size

[hardy cipher]

that's not what you want

[clever verge]

I tried to reproduce but Comfy itself catches the error before my code even runs.

[indigo carbon]

you didn't resize it properly. with AIT it's slightly complicated changing AR

[gloomy barn]

I can't just change the size in the workflow?

[soft zealot]

oh well, Im sorted now ,. Just dont break it again lol

[indigo carbon]

you can, but there are multiple values that need to be changed when changing size with AIT

[gloomy barn]

How is AR changed?

I only see it in one place...Empty Latent Image node

[indigo carbon]

you need to do math for that. in order to change the AR you need to change the upscale size and latent size. this is the reason I made two variations of my workflow- one for 4:3 and one for 1:1

[ionic dragon]

what do i want

[gloomy barn]

How do I set it to change?

There are nodes for upscale, but they don't show any numbers

[hardy cipher]

probably no errors, lol. have you narrowed down what is causing the issue?

[gloomy barn]

there is nothing in the upscale node properties to change

[ionic dragon]

no will figure it out

[indigo carbon]

yes there is.. if you don't understand how the AIT workflow works, you shouldn't be messing with the nodes too much.

[hardy cipher]

try using just the one model for the face detailer. the top one, and leave the other two open

[gloomy barn]

I want to change the size...don't want square images

[hardy cipher]

I don't know the specifics, but that's how I got it to work for me

[clever verge]

I'll try my best! Think I'm innocent this time, or so I hope. Possibly limitations of litegraph.

[gloomy barn]

How does it work? If I don't understand now, I can never understand it or change it?

[hardy cipher]

like this

[ionic dragon]

lemme check

[hardy cipher]

when I attempted to use the other two model inputs it gave me a barrage of errors\

[indigo carbon]

you can, change the values of the upscale to being exactly 2x the latent size, that's it

[gloomy barn]

There are no values...at least I don't see any

[hardy cipher]

I'm sure the other two models help with things, but I really don't know how to get them to work

[ionic dragon]

i have these

[hardy cipher]

well just try running it to the one input like I did. it might not be ideal but at least you'd narrow down what the issue is

[azure oxide]

you could also just wait for comfy to implement AIT himself. it wont require any knowledge or learning or nodes. its either on or off is how i read his comment

[hardy cipher]

but what if I want to be required to have knowledge and learn and use nodes?

[gloomy barn]

The nodes for upscale have no number settings, nothing to change as far as AR upscale, times to upscale

[ionic dragon]

change ultralytics to mmdet?

[hardy cipher]

if you have that option. I don't know why, it's just the only configuration I was able to get working without error

[indigo carbon]

this works for everybody else.. you are the only one experiencing this. looks like a problem in your end

[gloomy barn]

What problem? Are there supposed to be numbers there?

[trim orbit]

from what i'm seeign, changing sizes with AIT is a huge pain in the butt. i'm fine with the speeds i've got for now. i don't want to add more speed bumps to my workflows

[indigo carbon]

yeah, there should be numbers.

[gloomy barn]

pain in the butt, but how? What steps are taken to do it?

[gloomy barn]

Odd that there aren't. Ive tried in three different browsers

[trim orbit]

i dont know. i just see this big back and forth about changing a size being difficult and needs to be done in multiple places. i'm sure i could get it done because it sounds straight forward, but it has to be done in the first place is my issue

[hardy cipher]

there are different upscaler nodes. some have the option to adjust. some just use the default of the model

[gloomy barn]

what browser do you run it in?

[indigo carbon]

@gloomy barn what AR do you want, I'll make another version of my workflow for it.

[gloomy barn]

I want to be able to change it based on the image

not have to have someone else make specific ARs

[hardy cipher]

I believe by default they mostly do 4x

[trim orbit]

well i often change it multiple times in my sessoins. thanks for the offer though. I'll keep an eye on what you're putting down you can be sure of it.

[gloomy barn]

4x...that should be 4x the settings for width and height, no?

[indigo carbon]

no, it's 2x

[gloomy barn]

it doesn'\t matter the x

[trim orbit]

a 4x upscale means 4x the pixels. 2x each dimension on 2 axis

[gloomy barn]

If I set 768 width and 1024 height, then it's whatever x those, no?

[indigo carbon]

I think I can fix this by using some kind calculation nodes

[gloomy barn]

instead, it seems to be using some pre-set value for the upscaled image

[hardy cipher]

yes. what I do is 4x, then use "image scale by ratio"

[trim orbit]

ooo oyou're talking about automating all the different changes and only setting it once? brains

[indigo carbon]

its 2048x1532

[gloomy barn]

so that's the issue

the upscale is hard coded AR

[hardy cipher]

so it upscales by 4, then I multiply by 0.5 on each side

so brings out detail and then shrinks it

[indigo carbon]

yeah, we might be able to predate comfy's solution for AIT

I just always do 4:3, so this didn't effect me

[gloomy barn]

I dont mind having to set it....but how?

[trim orbit]

i do like 4:3 too

i like those tall mona lisa ar's and 5:4 sometimes

[gloomy barn]

4:3 tends to cut off people...not show them entirely

[trim orbit]

tall ones i just sorta eyeball in.

[trim orbit]

with 1.5 maybe but not with sdxl and king level prompting. maybe even lord level.

[hardy cipher]

try that

[gloomy barn]

eyeball what? how is it set? How is the final output size set if not calculated from the latent input numbers?

[trim orbit]

hell i bet a landed gentry could get away with uncropped xl generations at 4:3

[hardy cipher]

4x then 0.5x

or whatever you want I guess

[gloomy barn]

I will try some with the default values

[hardy cipher]

I like what the upscalers do, but then I don't necessarily want 16 megapixel image

[trim orbit]

this is a weird personal workflow i've developed for finding a good odd cropping i like. get a sweet phoot with a sweet subject. start+shift+s to get a screenshot selection, copy the subject with an eyeballed framing. save it. take that image's AR and calculate from there

[hardy cipher]

sweet phoot

[gloomy barn]

I cAN CALCULATE

[trim orbit]

mathematical!

[gloomy barn]

How do I set it? How do I put in the numbers in the workflow?

[trim orbit]

you gotta have that sweet phoot first tho

[hardy cipher]

search for "image scale by"

in nodes

[trim orbit]

the entire workflow breaks without it

[gloomy barn]

I know the AR I want

[indigo carbon]

I'm making a version of my workflow that doesn't need me making different variations for different ARs

[trim orbit]

sometimes the image will make an AR thats weird like 4:13.48 so i'll just round it

[gloomy barn]

ok

[indigo carbon]

just be patient, give me a sec

[hardy cipher]

the pre-made workflows are a double edged sword

[molten gull]

does anyone know how to call a comfyUI json file from a python program ?

[hardy cipher]

on the one hand they make things more accessible. but then people have no idea what they're really doing, lol

[trim orbit]

sometimes i like to do that golden ratio plus one more square. that's a nice tall AR

i forget the math on that one

[gloomy barn]

The fing hat

[trim orbit]

easy to not cut yourself on a blade once you know it's a sharp blade on both sides. just use it right.

people though.

[soft zealot]

or if you're pervse like me then do it this sort of way so on your front end you put in a whole number to multip[luy by rathe rthan thinking about converting 🙂

[hardy cipher]

part of me wants to learn more and make workflows like that. but then I feel like I'd have to deal with a bunch of irate people, if anyone actually used them

[crisp owl]

My img2img one on civitai have something like 330 downloads now and I've not had a single person come asking questions

[hardy cipher]

maybe the key is using common nodes

[hoary saddle]

mine are a mess

[crisp owl]

I made sure to include a bunch of notes though

[ashen oracle]

can you share a link to that one soles? i'm looking for a new img2img one to try, sd ultimate has too much going on

[gloomy barn]

no such thing in this workflow

[crisp owl]

https://civitai.com/models/123048

[trim orbit]

why won't it do the fingers i want!? make it do the fing!

[hardy cipher]

well you don't have to use that workflow, but do what brings you joy

[gloomy barn]

I dont know how to make one from scratch.

[hardy cipher]

well you also don't have to use every node in it

most of the workflows I've downloaded had some sort of conflict or node I didn't have, so swapped out for things I could use

[soft zealot]

oh you've noticed some of my response to people then 🙂

[hardy cipher]

nah, there's a right and wrong way to approach it I think

[trim orbit]

https://www.youtube.com/watch?v=AbB33AxrcZo detweiler's intro to comfy is essential reall

has all the pro tricks

[hardy cipher]

it's perfectly fine to ask questions and try to figure out issues

[trim orbit]

other youtubers are just looking for clickbait and "like an subscribe" junk it seems. det puts out real lessons

[soft zealot]

my biggest trigger is people who cant or wont read

[gloomy barn]

I dont see in the workflow how it's setting the final AR after upscale. It's not calculated - it's set. However, I can't see where it's set.

[ashen oracle]

can be difficult to get all the custom_nodes installed if you aren't technical. when i set up winston's workflow i had to use chatgpt4 to rewrite some of the bits where things wouldn't import - but it only took a few minutes for each one to fix. and worth it

[trim orbit]

this is the video where i learned how to drag out from a connection and then it'll give you a menu to make the new node there, with all the ones specific to that output in quick pick

so that helped

[gloomy barn]

______________ CHANGES EVERYTHING!!!!!!!!!!!!!!!!!!!!!

[hardy cipher]

people have trouble thinking abstractly

[ashen oracle]

this is why prompt engineers are the future

[trim orbit]

abstract people have trouble thinking

[hardy cipher]

I don't know about all that

[gloomy barn]

I remember nodes like this...but can't recall if it's from the past or future

[hardy cipher]

fragmented people maybe, but I think ther'es a balance

[soft zealot]

Days of Future Past

[trim orbit]

nodes or nodules. whats more fun?

[gloomy barn]

There is a place for setting the AR: Empty Latent Image. But that's it. If I set it to anything but what it is, then the resulting image is still the same AR as original before I changed those numbers - it stretches

[soft zealot]

@clever verge Im an idiot, Was just trying to work out why I wasn't getting my .txt file anymore.

Guees I forgot to connect from the new prompt nodes lol

[trim orbit]

https://youtu.be/kkYaikeLJdc 3 hours old. he's on this one a little late.

[gloomy barn]

I am making a video: Introducing AI Image Generation

Soon, something like this will be available to use both online and run locally. I expext it will take substantial VRAM to run this software, to generate images. It is coming soon!

[ionic dragon]

i didnt get you, i am very new to face optimizer

[late marsh]

Yeah I follow this channel and he's been pretty late with these. Like even weeks late

[hardy cipher]

in order to be successful on youtube you must be prepared to make extremely uncanny thumbnails of yourself for each video you make. so when people scroll they'll see your creepy exaggerated expressions and click

[trim orbit]

he might've been waiting on a real paper but there's just a report

[late marsh]

I hate those stupid faces

[trim orbit]

since his channel is all about papers afterall

[hardy cipher]

tilt shift

[gloomy barn]

Here's another video I will make: Introducing the Internet

Soon, we will be able to share information on a world wide...call it a web....of connected computers.

[trim orbit]

would love a HOMM game in this style

[unreal plover]

Mee too

[hardy cipher]

well I actually just tried to use it for the first time the other day. just use "bbox_detector." don't "use _sam_model_opt" or "segm_detector_opt"

it might not be ideal but it'll work

[gloomy barn]

I want to know how to do what's shown at 4:09. Make a video of me, use it to make a video of some character.

[trim orbit]

when neuromancer came out in 1982 describing a global cyberspace that a significant amount of people spent their time in for the near future, William Gibson was called a mad man

[hardy cipher]

qwerty, I just tried to connect to the other two model inputs in all possible configurations that my install will allow and I got errors each time. the same errors you got. or very similiar

[gloomy barn]

yes

[trim orbit]

he's talking about what controlnet has done for 1.5 , referencing another video about it.

[gloomy barn]

In the future, you will think things that become real. We are in the very beginnings of such generation.

[soft zealot]

why not call it a mesh?

[gloomy barn]

I see. However, I'd like to know how to get such good output out of controlnet with 1.5

[gloomy barn]

Good idea.

[trim orbit]

that video was a popular one on reddit. i don't think the author broke it down. same techniques everyone uses though. controlnet and ebsynth and tailoring it up a bit

[gloomy barn]

the outputs I've seen have been much less stable than that one

[trim orbit]

there's no pushbutton results with those. the unstable ones are just half assed efforts

[hardy cipher]

the stable diffusion subreddit really doesn't appeal to me these days. maybe I just have a skewed perspective. but it seemed a bit more constructive several months ago

[gloomy barn]

people are making such videos. I can't even get the AR changed on a workflow

[trim orbit]

good results are skill and eye for detail

[hardy cipher]

good results are the culmination of many bad results

[trim orbit]

yeah the most talented people don't often show off their failures or rough works

[crisp owl]

many, many failures

[hardy cipher]

I think that's the case with about anything

but then people from the outside only see the wins and think there's some magic ticket

[gloomy barn]

we see what's on stage for others, then we compare what's going on backstage in our production, find ourselves lacking

[ionic dragon]

can yuou share the workflow

[hardy cipher]

one minute

[gloomy barn]

I have to get this Brave homepage changed but cant figure out how to change it.

[indigo carbon]

done

this calculates on its own

[upbeat summit]

what does?

[indigo carbon]

no need for me making new workflows for different ARs, now it adjusts itself for every AR

[unreal plover]

I want this videogame too

[upbeat summit]

wohaaa - those are awesome!

[unreal plover]

voxel syle, lora:VoxelXL_v1:0.9, None A group of madmax shoting at a giant mecha in the desert with dunes and sandstorm, atomic explosion in the background, miniature, tilt-shift, film, bokeh, professional, 4k, highly detailed, lora:pixel-art-xl:0.75

[upbeat summit]

cinematic 3d voxel

[unreal plover]

atomic explosion missing sadly

[upbeat summit]

what a great prompt. nice work!

[hoary saddle]

i'd play the hell out of it

[indigo carbon]

@gloomy barn this should fix your issue.

[gloomy barn]

Thanks for making it. I am trying it now.

[hardy cipher]

lots of nonsense there, but just look at the face detailer

linear node paths

very efficient, guys

[unreal plover]

ConfyUI don't have an auto Lora loader from the prompt ?

[upbeat summit]

new arrangement?

[gloomy barn]

it works. For some reason it takes a long time to generate once it gets to ksampler

[hardy cipher]

that was from a few days ago

[upbeat summit]

ah I see - it mutated 😄

but this looks very clean

[autumn forum]

I feel like I’m a few months comfy ui workflows are gunna look like rgb rainbow with the amount of modifications some people have😂 I’m all for it. Love customizability.

[strong field]

what card you using

[gloomy barn]

4070ti

[strong field]

you should be double that mx

[hardy cipher]

I usually just clear it and start anew, lol

[ionic dragon]

which pack to get?

[hardy cipher]

searge nodes

[indigo carbon]

that doesn't make sense. I didn't change the K_sampler settings. that's a problem in your end, I built the workflow in a near perfect way for AIT..

[gloomy barn]

maybe I have to reboot

[hardy cipher]

looksl ike tinyterra nodes as well

and maybe impactnodes?

once you install node packs restart comfy and refresh your browser

[indigo carbon]

this is about the speeds you should be getting =]

[ionic dragon]

can you send the link

wait

[hardy cipher]

man, they're in the comfy manager

[ionic dragon]

i'll check

[hardy cipher]

btw, I don't actually know what the node packs are by heart. I just google the names of the nodes

[gloomy barn]

I had sdnext and comfy running at the same time. I guess it doesn't like that.

[hardy cipher]

fighting for resources

[high skiff]

Man, I love pseudo lmao

He's so brutally honest, and I love him for it lol

[elfin flare]

What's SDXL's max coherent resolution?

[upbeat summit]

it depends - everything that fits in 1024x1024 is in spec

[elfin flare]

Of course, so 1280x720 or vice versa

or even 1536x512

[upbeat summit]

correct

right now I'm getting great results in 1280x1024

[elfin flare]

Intel arc's version for WebUI is heavily improved atm.

I can now generate ridiculously high resolutions.

[upbeat summit]

I made many images in native 1920x1024 - you get duplication and mutations, but when it works, the details are really good

[elfin flare]

I assume this is with a combination Pixel Art+Voxel LORA

[trim orbit]

i had a minute so installed ait and dragged one of @indigo carbon 's images into it. big huge boy generated in 40 seconds

that is quite remarkable

[indigo carbon]

40 is alot for my workflow.. what gpu are you using?

[trim orbit]

it must've jsut been warming up the first time. this oen took me 16 seconds

[indigo carbon]

16 seconds is great, AIT is definitely functioning correctly

[ashen oracle]

AIT is still only being used for workflows without the refiner stage, right?

[indigo carbon]

nope, it works on the refiner in my latest workflow =]

[ashen oracle]

gentleman and a scholar

[indigo carbon]

also my latest workflow supports any AR, unlike my previous AIT ones, same efficiency as the others built for specific ARs.

[ashen oracle]

AR = ?

[indigo carbon]

aspect ratio

[ashen oracle]

testing now

[unreal plover]

no the last one is only pixel art at 0.75

[unreal plover]

If you make it, you will be rich

[indigo carbon]

?

[ashen oracle]

he means the lamp, it's awesome lol

wtb for corner of room

[indigo carbon]

ah ,I see

[upbeat summit]

biggest lava lamp

[unreal plover]

IRL, this kind of stuff dont exist at this size

[ashen oracle]

it also doesn't look as cool

[ashen oracle]

impressive.

[indigo carbon]

by how much did it speed up for you? I'm making a table for average speed up caused by AIT

[ashen oracle]

i'll try to do a side by side test

is there an easy way to "turn off" AIT happening in this workflow so i can do an easy side by side test?

this should do it, no?

[indigo carbon]

yeah

[ashen oracle]

ok will do it 3 times after model loaded and give results for each

``rtx 3090

without AIT (4.32 - 4.60 it/s)

29.95
29.88
29.68

with AIT (5.65 - 6.55it/s)

24.42
22.40
22.25``

[ashen oracle]

@indigo carbon

[vale eagle]

Similar result using 3090

[unreal plover]

Coming soon

[elfin flare]

Trying to make a low-poly lava lamp.

[unreal plover]

https://www.reddit.com/r/StableDiffusion/comments/15oeymz/depth_sdxl_controlnet_coming_soon_brace_yourselves/

https://huggingface.co/SargeZT/controlnet-v1e-sdxl-depth

[indigo carbon]

what CUDA version are you on?

[rose echo]

Does anyone know a reliable way to prompt for an image to look down on a character to up at a character? I've tried all kinds of variations and tried using more natural language ect, but it don't believe me.

[steel mist]

I ran into that a lot, used a lot of negative prompting

[ashen oracle]

Cuda compilation tools, release 12.0, V12.0.76
Build cuda_12.0.r12.0/compiler.31968024_0

[indigo carbon]

you can get a bigger boost from AIT by switching to CUDA11.8

[ashen oracle]

think that will break anything else? lol

[indigo carbon]

no, you might need to reinstall pytorch though

[vale eagle]

I don't think so. I get similar result with 11.8

[indigo carbon]

but somewhat faster, aren't they?

[vale eagle]

4.x to 5~6

[trim orbit]

my cobbled up sytan workflow is on left. ait on the right. i was wrestling with both and i really think there's a big quality tradeoff for the speed on ait.

[ashen oracle]

it did seem like i was getting different results generally, didn't take time to evaluate better/worse

[vale eagle]

just apply ait to sytan workflow

[indigo carbon]

those are different ARs and gen settings. AIT is reproducible, it should be about the same image by using AIT with same gen params

[trim orbit]

well yeah i was tuning and tweaking where i could on each. i just wasn't getting anything i thought was good quality. it all looks like low resolution bad upscales. probably just doing it wrong. maybe xiao is right and its the settings not the ait

[vale eagle]

I was able to apply AIT to my custom workflow and also speedup with correct setting.

[indigo carbon]

..

[trim orbit]

those have better consistent details. i guess it just doesn't like how i approach settings. i don't understand the step counts being used anywhere in comfy really.

i still think there appears to be quality tradeoffs on yours

looks more like a gan upscale than a diffusion

[elfin flare]

I do wonder what comfyUI is like.

Since I still use Vladmantic for Arc.

[molten gull]

does anybody know how to call a comfyUI json-file via python?

[indigo carbon]

I already made a comparison of that. image with AIT looked slightly higher quality

[trim orbit]

i've seen yours. i can't replicate. lk99 situation i guess.

i've only spent the psat hour with it of course.

[molten gull]

lk99 is the new "cannot reproduce" meme, is it ? 🙂

[vale eagle]

I made my comparison on human face and texture. I don't think they are much different.

[indigo carbon]

idk man, mine come out consistently extremely high quality..

oh wait

I know what's happening to you

[indigo carbon]

you are using CPU ARG, aren't you @trim orbit

[vale eagle]

Are you using GPU only option? @indigo carbon

[indigo carbon]

no, GPU ARG. it interprates seeds in a better way

[trim orbit]

no idea what that is. it's telling me that ait is just not ready for workflows. i dont think i would've set any option that said use cpu for anything

just default comfy. i've never touched params or nothing

[indigo carbon]

it's automatically like that. you can set in ComfyUI\Nodes.py to have the GPU handle seeds. it's way better that way IMO

[trim orbit]

but.... why?

you're talking about the RNG to come up with the seed number?

[indigo carbon]

no, the RNG that interprets the seed number. the number itself is always by CPU

[trim orbit]

that's like "31337" as a seed delta mattering

[indigo carbon]

even with same seed, prompts, everything- the images with GPU ARG are slightly more crisp

[trim orbit]

i dont think that would matter and i trust comfy to have the best option enabled by default if there were any clear benefits to either. i think it's just a thing people need to worry about to reproduce results across tests.

crispyness isn't my issue though? it just looks like when i use AIT they're coming out of a gan upscaler

[indigo carbon]

maybe, it might be because im used to GPU ARG, it's what A1111 used

[indigo carbon]

this looks like something a GAN upscaler will make?

AIT never seems to make results worse, if not a little better. it's just an insane performance boost

[strong field]

once you find the right tweaks its zoom zoom zoom

[indigo carbon]

yeah, and my workflow seems to use most of them

[strong field]

honestly, i keep coming back to yours cause i keep breeaking it

minor tweaks for awesome results

[indigo carbon]

I did make it more flexible by making it work with any AR, but yeah

[strong field]

thanks for the tips

[soft zealot]

apologies to those dyed in the wool coders & devs in the audience but.......

You know that old chestnut of trying to work out which package a node has come from when you're slecting from already installed nodes/packs?

Just use f;lipping notepad++ and search in files for the name of it !!

Why didnt I thinkl of this before

[tender timber]

And with standard 1.0 checkpoint... I'm getting about a 50% speed increase over not using AIT.

[indigo carbon]

that means you didn't set it up properly. other people using AIT report making images in about 10 seconds.

[trim orbit]

sytans without ait and sytans with ait. i made that gpu change you said too. 19 seconds without AIT and 25 seconds with AIT. thats when i just transplant the nodes into sytan's workflow with the offset lora on the base

[tender timber]

I'm on a 3060 12 GB model. And... the only change I made to the workflow was to flip the checkpoint to the one I had

[indigo carbon]

it shouldn't decrease speeds.. something is wrong

[strong field]

Definitely agree no way this decreases speeds

[indigo carbon]

^

[trim orbit]

i just lose all fine detail when i have it in play. theres definately a trade off. i'm on a 4080 using xformers

[strong field]

FP16 right?

[trim orbit]

what does comfy use by default?

[strong field]

Yeah there are some detail workarounds for sure

[trim orbit]

i thought bf16 was preferred where hardware supports it and ADA definately does

[cyan crown]

there was a daily challenge in a AI group today themed Animal magician. SDXL did a great job!

[indigo carbon]

odd. for me it makes slight better results.. also for @upbeat summit and @strong field

[zealous viper]

did someone say animal magician

[midnight shuttle]

Is it possible that AIT uses slightly more VRAM and causes the 12 GB to be exhausted?

That would slow it down.

[indigo carbon]

I'm on a 4070TI, it increases performance over x2

[strong field]

3090 with same results

[indigo carbon]

generates all the images I share here in about 12-ish seconds

[strong field]

Prompting makes all the difference. I would say AIT is tempermental

[indigo carbon]

way worth it for the speed though IMO

[midnight shuttle]

AIT's speed is influenced by the prompt? That's interesting. Parts of the model are compiled and parts aren't?

[strong field]

I meant quality is impacted by prompt for me

I can’t run the same copy and paste prompts from other workflows

It’s weird honestly can’t say I understand that part

[midnight shuttle]

I need to learn more about what AIT is and how it changes the behavior of the model.

It seems like a model should be a model. If AIT affects the output of the model it would be useless for many tasks.

[indigo carbon]

no, it's not compiling any models, it's an optimized optimization

[strong field]

AIT not a model

It’s using a neural network to interpret cuda

[trim orbit]

when i up the step count to ones like yours on sytan's workflow, they make the exact same images with or without now. but i get 35 second either way.

the two workflows don't exactly match up settings well but i tried

i was using ddim before. i guess it doesn't do well there

[midnight shuttle]

A neural networkto interpret raw CUDA code? That's very interesting. I need to read up on that.

[indigo carbon]

it also doesn't effect prompts, the reason @strong field describes differences with the prompts is likely because they are not used to my workflow.

[strong field]

Definitely possible

[midnight shuttle]

AITemplate (AIT) is a Python framework that transforms deep neural networks into CUDA (NVIDIA GPU) / HIP (AMD GPU) C++ code for lightning-fast inference serving.
Oh, that makes more sense.

[indigo carbon]

it does very well with the workflow I built, idk why you claim it to be slower. it's about 2x speed for most people.

[trim orbit]

it was slower that one time. it's really all over the place. some cases it's faster some it's on par. that oen time it was slower.

maybe it's an ada architecture thing

[indigo carbon]

on my workflow? I get what you're saying, the first generation is longer because it starts up. but after the first one they should all have a consistent x2-ish speed up

[trim orbit]

its not drop in thats for sure. it's causing a lot of hitches

[strong field]

First gen is always slow. Gotta load model

[trim orbit]

your workflow i'll try again

[strong field]

It’s definitely a tinker toy

[indigo carbon]

like, my GPU should be slower than your 4080, (its a 4070TI) and I'm generating stuff like this in about 12-ish seconds

[strong field]

But I’ve had much better results after understanding what’s going on

[rustic surge]

what is AIT？

[midnight shuttle]

AITemplate (AIT) is a Python framework that transforms deep neural networks into CUDA (NVIDIA GPU) / HIP (AMD GPU) C++ code for lightning-fast inference serving.

Basically you write code similar to PyTorch and it converts it directly to CUDA or HIP.

https://github.com/facebookincubator/AITemplate

[rustic surge]

ok thank you

How can I use ait to accelerate in stable diffusion

[midnight shuttle]

It's not something you can just enable. The developer of the UI needs to change a lot of their code. Maybe Comfy supports it. I don't think A1111 supports it yet.

[rustic surge]

sad

[trim orbit]

with your workflow i get 17 second generations with ait and 25 second generations without. but this is not something i'm seeing transfer to any other situation

[cyan crown]

I must stop doing these 😄

[indigo carbon]

that's still a dramatic speed up

[upbeat summit]

why? they are really great 🙂

SDXL's magic special effects are so good

[strong field]

I have successfully made other workflows using AIT with significant increases in generating

[trim orbit]

i addeda lora to your layout and it broke it. what's with the clip skip on sdxl? isn't that a novel ai thing that doesn't matter

[cyan crown]

Because it's like a drug 🙂

[upbeat summit]

than you are right at home here 😄

[upbeat summit]

the camel (giraffe?) is awesome

[cyan crown]

thanks

[high skiff]

So my realism LoRA is a huge success

[indigo carbon]

no, just different interpretations. no quality difference what so ever. but hey, 17 seconds is much faster than 25 seconds.. this means it worked =]

[high skiff]

and it had the unexpected side effect ot also helping hands out a ton

[strong field]

I haven’t been able to just drop this into Searge, Syrian, or Winston workflows. I had to start from scratch and slowly experiment.

[crisp owl]

nice, even has correct finger crease lines

[high skiff]

testing some more

[cyan crown]

you solved your problems so

[high skiff]

also, damn my new LoRA is so much better at easy realism than SDXL base lol

[strong field]

Lol

[cyan crown]

what was the problem at the end?

[trim orbit]

loras do a really great job of shoving the model towards latent spaces it already knows very well. so a little lora goes a long way

[strong field]

That’s a relief. sDXL at its worst right now. Only improvements left

[trim orbit]

i got a theory that even the most basic lora with minimal training, will improve outputs

[high skiff]

not too sure, either CUDNN issues, or a bug with my captions

either way, it was hell to figure it out

base SDXL

[high skiff]

with my LoRA

[high skiff]

migth I add, I asked specifically for dappled lighting on her face through an open window

[cyan crown]

can you share the json? I'm still having problems

[crisp owl]

can't tell you how many times I go to scroll zoom-in with my mouse scroll on discord after working in the ComfyUI....

[high skiff]

my settings are not for general use, they are VERY specific to my LoRA, as I have multiple trigger words, caption droput, and its all tuned for my exact number of images, one more or less and it will destroy all of the gradient checkpointing and accumulation

it also doesn't appear to be ideal yet

[strong field]

The panning and zooming!

[shy kelp]

😂 male chicken magic

[cyan crown]

ok. I'm looking to train a style but it comes out very different from original images

[crisp owl]

having fun with these

[trim orbit]

most loras are baked with meta data the settings are in them. if you dont want to share them @high skiff keep that in mind

[high skiff]

I know, the LoRA is far from done

[shy kelp]

i see everyone having fun with loras

[high skiff]

the current dataset it 90 images, I wanna go for at least 200 if I can

[shy kelp]

i cry

😭

[high skiff]

oh, i also trained it on a few animal images, so it also understands how to apply realism to other scenes too

like here

[cyan crown]

I can't train a good Lora. Not that fun......

[trim orbit]

https://tenor.com/view/magic-man-jerk-adventure-time-eat-it-gif-11617931

[upbeat summit]

@indigo carbon Do you think Loras can be integrated into an AIT workflow at some point? They can be used now, but for every Lora the generation process takes longer to actually start.

[shy kelp]

love this thing

no more out of memory errors

[high skiff]

I asked it for a medium close up photograph of a pretty french woman with brown hair standing in a forest at twilight with a black dress on

Base SDXL vs my LoRA

[indigo carbon]

possibly. might need to make more modules though

[high skiff]

both got the shot right, mine got the time of day right, mine looks more realistic

[upbeat summit]

it takes almost 10 seconds before the gen starts for me with a lora chained in the AIT pipeline

[high skiff]

raccoon in a pine forest

SDXL vs my LoRA

[trim orbit]

i like the base better but that's my preference for warm tones instead. kodak baybi

[shy kelp]

why does it look so baked

[high skiff]

I can agree, but I asked for twilight, not sunset, which is an issue

[high skiff]

which one, base?

[shy kelp]

yea

[high skiff]

The whole point of this LoRA is to prompt it as easy as possible

[high skiff]

just a linguistic prompt

"A medium shot of a pretty woman in a forest"

Nothing else needed for realism

[high skiff]

SDXL needs a lot of hoopla for good realism base images

[trim orbit]

i'll credit base with twilight. the sun is so low on that tree canopy that it's minutes from

[shy kelp]

can you do a green/purple one ?

[trim orbit]

long shadows are the harbinger of the twilight zone

[wet nacelle]

woman

[high skiff]

thats not twilight, twilight is blue hour, it depicted golden hour, which is sun above horizon, in all ways its incorrect

here, let me get some more examples

I only did a few before I had to share haha

[trim orbit]

" which is sun above horizon " uhhhh

thats the opposite

[strong field]

I’ve already achieved Lora injection into the workflow

[trim orbit]

oh i see what you're saying

[high skiff]

yeah

[trim orbit]

yes. it's minutes from. i live in the woods. that evokes twilight to me because it's about to begin

[high skiff]

I do like the sunset flare in the image, but I didn't ask for sunset

[trim orbit]

gets my stamp

[upbeat summit]

but does it freeze for you too? It takes 5-10 seconds before it starts generating if a lora is used

[trim orbit]

very subjective takes either way. both images are fine and dandy. i don't know why you think yorus is a slam dunk. honestly

it looks no different than another word added

[shy kelp]

what did you use to upscale ?

[strong field]

Agreed

[shy kelp]

latent, iterative or just upscaler ?

[cyan crown]

I do everything in ComfyUI

[shy kelp]

i know i saw the filename 😂

[shy kelp]

i switched to comfy aswell because its so much faster

[strong field]

I didn’t notice a slow down. Not at comp right now but I can check later

[trim orbit]

again i like that first one with the warmer kodak tones, but that's ENTIRELY a subjective preference. color temperature is something people have argued over since photography began. second coon's skull looks wrong too

[midnight shuttle]

If A1111 ever updates the way they handle VRAM it will be the same speed as Comfy.

[shy kelp]

whats an AIT workflow

🤔

[ashen oracle]

this https://github.com/FizzleDorf/AIT/tree/sdxl

[shy kelp]

ah so llike tensorflow

or tensorRT

i forgot what it was

[strong field]

Beware, it’s a rabbit hole

[strong field]

I was also able to achieve sdxl style injections with good results

[trim orbit]

i dont know. i tried using it in a couple existing workflows i have. there's a whole different theory behind the step counts and i can't recreate quality when i try ot do what i already know

[shy kelp]

how much more juice are you talking

like %

[trim orbit]

looking at 70 step count images like "what?"

[ashen oracle]

30% (22% to 37% for me) depending on what i'm doing

[crisp owl]

I want to go down the rabbit hole, but they still don't support SM75, so, I wait.

[high skiff]

I didn;t say it was, tho the LoRA is supposed to follow realism, which it does considerably better

Tho these images are not chery picked, and are instead first gen comparisons, so I don't always show the best

[indigo carbon]

depends. on my 4070ti it is over double speed..

[strong field]

Yes I noticed that too. The step counts matter ALOT but I don’t yet understand why

[shy kelp]

would you look at that, supreme being making a lora

[crisp owl]

What, how did you get footage of Sytan?

[strong field]

Lol

[shy kelp]

the skinwalker forgot to put on his skin

[trim orbit]

at lower step counts, numbers like 20, 30 being high, seems i get garbage when ait is enabled

[strong field]

From 3it/s (ish) to 6.9it/s

[trim orbit]

looks like gan

[shy kelp]

so 2x

more or less

[shy kelp]

but how much work is it to do tho

[strong field]

50 / 25 are the lowest I could get

[indigo carbon]

as of now if you have a 4000 series GPU it's as simple as adding a normal node

[shy kelp]

i got a 1080 Ti

https://tenor.com/view/poor-man-gif-23343928

[indigo carbon]

oh. sorry, not supported..

[shy kelp]

https://tenor.com/view/noooooooooo-gif-22984243

[strong field]

Pretty simple. Git clone the extension and use TDGs workflow

[shy kelp]

seems easy

now i just gotta get a 4090

[strong field]

Oops yeah Sm80 only

[shy kelp]

sm whjat

[trim orbit]

sm dn

[indigo carbon]

SM80= 3000/4000 series

[shy kelp]

https://tenor.com/view/chicken-piano-keys-chicken-head-beak-beaks-gif-22090521

[high skiff]

I cannot believe the background coherency of my LoRA

its almost more impressive than the actual subject realism lmao

[somber burrow]

/sdxl generate a cat sitting on a couch

[shy kelp]

what does it do ?

[high skiff]

base SDXL vs my LoRA

[midnight shuttle]

I'm holding out for better AMD support before I upgrade from my 3060. $1500+ for 24 GB VRAM is too much.

[high skiff]

imporves things not dying in the background of images haha

let me get a more extreme example

[shy kelp]

2024 also brings a new gen

no point buying anything now unless you got too much money or no gpu at all

[candid walrus]

@high skiff you keep saying realism but that's an art style that i don't think you're going for right?

[autumn forum]

Ima be honest, from other Lora’s that I’ve worked with and made it seems no matter what, Lora’s improve coherence and uniformity. Lol

[high skiff]

photo realism

[high skiff]

oh, I'm sure

[high skiff]

I am just glad in this case that its at least realistic coherence, rather than the plastic look of normal base SDXL

[shy kelp]

so overworked i forgot my skin at home

[high skiff]

base SDXL vs my lora

background coherency in images like this really shines