✨｜sdxl

[hardy cipher]

I wish I could find some sort of explanation for them beyond a vague description or the code. because neither of them have really enlightened me to how I would gauge what to use and when

[crisp owl]

Yeah same, I've tried to make some sort of understanding regarding it, but yeah it's just been pretty vague really

[hardy cipher]

it seems like in general 4096 works. tried 8012 but not sure if that improved anything

also wonder if sometimes using different aspect ratios would be preferable

[crisp owl]

I think (mobile atm so can't verify till at pc) I've been testing sticking with matching my latent ratios, but I've not been doing any serious images yet as I've been finalizing my main workflow

[hardy cipher]

man, this ipadapter setup I'm working on is decent

still trying to balance things. the cats slot is kind of overwhelming the other 2. but getting there

what I'm working with currently

can't deny the cat is looking good, but needs more spock and dinosaur

using stylers but nothing that's going to alter the style of the output too much

top left has transcended and taken on it's next form

[crisp owl]

I would have guessed a porcelain plate, river rock art, and a cat were your inputs lol

[hardy cipher]

i have it on hdr photograph and enhance for stylers, so the hdr photograph one might be causing them all to look like ceramic figurines, lol

probably isn't a magic ticket ideal setting for the parameters either

these are pretty quality though

[crisp owl]

kitty

[hardy cipher]

switched it up

something is nanning

pathetic

2 out of 4 came out right. better than 0 I guess

[mild birch]

Do we have a tile controlnet for SDXL yet?

[indigo carbon]

dall e 3 just got announced

SDXL might be in trouble, sadly

[crisp owl]

Their results looked pretty good

[indigo carbon]

I suspect they did move to latent diffusion, pixel diffusion has a certain look you can notice

I think it similar to SDXL, but maybe more parameters. the text in images might be more consistent

[hardy cipher]

I don't think stable diffusion will be in trouble. they have different approaches to things

dalle is like a nice car that doesn't give the driver access to it's engine

[crisp owl]

Yeah I've no concern for the usecase of SDXL

[hardy cipher]

it certainly looks like it works well, but different target audience I believe

maybe some overlap

[indigo carbon]

idk, I think SAI might need to keep developing SDXL if we want to keep up with OAI

[indigo carbon]

plus, their damn name is OpenAI, wtf man

doesn't that mean open source? lol

[hardy cipher]

sam altman seems like a high level scumbag. but decent pr team. hopefully they sort of fizzle out over time

[indigo carbon]

idk. I think if Dall-e 3 will actually outdo SDXL we might need to figure some stuff out

[hardy cipher]

not bad

I don't think stable diffusion, or anything else, will ever take the runaway lead because they all feed off each other. the tech and ideas spread around so quickly

[hardy cipher]

btw, this is a new setup. took out all the non-ipa stuff

[indigo carbon]

idk what to tell you. SDXL did take the lead, but now not really sure about that because dalle 3

[hardy cipher]

I don't see why we should be concerned with dalle. it's not likei t's going to cause sdxl to cease to exist

and whatever they're doing will get ported over eventually anyway

[indigo carbon]

clarify "ported over"

[hardy cipher]

well I didn't mean it as a literal thing. I just mean whatever they're doing will eventually get picked up or at least considered by all the competition

and it's different target audiences anyway. so I don't know. I really don't know what they could do to impact what I'm doing

in a negative way at least

[indigo carbon]

I did remember Emad talking about an SD3 in the works; that might possibly be an equivalent eventually

I believe that was mentioned on the SDXL release show stage?

also, I think OAI might need to figure out something similar to IPA to use Dalle3 for blending images. since it's (probably) not pixel diffusion; they would need to figure out that component

[hardy cipher]

well I'm sure they have people that understand the concept thoroughly

if I can sit here at my crappy computer and do what I'm doing surely they're able to do a whole lot more

[indigo carbon]

and SAI doesn't? IPA wasn't made by SAI

[hardy cipher]

no, where did I say that?

I don't think stability had much if anything to do with any of the ipa stuff so far have they?

[supple knot]

https://www.youtube.com/watch?v=vKV3vHHkRzY

[indigo carbon]

I mean, that's the only way to blend images using SDXL, so I suppose they might have

[hardy cipher]

I'm guessing they might have stuff in the works. but I actually don't know anything. so who knows

[supple knot]

Just think there always working on it, its there baby

Looks like alot of the process is sorting pictures with word tags

Then telling it if its right or wrong

[indigo carbon]

not really, there is more stuff going on, with things like IPA and MJ blend the subjects in outputs look like imageA or B

like this for instance, obviously- the bench is really similar to the bench in image input

[hardy cipher]

what's going on there?

[indigo carbon]

so it's not [BLIP OUTPUT FROM IMGa]+[text]

[supple knot]

https://tenor.com/view/forrestgrump-stanrd2020-john-travolta-gif-20446988

[indigo carbon]

my theory is IPA might be injecting image inputs in to the model itself, while mixing with CLIP_vision

[hardy cipher]

what do you mean by "image inputs?" I'm slow sometimes

[indigo carbon]

provided image to be encoded and used as prompt

for instance, here it was [IMAGEa] + [in minecraft]

[hardy cipher]

well it does right? that's how it works

inserts modules

[indigo carbon]

and with this it's [IMAGE_A] + [IMAGE_B]

the injection process can cause the model to lose many of it's qualities, so the IPA is meant to be pre-sampling then passed onto a sampler with the original model with BLIP output from the unfinished samples as conds, with denoising strength<1

[hardy cipher]

well it inserts between layers as I understand it

[indigo carbon]

so IPA is meant to give the normal model an idea and the unfinished latent that's a blend of imageA+B

this way it allows for AIT and style nodes to be implemented for better use

I'm assuming OAI also knows all this stuff, so dalle3 might have the ability for image input in prompt and image blending. so the way to outdo that with open source will be to improve SDXL itself

speed isn't as problematic due to things like OneFlow and AIT

which I'm also sure OAI has closed source alternatives of

[hardy cipher]

of course

I think their biggest weakness is also their biggest strength in a sense

they have access to so many resources and things. huge amounts. but then, something that big can't move too quickly either

[indigo carbon]

Yeah. Again, the thought of closed source AI outdoing open source AI sickens me and probably most people in here. So if dalle3 will actually beat SDXL we would need to take action

And the worst part is, OpenAI was intended to be open source in the first place. So they are defeating their own purpose like this

[hardy cipher]

I don't think it's a winner take all sort of thing though

wish I knew how to pinpoint what causes things to get cutoff

should probably start using those solid masks

[indigo carbon]

Always to the right?

[hardy cipher]

always

[crisp owl]

same

[hardy cipher]

those did well

[indigo carbon]

I also encountered that when working on my AIT blend workflow I used a second ago. Are you using IPA VIT-H or the normal one?

[hardy cipher]

normal one. they're moving to vit h though

[indigo carbon]

This is IPA and never gets cut off

[hardy cipher]

well does it make snowglobes like mine?

kitty snow globes

[indigo carbon]

For sure, it works really well. I'm not at home at the moment - but I plan on releasing it very soon

Also the img+txt version

Both take ~16s per image for me, and they beat MJ's stuff most of the time

[hardy cipher]

well I think mine is doing alright. maybe approaching a bit differently though

how do you take care of things moving to the right?

[indigo carbon]

You'll have to wait and see ;]

[hardy cipher]

my flow is goofy, but it's getting there

[crisp owl]

I'm in the final stretch for mine

the behind the scenes logic nodes

[hardy cipher]

I don't know if it's really possible to perfectly balance 3 images without adjusting things somewhat for different combinations of things

[hardy cipher]

yeah, you have a lot more nodes I believe

keep mixing it up to see what happens

interesting how it transfers the concepts to the output

[nimble heart]

pretty sure openai was originally made to be open source but they changed their mind once they realized they were the current best and could make hella money off it

[vital ermine]

Scum suckers

Now DallE-3 is to arrive

Dall-E 3

[hardy cipher]

kitties incoming

[indigo carbon]

Dalle3 isn't pixel diffusion I think

We could outdo it if SDXL would keep getting developed

[indigo carbon]

I fucking hate how a company called OpenAI is always making closed source AI though

I did hear about an SD3 being in the works, Emad mentioned that on the same day SDXL was released, so that might be the next phase.

[supple knot]

What size images was SD 2.0 train on ?

[indigo carbon]

768^2

SDXL is 1024^2, but insanely more params

I believe dalle3 is somewhat similar to SDXL, judging by the outputs they showcased, it's definitely latent diffusion similar to SDXL. But they said it would be used via a multimodal inference. We could easily achieve this if there would be an interface for LLaMa2 x SDXL

Like giving an idea to LLaMa, then have it make a good prompt, send to SDXL - and boom shakalaka, multimodal inference

[supple knot]

Have you done any video with sdxl?

[nimble heart]

trade the refiner for an LLM

[indigo carbon]

Oh yeah, for sure. With modern SDXL finetunes the refiner would degrade results

[nimble heart]

refiner also breaks down really fast at higher res

[indigo carbon]

I think LLaMa 3b would be a great fit with SDXL

[nimble heart]

deviate more than like 15% above 1024 and it just turns into soup

[indigo carbon]

Also on 1024 stuff, refiner isn't necessary

Slows down speed and worsens outputs

[nimble heart]

yea I run it without refiner by default and only turn it on if I wanna re-render the seed and see if it like fixes the face or something

life's been easier since I started doing that instead of fiddling perpetually trying to make the refiner objectively better

[indigo carbon]

The latest AIT workflow I made doesn't screw up faces, I usually bypass the refiner haha. But results are insane and equally as impressive speedwise

[nimble heart]

I am liking XL base though so I haven't even used any tunes yet

waiting for the civit.ai space to mature a bit more

[nimble heart]

what ait nodes does everyone use I kinda wanna try it on rocm at some point

assuming compiling the kernels for my gpu isnt too involved

[indigo carbon]

You would have to compile AIT modules for AMD, it wasn't done yet.

[nimble heart]

yea but like is it a whole 50 step process or is it just python compile.py models/sdxl-base.safetensors

[indigo carbon]

It's easier than that even. Especially on Linux. When I'll be home I could help you compile for AMD if you want

On windows it is a hell though, I'm the only person that could provide the precompiled modules for batch range 1-4 on windows

[nimble heart]

well rocm doesn't even support windows yet so guess that makes it easy

[indigo carbon]

You would need VRAM>12G to compile for batch size>1 though

[nimble heart]

the main hangup might be idk if amd's AIT fork for my gpu follows upstream AIT

or follows it closely enough to have the required updates for whatever the nodes use

[nimble heart]

24 gigs but no flash attention so make of that what you will

[indigo carbon]

You need flash attention 2 for compiling AIT

[nimble heart]

I dont even have flash attention 1

[indigo carbon]

No Xformers?

[nimble heart]

AMD has 0 flash attention support for anything

they have a fork of it but its out of date and fails to compile on newer ROCm versions

that's why my gpu uses like twice the vram of other people for SD

[indigo carbon]

24 gigs might be enough to still compile though

[nimble heart]

so 24 gigs behaves like 12

maybe if it can fallback to a math impl instead of hard depending on flash attention

[indigo carbon]

Yeah, you might be able to compile anyways

[nimble heart]

ExLLama uses flash attention a lot but it works great on my card with the fallback method anyways

I could run 30b @ 4 bit pretty fast

I quantized mythomax 13b into 8bit exl yesterday and I get like >40 token/s

[indigo carbon]

I think exLLaMa is similar to AIT? I heard it's a massive speed boost

[nimble heart]

yea its automated though. it compiles the kernels for you

[indigo carbon]

Jesus, nice!

[nimble heart]

so you just feed it a model and when it loads it hot compiles the kernels

so the first load takes like a minute then after that its instant

[nimble heart]

I also made a 13b 4bit version that hit 65 token/s

thats not the hot compile though that's requantizing the whole model specifically for exllama

hot compile from a gptq based model is slower

[indigo carbon]

I thought exLLaMa is only for GPTQ, how did you use it on 8bit?

[nimble heart]

exllama2 has its own custom quantization format

exl2

so you can requantize base models into exl2 and it does mixed bits between 2 and 8

so you can have like 6.5 bits average or something i guess

only problem is taking the measurements to find the error rate of all the different bit quantizations takes ages. mythomax L2 13b took like two hours just for the measurements...

so if you have a model you really like and want that huge speed boost I'd give it a try sometime

the exllama dev has pre-quantized versions of the llama2 base models

ig the 70b can run entirely on a 3090 with like 2.4 bits average or something

[indigo carbon]

Oh cool! Is that for Oobabooga?

[nimble heart]

yea

works out of the box on rocm too

just yells at me for not having flash attention lol

[delicate grotto]

wonder how much exllama2 takes

[nimble heart]

how much what?

[delicate grotto]

vram wise

[nimble heart]

depends on the model, how you quantize it, and your context

the llama2 based mythomax 13B quantized to an average of 8 bits with 4k context uses like 17 gigs of vram without flash attention

[delicate grotto]

oh wow, went down quite a bit

[indigo carbon]

Oh, so not the stuff TheBloke uploads? You quantise yourself?

[delicate grotto]

i thought it would take ~24 45

[nimble heart]

and and inference for me is 40 tokens/s

[delicate grotto]

ooh woow that's fast as hell

[nimble heart]

I use his GPTQ stuff but he doesnt make exl2 versions

I quantize it against the same calibration data he does so I assume they're comparable quality

idk what most of the settings do though lol

the convert script from the exllamav2 repo seems straightforward enough
https://github.com/turboderp/exllamav2/blob/master/doc/convert.md

so you just convert.py -i mythomax13b/ -o /tmp/quant -cf mythomax13b-exl2/ -c wikitext-v2-test/ -hb 8 -b 8 then wait two hours and you have a quantized model

if you re-quantize the same model to a different bit depth its a lot faster cause you can re-use the bit depth error measurements

[indigo carbon]

Then you just load it using exLLaMa 2 through ooba?

[nimble heart]

yea the output folder from -cf loads like a normal model folder

i have 3 measurement files for MythoMax L2 13b and Mythallion 13B if you want them

[indigo carbon]

I'll look into this. Does the bloke share his quantisation setting or we can come up with better ones?

[nimble heart]

just says he uses the wikitext-v2-test as his calibration data but other than that not much. I think the EXL2 quantization settings are different anyways

[nimble heart]

measurements for mythomax/mythallion. First two are measured with 4k context and a higher line count, last one is just mythomax with default settings.

i have no idea how optimal they are but higher numbers are better I assume. the r400 one took like 3 hours to compute so it better be good lol

[indigo carbon]

Will TheBloke also include EXL2 or do we have to do this on our own

[indigo carbon]

Also, how do you make these files?

[nimble heart]

-om flag takes measurements, -m flag uses them. It measures the error for quantizing each part of the model in 20 different ways and then using the measurement data when you quantize it for real it mixes the bit depths for different parts to be less lossy I guess

[indigo carbon]

I don't have much knowledge about LLMs. What does R mean?

[nimble heart]

I just got started with all this yesterday so im new to this too

[nimble heart]

its the -r flag from the exl2 convert script

how many rows of calibration data it uses

default is 100 so r400 means i used 4x the default which was probably way overkill

[indigo carbon]

Does that effect performance?

[nimble heart]

no idea

my hypothesis is it just affects quantization accuracy

like if you use more calibration rows it quantizes better

so token/s is the same but the quality of outputs might improve?

the docs arent very specific so im just guessing tbh

quick gen using 8bit AutoGPTQ vs 8bit exllama2

[indigo carbon]

Is 8bit really better than 4bit?

[nimble heart]

¯_(ツ)_/¯

it has a lower error in the measurements by a few %

for 13b models even 8bit doesnt come close to using my full 24 gigs so

guess 4bit is faster though

60 token/s vs 38.65

[indigo carbon]

Obviously, but by how much and is there degradation?

[indigo carbon]

Both will be instant so the one with better outputs will be the obvious choice

[nimble heart]

idk how to objectively measure degradation. The calibration err values were a few % lower for 8.13 bits vs something like 4.13 bits is all I can say

[nimble heart]

yea that's why I just went with 8 cause I assume its better

both are many times faster than I can read

[indigo carbon]

Interesting. I'll look into this. It might be a huge breakthrough for using SDXL and these things at a multimodal inference

[nimble heart]

coming from wizard-vicuna 30b 4bit I prefer the results of mythomax l2 13b 8bit so far, but there's many differences between them besides just bit depth ofc

the 30b is based on llama 1 so probably why

[indigo carbon]

Oh yeah, this will also work on 33b models

[nimble heart]

it works on anything llama based

[indigo carbon]

Is there any 33b models that beat Mythomax?

[nimble heart]

all the 33b models are llama 1 and mythomax is llama 2 so idk

depends what you're going for I guess

apparently mythallion is also better than mythomax for chat specifically but I havent played with it yet

[indigo carbon]

Also what about VRAM usage? Is 4bits half as much than 8bit with VRAM?

[nimble heart]

lemme see

[indigo carbon]

We might be able to fit SDXL and MythoMax at the same time if it's that efficient

[nimble heart]

9.7 gigs with 4 bits EXL2, 15.5 gigs with 8 bits EXL2

just loaded no extra context or anything which'll add a bit more

so not quite half but I mean close

[indigo carbon]

I have 12gb of VRAM. Maybe flash attention could resolve this?

[nimble heart]

one note is 8 bit is a target depth, not absolute. so if 6 bit 128g performs better than 8bit 32g for any specific layer then it'll use 6bit for that layer instead

so really the 8bit is probably more like 7.5 bits or so

[nimble heart]

you can quantize in any float between 2.0 and 8.0

so you could do 5.5 or 6.0 and easily fit within 12g

so the final model will be a mix of 4/6/8 bit layers depending on which layers benefit more or less from high bit depth

to get your target average

but yea this message implies that flash attention might lower your vram costs quite a bit

I checked the rocm flash attention fork and they cherrypicked some stuff from the flash attention 2.0 tag but still wont compile for me

and im not smart enough to fix clang errors

[indigo carbon]

Does EXL2 make engines for running? I'm thinking we could make it even more efficient with something like AIT

[nimble heart]

maybe? im not sure where the cache for kernels would be

there is a bunch of exllama2 cuda files in my torch cache dir so

[indigo carbon]

Anyways, I'll try EXL2 with 8bit when I get back; I'll update if it's better quality than 4bit

[nimble heart]

is there a way to benchmark relative quality?

like in some objective way with numbers

curious how my 8bit exl2 conversion compares to TheBloke's 8bit gptq

while you do that ig ill try to make AIT work on rocm

tomorrow its 1 am here

[indigo carbon]

It might not be as easy. If you fail to do so, I'll help you when I get back

[nimble heart]

what's the repo for the nodes & compiler?

im on Arch Linux so any random tools are just an AUR install away

[indigo carbon]

That's the thing, I had to make a compile script for scratch to make modules for SDXL

[nimble heart]

bru

that sounds super involved

python isnt that hard but idk shit about numpy/pytorch

[indigo carbon]

Yeah, but that compile script should be universal

I can give it to you when I get the chance

Oh right

I gave that script to FizzleDorf, it's included in the node's repo

So you just install Bdist wheel for AIT and run that script and if you're on Linux it should work right away

Interestingly, you don't need the AIT pip module to use the modules themselves, but you do need it to compile them. This is why the node is shipped with precompiled modules

[nimble heart]

so the ait package is just for compiling and the modules are ran with pure torch?

probably helps my chances of it working lol

[indigo carbon]

That's the thing, AIT enables the models to be loaded via MSVC engines, this is why it's universal

But yeah, it's only used for compiling the engines

[nimble heart]

so the ait package is only needed to load the modules on windows? Linux it just uses torch directly?

all this shit's confusing. maybe intel has the right idea with that UXL thing they announced

[indigo carbon]

No, on windows loading the modules is just as easy as on Linux. The compilation process is just much more complex on windows

[nimble heart]

ahhh

msvc compilation backend?

instead of clang ig

[indigo carbon]

Yep, and specific version of everything

A bunch of set commands

[nimble heart]

does nvidia cuda use clang too? ik rocm does

[indigo carbon]

Specific cuDNN and CUDA versions.. compiling AIT on windows is a leaving hell

Linux handles all that on it's own, so you just need the PIP package and the script

But after someone compiled the modules for Linux, no one needs the pip package

And that someone might be you when it comes to AMD

[nimble heart]

maybe. the rocm sdk packages on arch automatically pull all the compilers too so my dependencies should be fine.

hopefully its just run script and go

I compiled my own pytorch before it officially supported rocm 5.5 so im hoping AIT is easier than that

[indigo carbon]

It should be

[nimble heart]

when it comes to arch gfx1100. Dont think the kernels are cross-compatible between arches

so 7900 XTX and XT

[indigo carbon]

They are. The modules you will be compiling will work for all people that use Linux and has the same GPU architecture as you

[nimble heart]

yea so gfx1100? just the 7900 cards

rocm has separate flags for gfx1100 compared to other rdna 3 cards which is gfx1102 I think

[indigo carbon]

Should work for all the 7000 series cards really

[nimble heart]

so it might work but idk

rocm discriminates but maybe it doesnt matter as much for ait

[indigo carbon]

With Nvidia cards modules I compiled with my 4070ti work for people with 3000 series even

[nimble heart]

ah

probably good then?

[indigo carbon]

Yeah, should be

[nimble heart]

aight

guess thats my project tomorrow

when its not 2 am

rocm's ait fork is just python setup.py install so I assume since that goes smoothly the rest should too

[indigo carbon]

About that- to use the compile script made for SDXL you need a specific pre release, or maybe now the normal one works

[nimble heart]

its ait 0.3.0-dev0

thats the only version I get

take it or leave it

[indigo carbon]

Hmm. I used 0.3.33 for providing the modules, idk how this will behave with that script

[nimble heart]

there's no new commits on the rocm navi 3x branch so unless the mainline one works thats all I have

[indigo carbon]

It should work, but not 100% sure about this

[nimble heart]

we in the weeds now bois

thats assuming they even finished the AIT port and it doesnt just segfault lmao

I had read that someone else got it to work with the 7900 XTX on sd 1.5

[indigo carbon]

I do love AMD way more than Nvidia though, so I hope this makes AMD cards run SDXL as fast as Nvidia cards

[nimble heart]

im hoping flash attention lets me beat the 3090 at least

in blender I beat 3090's without raytracing by a decent margin

[indigo carbon]

Nvidia are so greedy, AMD needs to put them in their place lol

[nimble heart]

blender doesnt support amd raytracing yet so idk how it compares with optix

[nimble heart]

maybe intel arc will save us all

saw an a780 16gb on amazon for $320 today

granted their ML toolkit is even less complete than AMDs so

no exllama for them

[indigo carbon]

Maybe. Does that use CUDA cores or MATRIX cores?

[nimble heart]

its their own thing✨

[indigo carbon]

Pog?

[nimble heart]

it uses intel OneAPI

not compatible with rocm or cuda in any way

[indigo carbon]

One as for OneFlow?

[nimble heart]

¯_(ツ)_/¯

[indigo carbon]

OneFlow is faster than AIT but less compatible

[nimble heart]

no

Its just their compute stack

supposedly its cross-vendor compatible

[indigo carbon]

Ah, I see

[nimble heart]

hence OneApi

so the spec is there for nvidia or amd to use too

but AMD already tried that with rocm and you see how that goes

amd HIP can backend to nvidia cards fun fact just no one does it

[indigo carbon]

Yeah, I wonder if AIT gives AMD cards a bigger boost than it gives Nvidia

[nimble heart]

probably less

AMD has weaker tensor/matrix/whatever cores

so there's probably less optimization

also their AIT fork is stilll WIP so probably not complete

[indigo carbon]

I'd actually guess more, look at what exLLaMa did to your speed

[nimble heart]

hell supposedly ROCm doesnt even make optimal use of RDNA3's techyes

[nimble heart]

try it yourself before you make judgement calls lol

you might 5x speed too

with AutoGPTQ I was still behind Sytan's 3090 by a small bit

[indigo carbon]

Nope, I get 10T/s with GPTQ and 37T/s with exLLaMa

[nimble heart]

on the same model?

[indigo carbon]

Yeah

[nimble heart]

idk i never tested the same model on both, only my re-quantized models

I have a 30b 4 bit that should work on both

lemme test

[indigo carbon]

I think AIT and exLLaMa just "unlock" the potential that the software stack is limiting, so these should give a bigger boost to AMD @nimble heart

[nimble heart]

the thing is

[nimble heart]

this person said they went from 18 it/s to 25 it/s with AIT

so

[indigo carbon]

That's still huge though

[nimble heart]

but idk how optimal usage that was. it sure as hell wasnt comfyui

this was way before XL dropped

[frigid charm]

hello everyone 🙂 hope you could help with my doubts - I am in the phase of thinking/planning the project, I would like to fine-tune SD with ideally a dataset of few hundreds of pics, ideally few different styles - if it cannot work, I'll scale down to one stile less pics. I plan to use vast.ai, I have good python skills and knowledge in ML/DL. However I haven't touched generative in years and now I am a bit lost. I am looking at Dreambooth and Everydream but it's not clear the way I have to go, nor I find examples that seems good/relevant for my case

[nimble heart]

yea but nvidia cards like double with ait dont they

[frigid charm]

if you have any thoughts they I'd be very grateful 🙂

[indigo carbon]

They do, but AMD might get an even bigger boost when used correctly.

Like AMD gets a bigger boost than Nvidia from exLLaMa

We'll find out if that theory is true after you try it

[nimble heart]

Wizard-vicuna 33b 4bit
Autogptq vs exllama.

so not even 2x where you almost 4x'd

[indigo carbon]

I 4x'd on 13B, and so did you

[nimble heart]

with EXL2

[indigo carbon]

I didn't try EXL2 yet, I'll test that when I get the chance

[nimble heart]

what model was that specifically I'll DL it rn

[indigo carbon]

TheBloke MythoMax GPTQ

[nimble heart]

what depth

[indigo carbon]

I have no idea, I'll check when I get back

[nimble heart]

4bit 128g I'd assume for your card

[indigo carbon]

Yeah that one if I remember correctly

[nimble heart]

112MiB/s lets go

[half ivy]

I am here!!
What have I missed?!

[nimble heart]

what gpu do you have? 4070?

everything

[indigo carbon]

Ti*

[nimble heart]

dang. those almost cost as much as my XTX did

[indigo carbon]

Nvidia ripped me off, I know

No need to mock me about this

[half ivy]

Whats the latest in control nets for the XL’s? Wanna try make summin tonight 🤔

[nimble heart]

aight something's wrong

it ran @ 50 token/s on AutoGPTQ and just barfed german garbage

[indigo carbon]

Tf?

[nimble heart]

I wish I was making this up

hang on

[indigo carbon]

Screenshot model settings?

[nimble heart]

50 token/s, german garbage

[nimble heart]

its just defaults

bloke's files usually work with defaults so idk

let me try exllama

[indigo carbon]

This looks like something GPT2 would do when it was used on cleverbot

[nimble heart]

exllama works fine. 68 token/s

[indigo carbon]

Cool

[nimble heart]

exllama 2 specifically. I dont know if 1 works on rocm

2 just got rocm support like a few days ago

so now im converting all my shit to EXL

[nimble heart]

reminded me of when I tried increasing the max context on OPT

like the moment it gets passed its limit just barf

wonder if flash attention would make inference faster too or if its just memory

like 70 tokens/s is cool and all but it can always be faster

[indigo carbon]

It might. 68T/s is godly

[nimble heart]

'godly' on a 9 gig model

i'll have to try the llama chat 70b

the exllama dev uploaded exl2 conversions so I can just download that, no need to spend hours quantizing

[indigo carbon]

Might be a bad idea, I always wanted to try 33 and 70b and could never fit them in my GPU. I'll try it again with EXL2 when I get the chance

[nimble heart]

llama.cpp supposedly supports hipblas for splitting between rocm/cpu, and while it compiled fine and the model loaded inference crashed python

[indigo carbon]

Ohhhh is exLLaMa not dynamic?

I was wondering about that

[nimble heart]

multiple gpus but it cant split to cpu I dont think

might be wrong

according to the exllama2 dev, he has 70b running on a single 4090 with like 2.4 bits or something

ig I could just set my gpu mem limit to like 8 gigs and see if it uses the cpu too

[indigo carbon]

This is the reason why AIT is faster than TRT, TRT isn't dynamic so AIT can move around between GPU VRAM and CPU RAM

[nimble heart]

it just ignored my limit

[fierce hollow]

speaking of exllama, I made some nodes for comfy

[nimble heart]

I havent run a too-big-for-gpu model in exllama yet so idk how it handles that

what depth should I try...

the lowest, probably...

[indigo carbon]

Cool! I was thinking about including something like this in my AIT workflow

Multimodal incoming

[fierce hollow]

yeah it works well enough with a good prompt or lora

wish there was some good 3B model though

[nimble heart]

how recent is the llama2 data? will it know what diffusion prompts are?

[fierce hollow]

feeling the pinch with 10GB usage for Base SDXL + 7B

[nimble heart]

wonder if you could make a llama tune based on those prompt websites

[indigo carbon]

That's the thing. I think the goal is to make an interface where you just write "make me an image of (........)" then LLaMa will know how to prompt with SDXL, send the generated prompt to a workflow, then send back output images

[fierce hollow]

not sure about the llama2 dataset but it's not like it can't hallucinate a prompt without knowing what SD is

[nimble heart]

i mean even llama 1 can guess but I never got it to be good at it

it just kidna throws random words together

which I mean tbf that's what a lot of actual 'prompters' do but still

[indigo carbon]

I remember @ionic dragon saying they made a LLaMa2 LoRA for making SD prompts

[nimble heart]

maybe with llama2's 4k context you could just fill it with example prompts then have it give you some new ones

[fierce hollow]

oh right, I tested that lora with my nodes, it's pretty cool

older ss but you can see the difference

the top without lora is just hallucinating cause it has no idea what SD is

[nimble heart]

Oh yeaaa

i always forget LLMs have loras too

[fierce hollow]

exllama loras are also very easy to load/unload

[nimble heart]

yea

but I dont see many loras being used for anything

its always tunes and merges

[indigo carbon]

So we could make an interface that you would ask LLaMa for an image, then send generated prompt to SDXL and send back output images

[fierce hollow]

true that

[nimble heart]

on HF

[fierce hollow]

pretty much what the dall-e 3 page shows with chatgpt

[nimble heart]

holy shit it loaded

[fierce hollow]

there's already addons like that for ooba iirc (sillytavern too)

[nimble heart]

22 gigs vram

who needs flash attention anyways

[indigo carbon]

Yeah, but do they use SDXL with AIT?

[fierce hollow]

I mean no, it's just some basic api call

to use with comfy and ait uhh I'd guess you'd either need some template first or even have the llm generate the workflow in addition to image

[indigo carbon]

I say we could make something like Comfybox except it's multimodal

[nimble heart]

llama2-chat-70b @ 2.30 bits average EXL2

[indigo carbon]

WHAT

That's insane

[fierce hollow]

totally, just need somebody to make it kek

[nimble heart]

so is my vram usage

[fierce hollow]

are there prebuilt wheels for exllamav2 yet?

[nimble heart]

wheels?

[fierce hollow]

like precompiled binaries with the cuda extensions

exllama(v1) has them

[nimble heart]

on oobabooga you can just clone the repo into ./repositories/exllamav2/ and it loads it

no compiling or building wheels

no wheels that I know of

[indigo carbon]

It's inevitable I'd say.. just an interface where you provide an SDXL workflow, an LLM, then you could chat with the LLM and ask it to make images

[fierce hollow]

huh that's weird, pretty sure it errors out on windows without visual studio, ninja etc

[nimble heart]

no idea. im on linux with amd/rocm

hence exllama2 specifically cause its one of the few things that actually supports rocm well

llama.cpp segfaults

[fierce hollow]

ah that explains it, compiling exllama ext on linux is a breeze

[nimble heart]

I didnt compile anything

just cloned

for exllama2

[indigo carbon]

We might not even need to make a UI from scratch. You said Oobabooga has a feature for LLM+diffusion right? What if they make the diffusion part work on ComfyUI

[fierce hollow]

the cuda extension gets compiled on first run, it's just fast (when it works) so you probably didn't even notice

[nimble heart]

ah yea rocm does the same thing. first time you load a model it take a bit to compile a kernel

[fierce hollow]

yeah, iirc ooba uses the auto1111 api right now

but it should be possible to switch to comfy without too much effort

[nimble heart]

but on Arch Linux I just install rocm-ml-sdk and it pulls everything I need to compile everything rocm

[indigo carbon]

Yeah, then make some kind of node that places the prompt the LLM made in the prompt, then you could use any Comfy workflow with an LLM

[nimble heart]

i mean it did say it wanted tobe the auto1111 of llms

[indigo carbon]

It feels like it's almost in our grip, all we need is a way to execute this.. does Oobabooga plan to do this?

[amber musk]

Hi

[fierce hollow]

comfy does have some api since the swarm-something ui uses it

haven't really looked into how easy or hard it would be to implement in ooba/sillytavern though

[amber musk]

Which model has the best war/military/agents related things like extraction/mission impossible series? (best realistic)

[indigo carbon]

Shouldn't be too hard. By making it Comfy, we could use custom workflows even, just would need to specify where is the primitive with the positive prompt

[fierce hollow]

yup, there's an example here, looks like you can just provide any workflow you like and then plug prompts
https://github.com/comfyanonymous/ComfyUI/blob/master/script_examples/basic_api_example.py

[indigo carbon]

Does Oobabooga know about this? I feel like this is a goal they could focus on

[fierce hollow]

most likely, I can't find any plugin for it currently though

most focus on implementing llms in comfy, not the other way around (I'm guilty as charged too)

there's also the issue of gradio being hell to work with (subjectively)

I reckon this could be adapted to comfy https://github.com/oobabooga/text-generation-webui/tree/main/extensions/sd_api_pictures

[indigo carbon]

The only thing Oobabooga would need to change about the LLM+diffusion feature is to have the LLM unload before executing the diffusion part, most people can't fit SDXL and 13b models at the same time

Heck, maybe even keep one of them in CPU RAM at all times and have them switch each time being used

[fierce hollow]

that's also a problem on comfy's side I think, when you just want to chat you should be able to unload all the sd models

not sure it's possible to do that currently

[indigo carbon]

That shouldn't be too difficult.. it's definitely possible as of now

We just need to get Oobabooga to do this

Which they don't have to, so idk if this will get executed

[fierce hollow]

there's some complicated memory management stuff in comfy but no 'unload everything' button

[indigo carbon]

Maybe have Oobabooga do that? Pytorch commands should be viable on the GPU regardless of the environment

So having them switch places from VRAM to CPU RAM and back each time they are being used won't be too difficult

[fierce hollow]

no clue how that works exactly, but if you're calling the comfy api from ooba's side you should be able to also send an 'unload everything' command, since it could also be calling a remote pc

well, not that you can call the comfy api at all for now so just theory

[indigo carbon]

Or "send model to CPU" command to save even more time when the switching is happening

[fierce hollow]

yeah, same idea

[indigo carbon]

Same like ComfyUI does when VRAM gets full when using the refiner, have the refiner and base switch when being used

Having this done could be the future for multimodaling

[upbeat summit]

@indigo carbon I messaged you

[indigo carbon]

I'm sure this will happen eventually, even soon..

[amber musk]

anyone?

[fierce hollow]

no idea about that

also I just checked sillytavern and there's a bunch of apis already

(just not the one everybody's hoping for)

[indigo carbon]

Silly tavern is just Oobabooga wifeu edition, not really relevant

A ComfyUI extension for Oobabooga would be amazing imo

[analog roost]

Wow, this is beautiful!

[stone fossil]

Bann this guy to much 6 numbers.

[stone fossil]

It should be about eh 4h 10 min I release the lora then U can have a blast with it.

Still trying to collect better demo pics.

[crude sparrow]

took one of the dall-e 3 promtps and tried it on dreamshape xl (leg gone )

[stone fossil]

Waiting for @dapper dragon to get home come home raccoon so we can release this lora.

[indigo carbon]

@nimble heart how did you get EXL2 working? I got the pip package and built from source, but idk how to quantize using it

[uncut fiber]

Legs not gone, is under skirt.

[cyan crown]

The more I use SDXL, the more I love it!

[uncut fiber]

@cyan crown what model?

[cyan crown]

standard model

[uncut fiber]

o.k. nice!

[fringe barn]

How are you guys navigating terrible faces in fullbody images in SDXL?

[uncut fiber]

it is thing of faces in distance. Dont know what with it. Probably there could be workflow with codeformer or gfpgan.

[uncut fiber]

is it him?

[half ivy]

BRO 🔥

[shy kelp]

for some reason my router disconnected me for 10 minutes

during that time I've found that lama cleaner does not work without internet

[crisp owl]

Hey Picturesonpictures, how often do you get one of those right sided image outputs from using IPA?

[hardy cipher]

comes and goes. I started turning up width cropping. not sure what it does but we'll see

[crisp owl]

That's exactly what I was gonna test out

Was reading through the paper again to see if I could gain any other insight and came across this part again

[hardy cipher]

well I kind of ran out of other ideas. but can't completely tell what it's doing. it's not like it literally crops the image or the render

[crisp owl]

Under the 2.3 "multi-aspect training" section

[hardy cipher]

you know, I feel like I'm supposed to understand what's goin on there

I've got a better grasp of things now, but early on there were some things that were just so new and foreign to my mind that it took a little while for my brain to acclimate to being aware of their existence

like I'd read about some AI thing that was so far removed from anything I'd considered prior that it just couldn't be absorbed

[crisp owl]

Yeah I still want to know what the width/height/target/crop etc fully does, I feel like I barely have an understanding on them, to the point where I kinda "think" I start to understand, but then don't really know how/when I'd actually change them

[hardy cipher]

what really grinds my gears is how they always just put these images up with 3 word captions and we're left to infer what sort of magic is occurring

[noble shoal]

Magic

[floral island]

i just got the best worst possible image i ever have from sdxl, and i'm sure i can't post it here

[hardy cipher]

I'll call you out if it violates any rules, bud. hate to be that person

[floral island]

aight

imma post it, but i'll remove quickly after

[noble shoal]

Rule #4 is waiting

[hardy cipher]

yeah, that one doesn't bring me joy

[floral island]

it's insane how realistic it looks

[hardy cipher]

why did the ai make that?

[floral island]

0 negatives, 20 steps restart, with the prompt I feel alive. My self-portrait suicide Tonight is no time to die Now I'm awake on my model

that's it

nothing else

just let the ai figure something out from that prompt

[hoary saddle]

am i losing my mind (probably) but didn't at one point all images saved in "output" folder by default save workflow in image? dragging and dropping some now and none of them seem to pull in workflow anymore

[crisp owl]

are those lyrics or what?

[floral island]

yeah, they're lyrics

https://youtube.com/watch?v=pwmSHhapmHA

i'm addicted to lyricsprompting

that's why i have my verser script which converts songlyrics into perfect SD snippets 😄

a1111 -> dynamic prompts

| Slow, we step into the light. Inebriate in sights Unveil the truth that guides us home
| Look inside. With darkness my tears collide Sinking into the shrine of me I'm ready to be set free I'm sick as the dying, the air I breathe
| I feel alive. My self-portrait suicide Tonight is no time to die And now I'm awake
| Truth can break your soul in two. Just when you think it's you The trеachery will breed inside
| Embers, sweet and unrеfined. So bright but not so kind Will start a fire in your mind
| I feel alive. My self-portrait suicide Tonight is no time to die Now I'm awake
| Silence fuels the deadly cries. Ripped apart by knives in lovers' lies Curse my innocence goodbye Take my hand then take my life
| I feel alive. My self-portrait suicide Tonight is no time to die Now I'm awake Oh, now I'm awake Yeah}```

[crisp owl]

I mean, just seeing that snippet, I could see how it'd come to that. Some strong words in there are probably weighted heavily

[floral island]

yeah, running the prompt again... it keeps doing that

creepy AF

[lusty wolf]

Day for stupid questions, where do I find an updated good styles.csv for the WAS node?

[crisp owl]

You ever use the twri styles?

https://github.com/twri/sdxl_prompt_styler

[lusty wolf]

Not that I know of, thanks will give it a go.

[crisp owl]

It integrates your prompt into the styles.
And has like 70+ or something to choose from

[wicked frigate]

i finally got around to porting it: https://github.com/mcmonkeyprojects/sd-dynamic-thresholding#comfyui

[stone fossil]

https://civitai.com/models/149395?modelVersionId=166843

CircuitCraft - Infuse Electronics Into Your Visuals

Unveiling CircuitCraft, a cutting-edge LoRA (Text to Image) model that ingeniously converts everyday images into intricate printed circuit board (PCB) designs, adorned with capacitors, transistors, and electronic wonders. With CircuitCraft, you have the power to seamlessly merge the realm of electronics with your favorite visuals, creating a captivating fusion of art and technology.

Use trigger word: 3l3ctronics.
Trained on 3000 steps from a highly detailed by hand captioned large dataset.

Special thanks at:
@dapper dragon my SD buddy who helped me to make all my LoRA's and be able to output wicked stuff.

Also special thanks to my my upfront image generators and people who I care a lot about:
@masslevel
@osiworx
@mix
@abstract depot

[hardy cipher]

whoa! awesome!

this is going to be interesting

[crisp owl]

question for a buddy.
He has an AMD card, 8gb. radeon 5700xt.

I've seen snippets of people talking about some issues with AMD cards using SD, but mostly ignored since I have an rtx 2060.

Is there any limitations he'll hit if he tries to jump in?

[floral island]

my testing model rejects liked realism too much, it seems. they make great realism tho ❤️ too bad i can't get this ported to my main model without screwing up everything else ... or can i... sigh, another dumb idea for model mixing enters my head

time to start model merging again.

[wicked frigate]

AMD cards have all sorts of weird limits. Modern cards should work mostly, but yeah expect surprise snags and/or performance issues

[cyan crown]

I had 5700XT and switched to 4070 for Stablediffusion

[crisp owl]

Gotcha, I just didn't want to point him towards SDXL only for him to not be able to use it.

[stone fossil]

Mooie.

[floral island]

fuck. i needed this merge to fail instantly, not actually give me the most awwesome fire/ice/stone/water roses 😮

FUCK. i wanted to be done merging XD

no biggie. just gotta find a prompt i know this merge will screw up so i can discard it and go on with whatever except model merging again

[floral island]

I. FUCKING. WANT. YOU. TO. FAIL.

not give me a win 4/4

[hardy cipher]

almost cooked

[floral island]

@hardy cipher not almost, that's cooked 😦 i'm sorry

[cloud furnace]

@fair stag ah now i get it lol

[hardy cipher]

thanks. I guess I should give up

[floral island]

perhaps a certain weight too high?

[hardy cipher]

I'm experimenting with something new

but I also think a lot of people's criticisms in this regard are sort of evolving to mimic musical theory in the sense that people begin to treat subjective opinions as objective truths

[supple knot]

I went to look for the 4x Sharp upscale now there a whole Civitai type website for just different type of upscale

https://openmodeldb.info/

[cyan crown]

I use Auto1111 img2img to upscale

it works great

[supple knot]

I used A1111 with SD1.5, but with SDXL I only use ComfyUI so I can kinda see whats happen behind the scenes

[cyan crown]

I started also with Comfy but with 1.6 returned to A1111

[steady grove]

i love comfy's down to the gears interface. node graphs have a lot of benefit. Automatic makes it so easy to switch resolutions, styles, img2img, and just have a very dynamic workflow

[cyan crown]

they are both great

[steady grove]

comfy isn't very conduscive to creativity. it's a back end. needs a front end like stable swarm to have that grease for the process

[floral island]

be sure to give my model a try if you want ❤️

😄

shameless self-promo 😄

[cyan crown]

link?

[floral island]

https://civitai.com/models/148871?modelVersionId=166165

although i'm already mixing again, i want to improve it again XD

[cyan crown]

will try

[floral island]

you can disable refiner if you use it

[cyan crown]

yes I use

at the moment I'm exploring various painters and illustrator and there are tons in base model"

[supple knot]

I always got lost in the A1111 interface, But with comfyui what in your flow and the path its taking is great for me to get a more desired outcome

[floral island]

stick to what you're comfortable with 🙂

i'm very used to the [this:to this:0.5] output which doesn't work well, or even at all in comfy given i'm very often over 150+ tokens

[supple knot]

@Eface I hope one day you can have the Gundam deli tray in real life.

[floral island]

you saw my gundam deli tray? ❤️

that was actually such a hard prompt to pull off actually

[cyan crown]

SDXL is simply infinite

[uncut fiber]

@supple knot ChaiNNer is father of comfyui, some sort, only for resizing. And there are some good workflows.

[cyan crown]

man I'm naming every upload so you can reproduce them

[icy brook]

Aether gLitch

[livid cradle]

any expert developer with SDXL looking to get hired for a job?

[stone fossil]

JUST DO IT!!!!!!!!!!!!!!

[nimble heart]

I never built a pip package. If you clone the whole exllamav2 repo into oobabooga/repositories/ it can load it directly. I was able to use the convert.py script inside the exllamav2 repo with my oobabooga virtualenv active.

[indigo carbon]

I figured it out on my own. one thing, flash attention doesn't seem to be compatible with windows, so I guess it won't be any faster than EXL1 except it can do q8 now

[nimble heart]

it can only do q8 with the exl2 format which is why I'm converting everything

it can't do q8 gptq still

[indigo carbon]

so exl2>gptq, eh?

I'm currently compiling for -hb 8 -b 8 -l 4096 -r 400 -ml 4096

[nimble heart]

exl2 is supposedly better quality but for bit because it does that measurement pass first and produces a mixed bit binary

so -l 4096 always gave up on me. I had to remove it

[indigo carbon]

it's been going for over 2 hours.. I think it doesn't cause issues for me?

[nimble heart]

also I think -r 400 only affects the final compile, you need to set -mr to affect the measurement rows

[nimble heart]

yea the measurement worked with -ml 4096 but the actual compile failed with -l 4096

are you still measuring?

[indigo carbon]

oh, yeah, it failed haha

[nimble heart]

mythomax is 39 layers

save your measurements.json

[nimble heart]

that's still good, then you can load it with -m and skip the 2 hour measuring phase

[indigo carbon]

ok, how tf do I do that?

[nimble heart]

in output folder

[nimble heart]

file called measurements

add -m measurements.json

[indigo carbon]

can we go to DM? this channel isn't meant for this

[uncut gull]

I bet someone could calculate the total number of unique images SDXL could produce... Nope. You can always concatenate a longer prompt, and you can always start from a different img2img source. Because it accepts variable length, infinite inputs, SDXL is mathematically infinite, as far as computer programs go.

[crisp owl]

omg yeah no way to calculate it haha
Just with txt2img, img2img, ipadapter, revision, controlnet, plus any other combinations of those, then you add in the variables for seed, steps, schedulers/samplers, cfg's, and the infinite amount of prompting you could do, plus so much more I'm missing. Astronomical numbers it would come to

[cyan crown]

and that is the point.. Every possibile picture you can imagine is in it.......it's marvellous

[uncut gull]

Well, that would be impressive by itself, but consider this: If you had infinite VRAM, you could just plug in an infinite img2img source, and get out an infinite img2img. It is literally infinite, in the fancy way we define infinity in maths. 🙂
(You might get some duplication in an infinite image though, just saying.)

[uncut gull]

This system looked clearly harmful, so I used RLHF to make sure it's safe.

[crisp owl]

The crazy thing to think is how much variation you get with just a single seed number. Then just multiple that by whatever the maximum "if there even is one" number of seed numbers

[uncut gull]

Yeah, it would be interesting to know something like: What if you stuck to 75-word prompts, and just 1 sampler, and just 1024*1024 images? Use every combination of unique prompts + unique seeds. How many images do you get??? It's a mindbogglingly huge number. More images than there are particles in the observable universe.

[crisp owl]

Yeah it's like the card shuffling statement, that no deck of cards, shuffled together, has ever been the same. And that's just 52 cards

[cyan crown]

80.658.175.170.943.878.571.660.636.856.403.766.975.289.505.440.883.277.824.000.000.000.000

for 52 cards

[crisp owl]

hahaha so nuts

[uncut gull]

LOL I never thought about the deck of cards stats, but yes. Definitely true.
One caveat though: This is assuming you believe in God creating this as a unique universe. Obviously if you believe in a multiverse, there are infinite universes where you shuffled that exact same sequence.

[crisp owl]

https://tenor.com/view/thats-true-kramer-seinfeld-gif-6564373007119861775

[abstract depot]

❤️

[slender coral]

Lads, what m I doing worng in inpaiting, I'd like to remove this furniture but cant seem to do it:

[native knot]

Crank up your denoise

Also, try turning up your cfg

[slender coral]

What would you suggest for cfg?

[native knot]

8

Change denoise to .7

But you have to play with the settings a bit.

[slender coral]

Working thank you! 🕺🏿

[slender coral]

Final question, there seems to be an outline, is this fixable?

[native knot]

increase your blur a little

[slender coral]

Doing, would you reccomend masking JUST the object?

[urban fjord]

I tried experiment more with Controlnet-Lllite training and for this kind of out/inpainting I think I need a lot more training data and training time than I care to spend on it.
It was supposed to keep more of the input image, but instead it made its own Yann Lecun as "batman" as I couldn't run the controlnet at full strength. Still, it allows for easier composition so it's not completely useless.

[native knot]

Up to you, but I would probably want to mask just a little bit larger of an area. Just kinda something you have to mess with. Also, sometimes just re-running it will give you a different result you might like more.

[rustic garnet]

do you have a single gpu? You can load a 70b model in a single gpu with 24gb vram? 😮

[nimble heart]

thats with only 2.3 bits average quant

I could probably push it a little higher to 2.4 or something but any more it might OOM once the context fills up

[urban fjord]

How well does it work with that level of quantiziation?

[rustic garnet]

hm, I never got any of these 33b models with 8k context size running, not even with 4k context size

[nimble heart]

no idea I havent extensively used it. I just wanted to see if it ran

[nimble heart]

33b is all based on llama 1 which was trained for 2k context iirc

the llama 2 13b and 70b models can do 4k apparently

[rustic garnet]

yeah, there are some 8k lora finestunes

I just once tried a few of these models by TheBloke

I think they were 4bit quantisized

but as soon as I go over 2k context size I get OOM

[slender coral]

For inpaiting is No objects a positive prompt? Or should I worry about putting objects in the negative prompt?

[somber hill]

https://twitter.com/GozukaraFurkan/status/1704999167045571042

Furkan Gözükara (@GozukaraFurkan)

Wow look at this Dall E-3 vs latest Midjourney difference. Midjourney fails terribly

And the last one is from SDXL 1.0

“a giant ladybug looking disapprovingly at a mini elephant“

arts by : Logan.GPT and Tom Bielecki and Me

Twitter · Sep 21, 2023, 11:20 PM

[glad grove]

my fav part of Dalle-3 DALL-E 3 incorporates safety measures that restrict the generation of violent, adult, or hateful content. Moreover, it has mitigations in place to avoid generating images of public figures by name, thereby safeguarding privacy and reducing the risk of misinformation.

[west breach]

well they don't wanna get sued after all

[glad grove]

u can get sued for creating an image of two fictional ppl fightin each other?

[hardy cipher]

I did some weird things and ended up in cursed latent space

Demons

[glad grove]

demons made out of corn 🤣

[hoary saddle]

who's got the best controlnet setup thus far?

not having best of luck

cannot get shields or weapons to work in controlnet to save my life, canny or depth

can definately see controlnet working, did a 100 image batch, not a single shield

[west breach]

shields and weapons in general can be a pain in the butt

[supple knot]

if it was a full body shot maybe it will understand better

[west breach]

I don't see shield mentioned in the prompt

maybe start off with a simple prompt like viking holding a shield, then build on it

[hardy cipher]

Who's got the the best hotdog dog setup thus far?

[hoary saddle]

oops yeah i replaced it last minute from viking shield to space weapons, just trying anything at this point

figure this would be easy with controlnet, you can clearly see where the shield is supposed to go lol

[west breach]

might be easier to inpaint the shield section

[hardy cipher]

Corn cat

[urban fjord]

Train your own ControlNet-lllite on shields and weapon.

[hoary saddle]

yeah i really need to learn training, with the new cards no excuse anymore

[hoary saddle]

yeah i was able to that, but i need to batch run without user input, so maybe i need to try a generic enough mask that would cover all weapon sizes

[urban fjord]

I've not tried with 24GB VRAM, but you might be able to squeeze in Controlnet training on a 24GB card if it's that you have. If you've something smaller then you kind of have to train in the cloud.

[keen nimbus]

@native knot My prompt gen'd:

[native knot]

lol... doubt

[keen nimbus]

lora# 2

[native knot]

Well...using a lora in that context is kinda like cheating. 😄 But it's all good.

[keen nimbus]

https://www.youtube.com/watch?v=Xzex-nN2yM0

dont worry;

owned lulz

that music is actually good up loud

Interesting, a face shot finally

wtf, at the next one:

[hardy cipher]

Not the kind of thing I normally see

[frigid halo]

do I need to use the sdxl_vae with kohya_ss on sdxl based models?

as in the flag --pretrained_vae_path_or_name

[crisp owl]

I love me some good Zombie pics

[vital ermine]

🙂

[hardy cipher]

Made flowers for everyone

[crisp owl]

Anyone use depth controlnet?
If so, what preprocessor do you choose, and why?
I'm trying to decide on which option I prefer

[crisp owl]

I'm interested

[hardy cipher]

Same

[vale eagle]

MiDaS Depth Map

[crisp owl]

Any reasoning or just the first one you tossed in your mix?

[wicked frigate]

midas is kinda The Standard Option ™️

[crisp owl]

Yeah that's what I've gathered from reading around the web, but don't really see anything set in stone on why one vs the other offers better/worse performance.
Beyond leres offering remove bg options

[hardy cipher]

So close

[crisp owl]

zoe depth is out for me. Randomly will throw errors for no apparent reason, while all the other options haven't a single time.

[vital ermine]

Just grabbed this off civit

[crisp owl]

perfect

[glad grove]

smallest sd negative

[vital ermine]

hahaha

Yeah

disappearing legs made me lol

[glad grove]

heres my chungus negative with 27 neg embeddings (low quality, worst quality, lowres,third-party watermark:1.5),polymorphic, washed-out low-contrast (deep fried) watermark, cropped, out-of-frame, (mutated hands and fingers:1.4),blurry,Auroranegative,KFHB,bad_prompt_version2, ng_deepnegative_v1_75t,verybadimagenegative_v1.3,EasyNegative,bad-artist,bad-hands-5,bad-artist-anime,Vile_prompt3,Bad_quality,Bad-image-v2-39000, NEGS Bad Hands,negativeembed,NEGS Bad Image v4,NEGS Bad Prompt v2,badv5,deformityv6,wdbadprompt,badquality-test13-graupel-last ,badquality, colorfixerv1,re-badprompt,realisticvision-negative-embedding,an6,bhands-neg,negative_hand-neg,

[vital ermine]

I need to make some for XL as I had 2 I never did not use for 2.1

need to find their dataset if I can

[glad grove]

yea i only see like 5 on civitai 😔

[crisp owl]

Nearly completed.

[slender coral]

Could you guys reccomend a way to add more detail to a scene like this:

To me it just looks very blurry, not hyper detailed.

[solid rover]

"The Cake's Keeper" --soon on cinema 👻

[full idol]

when is scribble controlNET coming to SDXL?

[uncut fiber]

how you do it? blending?

[hardy cipher]

it's a conglomeration of quite a few things actually

ipadapters, blip interrogators, cfg scheduling, stylers, conditioning tweaks, other things

[uncut fiber]

something beyond my understanding 🙂 I tried A1111 again today

[hardy cipher]

how'd that work out?

[uncut fiber]

i like it. it is bit slower.

Also i have not much extension, which is must

[hardy cipher]

it's all I used until a couple months ago, but was glad to find something that gave me more options and control. I like to tinker with things, and comfy is ideal for that

[peak dove]

I love A1111 now it hs sorted some of its memory problems - ADetailer for faces is a cool extension

[uncut fiber]

@peak dove going to test adetailer

[hardy cipher]

added your flowers

[uncut fiber]

unfortunately it starts stucking...

[hardy cipher]

almost need sunglasses to look at these pictures

[peak dove]

How do you turn OFF Discord Notification Sounds? I have tried everything ... ?

[uncut fiber]

mute channel?

originality 0
i am in a1111 sorry. Cant check.

@peak dove you mean afterdetailer?

[peak dove]

Probably 🙂

Although its called ADetailer under Extensions

[uncut fiber]

yes it is it

[uncut fiber]

much better imho

testing adetailer, thats my problem not using loras and extensions

[uncut fiber]

cataracts

[hidden axle]

Is there any timeline for SDXL inpainting timeline releasing?

[noble shoal]

There is already an existing Model. https://huggingface.co/diffusers/stable-diffusion-xl-1.0-inpainting-0.1

[uncut fiber]

wanna try to see through spiderweb something, i must try it again and again.

[west breach]

https://civitai.com/models/141592?modelVersionId=167333

New update to my checkpoint model

[native moon]

hey has someone a video or something that explains how and why model merging works ( i dont want a guide on how to do it with a tool iwant to understand how it works) :3

[somber hill]

https://twitter.com/GozukaraFurkan/status/1705210084978991248

Furkan Gözükara (@GozukaraFurkan)

SDXL DreamBooth vs LoRA difference is amazing

Look at the tree details. Background details. Even 32 rank LoRA destroys the model

Last one is 256 rank LoRA

Same prompt
Same seed
Same ADetailer
Same training dataset

Twitter · Sep 22, 2023, 1:19 PM

[inland vessel]

how could i do some like this

[cyan crown]

I think is SD 1..5 and qr monster

like this

but here it's SDXL

[native knot]

Easy to do with ControlNet.

[cyan crown]

But not for SDXL

QRMonster isn't out for SDXL

BTX I Made a tutorial if you like

[native knot]

Yes, with SDXL

[cyan crown]

result is not the same

[native knot]

It certainly can be.

[cyan crown]

it's goodm but with QRMOnster it's great

jiuce : https://youtu.be/VoBrOy8g_Ho?si=Vt7wOtsq59QydzzE

[shy kelp]

i like this lol

[uncut fiber]

what is this?

[shy kelp]

upscaler lol

usually much bigger but i am testing smaller (the tests are not going well)

[somber hill]

https://twitter.com/GozukaraFurkan/status/1705242424287658128

Furkan Gözükara (@GozukaraFurkan)

DreamBooth is another level of LoRA for SDXL

Not a cherry pick I am just starting to test

Twitter · Sep 22, 2023, 3:27 PM