🏞｜general-with-images

[languid pebble]

I am a member in that server ... so should be A.I. 🙂

[gray canyon]

i got it thanks you

this is server

[lucid tundra]

2 hour finishing render... absolutereality with a finishing of icbinp

[nimble mason]

i successfully converted the pickles to safetensors and modified the UltraPixel code to be able to load those instead

it works

[wispy nest]

since you know a lot about cascade
do you think ultrapixel is a really big deal?
or is it not really

[nimble mason]

actually trying it for the first time now

[languid pebble]

Didn't really look like a real gamechanger to me

[nimble mason]

i'm pretty sure it was just dumbassery, but you never know... zero chance i'm touching a pickle that's pretending to be safetensors

[wispy nest]

do you think we can get similar resolution and quality with existing models

[nimble mason]

i doubt it

not in any reasonable timeframe

[languid pebble]

We can get hands with 4 fingers with other models, too 😄

[lucid tundra]

Pretty new to this stuff. I get that safetensors are made for not implementing malicious code, but what kind of code would they want to inject?
What is a pickle in this circumstance?

[nimble mason]

a pickle is serialized data from pytorch... all you need to know is that it's like the old days of invisible trojans on a thumb drive running when you just plug it into your computer

merely loading one is enough to execute any malicious code that may be hidden within

there's scanners that can help spot a lot of the simpler cases of malicious code, but like any binary, there's no way to be sure

safetensors are so easy to use that there's honestly no excuse for using pickles for literally anything anymore except your own convenience on your own system

i use them for saving checkpoints while training stuff, in case i need to back up and restart, etc

but you shouldn't ever be sharing them unless it's with someone that has very good reason to trust you completely

[lucid tundra]

So are models from civitai and huggingface generally safe? How does the user without coding ability discern between safe and unsafe when even those named safetensors may not be?

[nimble mason]

that's exactly the problem

i haven't checked whether comfyui will load a pytorch dump falsely named as a safetensors

i'm guessing/hoping it won't, but that's guessing

[lucid tundra]

We are always at risk with trojans etc. With ai image generation I am mostly concerned about the software secretly uploading my prompts and settings to steal my copyrighted work. I imagine just restricting the internet access of the browser is not enough when python is running in the background...

[nimble mason]

that should be the least of your concerns imo

[lucid tundra]

Perhaps. But still a concern.

[nimble mason]

many of us (myself included) leave everything embedded in all images so ppl can learn from the methods and recreate everything

so no one's going to be motivated to find yours if there's any barrier whatsoever (even just not embedding them)

[lucid tundra]

I do that most of the time for open work, but not for commissions.

[nimble mason]

yeah i just think there's too much low hanging fruit out there where zero effort is needed

myself my primary concern is ransomware

[lucid tundra]

Exactly. Good point

[nimble mason]

i'm careful to keep any financial shit confined to a separate system that i barely ever use cuz it's old and shitty

so i'm not worried about CC theft etc

but ransomware can be a fucking nightmare

[lucid tundra]

What is your method for expanding the outpaint without generating new characters?

[nimble mason]

yeah ultrapixel is pretty good

real light on vram

[limpid lichen]

while i love cryptocurrency as a tech, ransomware wouldn't exist in the automated form it does if it wasn't for cryptocurrency networks allowing them to collect money with pure anonymity. the potential for cyberattacks are going to get worse now that models directing attacks can just go on and on. auto-gpt was just the beginning of autonomous instruct agents like bitcoin was the first successful p2p digital currency. it's going to enable a LOT of bad

[nimble mason]

yeahhhh

true that

[whole gyro]

with safetensors, arent the model weights encrypted? so technically, the models arent really open source?

[nimble mason]

they're not encrypted

they don't include the architecture for the model though, unlike the pickles where you can just load the whole thing

but that's better anyway: the code is separate from the data

[limpid lichen]

right now, the people running a cyberattack have to run it. like they can automate a lot but, at some point, they gotta be there making it happen. authorities can enforce on that person. but language model agents don't have to sleep. you can facilitate their perpetual existence and they can operate large network attacks so long as one instance is running somewhere that knows the right encryption key.

[nimble mason]

in practical terms it makes no difference

yeah it's kinda a security nightmare

[languid pebble]

[idle meteor]

@royal monolith

here you see that it is 0%

[royal monolith]

okay, that shows your nvidia GPU carrying the load, which it should be

[idle meteor]

and it sometimes becomes 16% and then immediately 0% .. but if the CPU is intel, then it is higher than 0% always

[royal monolith]

i think, maybe, their stats might have issues. but, well - are you getting the sort of speed and results you're paying for?

[idle meteor]

I don't think so

[royal monolith]

then i might suggest you don't use them.

[idle meteor]

we are doing some video processiong. when the cpu is AMD, then it takes around 10 mins.
the same GPU (4090) handles it in 5 mins if the cpu intel is

[royal monolith]

why are you renting gpus?

[idle meteor]

what do you advice?

[royal monolith]

i need to know why you're renting them first

[idle meteor]

we have an AI project and I need to do some video processing with SD. so our machines are not enough strong. I have m2 max and it completes the job in 10 mins which is quite slow

that's why I thought that we need a good GPU resource.

[nimble mason]

https://github.com/2kpr/ComfyUI-UltraPixel/pull/27 for anyone that wants to use UltraPixel, here's the fix and the real safetensors i made

[royal monolith]

okay, to start with I'd suggest contacing their customer supprt, telling them what you are doing, and seeing if they can give you recommendations for the best options.

[idle meteor]

they said that they are not experts for SD.

that's why I joined here

[royal monolith]

then see if you can always get intel processors so you always get all the work being done by the gpu

your other options are things like Comfy online, huggingface, or googlecolab

[idle meteor]

yes. Maybe I need to install something special for AMD? I am beginner in this stuff.. I am a developer but firstime I am using SD

[idle meteor]

I need to call an API. so SD provides an API

[royal monolith]

it's running on THEIR servers, what would you install that would affect that?

[idle meteor]

I think, it is not their server(s). People put their devices on their systems.. but we use "templates". I don't know.. maybe Torch has a special version for AMD?

[royal monolith]

just try to make sure you always get intel

[idle meteor]

yes but the issue, there are not a lot of intel options 😄

this is my concern

[royal monolith]

it doesn't work like that. what you're seeing is the AMD cpu taking over the graphics processing.

[idle meteor]

exactly

the cpu usage is 99% and gpu is 0%

[royal monolith]

what would you install that would tell the bios of the AMD cpu not to do that?

[idle meteor]

but if it is intel, then gpu usage is high

[royal monolith]

maybe just buy a machine like this https://www.newegg.com/abs-aqa14700kf4060ti16g-stratos-aqua/p/N82E16883360436 and process at home?

[idle meteor]

it will not work. Because we'll create an app so if there are many requests then one pc will not be enough

one video process takes 5 mins and this is quite long. When there are 1000 request at the same time, I am done 😄

[royal monolith]

i promise you, i'ts more than enough, but whatever - just make sure your host is only giving you intel processor. you'll probably have to talk to them and get some sort of special set up

[idle meteor]

It cannot be.. how can I handle 1000 requests?

[royal monolith]

i think maybe you're in need of an enterprise solution if you are doing that. i would suggest yo utalk to AWS about this. that's not a single user setup, you're running a business

[idle meteor]

maybe the rented gpus are not strong enough? The vast.ai support told me that it is an entire gpu, not a core. So it supposes to be fast but it is not..

[idle meteor]

yes but we are kind of startup

I cannot effort high expenses for now

[royal monolith]

so what? you need a business solution. you won't get that by trying to rent what's been created for single users

[royal monolith]

you're looking at the single user prices and assuming enterprise soultions will use those. you need to talk to the hosting companies about their business solutions

[idle meteor]

I see your point

[wispy nest]

on Vast ai
tick the box "Secure Cloud (Only Trusted Datacenters)"
and in the drop down, select "On-Demand" not "Interruptible"
and make sure Unverified Machines is not ticked
this will get you much more reliable machines

[idle meteor]

I just want to increase my budget step by step

[royal monolith]

also, the hosting companies ahve not set up their single user solutions and expected a user to do that much processing, that fast - so you're likely causing them problems.

[idle meteor]

thank you for your tips. I am taking my notes.
I am also looking for another companies.

[spring mist]

there gonna get you into a Ferrari in no time

[royal monolith]

you also need to roll your costs into what the user pays you.

[idle meteor]

sure, but let me start with 1-2 rented devices.. If I see that I am earning something (at least my expenses) then I'll rent more, or maybe I'll buy devices

[royal monolith]

not for the speed you apparently want, and the number of requests coming in you said you had.

that's not alpha testing, or even beta testing. you're in production

[idle meteor]

if single process takes 5 minutes, and if 1000 requests come, I cannot handle.. then some users have to wait for hours

no one will use our app

fact

[royal monolith]

and that's not how it works, anyway. you structure your cost to the user based on what it's going to cost you +10%

you never pay out of pocket. unless you are a startup with investors that have fronted a few hundred thousan dollars

[idle meteor]

for now what I want to know that is there any special configuration/settings for AMD cpus?

if not, then I'll try with other templates

[royal monolith]

you're not luma, are you? cause that describes them perfectly

[idle meteor]

maybe it was a template issue

[royal monolith]

no. there are not. again, you need to make sure you are using intel

[idle meteor]

yes I got this. But AMD has an issue or it is normal? because I find that awkard.. end of the day I have a GPU and want to work with that, not with the cpu

[royal monolith]

can't help that. you can't stop it. there's nothing you, as the user, can do. you'd have to ahve access to their servers and be able to isntall code on them, not on the virtual machine they're giving you. just make sure you can get intel cpus. and don't try to scam the sytem and use a single user dynmic setup to run a business on. get a dedicated business solution set up.

[idle meteor]

I'll do what you say but not now.. because there is no guaranty that my app will be successful. That's why in the beginning I want to continue with rented devices.. and if I see, everything is fine then I can start to invest. Is that not logical?

[royal monolith]

you'll never get your app off the ground if you don't have the right hardware from the start. you're at an impass. either you get onto the right hardware, or you go nowhere. and then your postential users will walk off. you have to "spend money to make money" you are going at this totally backward

[idle meteor]

do you work at nvidia or intel? 😄

[royal monolith]

no. i'm just a computer tech with almost 40 years of experience.

[vagrant dust]

Still don‘t understand the whole discussion. The apple device even with the stable diffusion code from apple won’t get much faster. So you will need a configuration with a good gpu. If you want it the easy way just rent a NVIDIA gpu for the hours you need them. Enough gpu renting websites available.

[royal monolith]

do it right or don't do it at all.

[royal monolith]

AMD isn't apple

[idle meteor]

I see. I got your points. but buying premium cards are risky in my case. But I got your points

[royal monolith]

he's got an AI video creation application and he's trying to run it on something not designed for the usage he's asking for

[vagrant dust]

What really now I see the 40 years of experience. … he talked about his apple m2 at the beginning.

[vagrant dust]

That is why I suggest renting per use

[royal monolith]

i suggested getting an enterprise solution too.

[idle meteor]

here is the summary:

I rent 4090 from vast.ai. And if the device has an AMD CPU, then SD uses only CPU.
but if the device has an intel CPU, then SD uses only GPU.
I find that awkward

[royal monolith]

in the long run, @idle meteor - while you might spend more up front, you'll spend LESS if you do this correctly from the start

[idle meteor]

yes I am doing that. But then I have to find devices with intel cpus and unfortnately there are not a lot

[royal monolith]

you only find it awkward because you're not familar with the hardware.

[idle meteor]

so then I have to have a higher budget

[royal monolith]

find a venture capitalist, get them to find you invenstors, get a couple rounds of funding, get a couple engineers on teh team - do it right

[idle meteor]

Not fully correct. I am a senior software engineer and know hardward but of course I am not an expert

[wispy nest]

pls try the advice I gave you
for selecting a server on Vast ai

[royal monolith]

you're not familiar with, specifically, AMD/INtel+nvidia and stable diffusion

[idle meteor]

you are now opening other doors

[vagrant dust]

Simply rent only the GPU per hours or days. You just specify the GPU like a 3090 NVIDIA and the you pay a few bucks

[idle meteor]

Ok, I'll rent one by considering your advice. I noted that and I'll try it. Thank you

[royal monolith]

talk to yoheinakajima on twitter

[wispy nest]

Vast ai is the Wild Wild West
sometimes you get a good server sometimes not

[idle meteor]

yes but the issue is that if the computer has an AMD, then SD does not use GPU. This was my issue.
And there are not many intel options..

[idle meteor]

ok, I'll look at that. Noted

[vagrant dust]

Again: you select the GPU!

[royal monolith]

it's "if the computer has AMD, then AMD defaults to doing the graphics processing" - it has nothing to do with stable

amd has always been the cpu to get if you wanted to do graphics work, intel if you wanted to crunch numbers

[idle meteor]

how?

[royal monolith]

but if you ahve an nvidia gpu, you want INTEL

[idle meteor]

then this is the answer

that I am looking for

[royal monolith]

you're a senior engineer. i would expect you to know this

[idle meteor]

I don't know SD and graphic processing.. It is first time that I am doing something with AI..
I had experience with image processing but this is completely different

it seems that I'll have to buy my own setup

[royal monolith]

it's graphics in general. the same considerations are for running blender, or even daz studio. what do you normally program?

[idle meteor]

desktop, backend and mobile apps

[royal monolith]

that's going to save you the most money, and be the best solution, for a while. until you get to the point where you either need to build a data center or rent one

[vagrant dust]

Look for example at tensordock. You simply deploy a NVIDIA 4090 for about 0.4$ per hour. You use it while needed and the stop the instance again. Other sites offer equal offers.

[idle meteor]

thank you. I'll consider that.

[idle meteor]

I'll need constantly. Because we will develop an app that needs to access SD API. so I need it 7/24

[royal monolith]

you can also use google colab and perhaps even huggingface

[royal monolith]

i'm going to be very blunt here. "we will develop" - you're not even in alpha in that case. you don't need that much right now. you need enough for you and your team, and a few testers, to work on

you don't need up to 1000 requests flooding in right now. you're not to that stage

and that is perfect for google colab

[idle meteor]

my partner will develop the mobile app and I'll develop the backend. This is not the case. We both have enough experience. The issue comes from SD 🙂

[vagrant dust]

Others offer a 3090 for 90$ a month if you want to start with a low initial invest.
If you only want to create image (and considering your actual knowledge) it might be more useful to use an managed API and you pay as you go.

[royal monolith]

do not become yet another company that publishes an application and puts it into production, and then uses their users as alpha or beta testers. do it right.

[idle meteor]

if I need, then? I have to re-design the logic/structure.. I'll now have enough time than

[royal monolith]

so you redesign. you don't want to put out a product that's not ready

[wispy nest]

Vast ai offers reserved instances
you can reserve the data centre ones for over a decade even

[idle meteor]

where can I find that device for $90? 4090 processes a video in 5 mins.

[royal monolith]

how many frames is this video you're creating?

[idle meteor]

sure. I have to be ready for any case

[royal monolith]

and what's your FPS

[idle meteor]

yes I'll try that. Maybe it will solve my issue

[idle meteor]

25

[royal monolith]

you have got to stop putting the cart before the horse, here. you're worried about what is probably a year down the line, when you should be worrying about right now.

[idle meteor]

my partner is testing it. She reduces it to 10

[idle meteor]

haha you are right

[royal monolith]

25 frames? and you're doing FPS of 10? i do not even want to consider your videos. that's garbage

sorry

standard is 24 fps

[idle meteor]

it does not need to be a high quality video.

the aim is important

[royal monolith]

i'd have to run the clips through davincie and speed them up

[royal monolith]

look around you. everyone and their dog is doing AI video now. either you put something out that's better than pika at least, or you don't put anything out at all. no one will be interested

[idle meteor]

can I run a hugghingface model on davincie ?

[royal monolith]

davinci resolve is a video editor

i think that first what you and your partner need to do is learn video production

forget AI, you dont' know video

[idle meteor]

it will not help me. I need to automatize it

[royal monolith]

that's pika's problem too. they don't know video - thus their videos are junk

[idle meteor]

I'll tell it to my partner :)) thank you

[royal monolith]

you are up against byte dance, luma, kling, open sora - all applications that are being created by people that KNOW video first

you either learn video, or you don't compete

[idle meteor]

I cannot compete and not intend to

[royal monolith]

2 years ago, being able to do animations with ai was amazing. now if the video isn't hollywood quality, people aren't interested

[idle meteor]

we just want to create a simple mobile app that process a video with SD. That's all

[idle meteor]

our videos are not that s*it

[royal monolith]

those already exist. you should probably talk to purz

and look at what animatediff can do

[wispy nest]

can I ask
are you using an image model to process the frames
or are you using a video model?

[idle meteor]

I know animatediff, we tested it too.

[idle meteor]

actually both

two steps

one is an image and second one is a video process.

[wispy nest]

Stability AI offer Stable Video Diffusion in their API
as well as just the image models

you might be able to use that

[idle meteor]

can I use models from hugging face and also some extensions in Stability AI?

[wispy nest]

no

[languid pebble]

https://youtu.be/vxLnqgTCloY?si=vz3T-X1KxfbDF9Aq 🙂

[wispy nest]

but this company does offer that in a serverless setting
https://comfy.icu/

bare in mind serverless settings cos more per hour

than 24/7

and there can be warm-up times

[royal monolith]

they're not going to be happy with him using them to process multiple user requests like he's tlaking about. that's not what they're set up for

[idle meteor]

thank you. I did not know that. I'll look at.

600k credits ≈ 18 hours*

it seems that it will not be enough

[royal monolith]

none of these services you are looking at, @idle meteor , are designed for what you are doing. you need a dedicated, business, solution. you're trying to use solutions that are set up for single users. thats' why they are not really going to work for you - and in some cases you might make the company mad enough about how you're abusing their sytsem that they ban your account. you need to talk to these companies, tell them what you are doing, and get a custom solution set up

[idle meteor]

you are completely right. I'll talk to my partner regarding purchasing some GPUs

[royal monolith]

that's the best idea, yes

[idle meteor]

thank you a lot for your time

all of you

valueable inputs

[royal monolith]

welcome

[wispy nest]

I took a look at their FAQ

looks like they don't mind too much

Yes, if you're looking to use ComfyICU as a ComfyUI backend API, please refer to our API documentation.```

[royal monolith]

they're not expecting it to be used by a bueiness with a 1000 requests flooding in all at once or in a short time

[wispy nest]

would be worth asking them about capacity yeah

[languid pebble]

Good morning coffee(tree)!

[languid pebble]

Funny ... I was thinking about mushrooms prompting my good morning coffee 🙂

[nimble mason]

😄

[languid pebble]

Tired of life? Walking on the street 😄

[steep sleet]

Using Image Prompting (x4) in Fooocus - text prompt = lobsters bathing with astronauts and quarterbacks

[steep sleet]

Fooocus Image Prompting (x4)

[rare sluice]

Hey, I was thinking about using color segmented image (furniture encoded with colors) of interior with Stability-Control-Structure API. Did anybody have similar experience and can share some advice regarding prompt structure? Thanks!

[steep sleet]

Foooocus

[limpid lichen]

reddit balls https://www.reddit.com/r/mildlyinteresting/comments/1ednh4a/a_statue_for_sale_in_venice/

[royal monolith]

you are having a 'ball' with that lora, aren't you?

[limpid lichen]

thats not my lora its just a thing

[covert pagoda]

is that a clownshark in the back

[nimble mason]

just an average day on the beach

https://www.reddit.com/r/StableDiffusion/comments/1edoyy1/how_physics_can_improve_imagegenerating_ai_the/

[royal monolith]

at the very least, maybe gravity will be depicted correctly

[nimble mason]

lol

managed to splice out the B code from ultrapixel so i can use my own nodes for it

lookin good

[wispy nest]

wow that's awesome so we can just get regular cascade stuff working with the big resolution of ultrapixel?

[nimble mason]

not yet

you can just generate at huge resolutions as i've done for a while on here

but this allows you to use bigger resolutions for stage C internally (which ppl call "Lower compression" but i hate that term cuz i think it's confusing and misleading)

which means more structure to the detail

normally, if you were to crank the resolution up, you'd get mutations galore (try setting compression to 22 or something and see how that goes lol)

[wispy nest]

ah okay so its a bit like deepshrink or hidiffusion

I made a bit of progress on details and noise injection

you can get stronger and more interesting effects sometimes by injecting noise in places other than pixel space or latent space

so for example injecting noise into the self attention numbers

or into the control net

[nimble mason]

kinda but more sophisticated

[nimble mason]

that's interesting

are you dev'ing nodes for that?

that's something i never really thought about

[wispy nest]

there's one made already called CADS

https://github.com/asagi4/ComfyUI-CADS

sorry I mean cross attention not self attention

[wispy nest]

just going in and modifying the 2D cross and self attention maps directly has been done by most of the regional prompting and composition libraries I guess

it doesn't seem to be a miracle method

the dense diffusion paper, mentioned in the Omost repo, says that it lowers image quality without careful restrictions anyway

[frank drift]

Sublime

[frank drift]

Tremendous

[frank drift]

Wild

Forgive her for being sexy, but discovering the HiRes Fix function. Creates some sweeet results

Hires is on the left.

[wispy nest]

that one was noise injected into control net

[nimble mason]

oh nice

[wispy nest]

it interprets it in funny ways

this time it was a giant tree building with space pods apparently

and these were the CADS node injecting into cross attention

it just really ups the small details

[nimble mason]

interesting, i wonder what would happen if you injected something other than pure noise

something... noise-ish

[wispy nest]

its a bit different to pure noise its actually multiplied by the conditioning

its actually quite funny that this even works that well

[nimble mason]

oh wow

very intriguing

[wispy nest]

it takes your conditioning signal, that is CFG combined with any control nets etc
and then shakes it a bit
and thereby avoids the thing where a strong conditioning signal reduces the variety of outputs

this might be able to save some checkpoints that are overly over-fitted

[wispy nest]

there would have been a risk of the conditioning vector trajectory being blown completely off course
but they added a little procedure to do a linear transformation on the conditioning each step
to keep it on course

the node just reapplies the rescaling scheme every step though I think I might add a scheduler for that

[hearty violet]

new architecture study, sd1.5

[nimble mason]

really interesting

[nimble mason]

hell yeah! shark power!

[hearty violet]

guys what it this on comfy?

[ruby cypress]

@smoky vigil

I meant checkpoint and lora file tree storage

testing it on my windows machine first before I set it up on the debian

[smoky vigil]

This means you have missing nodes. Use the manager to install them.

[smoky vigil]

the way I set it up is that my comfyui is using my A1111 folders so I save a lot of space.

[ruby cypress]

oh?

interesting,

Guess I'll have to track down that manager you were talking about

is it an extension on the webgui?

[hearty violet]

0k thanks 🙂

[smoky vigil]

https://github.com/ltdrdata/ComfyUI-Manager

[ruby cypress]

Any tips and tricks on ComfyUI before I start get too far in?

[hearty violet]

I wanna too haha I came from a11 and I'm a bit lost

[nimble mason]

sounsd like your WF file might be corrupted

[ruby cypress]

its not that different from flowsheets that I do for datacenters

at least, visually

[hearty violet]

well I desinstall the node, let see

[visual hemlock]

Do anyone regonize this model? 🙂

[clever oar]

healing clay?

[nimble mason]

made out of paper

[quasi summit]

@royal monolith Checkpoint is AutismMix Lightning
Is it possible I accidentally changed something in the Settings tab that makes everything come out blurry? If so, is there a way to revert all settings to default?

[royal monolith]

are you planning to change the entire image or just what you've masked?

[quasi summit]

Only masked part

[royal monolith]

you've got whole picture selected

[quasi summit]

I've always done it like that without issue, has that changed?

[royal monolith]

not sure, but try changing that and see what happens.

[quasi summit]

Alright I'll give it a try

Didn't work with Only masked option, result is pretty much the same

[frank drift]

Super love her

[gray canyon]

I HAVE STARTED CREATING FREE NOTEBOOK AND WORKFLOW FOR YOUR COMMUNITY JOIN IF YOU LIKE, YOUTUBE VIDEO SOON MORE THINGS SOON

[lavish garden]

Prompt: https://civitai.com/images/21542182

"Blazing Ember" - Reimagined (OC: HerpderpIwork - Civitai)

[languid pebble]

Good morning coffee!

[nimble mason]

cookin breakfast over here

[languid pebble]

Sushi!!!

[nimble mason]

lol

[languid pebble]

God shave the king ...

[languid pebble]

They are really selling animal dryer machines ^^

[drifting lance]

#1237459938901491852

[wispy nest]

CADS node but with high PAG:

the CADS node adds some sampling variety and bias towards small details but can get too chaotic
but PAG can lower the chaos a bit

just CADS alone without PAG looks cooler but very chaotic:

[steep sleet]

Nice

SD3@ClipDrop

[nimble mason]

cascade is god

[wispy nest]

does cascade have advantages other than the latent compression thing for speed and resolution?
was wondering why you choose to use it over SDXL checkpoints

[nimble mason]

coherence at high resolutions

not needing multiple stages with upscales, which i find often degrades an image

tiled upscales result in loss of spatial coherence from one region to another

and latent upscales result in some things being fixed due to having more pixels to work with, but also introduce problems every time... usually with loss of complexity, smoothing, and introducing its own mutations cuz it's a poorly trained resolution

[wispy nest]

I see, thanks
yeah I've never been a fan of tiled upscales

or latent upcales

[candid surge]

sd15 - Cute Lunar

going to terraform into more lush utopian stuff

[candid surge]

terraform almost completed

[wispy nest]

I liked it more before the terraform lol

it is more habitable now though I guess

[candid surge]

yeah, look! houses are popping up!

forming complete. need to acquire new goal.

[wispy nest]

actually yeah that does look nice

[candid surge]

new suspect target acquired. need to review target. there seems to be a path. go to explore.

exporation log. surface hard to detect. anomaly ahead detected. need more input to cover for noise in data.

[vagrant wasp]

a pakistani beautiful girl

[nimble mason]

generated directly at 5120x3072

[wispy nest]

wow yeah this was amazing

was this ultrapixel or cascade?

[languid pebble]

Ultrapixel?

[nimble mason]

ultrapixel is just a cascade mod

it allows you to generate higher resolution latents for stage C

without mutations everywhere

[languid pebble]

Have you tried to use other models with Ultrapixel?

[wispy nest]

is ultrapixel quite a small mod?
in terms of code?

[nimble mason]

the code is pretty chaotic right now tbh

i'm trying to sort through it

it's only gonna work with cascade as it's trained and built on the cascade architecture

i see no reason a similar project couldn't be done with other models

but they wouldn't be nearly as vram efficient, i'd think

and you'd need a rack of a100s to train it

[languid pebble]

But we have some cascade models if I remember right

[wispy nest]

fine tunes yeah

I don't rly call fine tunes seperate models

but in some ways they are

[nimble mason]

if you have a bf16 version, and just the unet, i'd think it'd probably work

but eveyrthing's typically provided as the checkpoint and comfyui has nodes for saving clip and vae separately, but not the model itself

so i gotta sort that out

[languid pebble]

I wouldn't drive that one ^^

[quasi summit]

Ever since the inpainting tab looks like this (as of a couple days ago), inpainting has not worked for me (all outputs are blurry and have no variation between seeds). Has this happened to anyone else?

[languid pebble]

4090 desktop version?

[nimble mason]

yep

[languid pebble]

Mobile version only comes with 16GB 😦

[nimble mason]

yeah, really lame

[languid pebble]

😛

[nimble mason]

they cut the cuda cores in half or something too i think

[languid pebble]

Next computer will be a desktop again ...

[nimble mason]

yeah i'm very glad to have a desktop

the UW display is amazing too for comfyui, coding, etc

49" neo g9

5120x1440

[languid pebble]

Ahhh.... that is explaining a lot 😄

Your workflows for example 😄

[nimble mason]

yeah lol def explains why my WFs are so spread out

yuuup lol

[languid pebble]

My notebook is pretty good. Never been a friend of notebooks but this one works for me. Graphic card limitation is the only weak point ...

[nimble mason]

yeah

[languid pebble]

With buying the notebook I won a 5k € coupon for the ASUS shop ... so seling it after a year for a new desktop sounds like a good deal 😄

[nimble mason]

wow yeah no kidding lol

fortunately for you zen5 and the 5000-series are coming soon

[languid pebble]

I'll visit ASUS at the GamesCon as their guest ... hoping for some insider information 😄

[wispy nest]

does anyone know how many tokens is best amount for SD3

[languid pebble]

Sorry, not really ...

[sterile kiln]

can anyone tell me if there are local open source alternatives of the same caliber as Luma AI? (I already know SVD).

(this is Luma AI at right)

[royal monolith]

nope

[wispy nest]

opensora 1.2

needs 67GB VRAM but can run on rented datacenter GPU

but heavily lower any expectations

[jovial tiger]

great stuff

[sterile kiln]

67 gigs of Vram xDDD

I have only 12 (3080ti)

[jovial tiger]

sd3 medium

[wispy nest]

CFGrescale, SAG, PAG and FreeU experiments:

for the most part FreeU is too problematic it was too easy to burn the image

SAG is excellent at very low strengths

[nimble mason]

freeu_v2 is pretty good if you're really really careful imo

yeah agreed with SAG too

gotta be real careful with them

[wispy nest]

yeah it was v2

[nimble mason]

i sometimes like going in the "wrong" direction with the numbers

but only for part of the denoising process

[wispy nest]

PAG seems to be the best at fixing things but can also burn

[nimble mason]

it can give a real boost to complexity and a sense of dirty authenticity

[wispy nest]

oh going the wrong way with FreeU sounds great yeah I didn't think of that

[nimble mason]

yeah il'l do the first two numbers the "right" way and then the next two the "wrong" way or mix and match

[wispy nest]

I wish opposite of PAG was possible

[nimble mason]

casacde is just wild

[wispy nest]

maybe I should check the code and see if opposite concept of PAG can be done

[nimble mason]

well, maybe it could

i haven't looked at it at all but if it's perturbing it, there's gotta be an opposite direction to go

[wispy nest]

are these regular cascade?

[nimble mason]

cascade with the ultrapixel boost for stage C

[wispy nest]

ah ok yeah

[nimble mason]

and my samplers for stage B

which is what's allowing these to look decent at insane resolutions like 5120x3072

[wispy nest]

clown and shark? lol

[nimble mason]

looks like ass with eveyrthing else

yea

[wispy nest]

I still need to try to learn clownsampler

[nimble mason]

called em that cuz we were getting too confused trying to say which node was which for the settings

it was easier with clown comes before shark

[wispy nest]

I couldn't quite work out what the original res sampler does

different to normal sampling

[nimble mason]

it's the RES sampler with a TON of shit built on top of it, a whole ecosystem of stuff

a lot of rewrites wwith the code, it's a lot faster with some noise types and there's way more of em

basically think of it as dpmpp_2s_ancestral except better in every way

the only sampler that holds a candle to it (but is a lot less versatile and can't make nearly as clean of an image) is dpmpp_sde

at least from what i've tried

[wispy nest]

I feel like I've got a decent grasp of the deterministic ODE samplers
but I haven't learnt how the stochastic ones work yet

[nimble mason]

best place to start is prolly the clownsampler tbh

got the most options

set eta = 0.0 and compare to dpmpp_2m

then gradually increase it to 1.0 and see what it does vs dpmpp_2s_ancestral

[wispy nest]

yeah thanks this would be a good way to learn

the funny thing is samplers might go away one day
I read so many papers where they mention near the start that the long term goal is to do the entire thing with a single step of euler

[nimble mason]

the whole idea was to take the most accurate stochastic sampler we have right now (RES, afaik) and make it so you could control virtually every parameter at every step

you can even inject whatever noise type you want at every step, you can override the noise sampler

[wispy nest]

ah yeah I like noise injection

[nimble mason]

that's basically what SDEs do

just add noise prior to every denoise step

[wispy nest]

I kinda started getting the effects I used to have to use noise injection for from CADS

the little chaotic details

since CADS is essentially CFG noise

[nimble mason]

@languid pebble good morning coffee!

[languid pebble]

Good morning!

[unique condor]

damn that looks a lot like an actual painting in did way back. i wonder if my shit made it into a training dataset lol...

either way, shits awesome man

[nimble mason]

thx

[celest sigil]

https://tenor.com/view/shocked-sunglasses-double-sunglasses-look-remove-shades-gif-1883256838391659341

[languid pebble]

https://media.discordapp.net/attachments/989166678233464863/1178791361651167423/Its_magic.gif?ex=65776e01&is=6564f901&hm=7a08021cfaeac58d3a956589f3b275e27ddf21ceeea29df6bdaf01ed93ecdad7&

It's magic ...

[celest sigil]

We are both running 4090s with the same monitor! 😄

...but it's still no excuse for your workflows! 🤣

[nimble mason]

it's heaven for this stuff

[celest sigil]

I've fallen in love with Cascade all over again

[nimble mason]

good!

it's fn amazing

[celest sigil]

Much better than SD3

[nimble mason]

without any doubt

i'm generating this shit directly at 5120x3072... that's insane

[celest sigil]

Also unnecessary 😄

[nimble mason]

i did a bunch of tests, it's actually beneficial

in some cases it just looks blown up vs half that size

in some it gains real detail (the sampling is absolutely critical here)

but in virtually all, the artifact rate goes down significantly and the coherence goes up

it may be best to just gen at crazy resolutions and downsample x0.5, like supersampling

in a number of my tests the difference was pretty stark

[languid pebble]

stark?

[nimble mason]

very clear, obvious, etc

[languid pebble]

German word 😄

[nimble mason]

here's 2560x1536 with gaussian noise sampling

haha nice

[celest sigil]

https://tenor.com/view/tonystark-gif-20634708

[nimble mason]

and here's one that was at 5120x3072 then downsampled x0.5 with lanzcos to 2560x1536

sampled using my custom "cascade_B" pyramid noise

which makes a hell of a difference in many cases

the gaussian one generated at the lower res has a lot of noise artifacts

[celest sigil]

I notice that you don't change the sizes of "c" when you increase the latent size 🤔

[nimble mason]

yeah

C you have to be really careful with

that's why i hate the "compression" thing when inferencing

C is what's important

B just determines how much detail you're gonna get

but you'll get basically the same image, you can even change the aspect ratio and it'll just stretch it

[celest sigil]

Why do you have so many steps in your Polyexpo...90?

[celest sigil]

I'm using 60

[nimble mason]

prolly could get away with less

but i've noticed that 90 seems to be the peak for quality in general with res

here's the real q though, do you have CheckpointSaveFucked installed yet? 🤣

[celest sigil]

Nope...that's fucked 😜

[nimble mason]

lolo

[celest sigil]

5120x3072...forgot to downsize after that 😄

[nimble mason]

gorgeous

cascade does wicked watercolors too

[celest sigil]

I did get this with increased size though Warning: Ran out of memory when regular VAE decoding, retrying with tiled VAE decoding.

[celest sigil]

Look how clean those zips are

[nimble mason]

yeah the vae/stage a is always gonna be the limit here

it's worth it though

plus, what other model are you gonna be able to do 5120x3072 at in 3 min lol

and get anything decent at all

[celest sigil]

The limit is probably because I'm running it all through an LLM 😉

[nimble mason]

these are absurdly huge

ah ha

[celest sigil]

Doesn't matter what I set the eta to, they all look nice. Do you always use 1.0?

[nimble mason]

it's actually ramping from 4.0 to 2.0

the 1.0 is a multiplier if there's a scheduler hooked up to the etas input

otherwies, it's the absolute value

you can get away with lower but i've got it cranked up a bit to make up for the caveman sampling with stage C

[languid pebble]

What's that missing node?

[nimble mason]

this one?

[languid pebble]

Yes

[nimble mason]

don't worry baout that one, that node is fucked lol

[languid pebble]

It doesn't like me 😄

[celest sigil]

https://tenor.com/view/action-drama-sci-fi-inception-top-gif-3309861

[celest sigil]

The planners preparing to build...

https://tenor.com/view/stonehenge-at-night-stonehenge-starry-night-great-britain-prehistoric-megalithic-structure-gif-681031574320192287

[celest sigil]

That is one strange necklace

[languid pebble]

Expensive 🙂

[celest sigil]

Just made from old Grolsch bottles 😄

[languid pebble]

got prompt
Failed to validate prompt for output 112:

• StableCascade_StageB_Conditioning 83:
  • Return type mismatch between linked nodes: stage_c, IMAGE != LATENT
• ClownSampler 75:
  • Failed to convert an input value to a FLOAT value: guide_1, median_d, could not convert string to float: 'median_d'
  • Failed to convert an input value to a INT value: guide_mode_2, pyramid-cascade_B, invalid literal for int() with base 10: 'pyramid-cascade_B'
    Output will be ignored
    Failed to validate prompt for output 79:
    Output will be ignored
    [rgthree] Using rgthree's optimized recursive execution.
    [rgthree] First run patching recursive_output_delete_if_changed and recursive_will_execute.
    [rgthree] Note: If execution seems broken due to forward ComfyUI changes, you can disable the optimization from rgthree settings in ComfyUI.

I need a bigger monitor 😄

[vague grotto]

Is it possible to implement video object recognition functionality similar to track-anything (link: https://github.com/gaomingqi/Track-Anything) in comfyui or web ui?

[sonic bay]

stunning

[shut sinew]

@vague grotto you can do automatic masks like this
Not sure if thats exactly what youre looking for

[vague grotto]

@shut sinew This can indeed identify all objects, but I want to identify and track only a specific object. I haven't found a similar implementation yet.

[unique condor]

Try a vision transformer like florence2. I can draw shapes over things you're looking for

https://arxiv.org/abs/2311.06242

There are comfyui nodes for it and the models are tiny. In the node, you have to set it to the right mode and it has an image output pin that will composite the selections like this

[vague grotto]

Thanks for the tip, I'll definitely try out Florence2

[unique condor]

Yeah I'm not 100% sure how well it will do for solo tracking of one specific person in a group because I've only ever really used it for single things. But there are a bunch of other vision transformers like it and I'd imagine they have some specialized ones for things like crowd tracking.

Those might be your better option, but I've never tried them.

Security companies use them for crowd safety stuff and corporations use them for tracking individual workers to see who is slacking or working, as well as shit like seeing who is wearing a hardhat or not

[turbid stone]

Hello everyone, I am writing to try to understand why many images related to architecture are made with deformations? I have highlighted some of them in red, I have been trying to modify the NEGATIVE PROMPT for days but to no avail... here is what I wrote: cg, cgi, 3d,cartoon, sketch, drawing, anime, ((deform lines)), ((deformed contours)), low quality, (((low resolution))), mutation, jpeg artifacts, ((artifacts)), (((camera deformed))),bad proportions, extra limbs, flooring white.

I use architectureralmix as checkpoint and DPM++ 2M AS SAMPLING METHOD

[celest sigil]

Swim at your own risk.
Too early?

[unique condor]

Because AI doesn't actually understand anything it generates. It just associates concepts with image data and that image data for this specific image might have tens of thousands of images worth of data influencing it from the dataset.

If you want crisp lines and actual euclidean geometry that doesn't do all kinds of weird optical illusion stuff, model out a basic scene in a 3d package and use controlnets like edge or depth

It still won't be perfect, but it will be more consistent

[celest sigil]

This is not the tool you are looking for...

https://tenor.com/view/not-the-droids-kenobi-looking-for-new-hope-gif-16352920

[turbid stone]

ok understood even if in this way I still have to model something... I need it to show a starting point precisely to model after what is defined through these images to be realized with AI... thanks anyway for the answer...

[unique condor]

yeah, you can also model a basic scene out without textures and stuff, just simple colors, and then use controlnets+img2img to fill in the blanks as well. like if the simple 3d scene has brown cabinets and you set the denoise to like 60%, it might figure out that the brown should be wood or something. the controlnets will help it keep its shape though.

[gray ore]

what models come close to these? i think these were done in MJ. im trying to have similar results to these also is there a good way to img to prompt? so i can get a idea

[hearty violet]

did you see it? Hunyuan-DiT new chinese model better than SD3

Model Downloads:
models/checkpoints/: https://huggingface.co/.../HunyuanDiT.../tree/main/t2i/model

/models/clip/hydit/: https://hf-mirror.com/.../tree/main/t2i/clip_text_encoder
/models/t5/HunyuanDiT/: https://hf-mirror.com/.../resolve/main/model.safetensors rename to mT5-xl-encoder-fp16.safetensors

Comfyui Support:
Update your comfyui to latest version

well, have to test to see if really is it all that they are sayng

[nimble mason]

cascade just oozes style

[celest sigil]

Yes, but it has also been terrible at some things 😄

That piano is a mess

Had quite a few hands and feet turning into long tentacles.

[nimble mason]

it's just a vision of our evolutionary future

if you're getting mutations, try these settings

ignore the other crap, that's me fuckin with stuff

[celest sigil]

What's the relationship between those "c" dimensions and the latent size? I thought it was just a divide by 32

...and why are there 4 of them?

I lowered latent compression to 38

[nimble mason]

height_c and width_c are the dimensions for the upscaled latent for C

height_c_lr and width_c_lr are the idmensions for the low-res latent for C, that serves as the guide for the composition

right now i can only really recommend a few values for height_c_lr and width_c_lr
24x24
24x40
18x30
beyond that it's gonna be hit or miss

those are the most heavily trained resolutions

you want height_c and width_c to be exactly the same aspect ratio as the "lr" ones, except bigger

and for stage B, just do whatever you want

you can even have it not be exactly the same aspect ratio, it'll just stretch it

the problem with "compression" is the math behind it and the critical importance of the dimensions for latent C for the composition

if you're using the original cascade workflow, and you have a 1024x1024 empty latent with compression = 42:

1024 / 42 = 24.38... these always round down, so 24

that gives you a latent that is 24x24 for stage C

change that to, say, 40, and suddenly you're at 25x25, which isn't a resolution that it was specifically trained on

and it's mutation city time

[celest sigil]

ah, ok...thanks 🙂

I'm using your sampler for b

The tidiness may drive you insane 😄

[nimble mason]

hahah nice

yeah def mess around with that stuff, who knows what you'll find

my whole goal with those nodes is just to facilitate experimentation and discovery

[celest sigil]

What's "branch mode"?

Does it change much?

[nimble mason]

sometimes it's subtle, sometimes not

all it does is basically generate batches at each step, and select one of the seeds based on various simple comparative analytics

[celest sigil]

Those arms

[nimble mason]

sexy

[nimble mason]

crazy good

[nimble mason]

that keyboard is almost correct

really close

[celest sigil]

Yes, but still a problem with hands/arms 🤷🏻‍♂️

[nimble mason]

see what happens if you use 24x24 for lr, and... idk, 32x32, 36, 40, 42, 48, something ilke that for the other

i have def noticed before that the 1.666:1 AR has a lot more trouble with mutations than 1:1

24,40
30,50
36,60
42,70
48,80
54,90
60,100
66,110
72,120```
these are all 1.666:1 fyi

the higher end ones, espec the last two do some weird stuff sometimes

[celest sigil]

"I am not Groot!"

[nimble mason]

damn is that clean

wouldn't know if those are real characters to the left or not

but cascade with ultrapixel does seem to be better at text

[wispy nest]

```Had quite a few hands and feet turning into long tentacles.````having seen Clownshark's art for like a month he probably views this as a positive 😄

free cosmic horror

[celest sigil]

Sharp!

[nimble mason]

" Stage C is responsible for the content of the image, while Stage B acts functionally as a super-resolution model, adding details and increasing the resolution of the latents, but ultimately not changing the image in a semantically meaningful way"

^this is from the wuerstchen paper's supplementary material, and neatly summarizes why i prioritize manually setting the dimensions for latent C

[unique condor]

You can also decode the latents from stage C to see the small squint version of the image. Based on the compression ratio, it will be like 128 or 256 or something tiny like that. You can basically preview things before wasting further time on the upscale/refinement stageB

I always thought the concept of cascade was a really good idea. Wish it caught on more

[nimble mason]

no kidding

it doesn't even have to be the specific weights or the exact architecture... the general concept is fantastic

i see it almost as being like training a lora vs a finetune in a way

matrix A.B for lora

stage C, B for cascade

C and B dividing and conquering the training tasks in a way that reduces complexity in a superlinear manner

[wispy nest]

FreeU
b1 = 1, b2 = 1, s1=0, s2=0
CADS
noise = 3, apply to = conditioning, key = crossattn

[wispy nest]

same settings

somehow completely breaking FreeU with b1 = 1, b2 = 1, s1=0, s2=0
means you can use CADS without it being overly soft

[wispy nest]

how did you get so many subjects, all sharp?

is this PAG maybe?

[jovial tiger]

it's pixart 512 res upscaled many stages with aam xl anime checkpoint by lykon. it specializes in very sharp line anime.

[wispy nest]

ah ok yeah thanks

I guess the upscales are making it possible

with enough SAG/PAG/FreeU sometimes you can get there without upcale

but is tricky

[candid surge]

Cute Lunar, a sd15 model

[wispy nest]

perturbed the model with enough CADS noise and wrong FreeU settings that it started adding letters to the images 🤔

another time it made star wars into a stained glass window

[candid surge]

sd15 models at higher res seem to double a bit sometimes

[merry iron]

Thank you for using comcom analytics.
"comcom analytics" supports all community managers (moderators and server owners) by stats, visualization, and analytics.

If you have any questions, feel free to ask us!
Your dashboard
Help
Support server

Other languages
en: help
ja: help Japanese

[hearty violet]

guys stable diffusion have a logo?

I'm making a presentation about it I wanna put the logo there

[high carbon]

/credits

[hearty violet]

what?

[royal monolith]

yeah, stability.AI does https://stability.ai/ but you better ask permission before using it

[hearty violet]

ok stability ai

crystal I have to ask permission to put there logo on apresentation to talk about sd making advertising and credits they there?

onyl asking I dont understand why I have to do it, but if I have ok I will do it

[wispy nest]

finally got CADS working properly

had to add PAG, FreeU and Deepshrink to get it to work

[wispy nest]

it turns out you never really need both PAG and SAG together
PAG is meant to just be strictly better
the PAG authors spent half the paper roasting SAG lol

[nimble mason]

cascade is outrageously good, jfc

[wispy nest]

yeah I want to jump over to cascade, and separately Kolors at some point

[nimble mason]

i'm still ripping out code from the ultrapixel codebase

some functions and classes appeared in various versions like 8 times lol

god it's a mess

so much unused code

i've deleted at least 10k lines 🤣

two entire evenings just hitting backspace

getting close to something manageable

[pure monolith]

@celest sigil How do you get high quality and sharp quality generations. Many of my generations are blurred. What upscaling methods do you use. A lot of my generations are image to image.