🧣｜comfy-ui

[boreal stirrup]

Ty guys for the help I feel like 550 GBP is actually a really nice price for a 3090 refurb

[merry ermine]

yeah they are normally a bit over 600

[boreal stirrup]

Or well in that range of 550

[merry ermine]

is this CEX

[boreal stirrup]

Yh xD

[merry ermine]

I remembered the shade of red

CEX is great

[boreal stirrup]

I love them

[merry ermine]

to be honest at this VRAM level of 24GB, its better to buy
I use cloud because I am going 40GB+ and I don't want to buy that

[boreal stirrup]

Fair enough ik some1 that owns a RTX A series of GPUs

But the price is a bit much 😅

[merry ermine]

yeah prices get crazy

[boreal stirrup]

Wait what cloud service u use?

My fren was like "get a 4060 ti 16gb" I was like I would prefer having 24gb and better raw performance for gaming I'm mainly a comp gamer so I don't use RT or upscaling

[merry ermine]

I use a massive mixture
a lot of runpod and vast before
currently I am using Rundiffusion

Rundiffusion costs a lot more though

[boreal stirrup]

Yh I've seen but does it offer more that's worth the price increase?

[merry ermine]

essentially no

I should put more effort in and always use the bare clouds like vast or runpod

[boreal stirrup]

What would u say is better vast or runpod?

[merry ermine]

its more about what you can get at any given time
runpod instances are better on average

rundiffusion costs about 10 times as much

to put it into perspective

but they have fast cloud storage so you don't pay for download time or have to wait

[boreal stirrup]

I see I'm looking at runpod they seem pretty good I only have the download speed issue

[merry ermine]

there is a third option
google colab pro with cloud storage

[boreal stirrup]

I've heard of colab

Is it any good the paid option?

[merry ermine]

again its more expensive than vast or runpod

its more like rundiffusion prices

but its less than rundiffusion itself

[boreal stirrup]

I see what gpus does the colab pro have

[merry ermine]

well colab is the weird one

they don't have gpu they have google tpu

google kinda did its own thing

there is also kaggle, tensordock llamdalabs paperspace

[boreal stirrup]

So many

What's like a good one for my style of use?

[merry ermine]

how many hours per month of stable diffusion?

[boreal stirrup]

Prolly a month very minimal prolly like 2 to 15 hrs maybe slightly more

Mainly cuz I use m2 pro macbook pro and well my 6600 xt on pc

[merry ermine]

ok if its just 15 hours per month then go for rundiffusion

it won't cost too much and it is a big luxury

[boreal stirrup]

The professional plan?

Is that what I should go for

[merry ermine]

you don't need the monthly plan possibly

without the monthly plan you get 10GB storage, which deletes itself every 3 days

with the monthly plan you get 100GB storage, which never deletes itself

but their internet is very fast, its not like vast/runpod internet
you can download models quickly

there are a few more differences that might matter for example I believe to use custom nodes you need the plan

or to download from anywhere other than civit for image ai

this can be an issue if it is not on huggingface

[boreal stirrup]

Tried fooocus on mobile with run diff rn using free 30 min cuz not on pc speed is quite nice I will say

[merry ermine]

ah yeah its good for mobile cos less configuration

on mobile rundiff is literally the only thing I would recommend

don't want to fight a vast server with shell scripts on phone

[boreal stirrup]

Lmao fs 💀

[smoky wave]

Anyone else have an issue with comfyui where the search box randomly closes as you type?

[steep marlin]

Yeah it's super annoying and has been an on and off issue for me for months. What browser do you use?

[smoky wave]

@steep marlin Firefox here. I have some plug-ins though so that may be the cause too

[steep marlin]

I have a sneaking suspicion that it's some kind of race condition or mouse update rate issue where the menu and mouse location are out of order or the browser reads the mouse as some 0,0 or something for a frame. If you move the mouse cursor away from the field, it closes. So my guess is the menu opens and doesn't get the correct current location of the mouse and thinks it's not within range

And closes the box

I've witnessed it on a fresh install and I use chrome

And it still happens with the new frontend

[smoky wave]

You on Firefox as well? I don't think I get it on Chrome

Although on Chrome drop down text is blurry because of scale(1.21) in the CSS 😛

[steep marlin]

It might be something out of control for the comfy devs if it's windows or browser related

[smoky wave]

Yeah. In the meantime the sidebar plug-in works for searching nodes

[steep marlin]

It doesn't happen all the time though, just often enough that it's mildly annoying. Like 2/10 on the annoyance scale lol

[smoky wave]

Haha aight for me it happens quite a lot

[steep marlin]

the only other thing i can think it could be is the human operator. like maybe i'm subconsciously moving the mouse up too far when i go to move my hand over to the keyboard? maybe they need a larger top safezone to prevent it?

idk, ive never been able to reliably reproduce it to actually pay attention

maybe i need to record or set up a last 15 second style replay or something lol

[true onyx]

is there a tool that can convert comfyUI workflows to python code?

[distant pendant]

there was this but no idea if it still works
https://github.com/pydn/ComfyUI-to-Python-Extension

[vital root]

https://civitai.com/models/627087/image-gen-flux-text2image-upscale-to-6144x4096-perfect-for-cinematic-shots-better-than-kreaai

this is my new workflow. enjoy 🙂

[obtuse latch]

Best settings to fix seams when upscaling?

[boreal stirrup]

Guys what's the best help guide or video on comfy ui (not installing but using it for beginners idm if it's like even a hr long I just want one)

[surreal whale]

very threatening language on GPL licensed outputs, staying away

[merry ermine]

a playlist https://www.youtube.com/playlist?list=PLH1tkjphTlWUTApzX-Hmw_WykUpG13eza

[obtuse latch]

TL;DR?

[boreal stirrup]

Ty

[merry ermine]

I generally try to avoid tiled ultimate upscales

[merry ermine]

I learnt comfy by just dragging and dropping around 300 workflows made by others

what I found was that people's workflows don't actually vary that much

you tend to see the same sort of flows like img-to-img with a couple of control nets and IP adapters, into a bunch of image quality nodes like PAG/SAG/FreeU/automaticCFG into a K sampler and a refiner K sampler and then an upscale stage

that describes the vast majority of the workflows I saw

and then you just keep adding image quality nodes, e.g. dynamic thresholding, deepshrink, CADS, vector sculpt, res-adapter

[wispy kelp]

Yes, also most of the time they cna be simplified a lot.

basically everyone is in search of the consistency holy grail really.

[vocal grail]

Is there a FLUX discod already?

FLUX is the best model holy S

[wispy kelp]

lol

[livid gust]

@vital root I agree with @surreal whale . When files are placed behind the sign-in wall. I don't use them.

[boreal stirrup]

I've yet to still buy a gpu

A refurb 3090 for these new models

[merry ermine]

is comfyui set to work for flux controlnet?

trying to make this work https://huggingface.co/XLabs-AI/flux-controlnet-canny

but getting this error instead

[surreal whale]

looks like an update went in an hour ago pertaining to mmdit cnet, maybe that, i havent tried it yet, computer crapy

[merry ermine]

hmm i updated comfyui some mins ago didnt see any update for flux controlnet unless i missed something, is that supposed to have specific flux contronet that i can look for in nodes?

[flat dirge]

Comfy released a version of the XLabs-AI\flux-RealismLora that works in ComfyUI
https://huggingface.co/comfyanonymous/flux_RealismLora_converted_comfyui

[primal otter]

So. I am going on a BIG learning endeavor and I have learned some stuff but there is some other stuff I haven't quite gotten. Right now, I think I am doing ALOT of struggling with Loras.

Take this Lora for example:
https://civitai.com/images/16973917

It has 1 trigger word, and the prompts in alot of the detailed images are pretty similar. So I try it and think "Well, I'll write my own prompt, plug in the Lora, and get similar results."

Mm. Not so much. Either the Lora didn't activate or did something wrong. So I took half the prompt of one of the images, and lo and behold, it looks how I am expecting.

"Hmm" I thought. "That'd suck to have to throw in ALL that stuff to use 1 lora... I wonder what happens if I keep the prompt aspects and remove the Lora.

And that resulted in a much less Lora-y, which is good!

But then you take something like this Splitscreen Lora I am trying to get to work which is COMPLETELY not working. I got it great with Auto1111! Buuuuutt.... I got some abhorrent abominations on comfy.
https://civitai.com/models/380125?modelVersionId=424387

So I guess I am just trying to figure out: How can I know if I am using my Loras properly? I have been doing lots of research into it and seen more than a few videos but I feel like I am just missing something. Maybe a better question should be something to the effect of... does Civitai give all of the trigger words it should be?

Pics for reference with the vector one btw:
Without Lora and ton of prompt
With Lora nd much of sample prompts
WIthout Lora and much of sample promps.

[flat dirge]

You're getting the strange doubling because your dimensions. 1296 x 1920 is too large.

There's a bunch of different custom nodes out there that will help you pick latent dimensions that is more compatible with SDXL. If you really want to make larger images then you have to do first pass, upscale, into another KSamplers at lower denoise.(HiResFix) or you have to use one of the other methods out there to avoid those issues. Like PatchModelAddDownscale (Kohya Deep Shrink)

Here's a playlist to learn Comfy. Ep. 3 shows latent upscaling (HiResFix)
https://www.youtube.com/playlist?list=PLH1tkjphTlWUTApzX-Hmw_WykUpG13eza

[primal otter]

Ahhh, that is good to know. That answers a very different question that will most certainly help me in teh future. I'll keep that in mind going forward.

[vocal grail]

I am having problems with insightface and I don't know how to install it using stability matrix

Can I get some help on which folder to write CMD because stability matrix installation of comfyUI is not clear

[vocal grail]

How can I fix my DWopenpose?

[marble kettle]

wow

Hi guys, is it normal to take this long? 3090Ti with 64GB RAM

[flat dirge]

You have CFG set to 4. This will almost double inference time and provide bad results. You should use only CFG 1 with Flux-dev. Flux-dev is what's called a guidance distilled model. They don't use CFG or negative prompts. If you still want to replicate the effect of CFG there is an added parameter the model is trained on called Guidance Strength. This is trained to give outputs that look like different CFG images of the teacher model.

[marble kettle]

Thank you! I'm very new to the community (just one day old), and advice like yours means a lot to me. Thank you! Do you have anymore advice for newbie like me? something I should know, or tips and tricks?

[sick badger]

have any luck yet?

[merry ermine]

not really, we might need comfyui node update

[sly sundial]

The USDU Mode will take a long time because you're upscaling an already large image. You've already upscaled it 4x and you're now doing it another 2x, which is why there are so many tiles it needs to do.

This is way too large for video.

[gusty mirage]

hi guys!! comments on my workflow pls.. am i using the latent upscale correctly? the leaves are a bit plasticky when i use upscale latent.

the first one with latent upscale. the second one ultimate sd upscale only..

[gusty mirage]

because of the tiled decode i think.. just use controlnet tile.. works for me.

[flat dirge]

.90 denoise on the usdu is a lot. I would try adjusting that and see what comes out.

[polar stirrup]

Anyone else got the RuntimeError: CUDA error: CUBLAS_STATUS_NOT_SUPPORTED when calling cublasSgemm( handle, opa, opb, m, n, k, &alpha, a, lda, b, ldb, &beta, c, ldc) for comfy with Zluda?

[wispy kelp]

There used to be an arrow id click at the bottom corner and it woudl clear my gpu usage... now its gone how do I access it again?

[wispy kelp]

I clicke dit and it said "resources or chache succesfull cleaerd"or something similar... and then my gpu would be emptied.

[wispy kelp]

It was where those coordinates are a tiny arrow...

now my vram usage never goes down even if i clear the workflow and load the default. I have to restart comfyu for it to go down

[surreal whale]

i dont remember that, if you have manager there is an unload models button in the menu, not sure thats what you need

[wispy kelp]

Nope unloading models still leave smy vram usage high

this button would make it 0

[fading mural]

im not even gonna bother trying flux with my 4gb gpu

probably 1m for 1it

or more

woop i dont have 32gb of ram

[vital root]

https://civitai.com/models/627087/image-gen-flux-text2image-upscale-to-6144x4096-perfect-for-cinematic-shots-better-than-kreaai version 3.0. enjoy 🙂

[gusty mirage]

hi!! how do you install Canny for Flux?

[primal otter]

124 hours... for comfy to unzip from 7zip?

Is that right? (Trying the non-manual method)

[merry ermine]

this workflow is crazy.

[steep marlin]

PSA: the latest https://github.com/comfyanonymous/ComfyUI/commit/413322645e713bdda69836620a97d4c9ca66b230 for comfyui cut my inference times in half. Used to get like 16 sec/it with flux, now I'm getting like 8 sec/it (i only have 8gb vram)

I see why, it's actually sometimes using all 100% of my CPU alongside it. Normally, it will only use the e-cores (usage sits around 40%). Was noticing that sometimes it would be 8 or 9 sec/it, then other times, like 12 sec/it.

Another note: If I set the python.exe priority to above normal, it seems to be able to reliably stay in that 100% range. Must be some kind of windows scheduling thing, PC is set to prefer max performance and all that jazz in the power settings, with all power saving stuff turned off. My CPU temps never get above 60C, so it's not throttling or anything. Going to test if maybe the Windows Security has any settings that might occasionally cause this.

Edit: so from what I'm seeing, it will only use 100% of my CPU if nothing else is touching the CPU. Basically, it will only do it if the rest of the system is idle. If I'm browsing discord or use a browser, it will only allow python.exe to use my e-cores like it's a background application.

@storm folio maybe you have some insight on these last couple posts of mine? Maybe there's some way to specify or make a command to do something like max cores - 2 so that it will reliably use more CPU instead of only when the rest of the system is idle or when the python.exe is set above normal priority?

[wispy kelp]

That! What is that and where did it go? I want it back.

[marble kettle]

Hi Guys, Am I doing something wrong again??? Super slow here! 3090TI 64gb ram

[drowsy compass]

hi guys, i have a only 16gb ram on my pc, is it possible to lower ram usage because i want to load 6gb checkpoint and i tried --lowram argument but it didn't work

[steep marlin]

i figured out a reliable workaround, but it's annoying. after launching comfy, i have to go in and manually set the processor affinity and disable all the e-cores (in my case, 12-19 since this 13600kf has 8 e-cores). i know there's some kind of app that can launch apps with preset processor affinity profiles, that can automate it.

whatever operations python or cuda are trying to perform, it's pretty much strictly attempting to only perform them on the weaker e-cores. i'm assuming this is due to the windows scheduler and i've seen similar issues with other things.

yeah, googling around for "python e-cores" and seeing a lot of people with similar issues. some people are suggesting that running in administrator mode will allow it to use all cores. seems to have started around 22H2 or so, or with the 12th gen intels. i'm going to test it out

alright that fixed it, set the python.exe in the embedded folder to run as admin in the properties, and then launched comfy with the normal .bat file(a second python.exe window will open up, the non-admin console can be closed afterwards). it will now use 100% of my CPU without issues.

I don't personally recommend doing this unless you 100% trust every addon author, but for now, this works for me.

basically, it turns out that it was a limitation in how user level python and windows interact

[drowsy compass]

Nevermind, i just found a solution just mount your hard drive as additional RAM

[wispy kelp]

I got a node that unloads memory cache and models.

The arrow was better as it unloaded only memory cache and left the models in.

some Comfyuis i see have this magical arrow most dont.

It's an elllusime thing that seems super important yet no one knows about it.

[flat talon]

Is this one？

[wispy kelp]

yes

😦

What is that magic. Please help me Obi Wan Kenobi. You're my only hope.

[sly sundial]

I think that's from a custom node pack, but I don't know which.

[wispy kelp]

Wel i did not uninstall it so why did it vanish

It seems I have inadvertently subtled upon a magical node ancient relic as a novice comfyu resident. =0

An ellusive magical stone...

[cursive ridge]

2 questions:
How to I add a SD3 refiner or enhancer (or whatever it's called) onto the end o this workflow? Yes I need that SDXL or 1.5 in there!

How would I make a Flux version of this? Just adding flux doesn't work lol, just adding a flux specific checkpoint loader doesn't work...

[flat dirge]

You will run into many different issues with packages that rely on many CUDA performance libraries.

ZLUDA offers limited support for performance libraries (cuDNN, cuBLAS, cuSPARSE, cuFFT, OptiX, NCCL). Currently, this support is Linux-only and not available on Windows.

https://github.com/lshqqytiger/ZLUDA
ZLUDA was mainly created to be able to run PyTorch and little else.

[polar stirrup]

OK
Thanks
Got it to work on Linux on native ROCm instead

[vocal grail]

My workflow for animatediff works well on my 12 gb vram computer.but in my 3090 24 gb vram goes black monitors....

I have to manually shut down the pc

[vocal grail]

I installed the motherboard chipset and now it worked

But I dont know for sure of I just got lucky

[vocal swallow]

6gb is not a lot for a checkpoint, 16 system RAM is plenty, lowram is misspelling of lowvram you nincompoop

[vital root]

https://civitai.com/models/627087/image-gen-flux-text2image-upscale-to-6144x4096-perfect-for-cinematic-shots-better-than-kreaai Version 4.0. Enjoy ❤️

[marble kettle]

Anyone tried to use HTTP Request to send prompt to ComfyUI?

[obtuse latch]

Thoughts on how to best fix these hands?

[obtuse latch]

req = urllib.request.Request(f"http://{server_address}/prompt", data=data)

[obtuse latch]

Anyone know why loras are not working (after testing they don't seem to do anything) in my Text encode?

[merry ermine]

in comfy generally you set the lora strength in the load lora node rather than using a string in the text encode node
a good general handfix method is SEGS detailer from Comfy Impact pack, combined with meshgraphormer control net along with some self attention guidance and perturbed attention guidance
if that isn't enough then fine tuning the ultralytics YOLOv8 model can make the SEGS detailer work a lot better

[obtuse latch]

Any way to just control it through the text?

[merry ermine]

yeah sometimes people do that with custom nodes
https://github.com/asagi4/comfyui-prompt-control

[steep marlin]

I guess the guy that makes Forge implemented NF4/FP4 support for Flux. Makes sense since these dit models function the similarly to how llms do. He also put in CPU offloading so you can offload layers to the CPU, just like you can with most llama.cpp based apps. Kinda cool, wonder if comfyanon will implement something similar. These models are just going to get bigger and bigger and typical consumer levels vram amounts just aren't going to scale as fast.

And NF4 can produce similar weight quality to fp8 due to how it works

[merry ermine]

this is really cool

cos those 2 things (changing number format and CPU offloading of layers) has been huge for LLMs at home

I feel like Forge and SD Next / Vlad Diffusion are both competing to be the "fancy A1111"

and then Fooocus is the "more simple A1111"

[steep marlin]

I noticed that forge isn't using the CPU at all, so it still ends up being slightly slower(even with nf4) speed than comfy where I can use all 20 threads for my 13600kf while also using my 2080

With flux

Oh and it ends up eating up a shitload more sysmem vs comfy that will drop to like 16gb after everything is loaded

[steep marlin]

Damn their quick:

https://github.com/comfyanonymous/ComfyUI_bitsandbytes_NF4

[distant pendant]

interesting, bnb usually results in some speedup unlike quanto so this should be pretty neat

if it doesn't reduce the quality too much at least

[echo jay]

Hi dudes, trying to install nf4loader for the new flux model, it gives me the red node, any tips? i already have the custom node in my folder

[steep marlin]

Nf4 is a neat format. It can have individual weights vary from like fp4/8/16/32 on a per weight basis. I think it converges around 4.something bits, but it can assign higher precision values where it needs them.

From my testing, the quality definitely seems on par with the fp8 version

[distant pendant]

yeah from what I'm seeing of people posting around it seems very good

can't try right now myself but exciting times

[steep marlin]

You might need to update or install bitsandbytes. In the python embedded folder, open a command prompt and run python.exe -m pip install -U bitsandbytes

[echo jay]

Can you share the extact Path? I installed It in swarmui so i struggle to find it

[steep marlin]

Ahh, not sure, just go into your folder for comfy or swarm and search for python.exe within it+subdirectories

once you find that folder, at the filepath at the top of the window, click into it and type cmd, it will open a command prompt already within the folder you're in. then run that command python.exe -m pip install -U bitsandbytes

[inner summit]

y'all probably get this q 1000 times but is it normal for flux fp8 to take 50 sec per it on 32gb ram rtx 4070 8gb vram?

or is there some trick im missing

[steep marlin]

the trick is to have 16+ gb vram

[echo jay]

Thx dude, gonna try later 🙂

[distant pendant]

where does one get fp16 all-in-one checkpoints to use with the nf4 loader in comfy? feels like there's tons of them on hf but every single one either some merge, already fp8, or some weird mix

or am I supposed to use the already quanted checkpoints, this is weird

[surreal whale]

yes the node supports the checkpoints linked

[steep marlin]

To use that nf4 loader, you need a prequantized nf4 model. Maybe eventually he'll add in some automatic detection stuff for the regular model loader or some kind of toggle switch to pick the mode, but for now, it's a barebones addon with limited support

[distant pendant]

would be nice if it worked with the unet loader at least so I could use the nf4 transformer and fp16 t5 or some other combination

can't see any obvious way to apply these model options from the nf4 node to unet loader looking through the code

this is great regardless tho, I can now generate an image from cold start in like 20 seconds, down from 200

[steep marlin]

You might be able to manually make a full model where you load the model with the regular model only node, dual clip and vae and then use the save checkpoint node. I know he put out an nf4 version of schnell but it's the transformer only.

I'll test it out tonight if there isn't already one out by then

[distant pendant]

ah, didn't think about doing that

that said the linked checkpoints seem to work well enough so probably going to keep using them

can barely even tell any quality difference from full fp16

[cloud pilot]

hey guys anyone want to help me im in trouble with control net

i cant use it

https://ibb.co/mb8p3N0

[surreal whale]

just use model only from NF4 node and load your other stuff in seperate nodes

[distant pendant]

that's a good idea too, thanks

[flint laurel]

hello

do i need to use any special node to save the metadata for the image? cuz it seems like my image doesn't have it.

[knotty quest]

Anybody knows why this inspyrenet rem bg is not doing anything for me?

[spring jungle]

has their been any update to flux nodes in adding step control ?

🤔

[flat dirge]

inspyrnet transparent-background outputs a mask. Combine the mask with the image alpha channel.

[rigid otter]

https://tenor.com/zQ3X.gif

[vocal grail]

Forge authot has made a FLUX dev model that works way faster than FP8

https://huggingface.co/lllyasviel/flux1-dev-bnb-nf4/tree/main

But I guess Comfy UI doesn't support this yet.

I just tested it on Forge and it works on my RTX 2060 if CUDA 12 is installed and used

[sly sundial]

Yes, it does.

https://github.com/comfyanonymous/ComfyUI_bitsandbytes_NF4

[sly sundial]

Does anyone know the name of the node that lets you choose a named film style and/or artist names from a list? I saw it a workflow a few days ago, and have it installed, but I can't find it now!

[wispy kelp]

CSV loader

[flat dirge]

Prompt Styler?

[sly sundial]

Thanks, but not that one. It has dropdown lists with films and artists.

[wispy kelp]

CSV Style Loader

[sly sundial]

No, I don't have that one

[wispy kelp]

Well that one load sin lists of those things.

[flat dirge]

SDXL Prompt Styler (JPS)

[sly sundial]

Those are the lists!! 🥳

Thank you 🙂

[merry ermine]

this one looks kinda good

[tranquil parcel]

In comfy if I do a batch of 4 images, how do I find the seed number of each image is it possible, is there a custom node to help find this out?

[steep marlin]

Should be in the meta data if you drag the image back in and load the workflow. You make sure to set any seeds to fixed before running it

There are add-ons that can also view the metadata as well

[tranquil parcel]

I'm taking a batch not a single, I know this possible Automatic 1111

[cold radish]

Yesterday I was using a model to generate images fine, but today the model was only generating black. I looked at the file checksum, and it looks like it changed--has anyone else had this happen? I don't see any file writes according to Windows Explorer--last modify date was 2 days ago, before the issue occured, and yet the hash still changed and the model's unusable

[verbal sky]

Newest video is now available! Flux EXTREME 🙂 https://youtu.be/aZt5CL3r1lQ

[tranquil parcel]

https://github.com/lllyasviel/stable-diffusion-webui-forge/issues/1001

[brisk yoke]

#✍🏼｜rules-and-tos meinv

[steep marlin]

They'd likely need to be converted on a per block basis to match the exact precisions of blocks they patch in the model, since nf4 uses variable weight types like u8, fp8, fp16 and so on all in the same model

Well not necessarily converted, but cast to while loading and applying the lora

So for instance, if a Lora is in fp16 or something, and the weight in a block it's patching in the nf4 model is in u8, it would need to cast the fp16 to u8 and then perform the adjustment

like this:

that's the flux realism lora on the right

note the wide variety of weights on the nf4 model on the left. this is likely why people are having issues with loras on the nf4 versions of the models. it shouldn't take all that much code to make it work

it just needs to check the lora weight format, check the target model block weight format, cast the lora weight to that dtype, apply the adjustment and then move on to the next block

i could be highly wrong though, but this at least makes sense. it's 3am, so my brain is fried

[merry ermine]

I hope we get IP adapter mad scientist node for this

cos it goes block by block

so you can kinda see what each block does

like for SDXL, block 3 is the main one for structure and block 6 is the main one for style

which is why deep shrink only shrinks block 3

and most style transfer stuff is going via block 6

[slim shell]

hey anyone have a fix for this? i have the exact same issue https://github.com/comfyanonymous/ComfyUI/issues/4329

[wispy kelp]

Comfy is releasing a new frontend on August 15th. <--------- what does this mean and how scared should I be?

[merry ermine]

don't update for a while if you want stability

if you are not worried and want the latest then just update right away

[deep rapids]

can I use Comfy UI and Flux.1 w/ my 1660 ti?

[wispy kelp]

So how do people load their flux models in the checkpoint loader? i can onyl load them unet loader

yes i updated everything. Everything

yeah like above

hmmmm ok i kinda see whats goin on

[surreal whale]

there is a mess of versions some load in unet loader some in checkpoint loader

[wispy kelp]

The "flux scene" moves fast.

[sly sundial]

There are different models to download for each.

[dim grail]

Hello, quick question, how can I implement node control in custom node?
e.g. I want to increment a variable by a user-defined step value after the render is done, so that I can queue multiple renders and have it automate the value

[livid gust]

Try adding a new seed node and change it from randomize to increment. It will increase after every queue. If you need to multiply that by a constant try the math nodes.

[distant lagoon]

you can try it, but that's a 6G card. I've heard forge might currently be a better option. obviously try the most memory efficient configuration , which I think is FP8 schnell, but given that I've only been using flux for literally hours...I'm not the expert just yet

[sly sundial]

You can switch between new and old with a startup switch. This is available now, if you wanted to try it.

[merry ermine]

starting to use diffusers today

gonna try to make a big X Y Z grid from scratch as first project

cos this is awkward in comfy but should be easy in diffusers (hopefully)

[tough hornet]

Hello how could i have the mask preview node on comfyui ? thanks

[sly sundial]

Install this custom node pack

[carmine copper]

Whos better controlnet model

Mistoline or xiner sdxl

[merry ermine]

definitely the pro max here https://huggingface.co/xinsir/controlnet-union-sdxl-1.0/tree/main

[steep marlin]

Just make sure to include the union node between the model and the controlnet node to pick which mode you want to use it as

[merry ermine]

funnily enough it worked ok without

but yes it helps

[sly sundial]

It does work, but it defaults to one type, so you have no control over what it is using.

[tough hornet]

Hi! I'm new to comfyui and I use FLUX.
I would like to be able to generate a logo. All by making the FLUX template be inspired by a logo I have already created.
How can I do this?

[merry ermine]

oh I see thanks
in that case it was probably not working in my workflow

[fading mural]

is there a node or something that'd allow me to stop everything and wait until i resume after im done adjusting stuff

[carmine copper]

Im using it for Ai product shooting only . So the union is multi controlnet , but mistoline or xiner sdxl depth that im confusing

[marble kettle]

Hi Guys!
3090TI 64gb ram here,
recommend Pytouch and Cuda version?

[steep marlin]

Click the show queue button and click cancel on the current thing in the list, make adjustments and click queue again

[covert bough]

https://youtu.be/jk0jKKdHZvo?si=M9bx2v-GrpjXxPWf

[lavish stone]

i have a rtx 2060 mobile. Flux works very well with fp8. But when trying to gen with nf4 with the new comfy node, it get's stuck at the ksampler step, just keeps loading and does not generate an image. Has this happened to anyone else ?

[merry ermine]

@lavish stonenf4 not supported on 2060

can you tell me the speed with fp8?

[merry ermine]

i am wondering if there will be breacking changes with webui thats uses comfyui as backend.

[wispy kelp]

idk

but i updated fast now

lol

just in case

[lavish stone]

only on 2060 or 2xxx on general ? I have 20s/it

[surreal whale]

i use it fine on a 2080TI

[tough hornet]

how i can solve ?

[hardy tree]

if i recall, cr nodes are comfy roll nodes, you have to search and install that from the manager

[tough hornet]

I'm trying to get my img2img stream to work but even when getting latent CR, my Ksampler shows red

[hardy tree]

yea but what is the error?

when it goes red, it usually pops up an error message

[tough hornet]

he cannot find my fluxdev checkpoint

but now I don't have the message anymore, I only have the window in red

[hardy tree]

are you using the older version that is separate components? or the all in one version?

maybe refresh or reload the whole comfyui/browser

[tough hornet]

all in one in think, ive been dl comfyui just now

[hardy tree]

did you follow these instructions: https://comfyanonymous.github.io/ComfyUI_examples/flux/

[tough hornet]

https://openart.ai/workflows/maitruclam/comfyui-workflow-for-flux-simple/iuRdGnfzmTbOOzONIiVV

i followed the steps :

[hardy tree]

ah now i understand the problem, the tutorial you linked is using the components version with the unet folder, and you are using checkpoint loader simple that uses the all in one version, but you didnt download that version... so again.. its better to maybe follow instructions from the link i gave you, cause it covers both variants, all in one or components, and even gives you workflows to load

it's up to you to decide what variant you want and then go from there

[tough hornet]

oh thanks, I guess I need to put the dev fp8 stream in my checkpoint folder...

[hardy tree]

yes, but it has to be the all in one version

so you have to download it if you dont have it

[tough hornet]

done ! 😄 thanks

[hardy tree]

np

[tough hornet]

i dont understand when i launch my Florence 2 , this box appear

[silver nebula]

Do I git clone the new ComfyUI FrontEnd into the custom_nodes folder?
And where in my startup script (ComfyUI/python main.py) do I place "--front-end-version Comfy-Org/ComfyUI_frontend@latest"?

[deep rapids]

Thanks but I guess no one is ever going to answer my question

[somber flax]

Hello, is anyone familiar with installing JoyTag? I use it like this but there is no output, and the Show Text node is unusable. What do I do?

[steep marlin]

In the Nvidia start .bat file or whatever it's called. You don't have to git clone anything, just edit that file and add in that launch flag to the line where you see other --stufflikethis

[silver nebula]

With the new frontend an intermittent error "type failure failed to fetch"

[merry ermine]

how much speed per image?

[silver nebula]

2 minutes 10 seconds NF4 (8Gb VRAM)

[merry ermine]

btw is flux support animatediff?

[steep marlin]

don't know that error, i suggest you check the github repos

[faint mountain]

What's your video driver , AMD or NVIDEA ?

[upbeat needle]

"Hi everyone,
I have all my models stored in ComfyUI. I'm currently testing the Forge web UI and I'm wondering how to direct it to use the same model directory. I want to use all my checkpoints, LoRAs, UNets, etc., without moving them from ComfyUI. I've checked the Forge config file but couldn't find a main models directory setting."

[wispy kelp]

Hopefully Nvidea. :/

[distant lagoon]

I just use symbolic links for the checkpoints and loras, but I'm a linux guy. can you do that in windows? no clue

[steep marlin]

Yes, very easily. Both for individual files and whole folders. The command is mklink

[scenic beacon]

yes，I learning comfyui

[tough hornet]

Hi, I have a little question that is bothering me: I don't understand why when I change the batch size, I only have one image that comes out.

[wispy kelp]

So how is the new frontend treating everybody?

[distant pendant]

I hate the new menu with a burning passion so not using it

the new search also somehow manages to be worse than the old one despite having pretty previews so not using that either

it's 'fine' otherwise (since it's the same thing without these two)

[wispy kelp]

[distant pendant]

there's probably some better code underneath since it's lagging less so that's an upside I guess

[wispy kelp]

ok

[distant pendant]

also the most annoying bugs like nodes resizing on browser refresh (primitives especially) still haven't been fixed, will they ever...

[steep marlin]

You can revert it back in the settings and the search menu as well

[distant pendant]

I mean, yes, that's what I did (that said the old search is now marked as legacy and I'm guessing it will be dropped at some point, and I really don't share the enthusiasm for the new one while it's taking my whole screen and not even showing relevant nodes)

[merry ermine]

you guys made some sweet UI changes with the most recent update... loving it.

[distant lagoon]

in my case update broke me last night, workflows wouldnt load,I had to load a prior commit

[fickle remnant]

Pretty badly, I have nodes I can not delete, I have connection I can not delete or overwrite. I have nodes and connections stuck to my mouse pointer sometimes...

[covert bough]

Matteo's putting a list together of all the nodes that the new update has broken. seems like the second comfy got a team, everything fell apart.

[fickle remnant]

Nah they got this, it's not easy to switch to a new UI/UX language

[spring jungle]

is it possible to make preview node alittle smaller xD

when you use search fuction

[fading badge]

guys do you know how to start comfyui with highvram mode?

[steep marlin]

--highvram in the launch flags @fading badge

[fading mural]

anybody knows nodes that speed up generation? i have a gtx 1650

[minor plaza]

Is it possible to get ComfyUI up and running on a Windows 11 PC with an RX 6800 XT GPU WITHOUT using Comfui-Zluda? An update broke the Zluda version for me upon running it's start.bat file to launch it (it also does updates) and I have ComfyUI up and running, but I can not get Flux to work with it properly due to this error in the process:

got prompt
model weight dtype torch.float8_e4m3fn, manual cast: torch.float32
model_type FLOW
C:\Users\G30\miniconda3\envs\ComfyUIPython311\Lib\site-packages\transformers\tokenization_utils_base.py:1601: FutureWarning: `clean_up_tokenization_spaces` was not set. It will be set to `True` by default. This behavior will be depracted in transformers v4.45, and will be then set to `False` by default. For more details check this issue: https://github.com/huggingface/transformers/issues/31884
  warnings.warn(
Requested to load FluxClipModel_
Loading 1 new model
clip missing: ['text_projection.weight']
Requested to load Flux
Loading 1 new model
[F dml_util.cc:118] Invalid or unsupported data type Float8_e4m3fn.

[solid fern]

First time attempting Comfyui. I got this error. What do I fix?

[dire sapphire]

is there way like in A1111, to have a tab or a loader node which can preview the lora/checkpoing thumbnail ?

i have hard time remberring all my embeddings/lora

[cursive ridge]

Does anyone have a Flux Dev workflow which includes a lora, that I can use?

Not nf4

[livid gust]

@solid fern None Type errors might mean you have a node referencing a model that you have not downloaded yet. This can happen when you try out new workflows.

[merry ermine]

yes but runs like that on swarm ui with comfy as the backend

[merry ermine]

nvidia error bud, update your cuda and torch files

and install the latest nvidia driver for your GPU , that will fix that erroe as your torch.py and cude are bouncing with each other

[tough hornet]

On comfyui, is there a way to open a node's folder directly from the interface?

[merry ermine]

tap the open grid twice

[tough hornet]

I would like to know in which folder and locate this node

[true onyx]

I wanted to run comfy today, and I saw it update torch. since then I get the issue about conflicting tokenizer versions. When trying to reinstall tokenizers I got these errors. could someone take a look and help me out please?
https://privatebin.net/?1126aaf8e5077231#6NCt7gQv7jvGryNTk5SAJj4dEi2AtQBVJSegRAbh8kUe

[merry ermine]

the actual folder is called ip adapter plus

[merry ermine]

StableSwarmUI\dlbackend\comfy\ComfyUI\custom_nodes\ComfyUI_IPAdapter_plus im in swarm but after swarm path there the same

[tough hornet]

he can't find my .bin in this folder..

[merry ermine]

your using the wrong node dude

you need the plus node

I had that very same problem

even on install missing nodes it dont find it

even double tap on grid and search ipadaptor plus shows the wrong nodes for what you need

[tough hornet]

But.. when I download the workflows, it says that this is exactly the .bin file I need.

[merry ermine]

yes the file bin is right

your ipadaptor node is wrong in your screen shot

[tough hornet]

1505 000
This seemed weird to me, because it's a workflow I found in a tutorial that seems to work.. Maybe an update issue?

qhat is the right nodes?

[merry ermine]

get you some of this, but make sure you read the gumpff https://github.com/cubiq/ComfyUI_IPAdapter_plus

you shouls be able to port it straight to bin with cmd git clone

[merry ermine]

lol thats a very lame bit of hackery my dude. that sort of infest was being knocked around by script kiddies in the 90s bro

[tough hornet]

So if I understand correctly, I need to reinstall ipadapter?

[merry ermine]

and the node

[tough hornet]

okok thanks

[steep marlin]

For those having a hard time understanding Flux's base/max_shift values and how they actually interact with the calculations, I threw together a quick interactive Desmos calculator for it. Basically, if you stay within 1024^2 pixels, base shift will do absolutely nothing and it only applies when the width * height product is not equal to 1024^2. So for instance, if you picked resolutions like 768x768 or 1280x1280 or some other combo that the product is not equal to 1024^2.
https://www.desmos.com/calculator/c0jburw7z4

[fading badge]

C:\ComfyUI_windows_portable\ComfyUI\comfy\ldm\modules\attention.py:407: UserWarning: 1Torch was not compiled with flash attention. (Triggered internally at ..\aten\src\ATen\native\transformers\cuda\sdp_utils.cpp:455.)
out = torch.nn.functional.scaled_dot_product_attention(q, k, v, attn_mask=mask, dropout_p=0.0, is_causal=False)

does anybody know how to fix this?

[steep marlin]

No need, it's just a warning. It's a windows related issue if I recall correctly

I see it every single time I make anything in most current diffusers like comfy or even with the diffusers library

In theory, if it were being used, it might make things a hair faster or use a tiny bit less memory, but I think that's mostly if the code uses flashattention2

[fading badge]

bro my workflow 100x slower right now

i just tried start my comfyui with high vram mode

and i guess i did something wrong

even chatgpt cant solve it

just repeats itself

i just want to generate 3 pics and it took 1 hour to complete

[steep marlin]

Don't do that, just use normal. You're running out of vram and it's using your system memory and probably your pagefile

Which will make things like 10 to 100x slower

[fading badge]

it doesnt matter i did nor mal too

same results

[sly sundial]

I see the line moving up/down, but it still doesn't explain what it does, or how that affects the outcome of the image.

[steep marlin]

Basically, it's a convoluted pair of settings, but it behaves similar to how regular model shift works with sd3 or aura. The difference is that if you use non-1024^2 pixel counts, it can modulate the overall shift value(likely for better behavior in the model so it doesn't break down with small or big MP images like 0.5MP or 4MP). That final shift output of the graph is the only number that actually matters to the model, since it's the only value that gets fed into the transformer.

Much like the single shift value that sd3 uses

[long lodge]

question > how can you check were python is installed when using A1111/COmfyUI?

[livid gust]

If you're using ComfyUI Portable, python is pre-shipped in the python_embeded folder.

[flat dirge]

This is my work so far in visualising the flux shift factor. It's still a work in progress. Feedback welcome. https://gist.github.com/Geeknasty/1998050917cac45c8745a74d31d166ed

[steep marlin]

if you want a far easier way to visualize it, just use the graph sigmas node and a basic scheduler like so:

my little desmos graph was just showing the relationship between the base/max and the resolution and how it ultimately just spits out the final shift value that shifts this whole graph like this example

and how if you stay at 1024^2 pixels(or any combination that the product equals the same), base shift does nothing

oh and do note that comfyui will automatically pick this flow curve with simple, even though simple should just be a diagonal line. it's hardcoded into the code for the dev model

good tests with that link though, i like it

[flat dirge]

wish I knew about this node lol

[steep marlin]

it's a super lifesaver

[steep marlin]

https://github.com/Extraltodeus/sigmas_tools_and_the_golden_scheduler

it's in this pack

a while back, i was kind of doing the same thing you're doing where i was manually calculating and graphing sigma shit until i discovered this node

[merry ermine]

best node

I never run a workflow without it

[long lodge]

question > how can you check were python is installed when using A1111 ?

[round zephyr]

Does anyone have an idea why I'm getting different results in color reproduction in a Mac M2 Pro vs an NVIDIA A10 gpu? Mac M2 Pro results look mostly correct and NVIDIA A10 looks washed out. I'm running the same workflow on both.

[bronze kernel]

I have a node that installs just fine but does not appear in the UI after updating ComfyUI yesterday. Is there a common reason for nodes not to be visible? There are no errors in the log file and it does not show as a failed node.

[fading mural]

how is it useful?

im not sure what sigma foes

[steep marlin]

In a nutshell, sigmas determine how much noise to remove from the image at each step. It's a really complex topic that would take a while to explain. There are a ton of great YouTube videos on the topic of diffusion that do a good job explaining it all, so I'd suggest you check some out.

[cursive ridge]

I'm loving the newest comfy, so much easier for me to make workflows! Though I'm stuck on this, how/what do I connect to that clip?

[fading badge]

from checkpoint or lora and vs.

bro why is this stuck here so long , it must supposed to be little fast

[cursive ridge]

How? There's no out/clip on the loras (I'm using it to do a lora merge). Do I add a clip out onto one of them, or the merger?

[fading badge]

i need a ful workflow picture

[cursive ridge]

the above is it

[fading badge]

thats it ?

where is your checkpoint loader?

[cursive ridge]

I did a checkpoint merge before, and the workflow for that was nearl yas simple... But flux is a bit diferent\

[fading badge]

so flux dont need a checkpoint loader

i never used

[cursive ridge]

I'm just doing a merge, not making any images. Merging loras in this case. Or do I add them as checkpoints instead? (I tried that, didn't work)

[fading badge]

but i saw some workflow examples

maybe u can use it

from civit>ai

[cursive ridge]

I searched for flux lora merge workflows and didn't find anything

and using the checkpoint mergers, didn't seem to work

Maybe merging loras just doesn't work afterall?

I'll try adding t he flux checkpoint before the merge aspect...

[fading badge]

sorry i never tried before

i dont know how to solve your problem

[cursive ridge]

Think I'll load my merged mosels I made, to get my old workflow then chenge everything to loras. Might work 😄

[merry ermine]

it either wants T5 and Clip G or T5 and Clip L

can't remember which

but in Comfy you can just put node to load each clip and connect

but yeah as the other guy said if you have loras affecting the text encoding then the clip yellow cable comes out of that

[livid gust]

@round zephyr Macs do that, they color correct your screen by default. Check out your Display settings. You can disable it.

[steep marlin]

Yeah it's probably the difference between one setup using srgb color space and the other using a wide gamut profile like dci-p3

[ember token]

I love the switch I made so I don't have to type resolution over and over

[steep marlin]

There are some good nodes in some of the packs with common sdxl(1 megapixel range) resolutions as well to quickly pick from things like 1024x1024, 1344x768, 1152x896 and so on, and with a swap dimension button to switch between landscape or portrait

They usually have versions of the node for common sd1.5 resolutions as well that are in the 512x512 range

[spring ore]

guys comfyui keep trying to download hallo.pth but get stuck because it no longer exist online

how can i stop it from downloading it each time i try to enter

[round zephyr]

@livid gust @steep marlin thank you guys

[peak geyser]

Is there an custom node that automatically detects the face and applies an alpha mask to face area? I need an automated process that requires no action from my side.

something like that

[peak geyser]

thank you! this is what I meant

[fading mural]

any node that cuts out character and make it a transparent sticker

[peak geyser]

https://www.youtube.com/watch?v=ySoIptW2huI i found that when i was looking nodes for my workflow

[fading mural]

thanks!

[peak geyser]

Work like a charm 👍 Thank you again!

[hazy cliff]

Does anyone know where I can download demo workflows for Advanced editing features in Promax Model?
https://github.com/xinsir6/ControlNetPlus

[crystal gust]

Can someone help me with the error that i got when i try to queue any prompt on comfyUI patientx's version ?

[desert rapids]

Hmm I can't get any flux loras to work in comfy, getting a ton of errors and CMD converting them. Guess most of the ones on civitAI aren't comfyUI compatible?

[sly sundial]

They are, I've used a load. Have you updated to latest comfy?

[merry ermine]

When I click on the run_nvidea_gpu.bat file the press any key to continue screen appears but after I press a key nothing happens

[limber light]

What nodes would I use to clone face and voice from one video to another? I am trying Reactor for face and it works OK, and DeepFuze for voice and its not working at all... but Im just trying random nodes without a clue as to whats best.

[grand pike]

hi peoples. so i've been confused about embeddings/textual inversions for a while, do i put them in the positive or negative prompt? some times i see people put them in the positive and most embeddings dont tell me where to put them. any ideas?

[steep marlin]

Depends on what they are embeddings for. If they are a bunch of anatomy "fixing" ones for a model like sd1.5, they usually go in the negative prompt.

But it depends, your best bet is to look them up on places like civitai and read about them, making sure to click any "show more information" or click any "expand" buttons on the page to show all the descriptions

[hardy tree]

@storm folio not that it necessarily matters, just noticed that v0.0.9 was skipped? https://github.com/comfyanonymous/ComfyUI/releases

[storm folio]

it wasn't skipped, v0.1.0 was a major release v0.0.x are minor ones

v0.0.11 could have happened

[merry ermine]

None there

Nothing is popping up to begin with

[obtuse latch]

Hey lads, what's the easiest way to update torch to 2.3?

I have manager installed.

[steep marlin]

You'd do it from the embedded python directory using python.exe -m followed by the pip version of the command from pytorchs website where you pick the versions and whatnot

The -m makes sure it installs to the comfy virtual environment and not to a global install

[obtuse latch]

Thanks I just ended up using the nightly build

[steep marlin]

No problem, I take it that it installed correctly then? Like when you boot up comfy, it says the new version in the console?

And I forgot to bring up that 2.4.0 has some issues, but I think they're fixed in the nightly builds. But I see you went with the nightly, so you should be alright then

[desert rapids]

Is there no comfy standard for flux Lora nodes? I'm using two different ones and some work in one and not the other. Some loras I download works in neither of them

[obtuse latch]

Lads, for the API what's the proper image format to set?

def load_image(image_path):
    with open(image_path, "rb") as image_file:
        return image_file.read()

    if image_path:
        prompt["268"]["inputs"]["image"] = load_image(image_path)

I tried both dir and .read()

[steep marlin]

Are you using the launch flag when launching comfyui? If not, you won't be able to make api calls

I want to say it's just --api and probably --listen as well

[obtuse latch]

Yes this is way past launhing

[steep marlin]

Maybe check the script_examples then, might be some json type stuff

[obtuse latch]

way past the sript eamples 😄

[steep marlin]

Well check the functions used in the examples for pulling the images

[obtuse latch]

https://github.com/comfyanonymous/ComfyUI/tree/master/script_examples

[obtuse latch]

That doesn't set the API

[steep marlin]

did you check https://github.com/comfyanonymous/ComfyUI/blob/master/script_examples/websockets_api_example.py?

like im seeing

    data = {"filename": filename, "subfolder": subfolder, "type": folder_type}
    url_values = urllib.parse.urlencode(data)
    with urllib.request.urlopen("http://{}/view?{}".format(server_address, url_values)) as response:
        return response.read()```

and there's a get images function as well

[obtuse latch]

Thats returning image from the ws

that's a ws response,

[steep marlin]

what do you mean by "the proper image format to set" ?

[obtuse latch]

I'm asking about the actual workflow, adding the image.

[steep marlin]

like explain what you're trying to do

img2img or something?

[obtuse latch]

yes, i2i and update the workflow with an image.

[steep marlin]

are you using websocket mode where it doesn't save the image directly?

like in the websockets_api_example_ws_images.py example?

"#This is an example that uses the websockets api and the SaveImageWebsocket node to get images directly without
#them being saved to disk"

[obtuse latch]

This isn't saing images, this is setting the image,

I'll figure it out I think it's just the dir

[steep marlin]

so you want to take an already saved image from your drive and then do img2img on it?

yeah, you just need the correct folder path syntax and it probably needs to be converted to PIL or a tensor

[obtuse latch]

Yeah that's wy I'm asking here what the format is.

[steep marlin]

actually, don't even think it needs to be converted, you just need to set the variables in the node for the json portion of the script

[obtuse latch]

It's a dir, the serer wasn't pulling the image initally.

[steep marlin]

["inputs"]["image"] = img basically

[obtuse latch]

it would be nie if we had the ability to stream image bytes directly

[steep marlin]

if youre trying to load an already saved image, you'd do it like i just said. it's the equivalent of placing a load image node and then putting a picture into it

[obtuse latch]

yeah the isue is I hae to hak my way:

    if image_path:
        prompt["268"]["inputs"]["image"] = load_image(image_path)
    else:
        image_directory = os.path.join("/mnt", "dataext", "ML", "Data", "Image", "Processed", "Creatures", "preprocess")
        random_image = get_random_image_path(image_directory)
        image_directory = os.path.join("Y:", "ML", "Data", "Image", "Processed", "Creatures", "preprocess", random_image)
        prompt["268"]["inputs"]["image"] = (image_directory)```

[steep marlin]

this is why i asked you to explain your actual workflow. i don't know what your actual workflow is

[obtuse latch]

all images in workflow are the same,

The issue is I was hoping there would be a way to strem in byte data instead of img path

[steep marlin]

is it a workflow like this?

[obtuse latch]

With a load image, yes.

[steep marlin]

then why do you need to stream byte data?

[obtuse latch]

Beause the images are not on the prompting mahine.

I needed to make a network drie and the hak I made up there to set the image dir

[steep marlin]

are they on your own personal local network?

[obtuse latch]

They are now.

[steep marlin]

just make a simple ftp server

you should be able to use web addresses in the filepath locations

[obtuse latch]

Thaat's not the best solution,

The best would be just to stream in a img.

I'll see if I ccan do it and make a PR

[steep marlin]

then set the image folder on the other computer to be a network drive and make a reference on your comfy PC. it will show up as whatever drive letter you want like z:

and behaves just like if it were a drive on your pc

[obtuse latch]

That's what I did.

[steep marlin]

that's the correct way then...

[obtuse latch]

    if image_path:
        prompt["268"]["inputs"]["image"] = load_image(image_path)
    else:
        image_directory = os.path.join("/mnt", "dataext", "ML", "Data", "Image", "Processed", "Creatures", "preprocess")
        random_image = get_random_image_path(image_directory)
        image_directory = os.path.join("Y:", "ML", "Data", "Image", "Processed", "Creatures", "preprocess", random_image)
        prompt["268"]["inputs"]["image"] = (image_directory)

[obtuse latch]

It's not the best. 😄

[steep marlin]

no, in your case, it is and is the least hassle vs making some ftp server or something

you have authority over both pcs

so you just workgroup network that shit together

[obtuse latch]

The best way would be to stream in byte data for an image.

[steep marlin]

it's 2024, a 1 megabyte image will load in like 0.00001 seconds on a local network

[obtuse latch]

It's not about that, its about haing the data easily read, in prodution I don't hae a network drive.

[steep marlin]

okay, this rolls back on what i asked earlier about what you're trying to do

[obtuse latch]

I'm going to try and bug the devs to make a PR to stream in byte data for an image 😄

[steep marlin]

you'd need to import another web library probably and then use some other pathing for the directories

should only be like two lines of code really

[obtuse latch]

Nodes need to somehow accept a stream, or a temp dir that stores an image that a node accccesses eacch prompt

[steep marlin]

there is a temp directory though

[obtuse latch]

I can out of the box stream data to it using the API?

[steep marlin]

what do you mean when you keep saying stream? are you trying to run some kind of webcam setup or livestream thing? these are tiny images... they don't need to be loaded in small tiny chunks with 2024 network speeds

if they are already saved to something liek a .png, just load it

you aren't and shouldn't be waiting on anything. it's not like using LLMs where you have to stream a resonse because it's moving at 5 tokens per second or some shit

[obtuse latch]

I don't even know what you're talkng about mate.

😄

This has nothing to do with requests, I'm not waiting on anything.

I can't "load" an image on a production server that doesn't have the image.

I would have to upload / stream the image then reference it (the path) in the current api.

[steep marlin]

okay, for the third time: this rolls back on what i asked earlier about what you're trying to do

[obtuse latch]

The ideal situation would just be able to pass a byte streaam to the API

[obtuse latch]

I don't think you fully understand what I'm trying to do.

I've done what I needed to do, I was simply saying it would be a nicce feature to stream in images, instead of referenccing the img path.

[steep marlin]

i do, you're just doing an extremely bad job of explaining it and aren't following the simplest of requests: explain the exact workflow and the variables of the situation

[obtuse latch]

As for an API you don't have local data on external servers.

[obtuse latch]

I've explained it multiple times,

and even showed the code.

[steep marlin]

no you haven't, you've been extremely vague and are only talking about snippets here and there

[obtuse latch]

I've literally explained multiple times what I'm doing.

[obtuse latch]

This is exactly what I'm doing and did.

[steep marlin]

so the jist is that you have some server you want to be able to rip data from, but you don't have control over anything on that server and you don't know what or where the images on that server will be

[obtuse latch]

That is usually the case for API's, yes.

You don't normally have access to the directory on a external API / system, so it would be good to be able to stream in a image to the api.

Not reference a path.

[steep marlin]

no no no, you're misunderstanding what API mode for comfy is... API mode for comfy is you set up a comfy server, that you have access to, then from remote connections and/or apps, you can access the comfy script and run it on that machine, and then it sends you the completed image(s) back

[obtuse latch]

Mate I don't misunderstand it, I'm a SE of 15 years 😄 I know what it does.

I'm saying it would be good to HAVE this feature 😄

When you set up a "

Server" it's usually on an external system, hence the reason why you have an api. In this case it would be good for the devs to allow (somehow) streaming of images for nodes.

So I don't have to connect a external network / file system:

    if image_path:
        prompt["268"]["inputs"]["image"] = load_image(image_path)
    else:
        image_directory = os.path.join("/mnt", "dataext", "ML", "Data", "Image", "Processed", "Creatures", "preprocess")
        random_image = get_random_image_path(image_directory)
        image_directory = os.path.join("Y:", "ML", "Data", "Image", "Processed", "Creatures", "preprocess", random_image)
        prompt["268"]["inputs"]["image"] = (image_directory)

[steep marlin]

what you're implying is that you want to run the API on the PC you're currently using... to pull wildcard directories and/or images from another server/network that you don't have ownership/control over, and then do stuff with them? yeah, you don't need api mode to do that... you can just make a simple node for comfy that can scan for directories and images, and run it in a regular comfy workflow. you're just basically talking about wildcarding...

you keep using this stream term and in the ML world, that implies something actively being generated or ongoing. like with LLMs, sometimes they are slow and you don't want to wait minutes for some 2000 token response, so you have the API use a stream mode that sends the tokens as they generate, since it's still actively working on the job

[obtuse latch]

stream means the process of loading image data directly into memory as a sequence of bytes (a binary stream) rather than a "image" like a png. This is universal in development.

Strem in the sense of a ws (what you're talking about) and that's just a stream of tokens.

[steep marlin]

yes, open image does the same thing...

[obtuse latch]

Either being written to a response or to cli

[vocal vortex]

did comfy remove the load via folder option? I sorted what was what into folders and there was something in the settings that would show the folders when loading models/loras/etc...

[obtuse latch]

No a stream is a stream in memory, an image is a encoded image (encoded byte array)

[vocal vortex]

instead of everything just thrown in the fly out

[steep marlin]

no, most of the time, they get decoded with utf-8 and you have a wide variety of formats you can use

you can also encode/decode image data with base64 as well

[obtuse latch]

It doesn't matter, a stream and an image are different.

The point is what I was talking about is it would be nice to stream in a image as opposed to a path, especcially for an API

[steep marlin]

yeah, you easily can with pil->base64encode->base64decode

https://www.c-sharpcorner.com/article/converting-image-to-base64-in-python/ here's a guide to save me some typing

[obtuse latch]

I already tried that it doesn't work:

def load_image(image_path):
    with open(image_path, "rb") as image_file:
        return base64.b64encode(image_file.read()).decode('utf-8')

[steep marlin]

it will make a long string variable like:
"WWKimAq4pnVA9TSt8KytrkRIMX4RVyZlw5f/B6UHbLU0f8wqLE7Xwh3i1XQHgiHMoMB+ucmnARTvPshHPa3MWRZz7P91LU7+wdVGSu5BY3Et5xlxmqKG9yGCChhjNU24qmBBLmiZ8V1fDebjt9QvLEO4cQ9G2k+UQvjJTNWvhcY=", which then needs to be decoded

yeah, you're doing something wrong then

[obtuse latch]

Well you're looking at the code.

[steep marlin]

yeah i just ran an experiment and it works fine

image = open('test.png', 'rb')
image_read = image.read()
image_64_encode = base64.b64encode(image_read)
image_64_decode = base64.b64decode(image_64_encode) 
image_result = open('test_decoded.png', 'wb')
image_result.write(image_64_decode)```

don't actually need the utf-8 for it it seems

and i also had a step to save it as a .txt file, then loaded the txt and decoded it still encoded and it worked fine that way as well

still worked fine

[obtuse latch]

Where are youo adding it to your workfile prompt?

[steep marlin]

i'm not, i made a python example to test it first. but i'd assume comfyui definitely already has base64 loaded

[obtuse latch]

I don't understand... Nowhere do I need to encode / decode, I mentioned I tried to pass the base64 string to comfy and it didn't work.

I know how to encode an image.

[steep marlin]

oh ffs... why were you even talking about it then?

[obtuse latch]

Lol, because I tried to pass the stream to compfy and it doesn't work, it only takes a directory to the image.

When you mentioned comfy takes base64

[steep marlin]

i said it might

i brought up PIL/tensor/base64/etc

[obtuse latch]

Ah okay, well either way it doesn't seem to.

Would be nice though

[steep marlin]

and said it might need to be in some format like that

[obtuse latch]

side note **Also to anyone wondering, I updated to comfy nightly torch 2.5 and I'm getting around a 17% speed increase. **

[steep marlin]

just make what you need? i mean you said you're a SE of 15 years and all man... i make a lot of random nodes for things i need at the moment. it's python, shit's basically like playing with duplo blocks

and there are a ton of libraries out there

that simplify things like you're wanting. like openAI (commonly used in LLMs, but also handles image stuff as well), might already have some function you're looking for

[obtuse latch]

😄

[flat dirge]

A few options, MTB ImageFromURL send it url it loads the image. Impact Pack ImageReceiver, send it base64-encoded string it It decodes, transposes, and converts to images

[obtuse latch]

It's not so much the image loading / uploading it's the node format accepting it.

The api only accepts json.

So im at the mercy of what is loaded through that.

If a node can GET external uri's then that's one thing, but it's still not ideal.

ffs: https://www.runcomfy.com/comfyui-nodes/comfyui-easyapi-nodes/Base64ToImage

This looks like it's the solution...

[steep marlin]

Rule number one of engineering: research what's already out there. I figured you had already looked for nodes and didn't find any that people had already made. That's why I said to just make it yourself. I'll bet they are using a handful of similar functions like I brought up and if it can do urls, probably includes import requests and some bytesio function

[remote sequoia]

I don’t understand the why some people use comfy UI , Flux, automatic111
What’s the benefit of each?
I would like to use stable diffusion to create an influencer. Also can I just take any photos from instagram of someone who already exists? If not what’s the best way to go about using a face that already exists?

[merry ermine]

the three "good" workflows are comfy, diffusers or raw pytorch in my opinion

swarm as well

some people really really want an A1111-style interface, which is more of a personal preference

the projects like SD Next and Forge are essentially "how much modern tech can we cram into A1111-style interface", for those people

there's also Invoke which is more like the Apple of stable diffusion (opinionated/curated, slower to adopt new tech etc)

you have to take what you read about the different interfaces with a grain of salt because there are various conflicts between the GUI devs for some reason

[steep marlin]

ComfyUI is the most flexible. You can do all sorts of wild things in it by using a form of visual node based "coding"(extremely similar to blueprint coding in unreal engine). You don't actually have to code or anything because the nodes take care of that, but essentially, you're doing a very high level form of coding. At any point in the graph, you can pretty much modify anything you want. You can make a simple diffuser setup or a complex 30 pass setup.

Gradio based apps like a1111(mostly replaced by forge now since forge tries to stay up to date with new models) are good for entry level users that just want a simple workflow with some basics like control nets and upscaling.

Under their hoods, they are mostly all doing the exact same thing. Comfyui just happens to be the best for experimenting and once you get the hang of it, for even simple workflows, you'll learn to love it

[mystic plaza]

where should these 3 safetensors go ?

[untold haven]

Hi, I don't understand how install ipadapter on comfyui someone could help me?

[livid gust]

You install the IP Adapter the same way you install other nodes. Use the Manager Button, then click Install Custom Nodes. In the search field type ipadapter as one word. Choose "ComfyUI_IPAdapter_plus." You can visit the Github, by clicking on that name. I highly reccommend you do that, and scroll down to watch the video tutorial.

[untold haven]

Need i even ComfyUI Essentials by the same author cubiq?

[livid gust]

You only need the nodes your current workflow requires at any given time. The image shows what I run on my 8GB card. A few, I've never used, however.

[untold haven]

Ok the last question that i don't understand, how model need i to install https://github.com/cubiq/ComfyUI_IPAdapter_plus here in installation?

I just wanna make some anime image

[livid gust]

Check out the video on the Github page. Mateo shows how to leverage the IP Adapter for several scenarios. Style transfer from one image to a prompted image is explained.

[untold haven]

He don't explain in the video

[desert rapids]

I have yeah, running portable. What node are you using?

[sly sundial]

They work just the same as any other Lora. Do you have a standard Flux model?

Maybe you want to share your workflow in case you've connected it incorrectly?

[merry ermine]

You need IP Adapter Model Loader and Load CLIP Vision nodes to load the necessary things. Then you need one IP adapter node and one ClipVision Enhancer node per image. if you use embeds then you need IP adatper encoder and IP adapter combine embeds nodes, feeding into the IP adapter embeds node specifically. I would recommend using K+mean(V) w/ C penalty at first, and remember you don't just have to average embeds as some guides say, you can also get cool results by concat/adding embeds or combining in other ways. For more sophisticated embedding combination workflows do it manually and then input the result to the IP adapter embeds node. Comfyui -> models -> clip_vision for the clip vision node and Comfyui -> models -> ip adapter for the file locations, creating files if they are not there already. However, as with any other model, make sure you open up extra_model_paths.yaml and make sure that the data here matches up with your actual Comfy UI installation directory structure.

[desert rapids]

This setup is working for some loras. The load flux lora node doesn't work with anything really so it's not used.

But this lora safetensor file: https://huggingface.co/nerijs/dark-fantasy-illustration-flux

Doesn't even show up when put in the directory. Maybe I'm putting it in the wrong directory?

[sly sundial]

In comfyui/models/loras?

[untold haven]

ok but what Kolors does? and bigG clip vision encoder? Need i?

[merry ermine]

Kolors is a certain model, so if you are not using the Kolors model

[untold haven]

and bigG clip vision encoder?

[merry ermine]

you need bigG for some and vit-H for others

[untold haven]

Sorry i don't understand

[merry ermine]

some IP adapter models are designed to work with the bigG clip vison encoder
and other IP adapter models are designed to work with the vit-H clip vision encoder

[untold haven]

clear, thank you

[sly sundial]

Works for me. This is with and without the lora, same seed/prompt

[desert rapids]

Weird, which directory are you putting the safetensor file?

[sly sundial]

#🧣｜comfy-ui message

[desert rapids]

Thanks, for some reason I put it in that other node loader (xlabs flux lora) directory. I thought the regular Lora dir was just for stabled diffusion.

Appreciate the help Galaxy!

[sly sundial]

You know you're using the wrong model too?

[desert rapids]

Yeah I changed it to dev

[sly sundial]

Does it work now?

[desert rapids]

working fine now!

[sly sundial]

Awesome! 🙂

[desert rapids]

pardon the noobiness

[sly sundial]

We all start somewhere 🙂

[desert rapids]

If only it could generate a fantasy elf:

😅

Yeah I come from Auto1111, comfy is a rough transition. But I'm used to Blender 3D nodes so I think this node system will be pretty awesome to work with

[wispy kelp]

Comy is the way. Even if in the beginning it's not comfy at all.

[obtuse latch]

Anyone know why on nightly when I insstall a node it's just saying "restart required" even after a restart?

[wispy kelp]

Well that's the dreaded "something nebulous is wrong and good luck figuring it out my man, reinstalling won't fix it, restarting won't fix it, it's just not happening. the programmer was on something when he wrote me. Or I just despise you. I know you just want me to work and you hate all this technical mumbo jumbo idiotic garbage and just want your artistic vision realised. But it's not happening so go jump off a bridge." error.

Some nodes just won;t work. They hate you.

Like Reactor nodes.

[obtuse latch]

I removed the custom nodes and tried one at a time and t worked. thx

question for ya'll, the first image is a generation off the second one, you can see it looks like shit, it's got that AI blur to it, any suggestions on fixing it?

[wispy kelp]

Yes custom nodes can be hazardous.

That clothing node pack was a disaster for me

[wispy kelp]

Well sometimes adding "crisp, sharp" helps!

[obtuse latch]

This doesn't seem like it's a prompt issue, it's a setting, even getting banding in the image.

[steep marlin]

what model and are you using loras?

if that kind of compression banding is in the dataset, it's going to be in the model/lora.

[obtuse latch]

lads what am I doing wrong here with my refiner?

[steep marlin]

endstep is greater than steps

if you want 80 steps total, ksampler1: steps=80, start step 0, endstep 60. ksampler2: steps 80, start step 60, end step 80

[obtuse latch]

I just want to get it looking good, not specific to 80 steps,

What woud you reccomend?

[steep marlin]

is it an sdxl model?

[obtuse latch]

yyep

[steep marlin]

80% is when you switch to refiner

so 40 out of 50 steps, 20 out of 25 steps, 80 out of 100 steps, and so on

you add noise on the first ksampler and make sure it is set to return with noise

you disable add noise on the second sampler and disable return with leftover noise

[obtuse latch]

so this is 100 ssteps and 80 end

without return with

and this 😄

[steep marlin]

the noise schedules are different

they have to use the same

[obtuse latch]

okay return without noise fixes is

[steep marlin]

karras and karras, normal and normal, etc etc

try what i just said and reenable return with noise on the first ksampler

[obtuse latch]

Thank you so much!

See anything else wrong?

[steep marlin]

looks good to me

[obtuse latch]

Killer, thank you very muh!

[steep marlin]

tested for sanity as well since i havent used a refiner workflow in a while

that's with realvisXL 4.0 and the sdxl refiner for the second pass

[obtuse latch]

going to try out realvis now

[steep marlin]

it's a great model, replaced juggernaut for me ages ago

even though it's kind of marketed as a good people model, it's excellent at all sorts of other random shit as well

[wispy kelp]

I always just check the newest models the last week or so and download those. I am not sure if this is the right approach but I think the latest checkpoints are always the best coz they had the most time in the owen

Natvis is my fav now

[steep marlin]

I haven't been using sdxl much lately, so I've trimmed my models down to realvisxl 4.0, sdxl base and sdxl refiner lol

[obtuse latch]

dumb question but will 2 refiners make it even better?

[steep marlin]

no, probably not

and with a lot of the more recent finetune models, like realvis that i brought up, refiners usually aren't needed

they were mostly handy when sdxl launched and before a bunch of quality finetunes came out

[lone cape]

Hi, I trying to use ic-light workflows, but all wf i trying getting error with ksampler:

Given groups=1, weight of size [320, 4, 3, 3], expected input[2, 8, 64, 64] to have 4 channels, but got 8 channels instead```

Somebody have very simple wf for testing?

[steep marlin]

As far as I know, ic light is an sd1.5 model, make sure you're using one

[untold haven]

@merry ermine sorry for annoying you, i need to ask you something about ipdapter, could you help me?

[merry ermine]

okay yeah

BTW if you join L2 discord, the guy who made the IP adapter advanced node it there (Cubiq)

[livid gust]

@steep marlin Which RealVis do you use? I've seen Lightning and Turbo models, but I'm not sure what the difference is between them? Do they require alternate node work?

[untold haven]

It's just a stupid question that i don't understand. Should i put CLIP-ViT-H-14-laion2B-s32B-b79K
CLIP-ViT-bigG-14-laion2B-39B-b160k both off this two models in the clip_vision folder?

And how should i rename that?

What is L2 discord?

[obtuse latch]

Anyone have a good inpainting workflow?

[lone cape]

I tried to use with SD1.5 Photon_1

[merry ermine]

putting both in the clip vision folder is ok

you can choose the name

L2 discord is another discord server

[covert bough]

it would really be nice if - when a workflow is loaded AND comfy detects missing nodes it just went ahead and installed the missing nodes

[cursive ridge]

That virus a bit ago probably slows them down from adding that feature 😦

Is it just me, or did comfy suddenly get faster (with flux dev) in the past couple of days?

[covert bough]

just you

[wispy kelp]

Noodletown has many surprises.

[steep marlin]

I use the regular one. And lightning versions usually just do stuff in 4-8 steps, but require low cfgs in the 1-2 range usually.

[steep marlin]

Then I'm not sure, I haven't messed with it in a while. Try checking its GitHub repo for issues

[obtuse latch]

Anyone know what module / node this is? Getting it after I started using the nightly build:

errorr occurred when executing AV_ControlNetPreprocessor:

[Errno 2] No such file or directory: 'C:\\Documents\\ComfyUI_windows_portable_nightly_pytorch\\ComfyUI\\custom_nodes\\comfyui_controlnet_aux\\ckpts\\lllyasviel\\Annotators\\.cache\\huggingface\\download\\dpt_hybrid-midas-501f0c75.pt.501f0c75b3bca7daec6b3682c5054c09b366765aef6fa3a09d03a5cb4b230853.incomplete'

Okay so I figured out some of the issue, I dn't ave the depth, midas, anyne know how I would redownload it?

I have it installed, it's just not downloading the checkpoint:

I got it to work by copying over a old comfy custom node, but anyone have any ideas why it wouldn

t be updaing on nightly?

[mellow lily]

any way to make a hotkey for "open in mask editor?" I do that hundreds of times per day

[steep marlin]

Probably just look through the code and see where the other hotkeys are stored, then make a new one and have it call the function that the right-click menu would have called. Or at least that's what I'm assuming. But keep in mind, it will get overwritten every time you update comfy unless he adds it officially.

It might get weird though if you press the hotkey without having an image node selected, so you might need a little logic to prevent bugs

[onyx acorn]

Sorry, Wdym @obtuse latch

[mystic plaza]

so whats the cause for this error image size or the video size is off ?

[mystic plaza]

using liveprotrait but getting this kind of results where the motion is not captured at all or is there a need for video resize and stabilize it for the bot to understand ? @steep marlin @dry rock

[steep marlin]

Sorry but I have next to no experience with any of the video stuff outside of basic knowledge in how they work

[tough hornet]

Hello, i want to load checkpoint on my workflow but this error appear when i queue front, do u know why ?

[fading badge]

guys i guess my upscale model doesnt work correctly

[covert bough]

it works best if you have a headshot for your video and the person is just moving their face, not their body. it's trying to take her dancing and map it all to the face - that's why it's called live portrait - it's just affecting the face

[obtuse latch]

Anyone know the module name that displays the images at the bottom of the screen and you can change the amount you see and the size?

[steep marlin]

I think that's rgthree, but I'd have to double check

[steep marlin]

Correction: https://github.com/pythongosssss/ComfyUI-Custom-Scripts

[mystic plaza]

so it only understands headshots

[covert bough]

yes

[mystic plaza]

so if i want to record a head do i need my phone camera to be very stable

[covert bough]

yes - you could use a web cam, or you can get a tripod for cellphones

[mystic plaza]

and what size should the video be in as in aspect ratio

[covert bough]

if you're going to record with a cell phone, you'll be using the cell phone's video aspect ratio.

[mystic plaza]

cause i would need to match the scale of the video for a smooth capture

[covert bough]

but you could record on a green screen, use davinci or runway to remove the green screen so you just had the talking head on transparent - then set an AR in davinci, put a new background in...

[mystic plaza]

this would be possible but cant do it right now so a white background would better

[covert bough]

you don't have to be that picky, you just need to use a live headshot and it'll map that to the image

[mystic plaza]

also i thought liveprotrait could capture body movements as well but i was wrong

[covert bough]

well - it's like face swap - it's only working on the face...

[mystic plaza]

hun ?

[merry ermine]

its hard to concat two images, give it time 🙃

[verbal sky]

FYI For those wanting a quick direct-link inventory for Flux Loras - https://docs.google.com/spreadsheets/d/1543rZ6hqXxtPwa2PufNVMhQzSxvMY55DMhQTH81P8iM/edit?usp=sharing

[sharp raptor]

my workflow worked last month, update comfy one month later after vacation and ksampler that does Ic-light gives this error

any ideas?

Why is the ksampler not working anymore? Here I reconstructed the IC-Light part only

[pearl marsh]

where would one find preprocessors

[candid tangle]

Seems like you need to connect the latent output from the IC Light Conditioning node to the ksampler

[candid tangle]

Try Comfy Manager, Model Manager if you haven't already

[sharp raptor]

that is not it

latent to the ksampler uses the light mask for lighting

[pearl marsh]

ty!

[sharp raptor]

updated IC-Light even though there was no new updates and that fixed it

[steep marlin]

Try loading their example and see if it works

And make sure you have the right models selected in the right spots. Someone else had this issue the other day but I don't remember how or if they resolved it

[sharp raptor]

It fixed after update

[obtuse latch]

Thank you!

[mystic plaza]

i keep getting this error and have also refreshed many times but still not working

[surreal whale]

is that the lora loader where you click and it expands for more loras? if so expand it add more lora slots and unselect the ones hiding there

oh nm i didnt look at the other picture

[errant karma]

I am working with Confyui i have loaded an image “zdepth”, i have loaded it in the Controlnet Model node “control_v11f1p_sd15_depth.pth, when the image “zdepth” has a resolution of 512 px width and 512 px height i get results as i expected, that is, good. When i change to another zdepth image with a resolution of 620 px width and 368 px height the result is that it does not take the reference image for creating images. Can someone help with this problem?

[willow glen]

Can someone walk me through how to use the Tiled VAE for ComfyUI? I want to use it to save VRAM, not to upscale an image or whatever. Do not be afraid to talk to me like I'm an idiot, I will not take offense lol. The "Tiled VAE Encode/Decode" are from https://github.com/shiimizu/ComfyUI-TiledDiffusion, while the VAE Decode/Encode (Tiled) are the defaults. I downloaded the ones from the github as I didn't know about the defaults 😂 I'm gonna assume I use those over the defaults.

I also have no idea how to calcuate tile size. Yes, I am clueless, but I'm also stubborn, so here I am. Video guides are all about using this to upscale so they haven't been too helpful.

[flat dirge]

Just use it wherever you were previously using vae decode. The smaller the tile size. The less vram it uses, but it will take more time. So pick the biggest tile size you can run for best performance.

[willow glen]

How do I determine that tile size? There something I can do or I just keep trying until I figure that out?

I don't need to use the "encode" at all then?

[flat dirge]

Tile encode if needed. It's for encoding humongous images. How often are you doing img2img on massive img size? Usually I do all details and the final step is upscale so I only use tiled decode at the end.

[willow glen]

Right now i'm not doing that at all, I just am trying to get comfyui to work. Prior to this, it would crash my PC whenever I went to generate an image (I have enough VRAM, I have a 16 GB 6800XT and 32 GB of RAM), but I was using the wrong VAE. I'm using the correct fixed sdxl.vae now and I'm trying it again, but also with the Tiled VAE, as that saves VRAM, it certainly works in Auto1111 so I wanted to try it here too.

So if I'm not encoding humongoous images, I presume I don't need tile encoding

[flat dirge]

16 gb vram can easily do 1024x1024 decode so at least make tile size that

[willow glen]

Oh, hell yeah. This is great. Thank you. I'm getting images in 20 seconds on ComfyUI, but it takes 45-50 in Auto1111. Don't ask me why, I have all the same settings that I know of.

Are there any commandline args I should be using to boost performance even more? I only have the autolaunch command in my ComfyUI-Zluda start.bat right now.

[flat talon]

ComfyUI will now automatically do Tiled Vae when your latent size is large enough.

[willow glen]

Should I be using a Ksampler SDXL (Eff) or a regular one? I know the SDXL Ksampler has "sdxl_tuple" but I got no idea what that is or what it does. All of my models are Pony/SDXL, so should I be using the Ksampler SDXL?

[willow glen]

Alright, so I've got comfyui to work perfectly fine but decided to try out these Efficiency Loaders I've seen. However, on switching to these, I get... well, that. What am I missing that is doing this?

[distant lagoon]

that's amazing abstract art there

this is what happens with too much pony

[willow glen]

I'm just tryna make it work lol. Doesn't matter what checkpoint I use, the result is the same here.

Here's my previous workflow, the one without the Efficiency loader/ksampler. I'm not sure what the difference is, but I don't get abstract art with this one, though I would like to use the Eff. loader/ksampler.

[flat dirge]

Here you want to change setting in Eff. Loader. base_clip_skip to -2

[willow glen]

Yeah I did that, didn't do nuttin'

[willow glen]

You're a real one.

[crystal venture]

help T_T forever loading

[willow glen]

I have no idea what happened, but I went from taking 20-25 seconds to generate a single image to over 2 minutes. I have changed no settings what so ever, nothing. :<

[mystic plaza]

is there a way to fix this bro @dry rock @covert bough

[covert bough]

stick the lora it wants where it can find it

[mystic plaza]

i have it in the lora folder only but it still cant find it

[covert bough]

i don't use that node, so i'm not sure what the problem is other than these possible things - the filename is wrong, it's not in the folder that the node expects it to be in, it's corrupt. who wrote the lora stacker node you're trying to use? maybe there's a limit to how many it can list

[mystic plaza]

i am following this tutorial https://www.youtube.com/watch?v=-5ifRNlEaec

[merry ermine]

would recommend sticking to default nodes as much as possible

for something like a lora or control net loader, custom is not needed

[covert bough]

i'm not familar with that channel. i do know, however, that you probably will need to contact the guy that wrote it

[merry ermine]

if this is the lora stacker from efficiency nodes pack, it might only work with the ksampler from that pack

also are you using the original, archived, efficiency nodes, or the forked version that has ongoing support?

the original efficiency nodes got archived in january, see here https://github.com/LucianoCirino/efficiency-nodes-comfyui

[mystic plaza]

i am using the workflow which he had created with the lora stacker and alll

[merry ermine]

would recommend re-making without the efficiency nodes

i.e. for each efficiency node, replace with the base node versions

and see if that fixes it

[mystic plaza]

so make it from scratch is what you are recommending right

[merry ermine]

yeah that's what I'm recommending

there is a decent chance it would fix your issue

and the base nodes are a significant upgrade over the efficiency nodes anyway

for example the base Lora loading method lets you choose clip strength, and the efficiency node doesn't. Also the base Ksampler custom advanced lets you take a guider node and the efficiency node doesn't

[dry rock]

you don't have 1.5\Floweria_yiu_v10.safetensors, install it or remove it from your graph.

[mystic plaza]

so i shall just follow the tutorial and try it again i guess

[merry ermine]

isn't that the opposite of what I am saying?
since the tutorial uses efficiency nodes

anyway you need to do the step that Aryetis said before anything else

[mystic plaza]

how to remove from the graph now like delete it

[merry ermine]

ye

just delete

[willow glen]

Still getting nuttin'

What did I leave out?

[mystic plaza]

and it worked was a simple fix and i was a dummy

[merry ermine]

nice

[mystic plaza]

and now its throwing this error

[merry ermine]

did you load the model

[mystic plaza]

yes the model is loaded and there

[mystic plaza]

see its right here

[merry ermine]

but there is a second step

you have to click in the loader node

and set the path

[mystic plaza]

meaning ?

[merry ermine]

in nodes that load things

they will have a box

with the name of the thing you are loading

you have to click that box and make sure the path goes to where the file actually is on your system

[mystic plaza]

so i have to click on this box and then ?

[merry ermine]

this node doesn't seem to have a loader
did you do this step:
*Download model files from BRIA Background Removal v1.4 or BaiduNetdisk to ComfyUI/models/rmbg/RMBG-1.4 folder. This model can be used for non-commercial purposes.

from the layerstyle github

[mystic plaza]

i downloaded it from the manager

[merry ermine]

yeah you have to do that step then

*Download model files from BRIA Background Removal v1.4 or BaiduNetdisk to ComfyUI/models/rmbg/RMBG-1.4 folder. This model can be used for non-commercial purposes.

[mystic plaza]

so in this what do i need to download now @merry ermine

[merry ermine]

model.safetensors

[mystic plaza]

ok

[merry ermine]

.pth and .bin can work, but .safetensors is most safe

[mystic plaza]

i have added the model there but its still throwing a error

[merry ermine]

what does the error say this time

[indigo cosmos]

so i was told to ask a question in this channel - can anyone help me with this problem #📝｜prompting-help message

or maybe to just give me a work flow that works for img2video that can work on my 4006ti16gb; 4x16GB RAM?

[mystic plaza]

@merry ermine

[merry ermine]

maybe its looking for the .pth file in particular

so you could try putting the .pth file

instead of .safetensors

[mystic plaza]

its still throwing an error

[merry ermine]

would recommend trying a different set of nodes then

[indigo cosmos]

can you maybe look at my problem also?

#📝｜prompting-help message

[ashen valley]

hey : D

i am new to comfy

is there a way on how i can check the triggerwords for my loras?

in automatic you can just click the lora and it shows the version and the triggerwords + preview image

[flat dirge]

There's many dozens of custom nodes that add some form of lora trigger word fetching. The one I suggest is pythongoss custom scripts. https://github.com/pythongosssss/ComfyUI-Custom-Scripts#checkpointloraembedding-info

[flat dirge]

You still have clip skip at -1 there. You want to use -2 with all Pony models. Also you can change it to use the VAE that's included with the model. I usually do that instead of loading a separate one.

[ashen valley]

kk thx

🫂

[thin flame]

i am getting trouble downloading advanced controlnet, can anyone lend me a hand pretty please?

this node is kicking my ase, what do i need to do?

[ivory orbit]

Hello 👋
Has anyone ever used IPadapter with SDXL?

[willow glen]

Thank you very much. Works just fine now. :]

[pearl marsh]

"Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver " how do i fix this on rocm

[livid gust]

@thin flame NoneType errors generally mean you're missing an input, such as a model, an image or video. Click on the word "undefined" to see if you have any controlnets. If not, you'll have to manually download them and place them in the Models/controlnets folder.

@ivory orbit You might be able to, but the problem with the IPAdapter, in general, is it only accepts square images for input. You can supply any size, but internally, they will be converted to 512x512, often badly scaled with no consideration for the composition. Using SDXL, you might want to start with a 1024x1024 input image.

[ashen valley]

hey

how do i fix this?

my brain is a bit exhausted rn

i tried installing onyx

but i still get the same error

[forest birch]

Does anyone have a similar workflow as nerdyrodent has on his video to auto scan images and auto prompt images to use with training loras?

[forest birch]

And is there a "img 2 img" workflow to effectively control light with a gradient image for flux?

[mystic plaza]

what does this mean now ?

[hoary nest]

Is there a node in ComfyUI that allows to expand a mask in a specific direction, such as downward or to the left?

[mystic plaza]

hmm now its throwing this error like its installed properly and everything but its still missing for some reason

[ashen valley]

i still cant work out how to fix

i tried googling but im not getting any smarter from it

[indigo cosmos]

i always use chat gpt

sometimes it helps

[ashen valley]

i just updated the comfyui

[ashen valley]

it got worse

[indigo cosmos]

copy paste it

and into chatgpt

[ashen valley]

ok

[indigo cosmos]

after his first response just write i do not undertstand pls go into details - and he will segment everything

[ashen valley]

ok thx

[livid gust]

@hoary nest Check out Ryan's masks and particle systems. Optical Flow might get you a direction. You can search for it in the manager, or pull it down from Github.
https://github.com/ryanontheinside/ComfyUI_RyanOnTheInside

[hoary nest]

Thanks, will try this.

[silent cargo]

yoo
so i've been doing a lot of new things on comfy ui lately (new to me) I really tried my best to do it alone but here are some questions I would love to get some help. (its all about control net and openpose)

1.)when im using openpose does the checkpoint clearify if its sd1.5 or xl? (actually a general question what clearifies sd1.5 and xl? 😮 )
2.)Is there a difference between xl1 and xl2, do i need different openpose models?
3.)In the picture you see anime lineart as the preprocessor, but im using openpose as model, it somehow works but is not the right thing to do right?
4.) where can i download next to openpose models for xl2? (like depth canny etc.)

[livid gust]

I thought open pose was only SD1.5, but could be wrong. Don't supply lineart as the input image. Try dropping down an OpenPose Pose node in its place. At that point, you'll see it defaults to the SD1.5 512 resolution. To experiment with SDXL you could increase that number to 1024.

[silent cargo]

thanks for the answers! I appreciate it, do i need to change the resolution(512) for 9:16 / 16:9 or other aspect ratios?

[livid gust]

That's the rub, I believe it has to remain square, just like the IPAdapter. But you can still leverage a 16:9 or 9:16 ratio for your output. Try various inputs to see how it performs.

[silent cargo]

alright thanks again for the help, the openpose pose was so needed i literally got hedache from searching the right preprocessor lol xD

[silent cargo]

yo one little question more
when i connect a preview image to the"openpose pose " panel, then i get a black image without a skeleton, I somehow also cant connect it into the middle point like in your example, what am i doing wrong here?

[livid gust]

Weird. Perhaps try an image with a single person..?

[silent cargo]

nice thanks, the picture was the problem 🙂

[livid gust]

I don't think OpenPose works for quadrupeds either.

[craggy basin]

I am trying to run Florence 2 on Comfy, and I have the nodes, but don't know what node to collect the output for an image caption query

[merry ermine]

anyone knows if lora is supported for flux nf4 on comfyui?

[distant pendant]

pretty sure it's not supported, the bnb repo seems to have been abandoned in favor of gguf (where loras work, but it's also slower)

[merry ermine]

yeah i couldn't get the lora to work in my workflow with nf4 model, but unet dev model seems to be working fine

and you are also right about gguf model running bit slower than nf4

[distant pendant]

could probably port the lora support for bnb from forge somehow but interest seems low overall

[merry ermine]

if the author really dropped the bnb project in favor of gguf, its' a dead end road for nf4

[distant pendant]

well, the author of the bnb node is comfy himself and he noted in the readme that "This is very likely Deprecated in favor of GGUF which seems to give better results" so pretty much dead, yeah

[merry ermine]

you mean comfy from stable diffusion?

the one with yellow nick ?

[distant pendant]

I don't know the color of his nick but comfy as in the person who made comfyui

he made the bnb loader from illyasviel's code afaik https://github.com/comfyanonymous/ComfyUI_bitsandbytes_NF4

[merry ermine]

ahh yes i know who you mean, he used to hang out here, but then he left stable diffusion i think when sd3 crapped out with internal poltics

afaik and if im not mistaken he is still part of the comyui webui community as a dev

[craggy basin]

Not Yellow anymore since he quit in anger the day SD3 Medium was released. Gave two weeks notice and explained why he was leaving too

[craggy basin]

not 'as a dev', as its creator

It supported and supports SD, but was never Stability's. WHich is why ComfyUI supports so many other AIs from so many sources

Sound, video, LLM, etc

[hard marsh]

my workflow feels clean 😎

its simple but i like it

i aint doing complicated shit with comfy