🐝｜swarm-ui

[dim sonnet]

for the anime style Canny might actually be ideal, but for more realistic styles depth tends to do better

[potent quarry]

Awesome. Thank you for the information! I'll give it a try after I wrap up a couple things.

So with this should I skip using the refiner/upscaler entirely and only us the ControlNet workflow?

[dim sonnet]

ye

[potent quarry]

I've installed the ControlNet alternative and put the models into the folder but I only see them listed under the preprocessor dropdown and nothing under the model (and also nothing shown under the ControlNet tab). Did I miss something?

Or maybe I didn't because the folder is empty///

[potent quarry]

Do I need the modles in the ComfyUI ControlNet folder or the Swarm ControlNet folder?

[untold cairn]

I notice there is no sampler in your metadata. That probably means StableSwarmUI used the default: Euler. Euler is not very detailed. Try change it to DPM3++ SDE with exponential scheduler (somewhere in "ComfyUI" tab of parameters).

[potent quarry]

What is the difference between sde and sde_gpu?

[untold cairn]

One with "gpu" probably optimized for GPU?

Not sure, always used that one.

[potent quarry]

The image result is super wonky with DPM3++ SDE, either option:

Although in the preview everything looks fine until the very last step.

[potent quarry]

It seems like if I have the ControlNet models in the ComfyUI ControlNet folder they'll appear in the preprocessor selection but not the model selection. If they're in the Swarm ControlNet folder (only) then the dropdown is empty. 🤔

[untold cairn]

Not sure. Looks like a thing you can get in ComfyUI with option "return with leftover noise".
Also note that all DPM samplers require at least twice as much steps as Euler. Maybe you tried it with, like, 12 steps? Try 40.

[potent panther]

That's how Pony Diffusion XL was trained
It's a very annoying quirk of that model which wastes at lot of tokens instead of using words like good quality etc

And yeah the model looks best when you use all the score tags

It would've stayed in the bronie community if not for the fact that it trained anime

[dim sonnet]

preprocessors come from the comfy custom node

models go in the models folder and are separate

[dim sonnet]

i went to check the metadata on that but it looks like you turned that off :(

SDE needs really specific settings to work

[potent panther]

Oh I thought Spandrel was already merged https://github.com/comfyanonymous/ComfyUI/pull/2146

Might be why it's not detecting my upscalers

[dim sonnet]

that looks weirdly similar to

[untold cairn]

Once I successfully got merge on a pull request with only one symbol added. Unrelated project.

That symbol fixes a big bug tho.

[potent quarry]

I figured out my ControlNet models issue. I pointed all of StableSwarm's model/lora/etc paths to the A1111 folders when I started to avoid moving everything and had forgotten I did that.

[potent quarry]

I see a fair few people using After Detailer with A1111. Does Swarm have an equivalent?

[potent panther]

#🐝｜swarm-ui message

[potent quarry]

Thank you!

I imagine if I poke around I can find a breakdown of other things the segment feature can do?

[potent panther]

What do you mean by that?

segment uses clipseg to segment parts of an image using a word

[potent panther]

That was my bad I forgot that I had symlinked the wrong folder

[potent panther]

Mcmonkey could you add a user setting to save before segment?

[dim sonnet]

as soon as you type <segment: docs will pop up on-page with some usage info

[dim sonnet]

er... what?

oh, like output the image

there was already a Refiner-specific param for this, I swapped it to a general param that applies to any form of intermediate image - whether refiner or segment

and now you can see the difference your a photo of a cat <segment:cat's face> dog face segment made

[potent panther]

Thanks a lot

[nocturne glen]

I think the outpath is getting confused with the prompt because escaping brackets create folders.
For example, if the outpath is [prompt] and the prompt is hori \(hori no su\), 1girl, it creates hori > (hori no su > ), 1girl.png

So currently I am using a Fooocus style outpath format.
[year]-[month]-[day]/[year]-[month]-[day]_[hour]-[minute]-[second]_[seed]

[cold flame]

Anybody know of a way to get autocomplete for booru tags in the prompt fields? I tried installing pythongosssss' ComfyUI-Custom-Scripts and that doesn't even seem to work in the Comfy Workflow Editor tab although it does work when I launch ComfyUI standalone.

[dim sonnet]

oo it doesn't remove backslashes -- fixed

[nocturne glen]

Well, that was quick. Thanks!
EDIT: Confirmed working. 👍

[dim sonnet]

oo, ima make a note to add that. I have the engine for it, just not a place to add in your wordlist.
re the extension: make sure it's up-to-date, it broke before and was fixed. If it's not working on current version, post an issue and ref this past PR about it: https://github.com/pythongosssss/ComfyUI-Custom-Scripts/pull/149

[cold flame]

Awesome! I'll keep an eye out for your update. In the meantime I updated everything and tested again. The other features from the extension seem to be working as expected, but the text autocomplete section is missing from the settings and does not function so I posted an issue. https://github.com/pythongosssss/ComfyUI-Custom-Scripts/issues/179

Thanks for the quick reply btw

[versed copper]

halp!
how do i exit the "Edit Image" thingie?
i can't find any way to save and/or exit

[nimble belfry]

There is not currently a way without refreshing the page. Image editing is still getting worked on.

[versed copper]

ahh. after much fumbling and a ton of error messages in the console, i suspected as much
thank you for confirming 🙂

interesting comfy workflow though

[nocturne glen]

Since StableSwarmUI doesn't seem to have a tagging tool in the main UI, is there a way we can display the resulting text from ComfyUI?

[wheat pawn]

Is it possible to call stable swarm via an API? Currently I am calling Comfy directly with websockets and wanted to know if there was a simpler way.

[dim sonnet]

Yes. Swarm has a very straightforward API and uses its own API for things -- you can just interact in the browser and look at browser network console to see how it makes the calls and replicate them. Alternately here's a simple reference that uses swarm api externally https://github.com/mcmonkey4eva/SimpleDiscordAIBot/blob/master/Program.cs#L74-L158

[wheat pawn]

Thanks, Ill check it out! Is there a mechanism to see if an image name exists in the LoadImage list and if not upload one?

[dim sonnet]

If you use the standard swarm setup (ie not a custom comfy workflow), you just use base64 for the image.
If you use a custom workflow, ... you're still probably better off using the SwarmLoadImageB64 node with a base64 image

[wheat pawn]

Right on... these will be full custom workflows with some server tweaks before the actual call, just wanted to see what the division of labor needs to be

[nimble belfry]

I think most of not all perams are named exactly like they are in the UI. So if you were wondering how to set something just test with how it's labeled in the UI.

I have one bot that uses the preview images. I think that was the only thing I had to search for.

[wheat pawn]

Thanks, when I get the traffic console open I am sure everything will make more sense. A tiny aspect of what I am doing is dependent on images being in the load image dialog, I just dont want to reupload images if they are already there. That said, my app can just check the inputs folder if need be.

[gray geyser]

1

[potent panther]

with Save Intermediate Images toggled on the images in the gallery aren't in the right order

the actual order in Z to A

This is batch size 2 and images count 2

[versed copper]

is it intentional that if you enable FreeU with the suggested values, your usable CFG values drop way below 3?
as it is, it's not a free lunch, but an extremely costly one

[swift drift]

freeu has a really bad name and isn't acting like the holy grail it showed up pretending it was. I think it's a case of developer hype

[versed copper]

lol. that is a distinct possibility 🙂

[swift drift]

these guys are farming grants because theres so much money to be made. so they'll hype up anything they can to boost their signal and catch those VC funds

[versed copper]

that sounds even more plausible

in the case of Dynamic Thresholding, i think even the demo pics are no improvement whatsoever

[dim sonnet]

the main value of dynamic thresholding (imo) is it could generate in extremely few steps in the era before LCM/Turbo existed

outside of that it's fairly subjective but a bunch of people like it so shrug

and yeah FreeU ain't great

[versed copper]

is a picture control UI element a possibility? (saturation, grain, etc, there's a nice comfyui node...edit: ColorCorrect)?

[potent panther]

I heard it enhances Sag a lot

[dapper mantle]

What does this error mean? I just got StableSwarm and I'm getting this when I try to use segments

[dim sonnet]

in the ui go to server -> logs -> and set view type to Debug, likely the source of the error is in comfy's raw debug output

[dapper mantle]

The comfy error that I got was this :o

[dim sonnet]

are you using comfy-api-by-url backend? Should use comfy-selfstart

[dapper mantle]

Yeah I'm using the URL version since I already had comfy up when I installed it :o

[dim sonnet]

You can use your existing install with selfstart, just let swarm launch it (ie and configure the launch params to point into swarm's nodeset and all)

[dapper mantle]

Alrighty, I'll try that

I assume this is bad? :o

[dim sonnet]

You have dependencies for nodes installed (assuming it worked before when launched manually), but not in the env comfy is in?

(Or you don't have a proper env for the comfy install)

[dapper mantle]

I think I don't have a proper env, though the segmentation seems to work now :o

[dire bobcat]

Hello. I am having trouble with the Sketch to Image function. Does anyone know a way to generate images that are more photorealistic?

[potent quarry]

Hello again, still working on understanding how to refine and upscale images for the best results with different workflows.

The first image here is a base size image from my prompt along with one that was upscaled with a controlnet to 4k. It doesn't really do much to maintain or enhance the quality or sharpness of the image.

Original params: prompt: score_9, score_8_up, score_7_up, score_6_up, score_5_up, score_4_up, 1 girl, steampunk mechanical wings, extravagant dress, big hair, long hair, art nouveau background, negativeprompt: chibi, loli, child, 3d, cgi, render, photo, sketch, realistic, source_pony, score_4, score_5, score_6, bad anatomy, ugly face, mosaic, pony, images: 2, seed: 68268927, steps: 50, cfgscale: 9.5, aspectratio: 16:9, width: 1344, height: 768

ControlNet params: steps: 20, cfgscale: 7, aspectratio: Custom, width: 3840, height: 2160, initimagecreativity: 0.05, controlnetpreprocessor: Canny, controlnetmodel: control-lora-canny-rank256.safetensors, controlnetstrength: 1

When I instead add a refiner/upscaler step instead of a controlnet, I get much sharper and more refined images but with increasing mutation as I increase the refiner percentage

refinercontrolpercentage: 0.2, refinermethod: PostApply, refinerupscale: 3.5, refinerupscalemethod: pixel-lanczos, refinerhypertile: 512,

[hoary wave]

[potent quarry]

Refiner .2

Refiner .3

Refiner .4

Refiner .5

I'm trying to understand what is happening and how to prevent it.

The mutation is most pronounced in things like hands but in some images I'll get multiple additional body parts, etc. You can see it become more obvious starting at refiner .4 with the additional hand and second torso along with the faces starting to be rendered in the wings.

But the increased sharpness is

[cold flame]

This looks like you're using some form of tiled sampling on a second pass with the full prompt as well based on the additional faces that are appearing in in the wings. If you share the full workflow I might be able to help trouble shoot the problem.

[potent quarry]

Unless you mean ComfyUI workflow in which case I don't have one.

[cold flame]

I'd try lowering the upscale to 2x and do a 3rd pass to upscale again if you still need the image at higher than 2688x1536

[potent panther]

3.5 upscale???

is this guy nuts?

[potent quarry]

Calling me nuts isn't exactly nice nor helpful. If there is a specific issue with higher upscale values, I don't know what it is.

[dim sonnet]

Large upscales aren't necessarily bad in and of themselves, but the larger an image is the more a model will struggle to understand what it's doing and generate properly. You can see this best if you generate directly at large size - most commonly it'll start generating tiled-looking content (ie your subject repeats over and over again)

You can counteract this by reducing the scale of detail it's allowed to generate (ie reduce refiner control percentage)

but you also want percentage higher to handle larger upscales

so you get a graph that at a certain point crosses, where you need to both increase percent to fix details and reduce it to prevent tiling, and you're trapped

[potent quarry]

Is it a fools' errand to try to get an image to 4k resolution, even through several rounds of smaller upscale increments or am I misunderstanding?

[dim sonnet]

at that point stepping repeatedly (ie generate, grab the output, and upscale+gen again, and again and again) is the only way

[potent quarry]

Something like a .2 refiner percentage for a 1.5 upscale (times however many rounds needed) ?

Assuming steps are something reasonable

[potent quarry]

Hmm, it looks like I could also generate an image and then use it as an init image (img2img) to generate a double-size output to then upscale.

Issues with the base image aside, this is moving in the right direction.

Thank you for the information, Alex!

[nimble belfry]

I dont know if this is exactly what your looking for but I have been using this workflow and I have been very happy with it. https://www.youtube.com/watch?v=CxB47DMEyYQ

I took your latest image and ran it through and it seems to have worked well. It took FOREVER but it upscaled it 4x from what you had. I literally cannot zoom out more in comfy to show you the full final image. but its 21504x12288 and 237MB.
I want to look into making this work using swarms UI and be able to call it from the API but I have not had time to look into it.

[dim sonnet]

• Set up Primitive nodes as inputs, and title them along the lines of SwarmUI: Seed to assign default swarm parameters to the workflow's configurable inputs
• use the purple save workflow button
• then call swarm api as normal and add the [ComfyUI] Custom Workflow parameter to your input set to whatever you named the workflow when you saved it

[nimble belfry]

You are a king among men.

[potent panther]

Sorry

[nimble belfry]

How do I make the InitImage the image that gets loaded into Comfy Load Image node? A primitive node does not seem to take an image input/output. (I have never messed with Comfy other than loading others workflows.)

[dim sonnet]

you can either just have a single LoadImage node and let swarm autoreassign it, or use a SwarmLoadImageB64 with a primitive to force it

the b64 is more trustworthy/stable, it will always send direct to comfy whereas with LoadImage depending on how it processes it might get sent to datadrive temporarily (since that's how default comfy does image loading)

[potent quarry]

Thanks for this. I'll check it out a bit later tonight.

I finally started to learn a bit of ComfyUI and it isn't as intimidating as it first looks though I'm having trouble getting the same level of detail from the refining step so I'm missing something. I thought I had the second KSampler set up to do refinement like it works in the non-ComfyUI workflow but apparently not.

It adds a little detail but nothing close to what the other refiner was doing with similar steps and denoising weight

I set up three different save images so I could see the original generation, the refined image, and the upscaled image since I was trying to isolate the different steps to understand them.

[nimble belfry]

I have also heard about some people upscale,, downscale, then reupscale for more added details but i have never tried that.

[potent quarry]

That seems like it would be excessive for what I'm trying to achieve for now. 😅

[potent quarry]

I'm coming up short understanding some of the additonal values that the SwarmKSampler offers over the regular KSampler if anyone can help with my vocabulary.

• var_seed
• var_seed_strength
• sigma_max
• sigma_min
• rho

The sigma values seem to replace the denoising setting but I don't understand what the two values ultimately do.

[dim sonnet]

if you search in the main Generate tab to find the matching inputs they tend to have a bit more details about em

in the case of sigma&rho: they're internal values to muck with the sampling scheduler... for the most part practically speaking they don't overly matter much

the defaults are fine, and they only even apply with specific scheduler choices

the main case where they matter is models with eg custom V-pred scheduling, or SVD benefits from a changed value (which will happen automatically in swarm when you use SVD)

[potent quarry]

When I import a StableSwarm workflow into ComfyUI, the CLIPTextEncode looks different.

What is the purpose of the two text boxes and should I be using these for SDXL workflows in Comfy? If so, I can't seem to find them in the node menu.

[potent panther]

Could be nice having this in StableSwarm https://github.com/Fanghua-Yu/SUPIR

[abstract rapids]

update StableSwarm and now getting these all over "07:28:29.792 [Warning] ComfyUI-0 on port 7821 stdout: requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))",

[dim sonnet]

• might just need to restart swarm + refresh the browser page
• are you running locally, remote server, colab, ..?
• is anything actually breaking or is it just shouting in logs?

[abstract rapids]

im running local, it was working 2 days ago and now getting also this error "08:40:43.406 [Init] Self-Start ComfyUI-0 on port 7821 started.
08:40:46.090 [Warning] Failed to load metadata for open_clip_pytorch_model.safetensors:
System.InvalidOperationException: Improper safetensors file E:\StableSwarmUI\dlbackend\comfy\ComfyUI\models\clip_vision/open_clip_pytorch_model.safetensors. Wrong file type, or unreasonable header length: 578712552184433488
at StableSwarmUI.Text2Image.T2IModel.GetSafetensorsHeaderFrom(String modelPath) in E:\StableSwarmUI\src\Text2Image\T2IModel.cs:line 74
at StableSwarmUI.Text2Image.T2IModelHandler.LoadMetadata(T2IModel model) in E:\StableSwarmUI\src\Text2Image\T2IModelHandler.cs:line 356
at StableSwarmUI.Text2Image.T2IModelHandler.<>c__DisplayClass22_0.<AddAllFromFolder>b__1(String file) in E:\StableSwarmUI\src\Text2Image\T2IModelHandler.cs:line 485". I did not touch that model and was okay. Image generation works fine, just comfy in midas nodes and ip-adapater, animateDiff and contron-net stuff.

crap -- it's not only Swarm, getting same remote connection closed in portable comfi and stable matrix package. i disabled defender, anitvirus and same remote connection error. --

[dim sonnet]

well that particular error - the model is corrupt/invalid

I've seen that happen if you store a .ckpt file with the .safetensors extension by accident

but it's also possible if that happened at the same time as a bunch of other things broke, that you might have an OS error or hardware fault

[abstract rapids]

its happenning in foocus now, it I think is just deeper than comfi. I am going to simply do a fresh win install and others. BTW, your product made me forget what auto1111 looks like..lol, loving it as my main generation tool.

[pastel obsidian]

Hey everyone! In SD 1.5, what is the best way to mix multiple object dreambooths in order to generate an image and keeping the complete fidelity of all the trained dreambooth objects (e.g. mixing a specific cat and a dog to generate the hyper-realistic mix of those two specific dog and cat)? 🤔

[dim sonnet]

not sure if you intended to ask in the swarm channel or you're just lost, but -- go to the Utilities tab -> LoRA Extractor, extract the dreambooth models to LoRAs, and then you can easily apply multiple loras on the fly while generating

[nimble belfry]

I'm sure he is not using Swarm. He is spamming other channels.

[pastel obsidian]

Thanks, I'm new. I'll try that.

[abstract rapids]

update-- Mr. Youtube helped me and cussing at the computer with really vulgar insults help.. "the 8088 better chip than you lazy i7 ass' helped.lol.. seems like one command line in DOS with admin rights did the trick.
C:\Windows\System32>gpupdate
Updating policy...

Computer Policy update has completed successfully.
User Policy update has completed successfully.
--- so far errors gone on remote cloing and cliptorch error gone. I would love to know how in the hell did this solve the problem! So far, fingers crossed all green lights

[potent panther]

This came out
https://github.com/lllyasviel/stable-diffusion-webui-forge

[dim sonnet]

that ... seems to bring some of auto a bit closer to comfy (fixing the performance gap and the need to decipher arcane cli flags)

still doesn't do the biggest thing for swarm that comfy does, which is make it wildly easy to hack around custom generation workflows and call em via API

i think lllyasviel has an obsession with making stuff in the SD space, which is cool, but not as much thought towards, like, end goals?

like, swarm's end goal is to be the one true ui to rule them all, a clean and easy and powerful UI that can cover the full range from basic users to crazy advanced stuff. Like, long term, be the lasting option that actually makes life easier for all types of users.

vs. llly made Fooocus ... and that does a decent job covering basic users, but is pointless to advanced users, but then has really cool custom sampler code that advanced users would love, but then it's trapped in the minimalist UI and not available to advanced users. And this Forge thing looks cool, but it's just trying to get the old auto halfway in line with what comfy can do, and ... to what end? Bringing back the era of a hacky Gradio tabfest? why?

the one thing auto webui has going for it anymore is the long history of extensions, and, well, the new forge thing's core design concept is to break compatibility with that. So what's the point? It's neat, but, like, why tho (noting that the extension unet patcher thing is just like, a subset of how comfy does things by default anyway)

I think if he focused in on his skill at optimizing code, making more advanced sampling/etc internals, making all sorts of cool magic that he can make, and then implementing them into existing platforms (like making comfy nodes) that'd be a much better long term contribution to the SD ecosystem

[abstract rapids]

Love Fooocus for really two things -- it's focus-control net spin off is a true img2img ( advanced inpainting second to none) machine and it's unbelievable high quality prompt engine( chat gpt 2 i think) -- beyond that limit usage just for images with little effort.

[abstract rapids]

trying to use SVD 1.1 but cannot refresh the video input checkbox. Darn up and down buttons don't work

[dim sonnet]

all else fails, refresh the entire page or the swarm server

[nimble belfry]

I had to refresh the page for it to show.

[dim sonnet]

oh, i see why, it's now included in Reload Parameter Values

[abstract rapids]

is there a tutorial or guide as an example how setup primitive nodes(or does import workflow into main screen do the same thing with the workflow loaded?) , 2. Is the purple save workflow button the SAME as the save on the comfiui manager save button or a different type of save.

[dim sonnet]

the purple button saves to swarm's files and allows it to be reused in swarm (API or the customworkflow param or for grids or etc), the comfy save button just exports a file for you

[abstract rapids]

sweet nice to know

[dim sonnet]

for primitives it's just a comfy node

[abstract rapids]

thats so cool...

[dim sonnet]

can right click to title it in the format "SwarmUI: Prompt" to link it to any default parameter

[nimble belfry]

🪃

[dim sonnet]

nice

[rapid harbor]

Türk yazılımcı var mı şuana

[hearty notch]

Can we have a nod in comfy to use the style from swarm?

Or is there any existing way that someone can share??

[dim sonnet]

huh?

[narrow blade]

@dim sonnet can I dm?
i have a huge private dataset

[dim sonnet]

wot

[narrow blade]

got around 700mil images

[dim sonnet]

what does this have to do with swarm

[narrow blade]

nothing with swarm, but something i can contribute to sai

[dim sonnet]

weeelll i'm not the person to talk to datasets ain't my job

[narrow blade]

who's the one?

[dim sonnet]

@crimson rune

[narrow blade]

i did dm him, we had a talk before
but i dont think he is active on discord these days, didnt get a reply yet

[hearty notch]

The presets on swarm, is there anyway to load them in comfyUI through a nod there? And not just type it in the prompt box manually?

[nimble belfry]

hmmm, on the flip of that it would be cool to save a workflow directly to a preset. slightly redundant though.

[hearty notch]

I agree, but sometimes working with comfy is just easier especially when the graph is a bit complicated

For an example, the one I am working with now, contains ip adapter for 2 images, and 2 chained controlnet ,

Personally I use presets purely for the prompt text, hence why I am wondering if there is a way to load them in comfyUI without the need of inserting them manually over and over in the prompt nods

[dim sonnet]

You can - save a workflow, and then make a preset that just selects the workflow as ComfyUI Custom Workflow param

this is why my discord's bot only input is Prompt

i can just preset:... on the prompt to switch to anything else i want up to and including comfy workflows

[dim sonnet]

presets do a lot more than just prompt text in swarm

[hearty notch]

I know, hence why I mentioned “personally”

[dim sonnet]

i believe there exists a prompt text from file node thingy in comfy somewhere, just not using swarm presets

[hearty notch]

Hence my question if there is a comfy nod that will read swarm presets,

🤓

[dim sonnet]

(if still unclear the answer is no, no such node exists)

arguably one could be made but it would be awkward, data flows backwards

[hearty notch]

Haha it’s clear, thanks for the response 😁

[sacred mason]

@dim sonnet this may interest you https://github.com/hiddenswitch/ComfyUI?tab=readme-ov-file#distributed-multi-process-and-multi-gpu-comfy

this means you can replace managing copies of the comfyui repo and dependenceies with

1. Author your custom nodes in a separate repository as per https://github.com/hiddenswitch/ComfyUI?tab=readme-ov-file#authoring-custom-nodes

pip install git+https://github.com/hiddenswitch/ComfyUI.git
pip install [e-mail]:Stability-AI/StableSwarmUI-Custom-Nodes.git

At this point, you now have a working ComfyUI installation with your custom nodes, without touching or munging the filesystem, and without having to install torch - the setup.py in ComfyUI will select the right torch for the system.
3. if you want to use comfyui with your own web server, as a library, see https://github.com/hiddenswitch/ComfyUI/blob/fc6ab798d601d932a042574879b53a8fbf9bb96c/script_examples/basic_api_example.py#L139
if you want to run it as-is, just run comfyui

if you want to support multiple comfyui backends, use the distributed backend instructions. rabbitmq is a very capable and extremely easy to use queue implementation.

this fork has distributed comfyui workload processing. it does not require you to specify a list of backends, fiddle with ports, etc. you can either submit a workflow to any frontend using the pre-existing comfyui API, or use a better API specified in the openapi.yaml, or you can submit a job directly to the queue.

you can also use the pyinstaller based single file .exe if you do not want to manage a python virtual environment at all

it will correctly start a comfyui instance without any faff

[dim sonnet]

swarm already handles all of this very effectively on its own, don't need an added layer in between

[dense crypt]

Is Swarm dependent on Huggingface to run? Huggingface is having an outage right now.

Trying to load/add a model but seeing the following in the backend server logs:

2024-02-10 20:18:34.074 [Warning] Failed to load metadata for OfficialStableDiffusion/sd_xl_base_1.0.safetensors:
System.InvalidOperationException: Improper safetensors file /Models/Stable-Diffusion/OfficialStableDiffusion/sd_xl_base_1.0.safetensors. Wrong file type, or unreasonable header length: 5789751444030890300
at StableSwarmUI.Text2Image.T2IModel.GetSafetensorsHeaderFrom(String modelPath) in /src/Text2Image/T2IModel.cs:line 74
at StableSwarmUI.Text2Image.T2IModelHandler.LoadMetadata(T2IModel model) in /src/Text2Image/T2IModelHandler.cs:line 355
at StableSwarmUI.Text2Image.T2IModelHandler.<>c__DisplayClass22_0.<AddAllFromFolder>b__1(String file) in /src/Text2Image/T2IModelHandler.cs:line 497

[last tinsel]

working fine here

[dim sonnet]

It is dependent on HF during install to download the model file, but other than that no

so if you just now installed swarm it downloaded a bad model to the Models folder, but you can just open civitai or any other site that has models to download it from manually instead

[dense crypt]

ah... that explains this:

2024-02-10 20:55:15.758 [Warning] Failed to load metadata for OfficialStableDiffusion/sd_xl_base_1.0.safetensors:
System.InvalidOperationException: Improper safetensors file /Models/Stable-Diffusion/OfficialStableDiffusion/sd_xl_base_1.0.safetensors. Wrong file type, or unreasonable header length: 5789751444030890300
at StableSwarmUI.Text2Image.T2IModel.GetSafetensorsHeaderFrom(String modelPath) in /src/Text2Image/T2IModel.cs:line 74
at StableSwarmUI.Text2Image.T2IModelHandler.LoadMetadata(T2IModel model) in /src/Text2Image/T2IModelHandler.cs:line 355
at StableSwarmUI.Text2Image.T2IModelHandler.<>c__DisplayClass22_0.<AddAllFromFolder>b__1(String file) in /src/Text2Image/T2IModelHandler.cs:line 497

Is there a good practices guide for SwarmUI.

I'm using the Docker image and noticed the Comfy install doesn't include https://github.com/ltdrdata/ComfyUI-Manager. don't want to install it if it defeats the purpose of using Stable Swarm.

[nimble belfry]

I use swarm but sometimes I like to try out peoples workflows. Having the manager installed helps easily install the missing nodes.

[dense crypt]

appreciate your knowledge around the product. I'll start with Scott's video.

https://www.youtube.com/watch?v=M-C5eeDN7Ew

I'm trying to understand SD more at the Jupyter Notebook level. There are too many nodes - at this point looking to go back to basics and understand what core python modules contributed to generating an image.

If I understand correctly Comfy was a backend guy. I can relate to that. I'd like to see ComfyAPI if he ever decides to release it. 🙂

Enjoy your weekend!

[dim sonnet]

comfyui can be used as an API, or from pure python, if you prefer

using comfy as an API is how swarm does it. From python there's a repo named "comfyscript" iirc for it

[dense crypt]

this Jupyter Notebook is almost exactly what I was looking for. This is pretty cool actually.

https://github.com/Stability-AI/stabilityai-demos/blob/main/Image_Text_Generation_Notebook.ipynb

I'm going to have to clean this up now, and add the code to call SD directly, but I was trying to do something similar here: https://github.com/donbr/visionary_storytelling/blob/main/notebooks/ai_vision_image_classification_ollama.ipynb

AI Vision in a multi-modal workflow is powerful stuff.

[sacred mason]

hmmm

[sacred mason]

there is a comfy api, that is what i am working on with comfyanonymous

[sacred mason]

https://github.com/hiddenswitch/ComfyUI/blob/master/script_examples/basic_api_example.py here is an example

this is not the same as the api that people are using. this is running comfy embedded in python. you can also use it remotely.

[sacred mason]

do you recall when i was asking about the backend "Swarm" features earlier?

[magic ravine]

Is StableSwarmUI still being worked on ?
I'm wondering especially since I think it lacks themes

[potent panther]

Of course

[magic ravine]

Nice. Idk why I thought it was abandoned.

[potent panther]

Maybe because there hasn't been commits for a few days?

[magic ravine]

Nah, I remembered not seeing commits for months, but I just checked and it's daily commits. Maybe I just hallucinated

[nimble belfry]

I don't think monkey is capable of not doing work for a couple of months.

[dim sonnet]

yes still being worked on, and it has 3 themes. Not sure why themes are your most important flag of being worked on or not lol.

themes are selectable in the installer and also in the User Settings tab

[magic ravine]

May be odd, but the the theme is the first thing you see and the thing you keep seeing while using the ui x).
I tried the two dark themes, but I think main problem is margins and alignments, like the ui seems squished on the left and two small. Also the down pane is hiding the generation settings which gives a very small place for it. I'd love an option to have it as a right pane

[dim sonnet]

(past couple weeks work on swarm has been slightly slower than usual as i have multiple other major projects being worked on. Slower than usual for me is still faster than most projects out there lol)

[dim sonnet]

the left and bottom panes are resizable, just drag the border bar

[magic ravine]

I am not able to completely hide the down panel even using the right arrow

[magic ravine]

Yes, we can see here everything small and squished on the right and even borders going through text.
I still think the project is very cool, just this UI thing is very disturbing

[dim sonnet]

... huh?

what's squished?

[magic ravine]

I mean it looks like a wall of text, not buttons, everything is left aligned and there's a huge lost space on the right

[dim sonnet]

i'm sorry i'm very confused

you replied to a screenshot that shows only the left pane and commented about appearances on 'the right', which i'm not sure what that's referring to

the contents of the left pane are indeed left aligned, i'm not sure the relevance there

it might make sense to center-align the group titles?

parameters would look really weird if centered

right-aligned content wouldn't make any sense

[magic ravine]

Yes of course not. I like the idea of centering the titles. I think the problem is actually the text being too small for the pane which makes it lost on the screen. Maybe make controls next to each other instead of only vertical, like tbe four controls in core parameters could take only two lines instead of four

Also, don't you find it weird having the border crossing through the titles ?

[magic ravine]

Maybe a better description of the feeling I had would be like when you load a webpage but the css isn't loading, I genuinely thought there was a display error at first

[dim sonnet]

i used to have multiple parameters on one row but it was a lot more awkward and got complaints, so it's now locked to one per row.

[dim sonnet]

no i like that :(

i also tried making it boxier like dreamstudio but that also got complaints

(screenshot is of dreamstudio)

[magic ravine]

It's just that when the sections are opened (like on your screenshot), itms hard to see which lines sre sliders, which are top/bottom borders

[magic ravine]

I'm actually surprised, imo both having multiple elements on row and boxier would be improvements

[dim sonnet]

the sliders are the ones glowing purple

[dim sonnet]

boxy works for the dreamstudio "this is just a simple two-input interface to make dreams :)" interface but doesn't work well with "and here's 500 more advanced options"

i'd love to just hire a UX expert and offload the problem of making it look nice and feel nice to somebody else but i don't have the authority to hire people :(

best i can do sometimes is be a lot nicer looking and feeling that comfy's noodlefest or auto webui's gradio jank and call it good enough

Beyond that it's just a mix of what feels good vs. bad to me and what generates complaints. Taking direct suggestions on UX topics usually leads to two different people telling me two opposite suggestions at the same time

[abstract rapids]

Is it possible to change SVD to portrait instead of landscape?

[magic ravine]

This feels less hunched and having models on the right make it not cover settings which is the most important with models browser so each should have their own place. Also language is way to big and no need for both flag and name, moved it next to quick settings

far from perfect just an idea

[dim sonnet]

Yes - SVD is trained only one one specific portrait res but it seems to (mostly) function anyway on other resses -- a feature in swarm to choose your video aspect/res was added a few days ago, so just update. Using your source aspect is now the default behavior.

[dim sonnet]

... did you do that in an image editor or by editing css

cause if you can do that in css ... pr plis? :D

[magic ravine]

css

[abstract rapids]

thank you

updated then this shows up

[magic ravine]

Wait does that mean you like it ?
It's mostly css, but some things need to be changed manually, for example moving the models tab at the right instead of down. Also "display:block;" is added to each .input-group automatically and I needed to change it to display: flex; for each and quicktools and language buttons could be moved/changed manually too instead of a css for that.

[abstract rapids]

I have a second install of Swarm in Stable Matrix so at least I use that till it gets resolved. Pretty slick implementation.

[dim sonnet]

Re the left param tab: I don't love the design personally but it'd be great to have as an option (Theme variant?) (And then poll users to see which should be default)
re the Language menu: that's gonna be moved elsewhere soon anyway so just don't bother with that bit for rn
re the models on right: makes the centerspace way too cramped imo but it's close to planned goal of being able to slap tabs into different areas at will. (At which point, the current right side area would just be a "Current Batch" tab that can move around too)

[dim sonnet]

This looks like you have an image thumbnail for a model with invalid/corrupt data in it - i pushed a commit that will catch the error and report which file it is that's corrupt instead of crashing

[dim sonnet]

I suspect the current state of that left tab might misbehave once you start using it - add a controlnet image or select a model, see how it looks. If you weren't very careful with the css it'll probably go wonky

already for example the Controlnet Preprocessor input looks a bit off

[magic ravine]

I will make a pr for left param tab. (improved version of what I showed)
Language menu and models I wait for your touch

[abstract rapids]

you must sleep like half a hour a day...inhuman lol

[abstract rapids]

your commit worked

[sacred mason]

the aesthetics of the editor workspace matter a lot to an ordinary free apps users. consider how many people use VS Code because it looked more attractive than PyCharm, and that jetbrains marshalled all its resources to clone the vs code UI for this express reason

i would use stableswarm as a frontend to comfy if it were essentially an automatic 1111 wizards-style UX built around comfyui workflows

if it were focused on the aesthetics of that experience and nothing more, it would help me choose in light of how much already exists

[magic ravine]

That's how it looks now (with wrong language and models placement)

Also quick question, why is positive prompt there ? It should be next to negative prompt ?

[dim sonnet]

you can user -> parameter configuration -> Param Prompt -> check "Visible Normally" if you want it at the left

the idea with it being in the center is the idea behind the "Fooocus" UI that does the same - here's your central prompt and interaction place, over on the left is all the advanced parameters, you can just hide those and focus in on prompting now

[potent panther]

Wow I love that

[magic ravine]

I made another one

[potent panther]

Great taste in color btw

[abstract rapids]

looks darn tasty!

Img2img needs a facelift too 🙂

[magic ravine]

https://github.com/Stability-AI/StableSwarmUI/pull/233

[magic ravine]

Where is that ?

[potent panther]

The edit image button?

But that’s still an unfinished feature

[magic ravine]

Can you show me ?

[potent panther]

You can generate an image and then in the gallery click on it and there'll be be an edit image button above the image

[nimble belfry]

Just keep in mind (unless this changed in a recent update) the only way to close img edit is to refresh.

[dim sonnet]

ooo

[dim sonnet]

https://github.com/Stability-AI/StableSwarmUI/commit/0713229fbc34cea80be574b04cde8e0d29e80f95

[nimble belfry]

Holy shit! "most requested feature 2024" made me lol

[magic ravine]

@dim sonnet I did a PR.
Btw, completely unrelated:

1. Do you think there would be a way to have Krita integrated with StableSwarmUI, maybe a tab where you can send images ?
2. How hard would it be for you to add custom api backend ? I need to send a workflow json alongside an apikey to an api endpoint running comfy

[dim sonnet]

1: Should be possible, there are krita plugins like it, and swarm's API is pretty straightforward
2: pretty simple! Backends are written as extensions, so just write your own extension and save it in the src/Extensions/ folder, can reference the Comfy-API-By-URL backend and just repurpose that with some tweaks for the sake of an external comfy-based API, or write a new backend class but just reference the comfy code for the workflow processing and all

actually also have exactly that concept - an alternate API handling comfy workflows - in an internal extension (uses a stability internal comfy server thingy)

90% of the code is specific to that internal API, not much that's part of the backend impl cause it's all offloaded to the main comfy backend code

[dense crypt]

@dim sonnet bit of a left field question, but based on your broader work may be of interest... Have you looked into DSPy for prompt generation?

[sacred mason]

are you trying to embed comfyui in another application? like if it were possible, do you want to "just" distribute a krita plugin that is capable of runnign comfyui workflows, without the complexity of managing a copy of comfyui?

that is it would still be standalone and self contained

not via remote APIs or similar

maybe describe your ideal way of doing this

or what your goal is

[dim sonnet]

the formatting in the theme PR seems off? It's definitely applied, the colors and spacing changed, just, not in the way you intended/screenshoted earlier

[magic ravine]

I don't have a GPU, but I found few paid APIs that would run comfyui json (technically stable horde which is free is based on comfy, but for now it only allows a specific workflow), what I'd like to do is use stableswarm same way normal user would but send generation to remote. Though now that I think of it, it may actually be possible to have a comfyUI extension for that and just use stableswarm as normal comfyui, so handle that at another level.

Krita question was about having a way to send image generated from swarm to krita, edit it, may be do other trings like generate on it, and send it back to swarm for further process

[magic ravine]

Try closing reopening the same setting

I think it happens when switching theme

[sacred mason]

okay, so you are primarily interested in solving the problem, "i don't have the hardware to run a comfyui workflow on my local machine, but i want a painless way to run comfyui workflows on remote machines, at least as flexibly as the comfyui workflow format itself"

does that sound right?

why do you need the comfyui json at all? are you developing a krita plugin?

[dim sonnet]

oh my god that does do it

that's wild why does that happen

[magic ravine]

That's a good question

[dim sonnet]

it gets very confused if i shrink the left pane

should probably have minimum sizes on the elements so it just snaps to vertical rows if it's too thin instead of trying to keep horizontal

[magic ravine]

I thought it did move down, but maybe we need to add min, right

[magic ravine]

Yes, I'd like to have same workflow oportunities

[sacred mason]

and is this for you to run comfyui workflow sremotely in your particular use case, or for what reason? why not run comfyui on a remote machine, and access its interface over the web?

[magic ravine]

No two questions are unrelated. I just think swarm and krita are two best ways of using comfy

[sacred mason]

or are you trying to solve that problem generally for other people*

[magic ravine]

Because I don't want to pay for idle time, and I want to have workflow and output on my machine

[magic ravine]

For me and others

[sacred mason]

okay, and what in particular isnt' good about adobe firefly in this case

which does everything remotely

and is pretty deeply integrated into photoshop in a good way

and is free for a lot of usage, and extremely affordable pay per use. it's functionally free

at the time being

[magic ravine]

Well it's not stable diffusion is it ? And not comfyui ? And not opensource, and photoshop isn't either

[dim sonnet]

i thought you were coming in here to advertise your pip-comfy-thingy but now you're advertising adobe products? i'm so confused

[sacred mason]

no i'm just trying to help the guy

[dim sonnet]

wait speaking of shrinking the panes where did the snapclosed buttons go

i know they look silly but they're handy :(

[sacred mason]

can you help me understand how you are running something to minimize "idle time"?

minimizing idle time is an orthogonal problem to everything else

and it's complicated to achieve in a general way

[magic ravine]

I removed them xD, I found them "silly" indeed 🤣

[dim sonnet]

but they're so handyyyy

1 click to close and 1 click to full open

[magic ravine]

But it isn;t complety closed, I still had at least a centimeter of it showing, that why I found them weird

[sacred mason]

right now it sounds like you would be well served by google colab, which gives you t4 instances for free

and in which you can mount your gdrive

so your outputs and workflows will appear locally eventually

[dim sonnet]

clicky clicky easy open happy :D

[sacred mason]

you can use vanilla comfyui's colab notebook

[magic ravine]

If I rent a gpu, or I rent a service like rundiffusion, I will be charged even when optimizing the workflow. It's a stressful to have the countdown. I think it make more sense to be charged by generation

[sacred mason]

colab with t4 is free

have you tried it?

you don't have to rent gpus

the T4 is extremely capable

[magic ravine]

Isn;t it banned ? I know whole sdwebui collabs were

[dim sonnet]

my plan for prompt gen is to run a small LLM like StableLM-3B or something in gguf - lightweight enough to run on CPU on a laptop without getting in the way of VRAM gens and still be fast, but smart enough that it can have some dynamic prompt/control, not to mention easily swapped out for other models if people want to

[dim sonnet]

yeah colab doesn't allow any usage that's outside the intended purview of simple research/experiment interfaces

(unless you pay)

[sacred mason]

punching it into google it looks like people can run comfyui in colab

[magic ravine]

We used to have an easy exploit by renaming a string, but they eventually fxed it

[sacred mason]

i am not sure because i have a GPU

[dim sonnet]

ye last i checked swarm still technically works (because i put the string edit in the file lol) but, yknow. They won't be happy if you use their GPUs too much

[sacred mason]

so it sounds like it can be run

[dim sonnet]

They definitely don't intend for it to be okay to abuse that way - imo it should be fine for short tests so the string ban is silly

the intended ban is people just sitting there generating on their GPUs

which quite possibly they have a detection that only kicks in after n many hours or something for a ban

[dim sonnet]

that's surprising. You seem like enough of an obsessive python developer that you could easily get a nice tech job that sends you free gpus and/or pays you well enough to buy your own hardware

[dense crypt]

I should have asked this question in the research area... I see you showing up over there as well. 🙂

Depending on what the vision is for swarm it would be interesting to have flexibility to send it transparently to different backends - SD or other... (I like the flexibility of picking your backend with Swarm... this is an extension of that)

[dim sonnet]

multiple LLM backend options would be neat to have ye

[sacred mason]

i think you are in a difficult position because unless you want to deal with the annoying parts of colab, it's hard to get GPU time for free

adobe firefly might be your best bet

or maybe when civitai releases a service

the tough part is that it takes a long time to download and install models. even if you did the work to use per-second billing in AWS for GPUs on spot instances, you'd need to do all the work to get a warmed-up image

and the cost for the storage of one EBS volume per month exceeds the cost of your runpod / rundiffusion usage

you could also pay for colab pro @magic ravine and maybe that will let you run comfyui unfettered

it is extremely hard to do all of this for less than $10/mo up front cost to you.

[potent panther]

Noooh it's the best ui feature mcmonkey ever coded

[sacred mason]

or if you use this a lot @magic ravine , an 8GB 2060 can run a lot of things, and it's like $100

[magic ravine]

I'm not sure you're giving him compliments with that

[sacred mason]

a T4 is roughly as fast as a 2060 super but with 2x the RAM

maybe that's your best bet

[dim sonnet]

i don't know how you got on this tangent, pretty sure quantumsoul has a gpu and is happy with what they're using?

[potent panther]

T4 is slower than my 3070 mobile that has 90w top

[sacred mason]

indeed.

[magic ravine]

No I don't actually, but I don't think I will use it enough to invest

[potent panther]

and who's this doctor guy that's saying random stuff that feel irrelevant to the channel?

I'm just kinda confused

[sacred mason]

finding out "why" is relevant

[potent panther]

actually why not have it be middle mouse click that you can do anywhere on the bottom panel outside of the buttons?

and it can be shown in the tips and trick when you open StableSwarm

[dim sonnet]

ooo

actually it should probably be specifically on the buttons

cause if you middle-click a button it breaks atm

didn't know that til just now lol

but if you middle click the main body you get the magic autoscroller

[magic ravine]

What do you think actually, is it better to make a swarm extension or comfyextension like described here #🐝｜swarm-ui message ?

[potent panther]

I didn't think they would be # links

[dim sonnet]

oh there were was an actual context to the random tangent i entire missed it cause i wasn't bothering to read the whole conversation lol

[dim sonnet]

i mean i like using swarm for things but i'm biased

i think a swarm-krita extension would be awesome

a swarm backend extension for a specific paid api service idk. Like you can and it's not hard. Maybe publish it and show the service you did it and ask em to hire you? then get your money back and also enough more money to buy a gpu with lol

[sacred mason]

i don't mean this as a criticism at all: it is hard to justify developing a backend for people who want a free service

it is very very hard to make any of this make sense if you do not have a GPU. for artisanal quantities of usage, there are many services already, and the constraints - like not being able to run generic ML workflows - is inherent to the economics of giving the free service

[potent panther]

best projects are passion projects

[sacred mason]

no amount of passion can manifest a GPU

[potent panther]

Why are you so stubborn about the gpu anyway?

[sacred mason]

i am not trying to be stubborn about it, i am trying to have an illuminating conversation about the "why"

[dim sonnet]

i dumped passion into a few SD related projects and then joe penna DMd me a job offer and manifested the fuck out my new 4090 in the process

[sacred mason]

i have been trying to have an interesting conversation with you about the engineering for a while

[magic ravine]

that's big brain xD

[potent panther]

Even a 970 can run SDXL on comfyui so honestly if you feel the need to do stuff locally you can go extremely cheap nowadays (especially with turbo)
https://redd.it/1anrobl/

[dim sonnet]

on the budget end i'd aim for a 3060 but yeah a potato can work, just slow

[cold flame]

If you're paying anyways then you should probably take a look at Paperspace Pro tier. You get effectively unlimited runtime. I think they close the session after several hours, but you can just open another one. You also get some persistent storage. I don't use it, but I know that's what many (most?) people who were using Colab before switched to.

[potent panther]

mcmonkey do you have any idea why <segment> always makes the inpainted area softer?

[dim sonnet]

softer?

[potent panther]

yeah hold on

before after segment

happens on every SDXL models but I haven't used any SDv1 model on StableSwarm yet

instead of feeling like adetailer it's more like a blur filter

[dim sonnet]

i mean. It's notl ike there's a software blur added in or anything

that... might just be what da modal do

the first image looks like it's oversharpened

what was your prompt?

[potent panther]

it's Animagine

[dim sonnet]

oh lol

you have no segment prompt

that'd be why

it doesn't autofill that for you

[potent panther]

[dim sonnet]

maybe it should when your prompt is literally blank

[potent panther]

please do :D

[dim sonnet]

added

[potent panther]

yay

thanks lol

[potent panther]

@uneven moth Looks like a discord newbie got hacked

[uneven moth]

ty!

[hearty notch]

yessssssssssss!!!

[magic ravine]

Actually I should put them back inside better_dark_dreams but leav then out in cyber_swarm since latter already has other opinionated ux changes

@dim sonnet Do you think there'd be a way to have the left panel of swarm behave a bit like this https://youtu.be/L3isi26qy0Y ? Basically, start by having no settings and only pinning the nodes we want to modify. There would be way less elements in the side panel, which for now has too much unused controls. You would then save the list of pinned elements alongside the comfy worklow as a swarm worklflow.

Eventually I was thinking about having possibility to load multiple swarm workflows in their own tab and send images between those tabs. For example one for txt2img one for img2img etc. Basically a mix between invoke workflows, comfybox and stabilitymatrix inference (sent images).

I also made a template draft about possible UI changes. (last image)
Tabs like StabilityMatrix, Output like ComfyBox, QuickSettings like Invoke

[dim sonnet]

maybe? I don't think the raw comfy workflow interface should be a primary interface tho. It'd be nice to integrate it more thoroughly to the main tab only-when-it's-selected

[magic ravine]

In my ui draft, it's behind "show graph" button

[magic ravine]

While waiting for your answer about above proposal, I fixed this

[abstract rapids]

How do we use stablecascade with swarm?

[abstract rapids]

looks like from the stable cascade forum -- cascade is a vram monster-- great, just what I needed, another model i cannot run

[potent panther]

Cascade hasn't been implemented in ComfyUI yet

[abstract rapids]

"waiting is the hardest part"--tom petty...

[obtuse falcon]

there's two different sizes, the smaller one's not so bad

and because it's a staged model you could get clever with loading only subsections of it into the GPU at any one time

release-day code is usually much more resource-intensive than it needs to be; there's always tricks to be worked out for reducing usage

[hearty notch]

I saw it just now on a reddit post, are there any updates on it?

[abstract rapids]

Swarm is on drive E:, so I reset C: and installed . net 7 and 8 sdk and runtime. Swarm does not see .net 7 and gives me a link, went to the link 2 times -- down loaded it , installed it and still tells me I need .net 7. What else do I need?

[dim sonnet]

comfy is likely going to have a native cascade impl within the week (currently has some other things to deal with before getting to that)

[dim sonnet]

uhh make sure you have dotnet 7 sdk (64 bit) installed, it should show up in the folder C:\Program Files\dotnet\sdk or if you type dotnet --list-sdks in a command line

If you installed the runtime instead of sdk, that won't work, or if you installed 32-bit (x86) that also doesn't work

swarm's installer tries to autoinstall dotnet 7 for you, you shouldn't need to do it manually unless you're intentionally bypassing the install script

[dim sonnet]

my response hasn't really changed here - that can't be the primary interface but it could be neat as a thing that exists

it can't be a "show graph" button on the main interface

The main interface doesn't work as comfy graphs. It can use a comfy graph, and it can generate a comfy graph, but it doesn't behave as one.

what that means in practice is for example when you click between SDv1 and SDXL models, it completely changes what the final comfy graph will be, it uses different nodes to do things for different SD versions. When you check the video option that adds and routes entire nodes. If you type <segment:...> into a prompt that will lead to generating a graph with a lot of additional nodes in it to handle the segmentation processing

right now if you build a comfy graph and use that in the generate tab, it disables all of that functionality as it's hard to apply that logic to an existing graph without breaking things

[dim sonnet]

nice!

[dim sonnet]

@magic ravine so i made some tweaks - i fixed the bit where you have to close+reopen a group to see it - it was because you were matching input-group-open and display: "block" both of which aren't actually set on groups until clicked. I edited the js to always emit -open from the start, but the display:block isn't meant to be targetable so I just flipped that to target not-none
... I also put the quickclose buttons back for now. Would love to have em be less ugly, but removing entirely shouldn't be done by a theme - maybe ashtaka's idea of a middle click or similar.
I also pulled the PR for now cause I really like it and want it available, though there's still some things to fix - one is with the thinner space when there's two params horizontally, range sliders go funky:

and that's happening because the number box is colliding with the name. I've made it less bad by reducing the size of the number boxes to be close to the actual width the number displayed, but there's still a point where it just jumps awkwardly

also the user settings menu has gone funky

also for cyberswarm specifically the white-on-green is pretty hard to read

also, look at the top of the ? bubbles

that's a weird subtle lil shift

oh it's a padding value that appears to change nothing but cause the bubbles to break 0.o

fixed (just removed qbutton from that since it appears to be formatting meant for the seed buttons)

[dim sonnet]

fixed the settings ui

i wonder if maybe it should be excluded from the param flex? lotta settings tend to be longer and not fit as well when shrunk like that

[nimble belfry]

I was not going to ask about this because its not important but if you are working on looks....
If you have hint format set to hover it yeets the info box a mile away instead of displaying it where the mouse is or on the ? box.

[dim sonnet]

it's meant to be over on the right side of the params menu

if it was on your mouse it'd be in the way

@magic ravine ... why did you hide the close button on the comfy menu 0.o

[nimble belfry]

I see what you mean, would be in the way when you are trying to adjust something instead of reading info.
I enjoy the thing I'm trying to read to be where my eyes were already looking. I guess I will just keep it on click.

[magic ravine]

Oh okay, so basically normal UI does a hidden dynamic comfy graph. So what I was talking about for the quicksettings on comfy graph can still be applied to the case where you use swarm on a graph you made, right ? I'm wondering since it's one of the best use cases imo even if it's not the default one.

Though other points I made in the draft can also be apply, you could have a tab for the default dynamic workflow, and other tabs for custom comfy workflows. I also think most of the tabs right now are just settings and should be behind a settings button instead of being part of the main tabs. The UI template proposal still holds (though you would hide "show graph" button in default worklfow)

[magic ravine]

Nice for all the fixes!

Indeed cyberswarm is a bit different than better_dark_dreams in the sense that it also has opinonated touches: removed quickclose buttons, removed close button since didn;t know where to put it and having it look good, made the tabs corners straight and tabs colorful, etc.

For the ? bubble, it was not vertically centered there was a weird gap, that's why I introduced the padding, maybe it's different between browsers?

[abstract rapids]

had the 32 bit installed. ran 64 and back in business, thank you

[primal tapir]

/prompt

"Use Stable Diffusion to generate an image of a professional cyclist climbing a high mountain outdoors."

[dim sonnet]

oh, yeah, maybe - those tiny pixel alignment things tend to vary

[dim sonnet]

yes

I also think most of the tabs right now are just settings and should be behind a settings button instead of being part of the main tabs. no

the tabs at the top are generate (not settings), comfy editor (not settings), utilities (not settings), user (contains settings and other things), server (contains settings and other things)

[magic ravine]

You told me about comfy editor probably moving when using default so not gonna be there anyway, and generate obviously still stay there as well as utilities. I was talking about user and server which are imo a bit of a maze of settings, i find it weird to have two different settings tabs, and to have it next to utilities tabs.

[magic ravine]

When you say yes, does that mean you agree with everything i've said (except for the settings tabs) ?

[dim sonnet]

i find it weird to have two different settings tabs the reason they're separate is swarm is designed from the ground up to be sharable

while auto webui / comfy / etc. are basically ... you host it externally you get pwned for free rip

swarm is built so upon release you can share an instance with your friends, or if you're a company you can use it to make a webservice for generation, or whatever else

the former case is of particular interest to me personally, i used to try to host an auto webui instance for my friends but it was a nightmare to make that remotely safe and still at the end of the day had a "btw friends i trust you to please not fuck with the server settings kthx"

[dim sonnet]

i don't know what you're expecting from me in terms of responses with the repeated followups here. As I said before, the proposal makes sense as its own thing separate from the main tab

[magic ravine]

Oh okay, that does make sense. I wasn't talking about merging client with server settings anyway, my point was solely the fact that I found it odd to have settings as much present in the ui as workflow.

[magic ravine]

I'm not expecting anything, I wasn't trying to sound rude, i am just discussing possible improvements since I like the idea of the project. Nice to hear you think the proposal makes sense

[versed copper]

a sort of bug (i think): if using a refiner set to "use base", the refiner model does not get saved to metadata, so "reuse parameters" will not reset the refiner model to "use base"

or the vae

[dim sonnet]

the not being in metadata isn't a bug

but the reuse params part

i suppose it should be setup to reset all params to base before applying the image-specific params

[dim sonnet]

fixed - now resets to base first before applying, so should produce an exact match for the inputs the image had

[versed copper]

thanks!

[dim sonnet]

using the comfy workflow for Stable Cascade https://gist.github.com/comfyanonymous/0f09119a342d0dd825bb2d99d19b781c in swarm and import to gen tab works great:

sad lack of variety in outputs though :(

also doesn't currently have an easy one-click setup in swarm, as it's... 4 distinct models using completely different nodes, gonna have to see if it gets simplified and/or cascade gets popular or ignored before seeing about how to make clean easy UI for it

[potent panther]

mcmonkey you should try to talk to the guy that made a gui for gligen to see if you could implement it in Swarm
https://github.com/mut-ex/gligen-gui

[dim sonnet]

it's funny cause the UI is basically what my plan was for regional prompting in the image editor

just... regional prompting no worky goody until gligen came out now

and making UI work good is hard

is the dev in here

seems like they should be

either named "mut-ex" or "illustrious yard"... neither of which i can @ here :(

[potent panther]

could try opening an issue on the repo

also it would also be best to tell them that they don't have a license on their code

[dim sonnet]

@magic ravine i'm testing on both firefox and chrome and you have a bunch of tiny alignment things that appear to not do anything - like not even just, not on my one browser, but according to inspector literally isn't applied anyway

not sure what's up with that but ima just remove them while i work on figuring out what to import to main or not

I made a few tweaks:

• removed redundant minor alignments
• naturally adjusted how/where the css is applied
• set the minimum width before it does multiple in one row a bit wider, otherwise it gets cramped and weird at small widths:

• use a var for the background color rather than being statically defined
• any functionality changing alterations (eg hiding buttons) weren't kept naturally

and now it's integrated to the main basis and applies to all themes rather than being an alternate of Dark Dreams

eyesear white remains an awful theme. One day somebody who cares about light themes will come along and replace it lol

also i integrated cyber swarm more fully, it's in the installer listing and all

[potent panther]

you could base yourself off of Adwaita

[magic ravine]

Weren't them just overwritten by site.css since not writeen !important?

[magic ravine]

Nice nice

[magic ravine]

Fellow linux user ❤️

[potent panther]

Yesh

[magic ravine]

No thanks 😩

[toxic shale]

@dim sonnet
Are there any easy ways of hosting/sharing infinite grids like github or other options?

[potent panther]

mcmonkey what about moving the toggles to be after the text?

like at the right of the option like most OS do

[abstract rapids]

could you change temp of gpu to Fahrenheit and Celsius?

[hearty notch]

Hi @dim sonnet any idea why I am getting this? I did install donnet, the latest one, but still getting this error when I try to install: App: C:\Users\mmm\Downloads\SDAI\StableSwarmUI\src\bin\live_release\StableSwarmUI.dll
Architecture: x64
Framework: 'Microsoft.AspNetCore.App', version '7.0.0' (x64)
.NET location: C:\Program Files\dotnet\

The following frameworks were found:
3.1.10 at [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
8.0.2 at [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]

Learn more:
https://aka.ms/dotnet/app-launch-failed

To install missing framework, download:
https://aka.ms/dotnet-core-applaunch?framework=Microsoft.AspNetCore.App&framework_version=7.0.0&arch=x64&rid=win-x64&os=win10
Press any key to continue . . .

[dim sonnet]

yeah you can shove em anywhere, eg i have my old demo one here https://github.com/mcmonkeyprojects/mcmonkeyprojects.github.io/tree/master/short_example which corresponds to this page https://mcmonkeyprojects.github.io/short_example/index.html

you can also save images from within the page at any time to share that way if preferred

[dim sonnet]

what?

[dim sonnet]

hardware temps are always in celsius

i'm american and it'd be very weird to do hardware in fahrenheit

it's not weather it's coretemp

it kinda coincidentally works the same too - in fahrenheit we understand 100 F to be the "it's too hot" point, and in hardware temps 100 C is the "it's too hot" point too :D

[dim sonnet]

You downloaded aspnet core 7 runtime (?)... that is not the dotnet 7 sdk

you also seem to have v8 and v3 installed, neither of which are v7

if you use the standard swarm installer it will literally install dotnet for you and you don't have to worry about it

https://github.com/Stability-AI/StableSwarmUI?tab=readme-ov-file#installing-on-windows

[hearty notch]

Sadly I have to install them manually, company restrictions

But it’s working now! Many thanks

[dim sonnet]

https://github.com/Stability-AI/StableSwarmUI/discussions/11#discussioncomment-8521943 Cascade native support in Swarm!

[potent panther]

Most user interface in OSes have toggles button on the right

[dim sonnet]

oh do you mean on parameters?

.. wh

why

why do i

have them on right for groups and left for params

huh

i mean ngl the clean even row is kinda nice

[potent panther]

I just thought that putting them on the right would look more space efficient

[dim sonnet]

?

[potent panther]

Oh I meant toggles on the right like on iOS where it’s sticking to the right side

My brain is kinda fried and I don’t know how to explain

[potent panther]

Btw https://huggingface.co/KBlueLeaf/Stable-Cascade-FP16-fixed

[cold flame]

If I'm understanding correctly I think you mean have the labels aligned left and the toggles aligned right.

[potent panther]

yeah

[potent panther]

Gravity Blue has an invisible arrow

[potent panther]

After 132 gens today my system started to lag and it was my ram being filled by comfyui

at least I think it's comfyui

[potent panther]

welp it sucks when I can't reproduce

[potent panther]

@uneven moth

[dim sonnet]

oop, fixed

[potent panther]

Oki

[edgy adder]

Hi All, I am newbie and got this UI fresh installed. When I try my first time generate, it show me refused or failed.

[edgy adder]

When I check in logs, I found nothing useful.

[dim sonnet]

swap 'view' to 'debug' at the top of the logs tab

(and please do share what the error ends up being - I'm trying to make the displayed errors in UI clearer. This one having nothing at all is weird so definitely something to add checks for)

[edgy adder]

Thanks for help. But I just indicated that before change it from UI, I need modifying the Settings file first to enable "Debug" level logs

[edgy adder]

I sill can't change the view to Debug. But I see message from console.

[edgy adder]

I found these message

And I added "--disable-cuda-malloc" to ExtraArgs to solve it. Finally I can gen image succesfully. Thanks.

[dim sonnet]

you don't need to modify the settings file for that

you can just click it on the page

[edgy adder]

I tried but no response from the dropdown menu

[last tinsel]

Can I use Cascade in Swarm? Thank you

[raw laurel]

Yes

https://github.com/Stability-AI/StableSwarmUI/discussions/11#discussioncomment-8521943

[last tinsel]

Much appreciated!

[raw laurel]

This inspired me to try it out. What settings should be used in swarm for 2x resolution upscale with Cascade?

[raw laurel]

huh

How to change compression on SwarmUI? My problem arose when trying to use the workflow in generate tab with different compression setting

[dim sonnet]

ngl i genuinely don't know what the significance or impact of changing that value is so i didn't make a param for it yet. if you do a custom workflow you can of course do whatever you want with it

if it's useful to control i can add a main param for it

[raw laurel]

It adds seemingly much more sharpness according to quick testing

Difference in whiskers is massive

[sacred mason]

it should be 32 by default

[raw laurel]

Not necessarily. Some people say it adds generation time

[raw laurel]

How should I do the custom workflow? If I go to workflow editor, import from generate tab, do nothing and press use in generate tab it fails on generation with error
2024-02-22 01:01:45.497 [Warning] ComfyUI-0 on port 7821 stderr: RuntimeError: pixel_unshuffle expects height to be divisible by downscale_factor, but input.size(-2)=19 is not divisible by 2

[dim sonnet]

it- wh

oh, do you have the wrong model selected

between B and C

it might be sensitive about that

it autocorrects when used directly but import-n-back might attach the wrong one

[raw laurel]

B model is loaded

[dim sonnet]

try selecting C

C is the initial model so it's the main one

B is the second stage

compression 32 => 11.1 seconds,
compression 42 => 8.6 seconds

slight speed difference but not a ton

[raw laurel]

lol yes it seems to run now!

Is the seed used in b and c same through the UI? And amount of steps?

[dim sonnet]

rn step count is same (it, uh, probably shouldn't be?), seed is offset (second sampler is + a few from the first)

I think it probably might actually be better to have Stage C as the model and stage B as the Refiner, and then all relevant Refiner settings apply

[abstract rapids]

From comfiuiworkflow discord. https://github.com/ComfyWorkflows/comfyui-launcher, it's still in alpha but looks fantastic. Does anyone know of another workflow manager? Getting hard to remember them all from just names.

[tulip lion]

anyone here know if its possible to get the stableswarmui to run behind a reverse proxy?

[raw laurel]

yes

[tulip lion]

this is the current messages I get when I do try to generate an image when accessing through the reverse proxy...

[Error] [WebAPI] Error handling API request '/API/GenerateText2ImageWS': Invalid request method: GET

and the UI message is: Failed to send request to server, Did the server crash?"

[tulip lion]

apache2:
ProxyPass / http://localhost:7801/
ProxyPassReverse / http://localhost:7801/
ProxyPass / ws://localhost:7821/
ProxyPassReverse / ws://localhost:7821/
ProxyPreserveHost On

[raw laurel]

@dim sonnet pls implement the compression setting! I'm not very happy with the default, 32 for example produces much nicer results

[potent panther]

mcmonkey you should add a noise multiplier to segment since it's img2img

[edgy adder]

Now I found what's the matter. The drop-down menu will not respond in Chrome. But no problem in Edge.

[dim sonnet]

I could give comfy workflows a proper browserthingy with icons/descriptions like models/wildcards/etc. have if you want that?

[dim sonnet]

you gotta proxy the websockets too

[dim sonnet]

are you using 1Password? I've been told the 1password chrome extension breaks dropdowns

temporary workaround is to hold SHIFT while clicking

[dim sonnet]

a what

[tulip lion]

Tried. I’ll revisit tomorrow. Too late now

[dim sonnet]

might actually make 32 the default but ye. I just noticed, img2img comfy reference used 32 and txt2img ref used 42, so even comfy prefers 32 when quality matters.

i wonder if i can set the parameter to show up exactly only when you're using cascade models and not when you aren't

added it for now under Advanced Sampling

and the default is 32 now too

[potent panther]

more noise

to add more details

noise multiplier is just the name of the feature inside auto

[dim sonnet]

is it just the Init Image Reset To Norm param in swarm

which in practice just emits this

if so i can easily add that to segment ye

[edgy adder]

05:05:04.945 [Debug] ComfyUI-0 on port 7821 stderr: 0%| | 0/40 [00:00<?, ?it/s]/opt/StableSwarmUI/dlbackend/ComfyUI/venv/lib/python3.10/site-packages/torchsde/_brownian/brownian_interval.py:608: UserWarning: Should have tb<=t1 but got tb=1000.0001831054688 and t1=1000.000183.
05:05:04.945 [Debug] ComfyUI-0 on port 7821 stderr: warnings.warn(f"Should have {tb_name}<=t1 but got {tb_name}={tb} and t1={self._end}.")

[edgy adder]

Seems bug about floating point rounding

[potent panther]

that should be it yeah :D

[abstract rapids]

that would be wonderful.

[raw laurel]

is there a way to save all the steps automatically when generating?

[undone basin]

Great question having the same problem, my init image is low res, it's downscaled

For example when try to get resolution from Init Image, my final result still will be only 512px high

[dim sonnet]

0.o why

[dim sonnet]

wh... what?

[undone basin]

This is just simplified example from my big workflow, but it is good illustration.
So in the workflow image i sent, I'm trying to get width and hight values from "Init Image" in StableSwarm that (as i understand) send image to the "Load Image" in my ComfyUI workflow.

So StableSwarm did sent my 1280-720 image to load image, then I got my width and height from that image and used it in Empty Latent image
In theory my result should have 1280-720 size
But in reality i recieve 792-512 image all the time
No matter what Init Image is

But apparently this way it works and it have same res as Init Image

If i will do like that

My result again is 792-512

[dim sonnet]

if you're using the Generate tab, init image will automatically resize to the current Resolution setting

[undone basin]

When don't have Empty Latent image to specify resolution.
It sets kinda default resolution with 512h max

My whole workflow don't have specific place where resolution is specified.
It's all based on resolution of uploaded image

So is it possible to use resolution from "Init Image"?

It's just face detailing,
I upload hi res photo, automatically detect faces, refine them with 512-512, paste it back and receive my high res photo with refined face

[dim sonnet]

if you have an image input that isn't the main Init Image iirc that will just do raw size

offhand the easiest way to do that rn (pending future plans for a management interface to control it properly) is just have a sacrificial load image input that does nothing but claim the init image slot

[undone basin]

My workflow works as intended and it great. I change "Init Image" photos from Generate Tab and it works.
They are being used in my workflow and I get results. The problem is only a resolution.

[undone basin]

Got it! Will try it now. Thank you!

[undone basin]

It feels like there is somewhere in code somehting like
"If no resolution param found just do this standart res"
Which kinda also could work if i can just fix in code this standart resolution to be bigger

[dim sonnet]

yes, default resolution is attached to the model

you're using and SDv1 image so default is 512x512

[undone basin]

Ahhh

[dim sonnet]

you can edit the model metadata to change that if you want

just click the

[undone basin]

Ohhhh wow cool ❤️

[undone basin]

Sorry to report that didn't help. Tried to reload everything, tried different param.
Maybe there is another way how i can fix that? Sorry to bother you. Just it's driving my crazy, can't do anything util it solved

Changing model to juggernautXL with un edited metadata also didn't help

So I attached workflow in this .png inside

When you will upload any hires image as "Init Image", it will give low res result back

[undone basin]

Finally i found the solution and maybe it will help somebody who will have same problem. Omg satisfaction! 😅

This node solved my problems. Edit: ok no actually it did not it just upscaled previously downscaled image and now it's blurry

[dim sonnet]

^

[undone basin]

Well now it high res yes
but now its not using "Init Image" it's just rendering like normal ComfyUI whats inside of "load image" node

I want to show to my client (hopefully) the prototype of the app, where people upload thier photo and get ready result. Whihtout drawing masks or writing prompts.

Stable swarm is perfect to send to people who don't know how to use comfyui
And also I don't have to setup a server, I just can keep my PC on

StableSwarm is Awesome, i will setup cloudflare and will just send them a link where they will have to only upload photo and press generate. And hopefully work with me. I'm not dev so i'm just testing concept here, presenting how it might work in a future

[dim sonnet]

essentially the suggestion i gave as a placeholder for until swarm has better control over it is do this:

which then makes the params like so

[undone basin]

Did it work for you?
I tried it and it's still giving me same results with low resolution

[raw laurel]

I took part in AI generated image competition that had rules of no inpaint/outpaint/starting image and required proof of the generated image by providing steps

[small moth]

Can I connect api url of a cloud Comfyui on stableswarm?

[dim sonnet]

technically yes, but it's preferred you run swarm in the cloud and connect that

[small moth]

Ohh... Okay 👍

[dim sonnet]

oh, and, docs about that: https://github.com/Stability-AI/StableSwarmUI/blob/master/docs/Using More GPUs.md

[small moth]

Thank you very much @dim sonnet ... I would really like to try connecting to external comfy using local swarm.

If any persists I'll just move swarm to cloud.

Once again thanks 👍

[dim sonnet]

connect external swarm to your local swarm

only use external raw comfy if you're extremely restricted on what you can run

it will cause limitations on what you can do with it if you don't have swarm node set and all that

[small moth]

Okay will do that. 👍

[winter forge]

Im a comfyui user with 2 GPUs on my local PC, i can switch to swarm without a problem? The comfy side of it works the same as the stand alone?

[edgy adder]

You could only use 2 GPUs simultaneously, when running 2 asynchronous jobs

[winter forge]

Yeah,im aware, thx for the tip

[nimble belfry]

Yes the comfyui part works the same.

[bronze shale]

speaking of users with multiple gpus (backends with comfy) I noticed that both backends are not being used to generate multiple images anymore unless I specify the exact backend each time (split in half, queue both backends seperately)

[versed copper]

having a problem that didn't use to occur
changing model on the fly while "generate forever" is running doesn't seem to release vram, generation time goes to 3x of normal because 24 gb of vram are occupied.
release vram button doesn't do anthing, only stopping generation and restarting fixes it

[dim sonnet]

are you sure that you're (A) generating more than 1 at a time, eg by setting Images to 2 or more, and (B) that the model has loaded on each backend? if you're only doing exactly 2 at a time by default it only loads on one backend (as it's just faster to do that), so you can either generate more images at once to tell the system it needs to load, or just click the menu next to the model and hit "Load Now" to tell it to immediately load on all backends

[bronze shale]

This was working for me properly until my latest restart/update

[dim sonnet]

that, uh, sounds like it'd be an issue on the comfy end if anything? comfy's entirely in charge of managing vram

[versed copper]

honestly, no idea 🙂
i just noticed this is now happening, and didn't a week ago

[dim sonnet]

oh but

if you go to user -> user settings

can raise that setting

might fix it if that delay is a bit higher

[bronze shale]

that fixed it. default might want to be set a little higher in source

thanks for finding that!

Still seems a little quirky, but I know development is happening so thanks

[versed copper]

a crashed and restarted comfy backend has the same effect, VRAM not released

[dim sonnet]

If comfy crashes and restarts it'll definitely release vram

if it errors out and doesn't restart then yeah it sticks until you restart

[abstract rapids]

in the generation screen for cascade --- how do we set the image size?

[potent panther]

@uneven moth :D

Compromised account

[dim sonnet]

same as you do on any other model, just select your res?

[novel charm]

is it possible to run stableswarmui as frontend-only? (so only a remote backend is used)

[dim sonnet]

yep

[novel charm]

is that the default behaviour, or do I need to pass a specific flag? I'm a little confused about what the different LaunchModes do

[potent panther]

StableSwarm is a front end

It doesn’t have anything such as ldm or Diffusers included

You can add a ComfyUI backend using the self start option in the server settings of the Swarm’s web interface

[scarlet shore]

anyone using multipge gpu´s to render a single SVD?

iam building my new rig tomorrow and preparing everything for multi gpu use

[scarlet shore]

anyone?

[dim sonnet]

it's one GPU per video

SVD has shared attention across the full video while it's generating, it can't split up

[scarlet shore]

oh damn

it cant be that complicated, dont know why there isnt anyone to make this happen

[nimble belfry]

If it was easy it would have already been done.

[scarlet shore]

i wasnt hating, was just saying

but ye, u right

[dim sonnet]

ye^

and trust me i'm no stranger to hacking ways to make models use multiple gpus at a time quicker https://github.com/oobabooga/text-generation-webui/pull/2100 lol

[cold flame]

@dim sonnet I heard you were looking for a way to index large numbers of images with metadata. This did it for me on 300k+ images stored on a HDD in a few minutes. https://github.com/SirVeggie/sd-browser

[cold flame]

made by @languid blade

[dim sonnet]

ah

i don't think swarm's is that bad, at least not on ssd

it's pretty efficient, i just don't build a dense multi-folder index

[potent panther]

that's hdd stats I'm pretty sure

[cold flame]

I didn't time it, but it felt like no more than 5 minutes for all of my 300k images and they were definitely stored on a spinney boy scattered a bit since I just symlinked the output folders from multiple uis into my A1111 output folder. So that I can access them all from the same place.

[languid blade]

I kinda want to test how fast it is on nvme, might not actually be that much faster

[dim sonnet]

swarm will build metadata store as fast as your drive can supply it, everything past that point is multithreaded quick C#

it's just a store to avoid having to reload the original files tho, not a sorting index

[languid blade]

same

[dim sonnet]

oo

from past tests on different data, sort funcs become not-so-realtime around when you hit the ~million scale of datapoints to sory

[potent panther]

it's extremely slow for me and I only have 12k images in my swarm folder :(

compared to https://github.com/zanllp/sd-webui-infinite-image-browsing although that thing makes huge .db files

[dim sonnet]

if you're on spinning rust i can probably make it a lot quicker for you

i haven't actually tested against HDDs at all but there's a lot of redundant fileops that are probably what kills it on hdds

[potent panther]

oh i see

that'd be great if it does improve the speed

[dim sonnet]

pushed commit that adds new setting:

prior to this commit, every single image file got an mtime check if it's been more than a data since it was added to database

now it only does that to a random % of the images it sees

and you can set it to 0 to just not do that

on an SSD randomly reading thousands of mtimes isn't bad, but on an HDD i bet that hurts

on my local setup the only lag in image history is literally on the browser side generating the list of divs to hold the images lol

probably should have it not prepopulate those past a certain point (it already doesn't load the images, but it does load placeholders for where the images will be)

(the displayed images in history view are thumbnails stored in db not direct view of the original)

[potent panther]

how do you push stuff that fast mygod

[dim sonnet]

the other potential speed bump that can be overcome is it does query the filesystem to list paths, it doesn't read that from datastore

but changing that would make it not notice new files added by external programs

[potent panther]

oooooh yes it's so nice

now swarm doesn't hang when loading the history

[dim sonnet]

:D awesome

[dim sonnet]

that was painful to run the first time as it scanned the metadata of 10k images

you can set it to a massive number, and set validation chance to 0, and just let it spend a while indexing the first time it runs

there's no, uh, proper optimization/smart handling of large index atm it'll just freeze the image history listing and churn your CPU & drive til it's done

the overscan count is how many metadata entries it'll pull into a list before sorting if you have sort by Date set

if you use sort by Name it'll still only use your max images in history value

(that value is still used for Date for how many actually get rendered by the browser max)

hypothetically i could add sorts for other values but i'm not sure of any others that would be relevant as a Sort key, most things you'd only want to Filter by normally I'd think

[potent panther]

noway

[dim sonnet]

I'd @ everyone who asked, but, uh, i don't remember who

so

people will find out when they update i guess lol

[potent panther]

:D

[languid blade]

you can add a file watcher to get events when files were added/deleted/etc

[magic ravine]

Hello, I can't seem to find which part of the code is reponsible for building the ComfyUI workflow api json

[dim sonnet]

https://github.com/Stability-AI/StableSwarmUI/blob/master/src/BuiltinExtensions/ComfyUIBackend/WorkflowGenerator.cs

[magic ravine]

Thanks!

Woah it's a BIG file

[dim sonnet]

yelol

there's a lot of space usage added by random side features

probably will need to reorganize that a bit eventually

to not all be in one file

[magic ravine]

you do you

[novel charm]

@dim sonnet re [here](#🧣｜comfy-ui message):

no, Swarm has frontend (webpage), and a middle (queue, engine, etc), and a backend (ComfyUI usually)
the part you're referring to that does job allocation is the high-performance C# middle engine, not the frontend

Is there a way to run these components in separate containers?

I don't see (based on the docs I've read so far) how these could be scaled independently - from what I've seen, the backend servers need to be registered in advance. If I added a load balancer in front of scaled backend servers (an ASG or replicaset) and used the lb address as my backend, would the queue know to send concurrent requests to it?

Equally, if there were multiple frontends running, could I have them all use the same queue?

[dim sonnet]

if you have a custom scaling solution, you can just hit API endpoints on swarm (AddNewBackend + EditBackend + DeleteBackend) to add/remove/etc them at will, (or write a short C# extension to swarm to have it manage this in-process). If you have an entirely separate load balancing solution for comfy already you can add that as a C# extension as a new backend type that just calls the comfy backend code but points at your alternate url and declares its concurrency support, it's relatively simple to do.

frontends are webcode, ie they run on client browser, swarm already manages and balances multiple frontend users cleanly

[novel charm]

So is the intention that the swarm instance itself is never meant to scale? A CDN distributes the FE, swarm handles the queue and the backends register themselves with swarm (either directly, or using a custom lb)

[dim sonnet]

ye

you'd have to be operating in the many thousands of images generated per second range if not more before Swarm is the bottleneck... at which point you can just, like, have a couple separated instances or something

but you're going to run into the "it's way too expensive to buy that many GPUs" bottleneck first

[novel charm]

Perhaps bottlenecking isn't a concern - but for high-availability, is it possible to replicate the queue across instances?

[languid blade]

nvm

[dim sonnet]

you can have multiple instances of swarm hooked up to the same set of backends or an overlapping set. It won't share a queue but it also won't break. If both are used at the same time they'll get a little confused on time estimates but otherwise function fine, if it's kept as simply a failover (instance 1 fails, redirect users to instance 2) it should work more or less flawlessly. That is not however equipped to store&share user data (eg custom presets, user settings, etc) properly atm (though it is planned to add capabilities to sync multiple instances cleanly)

[sacred mason]

would you be interested in some changes to address these issues?

[past arch]

Hey there. Quick question regarding stableswarm UI and inpainting: Does it not support this feature altogether, or is it just stablecascade / sdxl that doesn't support it? I attempted to draw a black bar over the object to be removed, but after generating, the bar remains and the image quality deteriorated. Any insights on this would be greatly appreciated :)

(left is the original image with the black bar painted over it, right is the image generated with that)

[dim sonnet]

(Note in advance: the image editor is a preview and not super friendly yet, takes a few more steps than you'd expect)
So what you're gonna wanna do there is:

• open the image in image editor
• click +Mask to add a mask layer, fill it in black (ie tell it to not edit anything but the parts you want)
• click +Mask to add another mask layer, draw a white line where you want changes
• enable the Init Image param and set the Image Init Creativity and the Reset To Norm values high (higher on either = more creativity replacement. The default 0.6&0 is to refine an image rather than total replace. If both are 1/1 it will do an aggressive total replace)
• then generate

[past arch]

Thank you very much for that description :D
The result looks much better now. However it lacks some context awareness. I guess I did something wrong here.

[dim sonnet]

lol

you can reduce the reset/creativity a bit to give it more of a clue

but also bear in mind SDXL Base is not an inpaint model

there's an ooold old SDv1 inpaint model around, which is, not stellar but does pay attention to context well

there are also controlnets for XL that sorta do inpaint but aren't as smart as the old one was

[past arch]

Thank You :>
I fine-tuned the values to get a much better version. The bar is still visible but I guess for SDXL Base not being a inpaint model it is as good as it gets. Would you recommend using that old SDv1 inpaint Model to fill the gap? I'd guess it would look really out of place with the picture quality.

[dim sonnet]

ooh, actually there's an SD2 inpaint model i forget existed lol, probably better than the v1 version

https://huggingface.co/stabilityai/stable-diffusion-2-inpainting/blob/main/512-inpainting-ema.safetensors

can give it a try, it'll probably do better tbh

[past arch]

Thank you for the suggestion. I've tried it but it produces even worse results. Changing "Init Image Reset To Norm" only affects if the original object I wanted to remove is being generated or nothing is generated (see image). I'm happy with how the last attempt (still with sdxl) turned out, so I'll probably just leave it as it is.

[past arch]

I've tried removing an object the same way, but with stablecascade as the model (for both generating the original picture and inpainting). This only causes an error: https://hasteb.in/vMnUwSJvTiQgZ8Z

[sacred mason]

differential diffusion is needed to make smoother borders. it is in a PR for comfyui

there isn't anything you can do

it's a scientific problem

[toxic shale]

is there some (optional) way to reuse parameters including model + lora? (basically a literal 1:1 replication of said image, which overwrites all settings)
something like a shift+click?

[dim sonnet]

that, uh, happens by default?

[toxic shale]

I feel gaslit XD this never worked on any of my pcs

[dim sonnet]

i tested just now and both model and LoRA got copied over when i hit Reuse Parameters on an image

if you asked in theo ther direction, how to exclude those, I'd tell you about this User Setting:

[toxic shale]

ahh! I think I see the issue. could it be that it just doesn't work for grid generated ones?

[dim sonnet]

uh gridgen doesn't do anything differenty about metadata

[toxic shale]

x_x

[dim sonnet]

if you have or had a custom comfy workflow loaded that might do something weird

[toxic shale]

is it because I'm moving them to a new folder?

[dim sonnet]

huh?

[toxic shale]

I'm misusing the image history as a sort of inspiration & memory thingie, and just made a bunch of custom folders like "examples" and "themes", and download, then save the images there. they show up in image history, and show all the metadata as well

but I think that breaks the model/lora loading part

it works on some, but not all x_x I'll figure it out. probably some issue on my side.
definitely works 100% for all standard uses. just tried it

[dim sonnet]

are you possibly moving models around too? It won't be able to apply models if the model name changes

(folder location is effectively just part of the model name)

[potent panther]

Interesting thing https://github.com/comfyanonymous/ComfyUI/issues/1992#issuecomment-1817797912

Speeds up loading checkpoints in comfy with a one line change?

I haven't tried

[dim sonnet]

so in short: yes but actually no

it speeds up loading on certain hardware

and makes it worse on other hardware

the direct data map that the regular method uses is a major speed boost upgrade in safetensors, that they added to make it go faster

offhand the main case i'm aware of where direct mapping would be slower is something like network drives, I wouldn't expect normal local drives to have issues

[potent panther]

is comfy aware of this?

could be a good idea to have a toggle

[dim sonnet]

there's no way to ever know,,,

is comfy even real? who knows

[dim sonnet]

@cunning holly

[potent panther]

does that mean comfy is a product of our imagination?

[cunning holly]

Yes I'm aware but the solution is to write a better safetensors loader not this

[potent panther]

Why not add an argument for the people that need it so they don't rely on hacking up the utils.py for now?

idk how many people I've seen make their life harder by making hacks in files that could cause conflicts after a git pull

[cursive prairie]

Not all of us know how to write our own safe tensors but a lot of us have strong hardware

[vestal meadow]

/Two robots dancing.

[toxic shale]

is there any way to open the 'edit image' editor directly?

Basically had lots of people send me links to the "instant drawing to ai" sites that recently popped up. And all of that can be done directly in stableswarm. but for the sake of explaining it to other people, it feels odd to say "generate some random image, then click on edit image, then draw all white over it"

just a short example of how damn well this works

(super useful for making DND scenes - especially since it's combinable with loras - as seen)

[vital thorn]

#✍🏼｜rules-and-tos ins

/game_statsdfdfbhcc

[wary whale]

What is this plugin?

[potent panther]

Image history

[dim sonnet]

ye, quick tools over on the side

[toxic shale]

thanksies! 😄

[toxic shale]

• Open Empty Image Editor
• Load any of the 4step lightning checkpoints
• write a rough prompt for what you want
• enable 'Generate Previews' in the dropdown next to generate
• have fun drawing

[wary whale]

Thank you very much!

[potent panther]

???

[potent panther]

Rip mcmonkey

[outer cedar]

Is it possible to train a Lora in stable swarm ui?

[potent panther]

You can extract but there no utilities for lora training

Although I recommend https://github.com/derrian-distro/LoRA_Easy_Training_Scripts/tree/dev

(It’s being rewritten from the ground up in the dev branch)

[dim sonnet]

i have a great self-standing clean impl of SD3 for testing and referencing now :D

(for sharing with private partners before launch and sharing publicly after launch)

that's what took my codin' hours the past few days

[obtuse tinsel]

Will you be my private partner? Pweeeese

[dim sonnet]

Yeah, sure, just go start a business, make it very successful in the AI space, make it directly relevant to stability where we'd want to share things in advance of public release with you, then form a private legal agreement with the legal & business teams here, then get back to me :)

[obtuse tinsel]

Sounds good, I'll get back to you in an hour with the paperwork! 😄

really though, i'm excited you're getting to get things ready before launch, instead of at/after!

[mystic granite]

I'm dumb, is there a way to add adetailer functionality to swarm?

[dim sonnet]

add to the prompt <segment:face> a pretty face or whatever. It's ClipSeg matching so it's not limited to just faces unlike adetailer

[mystic granite]

[dim sonnet]

can do multiple segments too if you want easily

[mystic granite]

and this would be chronological?

so anything after segment:face is for the face?

[dim sonnet]

yep

[mystic granite]

do i need to repeat any quality tags?

or like modifier tags

[dim sonnet]

you can also leave the prompt after the segment blank if you want to just use the original prompt

[mystic granite]

is there a way to control the denois factor of clipseg?

[dim sonnet]

generally the best is to use a quality prompt specifically focused on the face

[dim sonnet]

yes, just type <segment: and it will popup documentation on-screen telling you how to use it

[mystic granite]

ooo

[dim sonnet]

"the denoise factor" would be the creativity value, ie the first number at the end there

[mystic granite]

this was very helpful! thank you!

one more thing if possible!

i often do quite high res gens so it might be that

but

often times when im genning it's all fine and dandy

but then randomly maybe I might accidentally leave a video playing

or a gif plays

or something in motion

and the whole thing just hangs

I have to restart the whole swarmui

and restart the gen

it happens on all UI I have used

so that's no surprise

but it happens quite often on swarm

and quite soon too

is it a memory issue?

[dim sonnet]

0.o

i haven't heard of that before

[mystic granite]

o

if it helps, it only occurs if i run it locally

I generally run it remotely on my work server

so it rarely happens there

or basically never

[dim sonnet]

next time it happens, go to Server -> Backend -> hit the Restart button there

[mystic granite]

since all I do i work on those servers

plus the server is usually CLI

but at home on my local PC

which runs a 4090

[dim sonnet]

I'm guessing that'll work just as well as restarting swarm itself, ie swarm was never the issue it was the backend

[mystic granite]

it'll sometimes lock up

like

even the cancel button won't respond

[dim sonnet]

I've had things struggle a bit when there's too much going on but never a total freeze

[mystic granite]

well

maybe it's not a total freeze?

but I've walked off

done cooking

and got back

and it was still trying

[dim sonnet]

also next time that's happening, open task manager

and look at what resource(s) are in use

[mystic granite]

it like slows down the ENTIRE PC

like moving the mouse is real laggy too

[dim sonnet]

yknow what, that does sound like system RAM

[mystic granite]

I've had it completely freeze the entire PC before

that's wat i thought too

[dim sonnet]

i've had something like that happen when beta testing SDXL way back in the day before we optimized anything

[mystic granite]

I'm mostly a software blob

but I do think it's that

[dim sonnet]

i had only 32 GiB of RAM back then and it overloaded it

[mystic granite]

it's curious is all

since i rarely gen locally

why would i gen locally when i have access to a TPUv5

[dim sonnet]

when running locally you miiight be happier with --highvram in the ExtraArgs of the comfy backend

[mystic granite]

can this be applied in settings?

[dim sonnet]

it'll use more VRAM naturally, but you're running a 4090 so that might be fine

[mystic granite]

or only in the bat

[dim sonnet]

Server -> Backends -> edit the backend

[mystic granite]

or config files

[dim sonnet]

it's all in UI

[mystic granite]

gotcha

do i need the -- part

[mystic granite]

is this to avoid memory leaks

[dim sonnet]

yes

[mystic granite]

okok

[dim sonnet]

no

[mystic granite]

[dim sonnet]

comfy by default manages memory balance between system RAM and VRAM

but this assumes you're more VRAM limited than sys ram

[mystic granite]

oh

[dim sonnet]

if you have lots of VRAM and little sys RAM, telling it explicitly to use more VRAM will reduce the sys RAM hit

[mystic granite]

really?

so what ur saying is

it underestimates my vram

[dim sonnet]

that's the positive interpretation yes lol

[mystic granite]

how come it doesn't occur on my TPUv5 then?

well not mine

DeepMind's TPUv5

but mine

[dim sonnet]

the other interpretation is... why do you not have enough sys RAM but you have a 4090 lol

ram is cheap yo

[mystic granite]

I have 32 GB on my local machine

[mystic granite]

5950x3D

i can't be bothered upgrading

when all my work

is done on a remote server