🐝｜swarm-ui

[bleak cedar]

🐝

[dim sonnet]

SwarmUI:

Website: https://swarmui.net/
GitHub: https://github.com/mcmonkeyprojects/SwarmUI
Install Guide: https://github.com/mcmonkeyprojects/SwarmUI#installing-on-windows
Basic Usage Guide: https://github.com/mcmonkeyprojects/SwarmUI/blob/master/docs/Basic Usage.md
Follow For Feature Announcements: https://github.com/mcmonkeyprojects/SwarmUI/discussions/1
Dense Changelog: https://github.com/mcmonkeyprojects/SwarmUI/releases
Swarm's dedicated Discord: https://swarmdiscord.mcmonkey.org/ or inv code q2y38cqjNw

[dim sonnet]

hey @toxic shale @untold hearth and... idk who else to ping, but, channel here!

[untold hearth]

wup ho

[uneven moth]

Party time!

[dim sonnet]

i got the hive, who's got the honey?

[junior sierra]

[uneven moth]

She does!

[tranquil jetty]

Joined the Hive

[bleak cedar]

https://tenor.com/view/bees-oprah-honey-not-the-bees-release-the-bees-gif-5375443

[toxic shale]

https://tenor.com/view/mine-finding-nemo-mine-mine-mine-thats-mine-lulugifs-gif-20827668

congrats on having your own channel! XD

[thorn harness]

[dim sonnet]

I wrote an initial draft of a full noob-to-pro SwarmUI basic usage guide https://github.com/Stability-AI/StableSwarmUI/blob/master/docs/Basic Usage.md
Doesn't cover everything but covers enough that if you manage to follow it all the way through you'll be able to figure the rest out

Which naturally starts off by pointing out the fact that the UI is full of self-explaining features like the ? doc buttons so the guide is a bit pointless but~ lol

[signal perch]

Less than 12 freaking seconds on my RTX3070 what the hell is this sorcery ?

[dim sonnet]

gotta give credit to @cunning holly for the incredible speed!

[signal perch]

I'm using ComfyUI since a couple weeks with XL, but I don't think that I had THIS speed before, anyway for an Alpha it's very promising as it impressed me ! Upscales seem to be broken, but base images on my Nigerian woman are stunning !

[dim sonnet]

what's wrong with upscaling?

[signal perch]

I may have badly configured it, but my upscales look like this

[dim sonnet]

0.0

what were your settings?

[signal perch]

Oh wait I put Refiner Upscale on 2x thinking it was the normal way of upscaling, I'm generating a new image with refiner upscale to 1 then will launch a 2x upscale

Once an image has been upscaled it seems that next generations are upscaled by default as my next generations are not base resolution but twice

It may be related to aspect ratio, because it seems reversed when generating base image, I have to choose 9/16 to get landscape 16/9 images, so if upscale is in the good ratio, it may split the image in tiles ?

[sharp idol]

How much stableswarm is better than automatic in speed and vram usage?

[dim sonnet]

oooh, oops, I see why, fixed

the reversal was probably also a side effect of that?

[signal perch]

Is the launch-windows.bat auto updating the app or do I have to git pull ?

[dim sonnet]

it doesn't auto-update atm, you gotta git pull for now yeah

• added update.bat, the most complicated script in the world, definitely not just git pull in a file

(I need to make a proper 'check for updates - do you wanna update?' thingy)

[signal perch]

Yup, base resolution seems to be back when launching a new generation after an upscale. Though upscales are still weird on my side :/ Like the optical illusions hurting the brain lol

[dim sonnet]

what settings?

[signal perch]

Is there a way to share the settings like the ComfyUI workflow in image metadata ? If not here is a screenshot

[dim sonnet]

they should be automatically added to your images, there's a weird thing about the way you copy images that always trips people up when sharing tho

you might try a lower refiner control percentage?

... also full disclosure i've never tested localities that use , as a decimal separator so it's possible bugs are introduced from that

[signal perch]

I didn't change anything here and I'm just drag and dropping images that I saved from browser. Here is the original file from output folder.

[dim sonnet]

uh... if you select the image in the image history view, does metadata show up? like this metadata

i'm wondering if maybe there's a bug preventing metadata from saving for you, or if it's just a file management issue, cause that image you posted again doesn't have metadata

... or, wait, other theory

oh my god discord strips it rip

okay i have to find a way to make discord not do that

[signal perch]

Oh, yes :
prompt: a beautiful portrait photography of a sexy nigerian woman, ultra detailed, intricate details, 8k, 50mm, Panasonic GH5, negativeprompt: out of focus, blurry, ugly, weird hands, extra fingers, poorly drawn hands, illustration, 3D, painting, cartoon, out of focus, cropped, images: 1, seed: 490927054, steps: 15, cfgscale: 7, aspectratio: 16:9, width: 2688, height: 1536, refinermodel: sd_xl_refiner_1.0.safetensors, refinecontrolpercentage: 0.3, refinermethod: StepSwap, refinerupscalemethod: latent-nearest-exact, model: sd_xl_base_1.0.safetensors, comfyuisampler: ddim, comfyuischeduler: ddim_uniform, generation_time: 0.00 (prep) and 129.31 (gen) seconds,

Metadata of generated image seems to be here https://www.metadata2go.com/result#j=c008a880-8748-4697-a54e-52cb16f82579

The image saved from browser has the same output metadata

[tranquil jetty]

Wait again? Discord reverted this a few months ago

[dim sonnet]

i think they partially reverted it ie they remove less data

but swarmui's metadata goes into a real exif key unlike what other ui's do so i bet it's on the discord blocklist

[tranquil jetty]

Ohhh okay

[signal perch]

I can share ComfyUI images with all the workflow/metadata intact, but it's png, maybe they remove metadata from jpeg only ?

[dim sonnet]

leessee

oh shit yeah it's still thing in the png

it's just jpegs they hate them

oh wait no

lmao

they stripped my metadata, but it has comfyui's prompt metadata

so yeah it's the key okay

[acoustic briar]

Just opened a minor issue on the GitHub - I ran into a filesystem error during setup that I traced to some counterintuitive logic around ModelRoot and SDModelFolder settings. https://github.com/Stability-AI/StableSwarmUI/issues/12

I could code up a fix, but I'm pretty sure the fix I have in mind (checking ModelRoot instead of SDModelFolder for an absolute path) would be a breaking change for some users.

[acoustic briar]

(I've coded up a fix that I think avoids breaking any current users whose paths are working right now. I just check whether either setting is an absolute path. If we decide that specifying an absolute SDModelFolder setting should override a relative or absolute ModelRoot. If that sounds like a good idea I can open a PR.)

[untold hearth]

ok so since yesterday Ive rebooted into my W11 setup onthis PC which meant having to install dotnet7 here as well.

No big issue all seemed to install fine, had to do a reboot as it identified some stuff that had some of the files in use .

Went to launch SSUI

grrrrrrrrr bloody dotnet!!!

now was there a specif reason I have an older dot net version in W11 but not by W10 install???

[hearty notch]

when I use refiner upscale, I am always getting these artifacts, is this option still under development? or I am missing something out? (upscaled is the right side)

[native basin]

Can you please explain what exactly is the difference between stableswarm and stablestudio?

[signal perch]

upscales are pure nightmare fuel right now lol

[hearty notch]

let's wait for the reply, already asked, no need to flame the conversation with the same question 😄

[signal perch]

Yes I already reported that upscales seem to be broken, being refiner or the "2x upscale" button under images #🐝｜swarm-ui message I'd probably should open a Github issue though

[untold hearth]

@dim sonnet Just popping by to say , glad I could provide some goofd feedback for you on initial install /setup of StableSwarm yesterday .

TBH I doubt I'm going to be using this as a daily driver as I'm not really the traget demographic IMHO.

This would be ideal to manage multiple installations/instances (and I think you said that on github somewhere) but for me I've now got a nice ComfyUI config that works for me

Have it setup so when I F11 my browser I have everything I need where I need it

[hearty notch]

I have to ask, why you are posting your desk?

and good lord your cables need some management

[untold hearth]

I have to reply: Because it shows the overall workspace and how the Comfy Graph sits in the centre monitor.

Its an holistic overview

[untold hearth]

theres a power brick don on the right hand side, theyre out of the say.

On the desk all thats floating around are various USB cables for charging stuff

[real chasm]

Excuse me what's this channel

[untold hearth]

mind you what you cant see from the front (this is the top down view of my tower) is that the radiators are chilling out externally, the cables are hanging out the back and the PSU is (mostly) iout of sight under the desk lol

[untold hearth]

Primarily to discuss usage of @dim sonnet StableSwarmUI

https://github.com/Stability-AI/StableSwarmUI

[hearty notch]

what are the options we have for naming, and can we use a number sequence for the image name?

[hearty notch]

also, where we can Find the presets? as for someone reason I can't see one preset I created, and the UI is not allowing me to create a new one with the same name

[hearty notch]

looks like the presets are not , friendly?

[dim sonnet]

either being absolutable sounds perfect, absolutely PR it ye

[dim sonnet]

main thing is studio is designed for simplicity n beauty n ease n etc, whereas Swarm is designed with a focus on raw power and features, (but tries to keep it accessible)

[dim sonnet]

they're not what?

[hearty notch]

using commas

[dim sonnet]

they're currently in a database file, i meant to make it controllable whether things store in user database vs. separate files, but have it just database rn (to test and make sure that works as intended)

[dim sonnet]

??

commas where

[dim sonnet]

oo gotta document that ye

[hearty notch]

in prompts, for an example: photograph , cat will cause an error, and you can not generate while photograph, cat will be ok, this is what I did notice when you are feeding the prompts in presets,

I thought to bring it up

[dim sonnet]

that... shouldn't error?

can you post the exact error you got?

[hearty notch]

well it did, haha this is why i am sharing with you

[hearty notch]

sorry did not copy that error, as I clean out the presets, and will be a bit of hassle to recareate it again

btw, really amazing UI, the VRAM load and power pull on GPU is amazing

also, can was wondering if there is a lora support? and if there is a way to use upscallers too?

[shell hamlet]

@hearty notch I broke it! 😄

Too big que 😛

[hearty notch]

how? same issue with presets?

[shell hamlet]

I don't know.. I was queueing like 40 images.. then it threw a couple of those errors. Didn't use presets. Just saying that "you told me so" 😛

Evening snack time -->

[dim sonnet]

you can do lora atm via the comfy workflow tab, it'll be added to the main tab soon ™️

[hearty notch]

and I really love that {Value} in presets! really nice touch

[signal perch]

I moved Swarm from my SSD having ComfyUI (D:/) to my main Nvme drive (C:/) to get more POWER lol, but now I have this weird "[...]StableSwarmUI\main.py': [Errno 2] No such file or directory" and it now refuses my server backends ComfyUI path, like if it was stripping drive letter

[dim sonnet]

uhh there's a setting for max simultaneous somewhere that... may be set too high? iirc it should automatically reduce itself to prevent that

[hearty notch]

hope that we will see some batch size also in main, I can see only batch count

I will try to recreate that error I mentioned now, give me few

[signal perch]

is there any perks of using batch size ? Personnally I already made some performance/speed benchmarks and I see no difference between for example size=4, count=1 compared to size=1, count=4

[hearty notch]

I am running a batch size of 12 renders with 4090 with vlad, I am saying it's good to have the option for people with large GPU

[hearty notch]

not getting it anymore, I tried to recreate it but seems it's fine, I think it was just some random bug caused by some nested preset, anyways, all seems fine now I see no issues anymore

[signal perch]

Moving back Swarm folder to the ComfyUI drive and it's working again, I have complete path in my server backend config so I don't see why it's breaking when moving it to another drive

[dim sonnet]

ye there's a few different path-handling issues brought up today, I'll have to look into it

(will be diving into code in a few hours here)

[signal perch]

no pressure from my side 🙂 I'm playing with all the settings to see what it does (I'm using SD since the beta like a year ago but each time there's a new toy it's like I'm discovering SD again lol)

I just bought 16Go RAM to push my system to 32Go and it's a pleasure to be able to listen to Spotify or watch a Youtube video without it stuttering every 2mn while generating lol

[hearty notch]

@dim sonnet if you can add presets to Grid, that will be a very nice option too,

also, did notice that presets are not saving image preview when selected,

hope you don't mind these comments I am sharing,

[dim sonnet]

oh yeah don't worry, bug reports and general feedback is very much appreciated

[dim sonnet]

ohmygod i think i see why, windows batch is silly and doesn't cd to different drives unless you /D -- it should be fixed now if you git pull

[signal perch]

Working perfectly, thanks !

[dim sonnet]

fixed

[dim sonnet]

it's an option already, it's specifically [Grid Gen] (as the parameter doesn't exist as a dedicated parameter outside of gridgen usage)

[hearty notch]

oh my bad, i will have a look now

[dim sonnet]

I have a plan for that, just haven't gotten to it yet - rather than defining the batch count as a number directly, in server settings you define a max-batch, and then when you generate many batchable images it automatically batches them together up to the limit (so it applies to eg grid usage, and in theory even applies on a shared instance if multiple users have the same settings and only differ by prompt/seed, as you can batch different prompts together)

(also low priority as SDXL doesn't get tooo much benefit from batching)

[hearty notch]

looking forward to see hi-res fix and upscale working with SDXL

loving your UI, really simple and straightforward

[dim sonnet]

yeah im doublechecking code and i'm very confused how you got here. It limits max simultaneous by backend count, and the timeout for waiting on backends defaults to literally 20 minutes, so how

you somehow had an image waiting to generate for 20 minutes (1200 seconds)

(noting that having a batch going that long doesn't count, an individual gen within the batch was waiting that long)

[shell hamlet]

The que goes haywire when you que different prompts. They come out in random order. Maybe that has something to do with it

[signal perch]

Yes it's like A1111 WebUI and ComfyUI just had a baby lol

[shell hamlet]

I threw in 5-10 to que with same prompt, changed the subject, 5-10 to que etc

Then the newer ones started to appear before all the old ones were completed

[dim sonnet]

oo

yeah tragically bad luck with the async random ordering

gotta fix that already

[hearty notch]

seems like it's picking one preset only, and generate the same photo based on the number of the number of presets I have,

[dim sonnet]

... you are correct, oof

it knows they need to be different per image too wtf

oh wait, it's just ignoring them lol

there's no applier for those, i see

[hearty notch]

I will avoid updating, and try to play around more with the UI, will make sure to share any anomalies I will encounter, again, really appreciate the nice work, and for sure there is a huge potential

[dim sonnet]

fixed

[dim sonnet]

hi-res fix in swarmui is handled via the refiner settings, just don't enable a refiner model and do enable upscale

like so

(you should probably enable the refiner model still tho, better that way)

[hearty notch]

ah also, the grid, is there anyway to get a get a stitched img? like XYZ plot in WUI?

[dim sonnet]

this button

[hearty notch]

#🐝｜swarm-ui message i am getting this weird renders

[dim sonnet]

oh, you might just need less upscale or less refiner control or a different upscale method

same issue has been a topic since ye olde auto webui hiresfix days

gotta pick ideal params

[hearty notch]

are there any plans to use upscaler like ultra shapr and remarci?

[dim sonnet]

i might need to do some testing and find clear defaults

oh yes i just need to figure out how to scan the list of options for that cleanly, comfyui supports it but has none included out of the box

[hearty notch]

looks like all laten upscale is causing these artifacts

and pixle upscale is a bit better, but killing many details,

i think will just wait for your update on that

[dim sonnet]

Documented: https://github.com/Stability-AI/StableSwarmUI/blob/master/docs/User Settings.md#path-format
[batch_id] added as well for if you want that, otherwise sequence IDs self-append if you use the same filename repeatedly

[dim sonnet]

Yep that sounds about right

latent upscale does weird things, i should probably change that to not be default tbh

if pixel upscale is good but not detailed enough, increase refiner control

(and make sure you enabled a refiner model, the sdxl base struggles with small details a bit, refiner is better at that)

[dim sonnet]

oh it's not default, pixel is

OH a typo is making it load the default wrong that's why

[hearty notch]

oh this is helpful, many thanks, at least we can use day and time to find the most recent or so,

[dim sonnet]

oh, yeah, time sort is the usual way for me, i didn't even think about sequence numbers for sorting lol

[hearty notch]

pixel-bislerp is causing an error:

[dim sonnet]

oh oops was meant to be latent bislerp not pixel

fixed

[hearty notch]

I guess I will do a pull on your mark, once you push things that you are working on at the moment, and then I will give it another try and see 🙂

correct my observation if I am wrong, with gird render, the main image out of base is like saved somewhere then only grid variants are rendered for the grid, this is why I am seeing super fast grid production???

if my observation is correct! that is a very nice one!!!

[dim sonnet]

i pushed it all as i was going

[hearty notch]

no worries, I did a gird for upscale

[dim sonnet]

i'm not entirely sure what you mean but yes there's several optimizations involved in how grids work combined with comfy's caching optimizations, so it's pretty stronk as long as you have the axes in a good order

oh a grid of different upscales! yeah comfy's cache does wonders for that

[hearty notch]

yes, it's not rendering the same image from zero over and over

with refiner activated and upscale X2 these are the best usable values

so for anyone who like to get some decent qaulity-ish, you can use any pixel upscale with refine control percentage up to 0.4

anything over that will start to break the image

hence why I did not include the results

btw, a seed reset button to random will be nice too,

[dim sonnet]

good idea - added, Random & Reuse buttons

[signal perch]

I have a cool extension in A1111 that is updating width/height according to chosen aspect ratio

[shell hamlet]

Aspect ratio helper is nice

[signal perch]

Yes I searched my installed extensions and found it's this one lol https://github.com/thomasasfk/sd-webui-aspect-ratio-helper it's very handy

[dim sonnet]

swarm has aspect ratio built in already

[signal perch]

Yes but can I change resolution ? I didn't find a way to enter a pixel value somewhere

[dim sonnet]

set aspect ratio to 'custom'

[signal perch]

Yes but then we have to calculate the other dimension if I want to stay at like 16/9. With Aspect Ratio Helper you select your ratio, but still have access to width/height, and when you change a value the other one is updated according to the chosen aspect ratio.

It's like if custom aspect ratio is default choice, then if we select a fixed aspect ratio one field is updated according to that ratio

[dim sonnet]

ooo

[signal perch]

it's a very cool extension, and in img2img we have an additionnal choice of image based ratio, it takes input image size and if you put like 1024 in width, it will update height according to input image ratio. Very handy for upscaling img2img 🙂

[hearty notch]

aside from that, and in my honest humble opinion there is no need for such calculator, I have to ask if there are any plans to support the extensions like controlnet? or this will be also via ComfyUI workflow?

[dim sonnet]

comfy is the general purpose "if it's not in the main ui, it's probably possible via the comfy node view"
ControlNet not currently supported but is a goal yes

[dim sonnet]

if you git pull it should keep queued requests a bit more orderly.
If that is insufficient you'll probably just need to raise the max backend timeout to be high enough to never hit you either way

[shell hamlet]

Do you have a todo list publicly available if people come up with ideas they could check that if it's already on the list?

[dim sonnet]

not everything but the major ones are in the readme https://github.com/Stability-AI/StableSwarmUI#status

[dim sonnet]

added! If you have the upscaler in comfy's upscalers folder, it's now available as an upscale method

[shell hamlet]

Nice!

[barren moat]

Hey Alex is it possible to use wildcards in your UI or is that not yet ready?

[dim sonnet]

that's at the top of the todolist!
So not yet, but definitely an important feature

*(note that the order of the todolist is the order they came to mind while writing, not necessarily the order anything happens in)

[barren moat]

Sounds good! Thank you very much!

[dim sonnet]

Also! Feature update: https://github.com/Stability-AI/StableSwarmUI/discussions/11#discussioncomment-6585698

[barren moat]

That's fantastic!!!

[hearty notch]

is there any certain folder i need to use? or is there any documentation helping with sorting this?

[dim sonnet]

if you used the installer to install comfy, the folder is StableSwarmUI\dlbackend\comfy\ComfyUI\models\upscale_models

[hearty notch]

this is where I placed them

not showing

[dim sonnet]

have you restarted swarmui since adding them?

atm it only reloads that data when a backend loads, which is normally only during startup

[hearty notch]

I did restart

[dim sonnet]

uh... does it show up if you refresh the page?

[hearty notch]

this is why i amwondering what i am not doing correctly

[hearty notch]

oh haha got it

[dim sonnet]

i needa find a way to ... not require a refresh huh

it's, uh, complicated - it doesn't know what the options are til a backend is loaded, and you can load the page before backends load if you're quick enough

maybe an automatic refresher thingy

[hearty notch]

haven't starting playing around with backend and other options yet, was taking my time with the main generate page all day yesterday but I will start today

so for the upscale now, once they are loaded it's just a normal workflow, prompts and hit generate, no further steps are required ?

[dim sonnet]

oh, that was really easy to do actually, applied

it refreshes the value list after backends load

[dim sonnet]

Yep! As long as you have one selected, and refiner upscale is enabled and >1, and refiner control percent is >0, it will use the upscale

... actually wait

heh, neat! if refiner control percent is 0 it still works :D as just a blind upscale of the final output

need to make it not do VAE encode if you do that tho

[hearty notch]

see with default of 0.3 its doing a good job, but breaking things a bit

hope that we can see something close to what hi-res fix used to do with eyes for an example, unless this is an SDXL model thing with bad eyes

[dim sonnet]

done, you can now just do this

to have the final output upscaled

[hearty notch]

i am always keeping the refiner on, is this wrong?

[dim sonnet]

refiner is good to have

i like running base-only but i don't generating pictures of humans often tbh

[hearty notch]

and keep it always to zero??

[dim sonnet]

refiner's biggest benefit is humans

not always nonono

read context

I made an option to upscale raw outputs, and that screenshot shows how to do that

[hearty notch]

i use -.2 to 0.4

[dim sonnet]

that is not recommended settings for anything

[dim sonnet]

this is a reasonable range for refiner usage ye

[hearty notch]

hold on let me update and try

I still need to hit refresh to get the upscale models but this is not an issue, and I think you are also updating documentations? so I can read it later?

is this normal: D:\StableSwarmUI\dlbackend\comfy\python_embeded\Lib\site-packages\torch\_utils.py:830: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage() return self.fget.__get__(instance, owner)()

[dim sonnet]

yes that's normal, you can just ignore that

[dim sonnet]

have you pulled the latest updates for the fix on that?

[hearty notch]

I just did

[dim sonnet]

if you have this yellow message:

then it isn't reloaded yet

models won't show up til the yellow box goes away

[hearty notch]

see still getting these odd things when upscale even with reinfer off

[dim sonnet]

you probably want refiner on

[hearty notch]

with on is the same too

[dim sonnet]

which upscaler are you using?

[hearty notch]

ultrasharp

and remacri

looks like when you go more than 1.5x upscale things start to get weird, is it possible that the upscale model is blowing the render?

it's like doubling the refiner job or intensity, hence why can not do more than 1.5x

[dim sonnet]

if you want refiner method to the other option is might do better?

[hearty notch]

can you elaborate?

[hearty notch]

i am trying this as we speak hehe

[dim sonnet]

stepswap might be blowing up raw noise too much, and postapply might work better because it's upscaling a proper image

[hearty notch]

i can confirm that remacri is doing a better job with post apply for sure, today I will be using the comfyui workflow, is there any tip or a youtube you can recommend?

or just stick to your github and I can read things from there?

still tho upscale is killing lot's of details, I am not sure how this is working, but is there any denoise strength for the upscale?

[dim sonnet]

denoise strength is Refiner Control Percentage

[hearty notch]

I will try using the init img2img also with only upscale to see if there are better results

[dim sonnet]

that's what PostApply is, img2img

[hearty notch]

i mean only for the upscale not the refiner, like i was sharing I do not know how things work in the coding world haha

[dim sonnet]

oh, upscalers actually just don't have any inputs but the image for some reason

[hearty notch]

then what is init image actaully does?

[hearty notch]

noted

[dim sonnet]

img2img

[hearty notch]

this is what I am trying as we speak, generate normal img with normal workflow no upscale only refiner, then use the output in ini image only for upscale with no rfiner,

[hearty notch]

these are decent, really job well done, I wonder what are you hiding for us in the offcial release haha

will give it a try with other styles too, cuz I really love how presets are managed 😍

side note, maybe also a good option to have the shift+arrow keys to control weights on prompts? it's a bit hassle to do it manually also

i know that many of us comes from webui background as it was the mainstream way to SD for quite a while hence why I am sharing this too

[dim sonnet]

ooo yeah good point

[hearty notch]

you see this is what I am confused with, the right one is no upscale with 0.35 refiner control, once you have the upscale with 2x it's like rendering things too much and you start to lose all the details,

the more we push the upscale the more things get weird, I do not know if this is how upscale supposed to work?

oh, could it be that these upscale models are meant for a small base res? like 512?

[signal perch]

the refiner upscale is working great now, I'm using UniversalUpscalerV2-Sharper which is the sharpest upscaler that I have and I can 2x without any "tiling mutation" things

[signal perch]

4x is super slow on my system lol, going from 89sec for the 2x, it announces me 40mn for the 4x 😅 I hope that it will work

[hearty notch]

can you share a link

[signal perch]

https://drive.google.com/drive/u/0/folders/15dV3rhMcq_LXg5Vrn7DG6lwo4M6cXt6C

[hearty notch]

non google drive?

original source?

[signal perch]

That's the link from the upscalers wikipedia page

https://upscale.wiki/wiki/Model_Database#Universal_Models

[dim sonnet]

yeah unfortunately upscaler models have some real sus distribution methods rn

[signal perch]

4x refiner upscale kind of worked though it burnt every details (and took almost 3900sec to generate on my RTX3070 lol)

Here is the 2x refiner upscale for comparaison

I'm thinking about something : upscales should not take the image prompt as positive but only the quality tokens like "ultra detailed, 8K" etc. , so maybe we should have 2 separate positive prompt inputs ?

[shell hamlet]

This is how I always used upscaling in A1111. Or completely removing the positive prompt. It is not needed and can cause artifacts. Of course it depends how strong is your denoising but without positive prompt you can do pretty strong.

[dim sonnet]

can do that via running it manually through init-image afterward, or with a custom comfy workflow

[hearty notch]

@dim sonnet when i use a comfy workflow I get this error

when I run the graph in comfy, it's working, but when I use it in SSUI i get that error

am I missing a step?

oh nv mind, I restart the server and it's working,

however, looks like we can not use presets in this workflow?

[dim sonnet]

presets should work insofar as they match

so a prompt preset should apply for example

(if you have a default prompt input)

any nodes that don't match standard, will not work

or rather: nodes that don't match standard inputs, won't match standard presets

you can create presets for workflows

[hearty notch]

did not work, as parameters name changes when the CUI workflow copied across to main tab

which is not an issue tbh, we can still recreate presets, or create a node that match standard which I am not familiar how to do and that's fine

so If i am now understanding this kind of workflow, we basically convert all nods into the main generate tab and control them from there, can you share with me what kind of advantages this will grant us over using comfyui directly?

[dim sonnet]

yes, that happened because you didn't use the normal prompt node, you used a custom one (That defines G and L separately)

If you used normal prompt node it would apply as expected

[dim sonnet]

well, A, the interface is just ... nicer to work with for the most part - it's designed to rapidly change things like res/aspect/etc that are very slow to edit in comfy, and to have your prompt/etc right next to the image output for real quick generating, and a persistent clean image history, and etc. B, you can use presets - there are some limitations you're encountering, but you can still use presets. Worst case scenario define it unique to the workflow.
C, you can use grids and other tools like it.

[signal perch]

I'm making some max resolution bound tests with my 8GB RTX3070 and how is it possible that I still didn't had a CUDA OOM crash I'm now generating 2048x2048 images 😱 (without upscaling)

[signal perch]

If I launch with "--loglevel Debug" will I get every step output in the cmd ? Because I started a 3096x3096 image to see if it will run without OOM crash but it's stuck after the base steps for now more than 10mn and I don't know if it's doing anything and if it will start the refiner steps
Edit : closed the server and restarted with the Debug loglevel and now I have my answer on how is it possible to generate this large without upscaling lol : " [Debug] ComfyUI launcher: Warning: Ran out of memory when regular VAE decoding, retrying with tiled VAE decoding." 😄

[hearty notch]

I strongly agree on the interface for sure, and looking forward to see more support on the webui extensions too, as highlighted before, huge potential and wish you all the luck with that, and sure will be sharing more things as I am enjoying the dive in this UI 😊

[native basin]

Yeah, But Studio allows the use of API, does Swarm also allow us to use API? Or should we download the complete models and use it locally based on the system and CPU?

[hearty notch]

I think it does there is a tab called server, have a look at that

[native basin]

Oh Thanks, yeah i check it out 👍

[dim sonnet]

swarm lets you choose between remote API, local, or personal api (ie host your own remote server)

(designed for local first tho)

[native basin]

Thanks 😊

[signal perch]

[Debug] ComfyUI launcher: Warning: Ran out of memory when regular VAE decoding, retrying with tiled VAE decoding. It doesn't seem to retry with tiled VAE decoding, base steps took 24sec, refiner steps took 1.11mn, and it's now like 14mn that it's stuck on this message. GPU is almost full speed and RAM/VRAM is still used. Is there a way to see if some tiles are being decoded ? (In ComfyUI we can see the status bar moving on the tiled VAE decoding node).

[jolly sentinel]

How does multiple GPU works with SwarmUI? Does it renders one at a time per GPU? Or can it spread one generation across all GPUs?

[dim sonnet]

any one image is on one GPU, but you can easily queue up many images that will spread between different GPUs

[jolly sentinel]

Ooh... I was expecting spreading one job across all GPUs
Something like accelerate, I don't think I've seen any UI implement it

[acoustic briar]

An unexpected behavior on StableSwarmUI's part cost me a few minutes of confusion today:

When the output filename format contains [seed], the filenames of all images a given job receive the same "seed" value, even though the actual seed used for each image is different. Thus, the seed displayed in a given image's filename might not be the seed used to generate it.

Is this by design, or a bug?

[austere epoch]

Hello, I've just installed stableswarmUI and I get this error, can anyone help me?

[signal perch]

You have to also add "\main.py" in the startscript field 🙂

[austere epoch]

Oh, yeah that was the problem, thanks!

[signal perch]

Np, I have been there too 😅

[hearty notch]

you can change your setting from user tab, this is not a bug, and if you scroll up, you will find more info on this,

@acoustic briar #🐝｜swarm-ui message

[acoustic briar]

Cool, thanks. Surprising default behavior, I suppose, given that the most popular UI always names files with the actual seed used.

[hearty notch]

no worries, and I agree with you, and yet it's an easy thing to change, for now I am using time as my naming method so it will keep things neatly organized

[dim sonnet]

unfortunately no, that'd be very difficult to implement efficiently - even HF accelerate for LLMs has massive overhead, to the point that i was able to beat its performance but replacing it with just calculate half on one GPU then half on the other https://github.com/oobabooga/text-generation-webui/pull/2100

to do it with SD the best bet would be a tiling based approach, where different GPUs handle different tiles

or the best approach for quality minded generation is actually already implemented! Generate multiple images on each GPU and use an automatic aesthetic score tool to pick the best ones and keep only things - you can do this with swarm via the Scoring parameters (haven't properly documented how to use this yet but you just gotta install the python thingy and the rest is straightforward after that)

[dim sonnet]

that'd be a bug

[dim sonnet]

I think you misread the question?

the [seed] value shouldn't be wrong

[dim sonnet]

ooo i'll tweak the docs & error message to make that more clear (EDIT: done https://github.com/Stability-AI/StableSwarmUI/commit/51e4c587d11d161612544e9c940f505a6de4fcbb nobody's gonna be confused on that anymore I think)

[signal perch]

Speaking of score, how do we use the score part in Swarm ? Both models get me an error (can't remember which I'm at work)

[dim sonnet]

atm the score implementation 'works' but isn't 'clean', requires a built of manual work:

• presumes you have python3 installed
• open up src/BuiltinExtensions/Scorers
• create a venv if you want to do it right, don't technically have to
• pip install -r requirements.txt
  after that it should work on its own. Atm only uses a single local GPU, no splitting yet

[jolly sentinel]

ooooooh nice! But my problem about generating one image per GPU is that sometimes one GPU has not enough VRAM, 4GB-6GB. It would be cool to have a tiling based approach spread across all GPUs.

[dim sonnet]

oh ooh uh idk that tiling would solve very-low-VRAM limit issues. It'd help a little, but most of the VRAM is the model itself.
Maybe a rapid-switcher ie half the unet on one GPU and the other half on the other... it'd be fairly slow still (not as bad as hf accel, but it'd have to gpu-swap twice per iteration which'll hurt a bit)

(very similar to the LLM approach in the PR i linked above, just it has to switch more often than LLMs have to)

[dim sonnet]

yep, simple bug, wrong copy of the variable was sent through, fixed

[jolly sentinel]

Would it be possible to use one GPU to keep the model loaded and another for tiling?

[dim sonnet]

no, the model has to be on the GPU that's executing the model, if you don't want everything to be horrifically slow

[thorn harness]

Is there any way in comfyUI to save the images with the prompt in the name for example like in A1111 or at least the seed maybe? To make them unique, that would make it easier for me to store them instead of them all being named the same with numbers

[dim sonnet]

well you asked in the swarmui channel, so... yes swarmui does that by default and makes it easy to configure via user settings lol

[signal perch]

I think that latest commit broke the UI :/ (or I made a mistake somewhere)

(no error message in cmd)

[thorn harness]

I thought you work together with comfy so I could ask here
I'm using comfy standalone atm cause the weird issue with the comfy manager

[dim sonnet]

wtf? can you uh restart the server, and once the page is open (if it hasn't fixed itself) CTRL+F5?

[dim sonnet]

well yes, but. lol.

[signal perch]

it seems to have fixed it, I think that the browser is opening before everything is loaded

Yes it seems to open the browser too soon

[dim sonnet]

it's been brought up enough times that i said screw it and just https://github.com/ltdrdata/ComfyUI-Manager/pull/43

fixed it meself

if you use that PR, Manager shows up in the swarmui tab for comfy

(actually might need to edit a bit more for it to actually work fully too hold on lol)

okay now it actually works too lol

[thorn harness]

[signal perch]

I deleted my Swarm folder and git cloned to make a fresh install and it's now working fine, I don't know what was wrong 🤔

[dim sonnet]

uh... okay then

ideally that should never be needed, but, if it's your preferred quickfix when you see a bug, why not i guess lol

[signal perch]

Yes it's not like if we have to make dozens of changes in settings etc. 🙂 Just the main.py path, models and SD models path, done.

[signal perch]

to import a ComfyUI workflow we just have to import it in the workflow editor, then click on "use this workflow in generate tab" , or once it is in the generate tab we have to tick every node that has been added ? (I'm trying latest Sytan workflow and there's a lot of added things lol)

[thick remnant]

can I install Swarm on a potato and use DreamStudio for generations? swap for a ComfyUI colab server when Im running one?

[runic horizon]

is there any support for AMD?

[dim sonnet]

yes, just import and 'use this workflow'. Any node you don't tick to enable is left as default values (ie: whatever you had in the workflow)

[dim sonnet]

Yep - it's referred to as the StabilityAPI backend - it's an option in the installer, and also in the server->backends page.
atm it's not perfectly integrated - main limitation is just you have to download at least one model so it doesn't complain that you don't have any models, i'll hopefully fix that soon.
Also yes you can boot up comfyui on colab or anywhere else and use that as your backend too

[dim sonnet]

Yes, insofar as comfy supports it, which largely comes down to, uh, how well AMD works - you need a relatively recent AMD card, and ideally be running on Linux or WSL.

[signal perch]

Why aren't all the nodes ticked by default though as if I import a workflow I'd expect to just hit "generate" and have the same output as in the ComfyUI tab ?

[dim sonnet]

Unticked = use what's in the workflow
Ticked = change to different from the workflow

[signal perch]

So Sytan workflow seem to not work in Swarm but I'm a bit tired today so I'll dig more into it tomorrow 🙂 (some null pointer crash thing and even if I disable ComfyUI workflow and load default settings, restart Swarm, I still get that pointer error. Nevermind 🙂 )

[dim sonnet]

if it happens again next you're trying it,

• make sure you're fully up-to-date
• launch with --loglevel debug
• send me the workflow + the logs from it

[signal perch]

This is the latest Sytan ComfyUI workflow he's pretty popular on the SDXL chat, I just generated this (workflow embedded), I'll try to get some logs tomorrow (I'm always up to date I added a git pull line in the bat file to update at each launch 😁 )

[dim sonnet]

oo, replicated

[split swan]

you mean width vs height tiles?

[dim sonnet]

update again, should work now

also added more debug so if it breaks in the future it'll be easier to track down

[signal perch]

Working, and lots of debug output ❤️‍🔥

Damn I should be sleeping... 😄

[thorn harness]

It works yes

Now I need to figure out how to save the images with a better name

[signal perch]

You have all that's needed in the server settings tab 🙂

[thorn harness]

I just dont get where this applies, if I use a save node in comfy it doesnt work

[signal perch]

I'm at work and didn't touched it for the moment but I'll check it in a few hours because I prefer starting with the seed

[thorn harness]

Yeah id like the prompt and seed in the name as well, model isnt as important

Maybe it only applies if you use the generate window

[signal perch]

yes I guess Comfy has its own settings for the names, here it is Swarm settings

in Comfy you can change it in the "Prefix" part of the save node I think

[thorn harness]

Yeah but stuff like [seed] doesnt seem to work

[signal perch]

I think that Alex fixed it yesterday https://github.com/Stability-AI/StableSwarmUI/commit/d83bb2805bbbf5a2ec6a4ad5b3a71a998e15fec1

[thorn harness]

No I meant in the comfyUI save node, my workflow does seems to do well when I take it into the generate tab sadly

[thorn harness]

with this as file name in comfy you get a unique name with the date and time [time(%Y-%m-%d)]_[time(%H-%M-%S)]
Not the perfect solution but works for me for now

[pastel briar]

will it be possible to use NVLink?
(for VRAM pooling - got 2x2080ti)
thank you

[inner sand]

Hi, new member here, a referral from the ComfyAI server of Olivio Sakaris. Following your project with interest.

[trim comet]

Hi to the Stable community, anyone on silicon mac had success with StableSwarmUI?

The UI start but show : Backends are still loading on the server...
i can select a model and type a prompt.

When i hit Generate button i get this error on terminal window Thanks for your help... :

*Exception in thread Thread-1 (prompt_worker):
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
self.run()
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/threading.py", line 953, in run
self._target(*self._args, *self._kwargs)
File "/Volumes/Stef-T7-2/StefAI/StableSwarmUI/dlbackend/ComfyUI/main.py", line 88, in prompt_worker
comfy.model_management.soft_empty_cache()
File "/Volumes/Stef-T7-2/StefAI/StableSwarmUI/dlbackend/ComfyUI/comfy/model_management.py", line 549, in soft_empty_cache
torch.mps.empty_cache()
AttributeError: module 'torch' has no attribute 'mps'

[split swan]

It's nice to see collaboration among competition between different UIs etc, to allow for saving and opening of various settings and so on. Rather than locking everything else out if you run a certain UI in order to "keep" users in

a bit like how you can import different files through different image or 3d software, audio, daw etc

[split swan]

how come stable wants webui.bat and not webui-user.bat? this means I cant add set PYTORCH_CUDA_ALLOC_CONF= garbage_collection_threshold:0.9,max_split_size_mb:512

[toxic shale]

like everything reads the file paths correctly, except for the image reader 🤣

actually fixes it - but still makes it impossible to use multiple drives for different folders

[toxic shale]

so... about that. it says that it's loaded. but it's not

but the 2nd image is with it actually loaded

you sure the model loader from swarmui, doesn't just skip all steps until the clip text encoder?

I take it back. it is being loaded. it's just working... wrong 🤣

found the issue. it lies in the part where the first ksampler doesn't show his values - and instead gets them from the 'core parameters' <- except they don't show a sampler/scheduler. So it auto defaults to euler/karras, no matter what was set in comfyui as a 'default'.
Solution was to keep an additional ksampler node that 'doesn't work'. Now everything works.

how do I use batch size, to more efficiently create data for multiple seeds in grid generator?

[toxic shale]

for the grid generation, we're missing a <prompt replace> - for when style words or trigger words are necessary
and when you have multiple prompt boxes - it just ignores all after the first, rather than cross combining them

[dim sonnet]

Hypothetically maybe? I'm not sure, never tried working with nvlink directly. Nvidia's claimed speed is nvlink is 7x faster than pcie transfers (except the way they write claims makes they very hard to believe at face value). So, uh, in a theoretical world where nvlink was instant it'd probably be a very powerful option. If it's only a few x faster than pcie, it's just the same problem as within nvlink but a little less bad.

[dim sonnet]

I know at least one user has it running locally on an m1 mac: https://github.com/Stability-AI/StableSwarmUI/pull/14

[dim sonnet]

If you edit webui-user.bat
where it has:
set COMMANDLINE_ARGS=
to instead be:
set COMMANDLINE_ARGS=%*
(I think?)

you should then be good to link the webui-user.bat in the backend settings.

The reason here is that the backend launcher passes some args through (mainly --api --port={PORT}) so if you pass those along it should be good

note that auto webui is a bit limited as a backend vs. comfy btw

[split swan]

I just want that second line through cause it makes sure I don't run into memory partition issues

set PYTORCH_CUDA_ALLOC_CONF= garbage_collection_threshold:0.9,max_split_size_mb:512 this one

[dim sonnet]

technically you could also just shove that in your swarm launch script and i think it still applies and works

but either way

[split swan]

ah cool

what about extensions btw?

how does that work its way in?

like multidiffusion, ADetailer, Control Net of course, Regional Prompt etc

[dim sonnet]

should be fixed now

[dim sonnet]

auto webui extensions aren't recognized at all by swarm atm - focus has been on comfy more than auto

[split swan]

ok

I would suggest seeing if you can work with invokeai to use their canvas painter

because it's amazingly stable and easy to use

inpainting, outpainting, doing fills where the box recognizes the current content and so on, adding layers

aside from a paid for plugin for photoshop, it's the best canvas I've had for SD

[dim sonnet]

Fixed - it now uses the sampler/scheduler you actually set. And also actually shows the parameter in the UI lol

[dim sonnet]

yeee, I've been wanting that (or similar), just haven't focused in on image editing interface yet

[split swan]

invokeai has one if you can get a colaboration going

[dim sonnet]

I have a plan for that, but haven't gotten to it yet - the idea is that batching lets you change several different params (eg prompt) freely within a batch, so I want a system where any time there's multiple generations queued with similar details, it will automatically group them to a batch (up to some limit defined in server settings and/or automated relative to available VRAM)

[toxic shale]

would it be possible to add a token word, like {grid}, which gets auto replaced from the prompt if it is there? (for prompt grid)

[dim sonnet]

oh, oops, fixed.
Prompt Replace was already very much a thing, it was just missing because custom comfy workflows clear out unrelated parameters, and those naturally aren't in the workflow itself. It now retains gridgen-related params (Prompt Replace & Presets) regardless.

prompt replace applies to any text input that has a name ending with prompt - so the default prompt & negative prompt, or any custom prompt primitive like

(note for primitives, you have to enable the parameter in the UI for gridgen to count it)

[toxic shale]

I temporarily solved it with a custom node that simply forced a prefix XD
glad I can do normal ui again

[dim sonnet]

Also!

you can now do this ^ - title a primitive SwarmUI: Prompt

[toxic shale]

le gasp!

[dim sonnet]

and it will use the prompt param

works (theoretically) for any param you want

[toxic shale]

also, how come the interrupt is always visible? even when there's nothing to interrupt

[dim sonnet]

Technically it's possible for the button to still be helpful even when there's nothing - if for any reason the client has forgotten it's generating something, but the server is still going, the interrupt will be recognized by the server regardless and still interrupt

i could probably fade it out a bit when not-seemingly-relevant tho

Done - fades out now

[signal perch]

to me the Generate Image button is gigantic, I'd prefer it the size of the Interrupt, side by side, it would free up a lot of space for the other settings. But maybe there will be styles with smaller button 🙂

[toxic shale]

could you make custom workflow + current settings persistent across refreshes?
I would love it across restarts too - but standard refresh is definitely needed, as I'm currently afraid of accidentally refreshing

[split swan]

I had a thought awhile back, about a "stage controller" where you can place nodes in 3d space where you want something generated. Kinda like regionprompt except you go into a 3d space and maybe place a box you can resize etc for how much space it takes up and where in the 3d space it is.

This might make it easier to create images with depth control of all actors and events

@dim sonnet You know what I mean?

I know Gyre AI for Photoshop already does this in their plugin

[dim sonnet]

it's "the big main button", intended to be large, but yeah that should be easily changed via custom theme (a couple lines of css)

[split swan]

https://www.youtube.com/watch?v=MWAeZxAlGxw here it is

that system just looks really good

@dim sonnet you see what I mean?

[dim sonnet]

Done - persists now. Parameter persistence is in your browser with a 12 hour expiration, however workflows needed to use localStorage API to not get broken by browser so they persist indefinitely instead.

[dim sonnet]

ooo. Yeah I like that

that's ... inpainting + multiduffusion+controlnet+a complex 3d interface over top of it all. Complicated, but would be amazing to have

... i'm gonna end up having to dive deep into custom canvas code to do the image editing engine now aren't i...

[split swan]

heh

that one up there is Gyre AI (formerly flying dog) plugin for photoshop

its for sale on adobe store

[toxic shale]

I have so many padding questions 🤣

[toxic shale]

getting there 😏
pretty alignment

[dim sonnet]

ooo

... y no main prompt in your workflow

[toxic shale]

cause I had loaded the wrong comfy affter restarting for the 15th time 🙈

[dim sonnet]

olol

have you updated to get self-persistence yet?

[toxic shale]

nop - I'll do it once I'm happy with layout

so... how do we feel about moving prompt + core params ... above the generate button

[dim sonnet]

mmm i like it at the top tho

seems like something that should be adjustable via Theme selection tho

[toxic shale]

kay. then I'll instead do a separator thingie between the most important settings, and everything below. - since there's significant information overload right now

[dim sonnet]

the information normally visible:

until you click into param groups

[toxic shale]

yes XD

[toxic shale]

sure its boring, but its quick to use and easy to look at

[dim sonnet]

feature update: https://github.com/Stability-AI/StableSwarmUI/discussions/11#discussioncomment-6615686

[dim sonnet]

oooo i love it

actually wait

why no slidebar

the bars are really really nice for quick adjustments

if you can make em more compact n clean tho, and fit them in with that setup, that'd be incredible

(either way this is awesome btw, i really appreciate the help with UX!)

[toxic shale]

for steps - yeah, I'll re-add it, but prob with a 5step interval.
for cfg I'd rather do a -- - + ++ kinda thing. where the double subtracts/adds one whole number, and the single subtracts/adds 0.25

since you'll rarely move by more than a few numbers

is there any reason that the first instance of resolution isn't in the core parameters?

[dim sonnet]

Resolution I specifically wanted the really clean preview thingy

if you can make a clean preview thingy not based on shoving 3 parameters into a group, putting it into core would make sense

[median crest]

How generate image ?

[toxic shale]

closed/open
is how I'd imagine it (quick mockup in photoshop)

[signal perch]

I like the second one and think that W and H can still be displayed when image ratio isn't custom, and when a value is changed, the other one is automatically calculated from the aspect ratio value (which is something I suggested a few days ago 🙂 it's like the A1111 extension "ratio helper" or something like that)

[toxic shale]

good point. since there's no longer the container showing the resolution, might as well leave the boxes open permanently

[signal perch]

Or a "+" and "-" button on the right to add/remove some frequently used resolutions to the dropdown list ?

[toxic shale]

would be cool - but relatively annoying to implement since you'd also need to store those values + save button, +delete button. Maybe if there's a preset manager in the future, under the user page where you can add all kinds of config presets

[signal perch]

Yes I only proposed that idea in case of people frequently generating non standards resolutions 🙂

[split swan]

@dim sonnet hmmm I don't remember if you had a checkbox or something to lock aspect ratio? So it will scale from whatever you have at that moment. Maybe that's in the aspect button. It would also be nice with a style editor, so I can go in and edit saved prompts and maybe delete them or something. Also to maybe save all prompts automatically to some secondary file and I can then confirm which ones I want to add to the primary.

Do you have a pix2pix feature somewhere?

[toxic shale]

edit presets?

triple dots should prob be replaced by a smol yellow box with wrench icon

[split swan]

oh are they called presets there?

does it save every prompt automatically?

[toxic shale]

they're his own implementation of it I guess - but a lot better than alternatives from previous setups, since it can do cool things
prompts are automatically saved within the output file

[split swan]

not to the main list but like, on the side so you can add it

[toxic shale]

if you drag a saved image into comfy, then you get all the parameters again

also yes, there is a full history with all details inside of swarmui as well

[split swan]

yeah but I meant saved in the webui itself so I can load old prompts

without manually going into files

[toxic shale]

yep - automatically via history tab.
manually via saved images

[split swan]

hmmmm there's no way to save the info to a txt file?

cause I never save metadata on an image

[toxic shale]

even then its still stored inside the swarmui database.
just generated an image without metadata - and history tab has all details

[split swan]

but is there a separate file?

like in automatic

[toxic shale]

no

[split swan]

otherwise that's a feature I'd request

@dim sonnet would it be possible to save prompt data etc to a txt file saved along with the file, sharing the filename? Like in Auto

[split swan]

thank you for the assist

[dim sonnet]

deefffinnitely need sliders for resolution

(ps do note my sliders for res have Power-of-Two scaling factor, ie they're modified range to make the slider naturally position at relevant sizes)

[dim sonnet]

you can save resolution presets with the preset system

literally just

and then enable these and set wherever

[dim sonnet]

what auto webui calls Styles, SwarmUI calls presets - they control a lot more than just prompt (can control literally any parameter, even res as shown above lol)

you can import auto webui styles directly to presets

pix2pix isn't directly built in, other than where a backend supports it (comfy I assume can somehow, auto webui treats it as just img2img with a relevant model)

[split swan]

okay

what about txt files to go with each generation that holds the meta data instead of the image file holding it?

[dim sonnet]

the metadata is in the image file, but i can make a .txt option if you prefer that

[split swan]

that would be preferable

I don't like having it in the image, but I like having it available

[dim sonnet]

if unclear, I'm replying to messages in order sent, so repeating questions isn't needed unless i've already gotten past it

[split swan]

right

[dim sonnet]

Added, under user settings

[clever wind]

is there already an inpainting editor and if not, are there plans to include one? That's the only thing I'm missing in ComfyUI

[split swan]

nice thank you

[dim sonnet]

I believe comfyui actually does have something for that somewhere - not really sure.
proper image editing tools are a key feature not yet implemented that definitely needs to be

[clever wind]

would be cool to directly connect that with segmenteverything such that you can click on objects in the image to mask them 🤔

[bleak atlas]

/Beauty Girl denoise strength is Refiner Control Percentage

[dim sonnet]

yeeeee

[dim sonnet]

... ??

[bleak atlas]

/特郞普打高尔夫

[bleak cedar]

@bleak atlas Use #1100170312106127410 for generating

[analog terrace]

Any thoughts on adding functionality of running "Stable Beluga" or other LLMs? Have a dedicated machine to prompt and analyze images while another diffuses images. Many potential use cases.

[dim sonnet]

i would love so much to train an LLM to directly output prompts & configs for images based on user request

needs a lot more work to be good than just shoving an LLM into the prompt box tho

hopefully we can pull off a collaboration between all the different stability teams at some point to build that

(or, well, at least the language team and mine lol)

(also requires, like, building brand new datasets to train it on and stuff like that too)

[analog terrace]

I have been making prompt assistants with Oobabooga, I have not tried the ComfyUI extension that allows them them to share APIs. Specialized models would be much much better but honestly you would be surprised what you can already get out of these general 13B models with minimal pre-prompt instructions, yet alone a LoRA. Thanks for the response! 🙂

[toxic shale]

could I request a refresh button, next to the question mark of the lora loader node?
so that new loras can be loaded without restarting XD

[split swan]

I don't have enough vram fora n LLM

well an LLM to be run simultaneously as auto

[signal perch]

I'd be curious to test something where you describe the image that you want, it's sent to SD, output is sent to CLIP, which is then sent to an LLM like "here is the request, here is the result, what should I change ?", and the LLM send this to Instruct Pix2Pix and a scorer until image gets a high score and all the required elements.

[dim sonnet]

top-right corner:

that button reloads all of em

would probably be useful to make that more obvious tho

lot of people miss that

[dim sonnet]

you would be surprised what you can already get out of these general 13B models with minimal pre-prompt instructions, yet alone a LoRA.
oh yeah dude don't worry I know. (Go look who authored the lora training tab for ooba's text-gen-webui lol https://github.com/oobabooga/text-generation-webui/pull/570). LLMs are wildly capable if you prompt em well.
imo the results of that are in the "cool foss project" range still tho, and upping from that to "StabilityAI published the easy mode ask an llm and it gets you an image all-in-one box" requires really perfecting it.
For more examples of the type of stuff the perfected version of it would do, check out https://arxiv.org/pdf/2305.13655.pdf
(I mean to send that link yesterday but got distracted)

god i want it so bad

[analog terrace]

You guys are on the right track. Gotta get all the foundations down before trying to mix all this new tech together.

[dim sonnet]

the way the ooba extension that does it works, it swaps an LLM into VRAM, generates text, swaps the model back to RAM, then loads the SD model, generates image, swaps it back out, etc.

It only adds a couple extra seconds to the process, it's not terrible at all

[dim sonnet]

yeee. I wouldn't use exactly ip2p for that, but the general concept of the iterative feedback loop is definitely key

there's a few projects (LLaVA, MiniGPT4, ...) that do direct image inputs to LLMs that are very useful for that

[signal perch]

Yes Instruct or anything else capable of analysing an image to edit parts of it via text input 🙂

[split swan]

ah I see, that's nice

[solemn rampart]

How are you supposed to create... subfolders I suppose, on the left side of the preset area?

[dim sonnet]

Right now what you do for that, is just: create a preset and name it with a / in it.
(I realize this is not the clearest system lol)

if you make a preset named My stuff/my preset that will make a folder named My stuff and put my preset inside it

[hearty notch]

Hi, still enjoying the UI so much

was wondering if there are any updates on upscale, out of using the comfyUI?

or there is any recommended workflow that you can share?

as I am still struggling to get as proper upscale

I tired few workflows from Comfy, but they mostly use a very complicated nod structure

[solemn rampart]

I've seen worse hacks by far, but you might want to put a note or something, or put it on the wiki! 🙂

[hearty notch]

you would love that wouldn't you ahaha, wikiperson

[toxic shale]

#🐝｜swarm-ui message
For all the install & guide for stableswarmui!

[torn canopy]

ok (illiterate PC person here), was able to clone the installer... what do I do at this step

[dim sonnet]

doubleclick on the first launch-windows file

it also shows up as an icon on your PC's desktop

[solemn rampart]

Oh, I did look at that.

[torn canopy]

is there a way to reference my models folder on a1111 to not have duplicate models?

[dim sonnet]

yep, just go to Server -> Server Configuration in the UI and edit "ModelRoot"

replace it with your stable-diffusion-webui/models path

[torn canopy]

thanks for your help! one last question dont think i got this right...

oops

i see

[dim sonnet]

just need \models on the end

[torn canopy]

bless you sir. this was indeed so much easier than my a111 adventure.

[dim sonnet]

:D

[torn canopy]

one moreeeee question... Is there a way to load my saved prompts from A1111

[dim sonnet]

Do you mean Styles in auto webui, or...?

if styles, yes, that's the Presets tab

[carmine cloak]

So does this use a LLM model as a text encoder or something like that?

[dim sonnet]

that's what the screenshot is about yes -- er, well, using an LLM to form inputs, not directly as the text encoder
(if unclear: that is not a swarm feature, just a discussion about an idea)

[carmine cloak]

That would indeed be very impressive and amazing to use. Imagine the understanding of complex concepts. I could imagine we could get really precise and accrete results with something like this.
Tho the thought of Vram usage gives me lowkey anxiety. Lol!

[obtuse falcon]

will this take a UNC path/path to a network share?

[obtuse falcon]

have experimented with this, my chatbots use the same LLM that powers them (a 33B LLaMA derivative, though that's massive overkill, 7/13-14B do just as well) to convert natural language descriptions into tag prompts for anime models; the same approach can be used to expand a short prompt into a longer one

works quite well but takes a bit of tuning

[dim sonnet]

iiiiiiiiiiiiiiiii have no idea

maybe?

if you mount it to a drive path, it should work

without that, idk, maybe

[obtuse falcon]

valid; if it's just using regular OS-backed filesystem calls and you're not doing anything shady with path handling it should work

only one way to find out

[dim sonnet]

ye

only limitation is depending on how you format the path it might get misparsed as relative instead of absolute

but if it's mapped to a drive it's fine

(cause then it's just Z:\MyPath or whatever and that definitely works)

(or on Linux it's just a folder path anyway since everything on linux always is lol)

[toxic shale]

grid generator currently ignores custom names for lora loaders
which while not important for one lora - makes testing 2~4 loras loaded at the same time quite hard as it's not always clear which lora loader is being changed

(even a parenthesis with Node ID would be fine)

[toxic shale]

also this is definitely not working - unless you implemented it in the last 2 days

(I'm 2 days behind on my local version)

for comparisons sake, would it be possible to add a none option, or support for leaving it blank in the lora loader?

while I can do a strength option for 0,1 - this would cause 2nd image per lora checkpoint, rather than once per grid

[trim comet]

Hi, is there any neew advice to follow. Hi hope test your stableswrm-ui on mac soon 😁

[blazing belfry]

How could I set external models folder for swarm-ui?

[toxic shale]

writing full path should also work if you're up-to-date with your install

[blazing belfry]

Not complain, just a suggestion. The description part could be handled better. I know it is in early stage.

Does it suppose to blank when using installed comfyui?

[toxic shale]

it should load in after like 3~5 seconds

[blazing belfry]

I could open the comfyui started by swarmui in another tab. But it couldn't load in this page

[toxic shale]

ah, you used the internal one

[blazing belfry]

I setup this one

[toxic shale]

yeah that's a question for mcmonkey. I moved over to using my main comfy setup

if you're using your existing install, it should look like this btw

[blazing belfry]

ok

[toxic shale]

except port is the port you're actually using

[blazing belfry]

It still blank...🤣

[toxic shale]

well rip. here's to hoping nothing broke in a recent update XD I'm still 2 days behind on my install

[blazing belfry]

Try to install with its internal comfyui

The only thing I want to try is the grid

Do we have the restart backend button?

[thorn harness]

Any plan to add color managment with color profiles?

[dim sonnet]

if you click edit then save (without editing anything) that will trigger a restart

[blazing belfry]

ok

[dim sonnet]

this should either load or error, staying blank white like that is weird. If it doesn't resolve itself after a restart, check browser console (CTRL+SHIFT+i) for error messages

[dim sonnet]

there are 3 themes and they are just css files which you can customize, but a UI for personalizing would be a nice future addition

[dim sonnet]

been thinking about removing that entirely - the actual info display is in the ?, the preview of it is just confusing to people

[thorn harness]

No I meant color managment like SRGB or eciRGB for example, since SD spits out images without any

[dim sonnet]

Er... it might not work properly with custom comfy workflow params actually, need to fix that

[dim sonnet]

uhhh does comfy let you leave it blank? cause that effectively just feeds the value to comfy.
Probably should figure out a way to like... myfile:1.0 similar to how auto does it to make it easier on that

[blazing belfry]

I also notice that the seed from primitive node of custom workflow could not set as randomized as in the workflow?

[dim sonnet]

(blank lora name will make more sense with the workflow autogenerator, as swarm itself will be able to process that)

[dim sonnet]

use -1 for random

there's also a lil 🎲 button to quickset that

[blazing belfry]

but -1 doesn't apply to the custom workflow

[dim sonnet]

in swarmui, if it recognizes the input as a seed, -1 will work

it pre-randomizes before sending along to comfy

[dim sonnet]

oh. Huh. Uh. I don't know enough about how that works tbh

i added a DPI setting on joe's request a while ago and that amounted to just setting a standard exif key lol

[thorn harness]

It would be nice cause then I wouldnt have to open images in Photoshop and resave them and loose all the meta data

[blazing belfry]

I know why. I used my custom sampler which has noise_seed as the seed

[dim sonnet]

oh oo i see -- fixed! if you update it will now automatically recognize that as a seed

[toxic shale]

multi lora support plz XD ❤️

[dim sonnet]

very much part of the autogenerator plan for that yee

[dim sonnet]

the image library i use has the ability to write color profile metadata, but not convert between em - there's a draft PR for conversion open since 2021 https://github.com/SixLabors/ImageSharp/pull/1567

might need to swap the lib anyway since it makes mucking with metadata more annoying than it should be

[thorn harness]

Oh yes that would be sick

But 2021 was long ago so probably dead

[dim sonnet]

the PR I linked is still the best reference i have for mac

[dim sonnet]

it's not dead, it's actually been in the works the whole time

there's commits all over the place in that

[thorn harness]

Geez

[toxic shale]

how do I use commas inside the prompt box of the grid generator?

it turns it into a new prompt, but what if I have a list of 20 full prompts, commas and all, that I need to generate?

[dim sonnet]

use || to split the prompts

if there's any || it will use that instead of ,

[inner vault]

#1011228667659178055 so i installed dotnet and swarms and for some reason every time i lunch swarms it gives me error that it didnt find path or something like that . i checked and dotnet is installed proprly ... tried reinstalling both but same error ... whats happening ?

[dim sonnet]

are you sure you installed exactly .NET 7.0 SDK for Windows x64 ?

[inner vault]

yep pretty sure

[dim sonnet]

huh

uh

... try just launching it again?

[inner vault]

same error

[dim sonnet]

what happens if you run dotnet --list-sdks?

[inner vault]

i copy it like this dotnet --list-sdks in cmd ?

[dim sonnet]

yes

[inner vault]

nthn

[dim sonnet]

wtf

so

it's... installed, but it doesn't acknowledge its own existence??

why does microsoft do this

• maybe a PC restart?
• maybe uninstall + reinstall

[inner vault]

already uninstalled but ill retry any way

be back in a min

yep same thing restarted and reinstalled but still same thing

we love microsoft

[dim sonnet]

wtf

[inner vault]

[dim sonnet]

what do you get from dotnet --version?

[inner vault]

i am confusion 🙃

[dim sonnet]

ooo

uh

what do you get for where dotnet

[dim sonnet]

AHA

okay hold up

[inner vault]

hold up .. wait a minute .. something ain't right

xD

[dim sonnet]

Okay:
copy and run this line:
SET PATH=C:\Program Files\dotnet;%PATH%
and then right after, do where dotnet again

[dim sonnet]

hell yeah!

okay!

[inner vault]

what now

it works now ?

nope

[dim sonnet]

okay, I pushed a commit - open the StableSwarmUI folder, and doubleclick the update.bat file

then after that closes, try launching it again

[inner vault]

holly shit xD it works

thank you soo much man

luv u xD

[dim sonnet]

:D woo

and thank you as well! Your help in figuring that out means it's now fixed for everybody :D

[inner vault]

what was wrong with it tho am curious ?

[inner vault]

i feel honered

[dim sonnet]

You had a 32-bit version of .NET installed in the past, and Windows was strongly trying to use that even though it's outdated and invalid

and completely ignoring the 64-bit one

[inner vault]

yeah but like why 🙃 windows is actually stupid

[dim sonnet]

:D https://www.youtube.com/watch?v=qyP-i9mfLLc

[toxic shale]

"secret project" 🤣

"totally not a thing yet grid generator thingie that may or may not be in development"

[trail nebula]

Has anyone else had issues with the Image History and Models not loading an an error saying "Missing required parameter 'depth'"?

[trail nebula]

I love the grid thing, I used it like crazy on A1111.

[dim sonnet]

hit CTRL+F5 on the page?

that's a new feature so you probably just updated but old code in cache

[trail nebula]

Ooh that worked.

Thank you. 🙂

[dim sonnet]

I fixed it now so that if you refresh on the comfy page, then use reload param values on the generate page, it will recognize changes to eg lora list and all

[tardy sundial]

I'm having issues installing the StableSwarmUI, getting error in the command line: "12:53:54.258 [Init] Self-Start ComfyUI on port 7820 started.
12:59:29.802 [Error] [WebAPI] Error handling API request '/API/InstallConfirmWS': Internal exception: System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host..
---> System.Net.Sockets.SocketException (10054): An existing connection was forcibly closed by the remote host.
--- End of inner exception stack trace ---"

Anybody had this issue?

[inner vault]

yo . how can i get the default workflow of the UI in nodes i used a custome node group and when i restored the default work flow on the the UI id didnt turn back in nodes

[cursive badger]

I'm having a hard time launching it. It's in portugues but it gives this "error NU1101: could not find..." etc

where dot

[dim sonnet]

er... it seems to be having some form of issue downloading nuget dependencies? 0.o

can you open a command line in the StableSwarmUI folder and type: dotnet restore?

[dim sonnet]

did you close your browser window while it was installing?

[tardy sundial]

No i left everything open

[dim sonnet]

not sure what you mean?

[tardy sundial]

might have to disable my firewall and try again?

[dim sonnet]

er... then, just try again, and if it breaks again make sure to screenshot the full output

i don't think firewall should be relevant?

[drifting helm]

i've seen that issue before; usually happens when something weird happens with browser or some unexpected disconnect

[dim sonnet]

it has connections to (A) your browser tab and (B) remote page to download (github for comfy, huggingface for models), that error message lacks any details on it so i'm not sure which of the two lost connection

[tardy sundial]

noob question but whats the best way to screenshot the comand line when its stuck? I can copy and paste the lines

[inner vault]

ill explain .... you know when you first load the ui and it comes with default workflow and node setup ?

[drifting helm]

to screenshot on windows? theres win + shift + s for snip n sketch

[drifting helm]

timeout error, hm

[tardy sundial]

Gets stuck there

Ill try disabling firewall and try again for the heck of it

[drifting helm]

some timeout error while downloading some file(s)

huh...

[dim sonnet]

that's one of the model downloads from huggingface

if you update, there's now a better error message if/when that happens

[dim sonnet]

in the comfy workflow tab? That's the comfy example default workflow

in the generate tab, if you're not using a custom workflow, it dynamically generates a workflow on the fly based on your inputs

[drifting helm]

the real issue is that its using dotnet lol

[dim sonnet]

there's been a couple issues from that yes, but there's so many issues avoided thanks to dotnet, not to mention the major performance advantage

[inner vault]

yeah exactly ... how can i restore it in nodes i want to add lora to it

[dim sonnet]

restore what

[dim sonnet]

(also the main issue, install errors, is now fixed :D)

[inner vault]

like i used a custome node workflow but when i wanted to restore the example default i didnt know how

[dim sonnet]

are you looking for the 'load default' button?

[inner vault]

nooo that only loads Confy UI default not the stableswarms

[dim sonnet]

there is no swarm default workflow

[drifting helm]

yeah, the fact that you have a human readable language & its very fast makes up for the few extra errors that you have to deal with

[and also the fact that dotnet is just 2+ languages in a trenchcoat]

[inner vault]

there is this work flow which loads automatically but i cant get it in nodes

[tardy sundial]

would it have anything to do with the fact that I already had comfyui installed on my machine previously?

[drifting helm]

its an issue with trying to download a model from huggingface; so it shouldnt be an issue with the pre-existing comfyui

[dim sonnet]

As I said: there isn't one workflow there, it dynamically generates on the fly based on your input

(it also generates as an API call, not viewable nodes)

actually for that matter with your settings it doesn't even use exclusively comfy, the scorer goes to a separate tool

[tardy sundial]

Ok ill try installing again but with a different model, I started with the SD1.5 version, this time ill try the SDXL model and see what happens...

[dim sonnet]

if it fails again, you might just skip model download during install and download it separately

might have better luck downloading in browser

[inner vault]

ok so if i want to add lora what do i do ?

[tardy sundial]

Ok, ill try that as next step if that fails.

Do i just drop it in this folder: ... StableSwarmUI\StableSwarmUI\Models\Stable-Diffusion\OfficialStableDiffusion?

[dim sonnet]

wait like, a few hours or so? lol i'm currently working on integrating loras to the main generate page

[dim sonnet]

ye

[drifting helm]

wait you're the only dev on this?

[inner vault]

ok got it

@dim sonnet yo 1 more thing soes the refiner work with SD models ?

[dim sonnet]

yes

[dim sonnet]

I'm the main dev, but i'm not all alone

eg most of the internal stuff is built by comfy

[drifting helm]

ohh, so something like comfybox?

[dim sonnet]

huh?

[drifting helm]

comfybox is a frontent for comfyui

it has the worst code ive seen

uses gradio for frontend

but it works

[dim sonnet]

lol

[drifting helm]

it has lora support, mobile friendly

could be useful possibly

[inner vault]

bro if this UI becomes the standard with more support to plugins like control net and all... like its so convenient and so so much faster and less demanding that automatic 1111 ... like on A1111 i cant even run SDXL for some reason but here with confyUI support i can even run 6 batches of a 1024/1024 pics

and its easy to use

[drifting helm]

comfyui already has controlnet support, just that everyone thinks nodes are too hard lol

[inner vault]

i mean not gonna lie it is bit harder and annoying to use thats why this UI is big deal it makes that proccess sooo soo much easier

[tardy sundial]

looks like that worked, thx

[hearty notch]

[Error] Error in websocket handler: System.AggregateException: One or more errors occurred. (Value cannot be null. (Parameter 'key')) ---> System.ArgumentNullException: Value cannot be null. (Parameter 'key') at System.Collections.Generic.Dictionary2.FindValue(TKey key)
at System.Collections.Generic.Dictionary2.TryGetValue(TKey key, TValue& value) at StableSwarmUI.Text2Image.T2IParamTypes.ValidateParam(T2IParamType type, String val, Session session) in D:\StableSwarmUI\src\Text2Image\T2IParamTypes.cs:line 378 at StableSwarmUI.Text2Image.T2IParamTypes.ApplyParameter(String paramTypeName, String value, T2IParamInput data) in D:\StableSwarmUI\src\Text2Image\T2IParamTypes.cs:line 406 at StableSwarmUI.WebAPI.T2IAPI.GenT2I_Internal(Session session, ValueTuple2 input, Action1 output, Boolean isWS) in D:\StableSwarmUI\src\WebAPI\T2IAPI.cs:line 88 --- End of inner exception stack trace ---

[dim sonnet]

the image editing tools are gonna be the big one to really cement swarmui in place, gonna have to make sure to get it right

[hearty notch]

getting this error now

[drifting helm]

theres already been an ui like that yeah, comfyui so optimized, amazing work by comfy and devs, and stableswarm making it easier for new people to use it [while keeping them seperate]

[hearty notch]

this is what I am getting now after the last update, @dim sonnet

[dim sonnet]

oh, oops, update again, should be fixed

[drifting helm]

if i knew any js, id try and help; but alas, i do not want to know the cursed language

[dim sonnet]

lol

[hearty notch]

now it's only stuck on loading and nothing after

[drifting helm]

eh why not, i have nothing better to do than to complain about the 1660S

[dim sonnet]

ehh... CTRL+F5?

[hearty notch]

@dim sonnet btw, do you think having the option to kill and restart the server as a button in the UI will be a good thing?

to avoid closing and opening many times

[dim sonnet]

why

do you mean for updating?

[hearty notch]

well as an option not in particular for this reason, sometimes, after downloading a model, or so, or just for the sake of resetting the server during a workflow that gone wrong

could a helpful option maybe? just thinking out loud here, I am not sure if this is code-able

[hearty notch]

this did the trick 😄

woah!! what is this sorcery!!! hahah we can load them directly now? with no need to use CUI backend???

https://tenor.com/view/deadpool-touching-myself-tonight-wade-wilson-ryan-reynolds-gif-15162737

[dim sonnet]

you don't need to restart after download models, there's a Refresh button

[dim sonnet]

not fully implemented yet, still in-progress

[hearty notch]

any specific folder structure that we need to build?

[dim sonnet]

folder paths are configurable via server settings

[hearty notch]

so what kind of workflow that we will be expecting to see for the new changes?

also, is this something major? D:\StableSwarmUI\dlbackend\comfy\python_embeded\Lib\site-packages\torch\_utils.py:830: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage() return self.fget.__get__(instance, owner)()

[inner vault]

@dim sonnet am having this issue with the bar of madels and VAEs and these stuff ... basically whenever i click on one ... shouldnt a tab with Vaes or or madels pop up ? its not for me

[drifting helm]

not something to worry about id think

[inner vault]

like what ever i click on the bottom bar it doesnt seem to do anything

[hearty notch]

btw, is it jts me or there is a change in the render speed? I am notcing a very long pause between each render that was not there few days ago

[inner vault]

do you change models between those renders ?

[hearty notch]

no

I am aware of the loading pause when you change models

but only using the same

this is why I am asking 😄

[inner vault]

like how long of a pause ?

[hearty notch]

like 5 seconds or more

regardless,

I am only communicating something that was not there

[inner vault]

idk 😶 for me its same as regular comfy UI speed

[hearty notch]

thanks for the feedback,

which gpu you are using?

[dim sonnet]

you can ignore that

[dim sonnet]

new vae/lora stuff is still in-progress, doesn't do anything atm just a viewer

[dim sonnet]

er... you need to bring it up

click the bar and raise it

you have it slammed to the bottom there

[inner vault]

3060ti

[hearty notch]

any idea about these pauses @dim sonnet ? before it was like there is almost no pause between renders

now there is a very noticeable one

[dim sonnet]

when does it pause?

At the start of a gen, at the end, in the middle, ...?

[hearty notch]

between 2 gens

[dim sonnet]

Gen once, and look at your console

[hearty notch]

its a bit long compared to few days back

[hearty notch]

I am always keeping my consol on the second secreen haha always looking this is why I am observing this

[inner vault]

i see .... it would be better if there is a 1 click button to bring it down and when ever u click on what ever you need it brings it up half way if it is already down

thank you ❤️

bro i love how devs are actually listening and consistently working on bettering the program

[drifting helm]

compared to auto

[dim sonnet]

lol

VAE is implemented, lora and embed still to-do

[dim sonnet]

yknow i was thinking about having a lil, like, arrow button to quick-close/reopen before

[hearty notch]

when you change anything in prompt, without changing any other settings there is a like a 15 seconds pause, this I never experience before,

[drifting helm]

as in:
img1 generated
prompt changed then queued
[delay]
img2 generated

or

img1 generating
prompt queued
img1 finish
[delay]
img2

[inner vault]

yeah it would make life easier xD but when ever you bring it up it needs to get back to last position you set it in before just a small touch but idk it might not be worth the programming time

[hearty notch]

lol you are really killing me over here, but I will play along, prompt in, loading models, img 1 gen with refiner,,, img2,,, img3,,,, >>>>>>>>> change prompt,, delay,,, gen img4,,, img5,,, etc

hope this is better?

[drifting helm]

seems like theres some issue with queuing prompts; not your fault

[hearty notch]

this is what I am saying,,, there is an issue

forgive my ignorance, are you one of the devs in there? @drifting helm

[drifting helm]

yea; thought you were talking about generation time for each batch

[drifting helm]

no .w.

though i could find the issue;

[hearty notch]

seems just like clutring the chat with answers that are not helpful

can we keep it clear a bit for the dev to follow up with us?

[drifting helm]

i'm just trying to help!

[hearty notch]

was not helpful, but thanks for the efforts 😄

[cursive badger]

Looks like it gives a similar error

[dim sonnet]

added

over on the right

if you click that, it slams to bottom

and if you click it again, or any tab, it pulls back up

[dim sonnet]

uhhh... can you try using the comfy workflow tab to generate images? To see if it happens when swarm isn't touching the process at all

which I assume it will, at which point it's a comfy issue

(or if it is specific to something in swarm, i'm very confused)

[hearty notch]

this is only with swarmui, no comfy

when you change prompts these is a long pause

before the batch start to work

[dim sonnet]

er... can you open the file %appdata%\NuGet\NuGet.Config in a text editor and show me what's inside?

[dim sonnet]

Do you mean... you're not using a comfy backend at all (if so, what backend are you using)
or the delay happens even without hitting "Generate"?
or...?

[cursive badger]

Sure!

[hearty notch]

stock settings, I mean I am not using comfy custom workflow, just the normal generate tab

also, @dim sonnet is there anyway to control the refiner from here in comfy workflow editor?

as when comfy workflow is used, this refiner part disappear and we can not control the method or the default upscale,

[dim sonnet]

wow okay. How did... that even... happen.
idk
But
nuget sources add -Name "NuGet official package source" -Source "https://api.nuget.org/v3/index.json"
run this command and it should fix that

[dim sonnet]

when you use the generate tab, it automatically produces a comfy api workflow and passes it to comfy

if you have comfy as your backend, comfy is what does the actual image generation

[hearty notch]

i understand that now,

[dim sonnet]

you can still control those things, if your custom workflow has a refiner and upscale in it

[hearty notch]

when using upscale nods they do not show upscale models,

using latent upscale nods will not give good results, and using model upscale nods will only upscale the image with very bad artifacts, seems like the default refiner tab you have in generate tab is the yielding the best results, I will run few examples and share to explain better

[Error] Error in websocket handler: System.AggregateException: One or more errors occurred. (Value cannot be null. (Parameter 'key')) ---> System.ArgumentNullException: Value cannot be null. (Parameter 'key') at System.Collections.Generic.Dictionary2.FindValue(TKey key)
at StableSwarmUI.Text2Image.T2IParamTypes.ValidateParam(T2IParamType type, String val, Session session) in D:\StableSwarmUI\src\Text2Image\T2IParamTypes.cs:line 389
at StableSwarmUI.Text2Image.T2IParamTypes.ApplyParameter(String paramTypeName, String value, T2IParamInput data) in D:\StableSwarmUI\src\Text2Image\T2IParamTypes.cs:line 403
at StableSwarmUI.WebAPI.T2IAPI.GenT2I_Internal(Session session, ValueTuple2 input, Action1 output, Boolean isWS) in D:\StableSwarmUI\src\WebAPI\T2IAPI.cs:line 181
--- End of inner exception stack trace ---`
@dim sonnet

using the workflow in the editor is working, but when trying to use in generate tab, will get that error

[ruby glacier]

Hi Guys, do anyone know if the new StableSwarmUI have a support for Apple Silicon Macs, I have Metal acceleration via ----> https://developer.apple.com/metal/pytorch/ , but I see no difference, it is even slower than ComfyUI...... Some people in the chat, claim that this new and official UI will have multiple GPU support, it is True ??

[dim sonnet]

Apple silicon support: yes
Multi-GPU support: yes it does
Slower than comfyui: that can't be right, comfyui is the backend generator used by default

If you have a working pre-existing comfy install you can directly link to it

[ruby glacier]

I had a pre-existing comfy install, but I made a new clean install with the idea to use only StableSwarmUI in the future, but A1111 works much faster on my MBP M1......I hope this SS UI to have native support for Metal GPU family, because they are great for photo or video editing, but unfortunately at the moment not even good enough for SD image generation :/

[cursive badger]

I did not recognize the 'nuget'

[dim sonnet]

oh

uh

[dim sonnet]

Prooobably this?
dotnet nuget add source https://api.nuget.org/v3/index.json --name "NuGet official package source"

[cursive badger]

Worked! Amazing thank you so much!

[dim sonnet]

loras now work

not perfect, still in-progress, but it functions now

[hearty notch]

what about this? #🐝｜swarm-ui message

any idea why this error is happening?

[dim sonnet]

eh... does it still happen after an update?

[hearty notch]

yeah, start getting this error today

the workflow is working in the comfy editor and rendinf images, but once moved to generate tab it will error out

[dim sonnet]

please update and give me the error message as it appears after the update

[hearty notch]

[Error] Error in websocket handler: System.AggregateException: One or more errors occurred. (Value cannot be null. (Parameter 'key')) ---> System.ArgumentNullException: Value cannot be null. (Parameter 'key') at System.Collections.Generic.Dictionary2.FindValue(TKey key)
at System.Collections.Generic.Dictionary2.get_Item(TKey key) at StableSwarmUI.Text2Image.T2IParamInput.Set(T2IParamType param, String val) in D:\StableSwarmUI\src\Text2Image\T2IParamInput.cs:line 146 at StableSwarmUI.Text2Image.T2IParamTypes.ApplyParameter(String paramTypeName, String value, T2IParamInput data) in D:\StableSwarmUI\src\Text2Image\T2IParamTypes.cs:line 429 at StableSwarmUI.WebAPI.T2IAPI.GenT2I_Internal(Session session, ValueTuple2 input, Action1 output, Boolean isWS) in D:\StableSwarmUI\src\WebAPI\T2IAPI.cs:line 88 --- End of inner exception stack trace ---

here you go

[dim sonnet]

ooo that's actually a different (but strongly related) error

fixed

[hearty notch]

btw, are you now using normal RAM?? I see models loading there and not in VRAM

what is this sorcery? hahah

[ruby glacier]

Multi-GPU support: yes it does, it means only CUDA or including Apple silicon or AMD GPUs?

[dim sonnet]

Comfy loads models into RAM, and only transfer into VRAM once it's running

you can force it to always use VRAM via --gpu-only in extra args in the backend settings

[dim sonnet]

Yes, all of the above, you can even mix different ones

can even be remote GPUs (eg colab or rented servers)

[hearty notch]

looking forward to see it using shared ram and vram that will be dope haha

[hearty notch]

the error is still there

[Error] Error in websocket handler: System.AggregateException: One or more errors occurred. (The given key 'OfficialStableDiffusion\sd_xl_refiner_1.0.safetensors' was not present in the dictionary.) ---> System.Collections.Generic.KeyNotFoundException: The given key 'OfficialStableDiffusion\sd_xl_refiner_1.0.safetensors' was not present in the dictionary. at System.Collections.Concurrent.ConcurrentDictionary2.ThrowKeyNotFoundException(TKey key)
at System.Collections.Concurrent.ConcurrentDictionary2.get_Item(TKey key) at StableSwarmUI.Text2Image.T2IParamInput.Set(T2IParamType param, String val) in D:\StableSwarmUI\src\Text2Image\T2IParamInput.cs:line 146 at StableSwarmUI.Text2Image.T2IParamTypes.ApplyParameter(String paramTypeName, String value, T2IParamInput data) in D:\StableSwarmUI\src\Text2Image\T2IParamTypes.cs:line 429 at StableSwarmUI.WebAPI.T2IAPI.GenT2I_Internal(Session session, ValueTuple2 input, Action1 output, Boolean isWS) in D:\StableSwarmUI\src\WebAPI\T2IAPI.cs:line 88 --- End of inner exception stack trace ---

[ruby glacier]

Really cool as an idea 🙂 but in the Alpha doesn't work .... I hope it will

[dim sonnet]

this is also a unique error lol

[dim sonnet]

it does work in the alpha

[hearty notch]

one of the main reaons I never pushed myself into learning coding lol

[hearty notch]

where to use this argument if I may ask?

[dim sonnet]

that one should also be fixed now

[hearty notch]

will try now

[dim sonnet]

extraargs in server->backends here

[inner vault]

bruv you are a blessing to the community xD

one thing i noticed tho is when i pull it all the way down when it was open it still considers it open

[hearty notch]

so what ram is recommended now?

like 64?

[dim sonnet]

it worked different than how you thought it did, but what you're thinking makes sense so i revised it to snap like that

[dim sonnet]

if you have 32 GiB or more you're golden, if you have 16+ you might be a little limited but it oughtta do for SDXL base. Below that SDXL might be difficult to load

in gpu-only you don't need as much RAM, but you need a lot of VRAM

[hearty notch]

I can see that my 32 is hitting the edge now haha, I just got a new 4090~ not i need more ram. will I am selling my leg today lol

[hearty notch]

ok it's working now, and renders imgs smoothly, still it's showing this in console : [Error] Invalid/null/empty image data from ComfyUI server for 'ComfyUI_temp_osppl_00002_.png', under { "images": [ { "filename": "ComfyUI_temp_osppl_00002_.png", "subfolder": "", "type": "temp" } ] }

anything to worry about?

how to sue Lora now? is there any wiki or documentation to read?

[dim sonnet]

did you use a PreviewImage node?

[dim sonnet]

• download loras
• shove em in the loras folder
• refresh page
• select loras you want in the tab
• generate and enjoy

[hearty notch]

there is one, but I usually close my loop with save img only

[dim sonnet]

lora weighting needs to be added still