I've seen some really interesting post processing ideas on YouTube
#ad_discordbot (Fork of Fork of xNul's bot)
terse folio
halcyon quarry
That character can be downloaded and fought against thousands of other open source characters
halcyon quarry
Maybe my two favorite things improvised on that project:
- I saw a youtube video of the Imperial March from Star Wars being performed by a pair of floppy drives. Had the idea to make Jailbot fly in like the Star Destroyer in the opening scene, to that tune
- When the match ends in a Draw, had Jailbot transform into an etchasketch and draw a gun - like in Toy Story (which was inspired by Buggs Bunny / Yosemite Sam)
halcyon quarry
The thing that complicates this Regen / Continue thing is that user prompts can be sent in the background (Flows), and text can be generated and saved to history, but not sent to channel (tags behavior)
Just need to consider there may be gaps where history items do not have any associated message IDs
the list lengths can't really be a factor (as I suggested earlier)
halcyon quarry
Lastly (really) - the new bot behavior will have the experimental feature to merge multiple messages into one prompt…
keen palm
How's that?
halcyon quarry
I was thinking of ways to make it possible for the bot to behave more humanistic, if people care about that… this is going to be controlled by a combination of a ‘responsiveness’ paramater (1.0 = current behavior, 0.0 = busy, distracted human) and maximum delay setting
Additionally-
People that don’t response immediately will usually read multiple messages before typing
keen palm
Hmm, that could be useful
You could type a string of prompts that would be combined into one
halcyon quarry
So the idea I had was that if the bot’s behavior allows multiple messages to occur before responding, it prefixes them with the authors names and sends as a single prompt
Or, that it may possibly cancel its current response and be reprompted with multiple messages if a new msg comes while typing
The history manager function already has this started a bit… it’s just not in use yet. If it receives a user prompt but no Llm reply it starts collecting a list which will be joined by \n\n (2 new lines)
keen palm
I assume this would only work in the main channel for the bot
halcyon quarry
Should work anywhere… if done correctly… again this is pretty theoretical
an option will be to have name1 parameter be your server name and all messages get prefixed with user name
A good ‘user_bio’ param could be a explanation of the scenario that it’s a chat server
I’m sure that parameter is good for something
halcyon quarry
Ok I understand the concern now. Yes, we will need to collect pending prompts into channel specific lists
terse folio
What's the difference between recent_messages and session_history?
oh maybe to skip results of flows? so it's only typed inputs and bot replies?
keen palm
15:27:53.554 #205 ERROR [bot.main]: http://127.0.0.1:7861/sdapi/v1/txt2img response: "404"
Any clue here? SD was enabled upon startup of the bot with no issues
terse folio
Check the SD console, maybe that's the wrong endpoint?
Do you have an old version of SD?
Did it work in the past?
keen palm
No messages in the console about that. I did recently update SD. And no, I've never really tried with this bot
terse folio
It's also possible the bot has an outdated version of the api
Hmm no, that's the latest
Ahh, I think you have the wrong port
try 7860
keen palm
No. 7860 is webui, and 7861 is SD
On my computer
The imgmodel command worked correctly to load a model in SD
terse folio
keen palm
Alrighty
keen palm
What am I looking for here?
terse folio
just ensure that's stable diffusion and that the correct api is there
sdapi/v1/txt2img
keen palm
Yes
terse folio
What kind of http method is expected?
POST?
keen palm
Yup
keen palm
I haven't tried it recently, but it did work at one point with webui as well as oobabot
terse folio
I made a push to display the error reason on error
oh, push was too the dev branch btw
terse folio
if you're using vscode, you can switch branches in the source control tab
keen palm
15:52:26.908 #205 ERROR [bot.main]: http://127.0.0.1:7861/sdapi/v1/txt2img response: 404 "Not Found"
404 Not Found is to be expected
terse folio
hmm, was hoping for more insight XD
Sometimes servers would reply what exactly is not found
I've seen it used in the case of missing endpoints and missing items
if you want you can add a print statement to show the json being sent to the server
it'll probably be a really big payload because this bot sends default values as well
keen palm
Is there anything you're looking for there?
terse folio
It might be of more insight to you, like what model is being loaded? Do you have that model?
Or maybe it's trying to use a feature you don't have
I would use cmdline to run the endpoint manually
see if it works on its own
terse folio
that's because txt2img is returning None due to the 404
if you run /imgmodels does the correct model list come up?
terse folio
from /docs you can click on an endpoint and execute it
you likely will have to fill in some of the placeholders
keen palm
Hmm
The default sampler was DPM++ 2M Karras, which I don't see on my sampler list in the SD ui
terse folio
can't imagine they'd delete a sampler, but you could search that in the bot settings and swap them to something like "Eular A"
terse folio
make sure to restart the bot, I don't think settings will load if you change them like that
keen palm
i did
terse folio
Hmmmm
terse folio
The payloads need to be cleaned up,
There's no need to send all the default settings, it can cause bugs later down the line as SDwebui adds/removes parameters and the bot sends the older versions.
Had this experience plenty with TGWI as I would update it
that would also make debugging easier, as we can see what the bot is sending that's different from the default
keen palm
I think I'll just run something in SD, make sure all those settings work, then move them over into the config
dict_Base_settings, rather
terse folio
That's where I would start too, i'd love to know if something is different
halcyon quarry
I know A1111 changed up their sampler handling… didn’t know they changed the backend values
halcyon quarry
Tbh I think the A1111 API is not as dumb as TGWUI - I think you can send inapplicable parameters
Whereas you send one param not on the list to TGWUI API and it says fuck you
But, invalid parameter values is another thing
keen palm
Doing it manually with the wrong sampler gave an error response, though
halcyon quarry
I think there’s an endpoint to get valid samplers…
halcyon quarry
There's not even much there, since all the extensions are turned off
Yeah, thanks for mentioning that because it's now clear I handle settings for those bass ackwards
fixing that tonight
half now and second half after book time
(instead of always making extensions args and deleting them later based on config setting-
Just initialize them or not based on config setting)
Probably still delete the keys if they're not enabled in payload though
Kind of spinning my wheels as I still want to leave all those as default in base settings
Will add a note that user can safely delete extension params if not using them
terse folio
#1154970156108365944 message
Had a question earlier about how history is stored,
Thinking about the best way to get into that
@halcyon quarry
halcyon quarry
What's the difference between recent_messages and session_history?
recent_messages are all messages regardless of whether they were committed to history or not - mainly useful for the Flows feature, but the variables that these messages represent can be used in prompts as well
terse folio
I see
maybe could add a parent/from attribute to the message class.
Generated from a flow or user interaction
Later on this could lead to doing sorting/selections on the full history if needed.
halcyon quarry
I have another feature on the backburner where these variables will also be useful
terse folio
hmm, messages could also contain replies in them.
so a message could have a method to get the history up to that point or to start a new branch
halcyon quarry
I want to have a syntax that will essentially make a flow... but call it something like "multi-prompting".
So you can use it to generate multiple consecutive responses
The main use case I have for such a thing, is with the Forge Couple extension.
It could be easy to request a properly formatted Forge Couple prompt
I'll get to it eventually 😛
halcyon quarry
It's basically a better implementation of Regional Prompting
terse folio
oh I see, searched it up
halcyon quarry
Yeah it's currently Forge specific, but I think once Forge gets obsoleted (seems to be happening soon), I think it'll get ported
terse folio
looks nice,
I wonder if there's a way to get the discord View that Midjourney has with the region/inpaint feature
Maybe it's only accessable to MJ because it's a discord partner
halcyon quarry
Yeah, I've been daydreaming about such a feature in discord... even today I was, I swear
Haven't looked to see if it exists. Must now
The ability to draw on a canvas would be super awesome
I have inpainting working great, only caveat is you need to upload the mask image 🙂
terse folio
mhmm, mhmm, can't really think of another way outside of creating a webui
halcyon quarry
If I had no job and no family I'd be halfway done with that lol
Don't think it'll ever happen though
I've done some gradio and yeah, I've got too many goddamned settings
This might do it https://github.com/LeonZ2019/dbm-canvas
erm maybe not
terse folio
That looks like it's just for rendering images using Javascript
Python can do that using PIL/OpenCV
terse folio
That's sad, a cool feature!
halcyon quarry
I found a free ios image editor that has layering, so you can load an image and paint a mask over it - then upload to the bot
still pretty clunky
What I am going to add, and probably soon, is extension support for Segment Anything
terse folio
that's the text2mask model right?
halcyon quarry
It has a method that is pretty good at masking something in the image based on prompt
terse folio
that would work nicely on discord
halcyon quarry
yes in the extension its called Grounding Dino
maybe some other extension has similar feature
I've tried it twice and both times it masked the thing nicely
terse folio
It works for some more abstract concepts too, like "distractions"
I was trying to build a comfy ui workflow to inpaint random imperfections in a photo such as mess on the floor in a background.
There was a mask, but it was partially transparent, so a little hard to work with figuring out the right weight
halcyon quarry
interesting
terse folio
Better aproach would be to use a vision model and figure out what the subject is, and invert the mask to clean up the background
But this only works for the use case of single subjects, like a portrait
halcyon quarry
I'll bet that Segment Anything can take a prompt like 'people'
I could add some custom user param to invert the mask
(if one doesn't already exist)
I'll probably do that anyway for inpainting / Controlnet maps/masks
One thing I did to make extension param management easier, is initialize them as dictionaries then convert to list before payload send
because most of them need to be lists
terse folio
I'll bet that Segment Anything can take a prompt like 'people'
Yea, that's true, I could supply a batch of images and specify what the subject is, pet/people/whatever ^-^
It would also be interesting to have segment by image, like feeding in an image for reference of the target you want to extract
halcyon quarry
Woot, got my first tgwui pr merge
#github-commits message
2 lines!
terse folio
Woo!
Still waiting on mine, it's been weeks!
halcyon quarry
It’s probably just a 2 week wait time for every line of code
terse folio
Yea, bet a lot of testing would have to go into it, figuring out what everyone's code does
Copying variables is only necessary for certain types.
Things like lists/dicts/sets where they can contain variables or change.
A list variable is a pointer to the list object so you can reference it in multiple locations.
But strs, ints and floats aren't pointers.
llm_prompt = text creates a new variable that will not affect the old ^-^
I'm cleaning this up a little, but thought i'd write a tip about it!
halcyon quarry
Im skeptical
terse folio
try things out in an ipynb file with vscode, it's a live environment to run python code
halcyon quarry
It’s been a long time since I did the copy.copy with that, but it sure seemed like as I changed one the other was affected
I must have been mistaken
terse folio
It does work that way with lists, like duplicating the all_img_models so you can remove one from the new one
halcyon quarry
I know this all too well 😛
terse folio
That was an interesting issue to come across as I was learning python.
Figuring out why things were being affected randomly
Also for the same reason you don't want to write defaults in functions like this:
def test(items={}):
do stuff
because as you do stuff to items, the next time you call that function, all your previous activity will still be there since the dict is created when the function is defined.
So the correct way to do this is
def test(items=None):
items = items or {}
do stuff
There’s a lot of iterating over lists / updating in the Tags feature - was tricky to avoid issues related to this
terse folio
Yea, that's where you want to use deepcopy 😸
as you do
iirc it recursively goes through everything and copies it
halcyon quarry
I just learned from trial and error, if the variable has some depth to it better do the deepcopy
halcyon quarry
I have an interesting dict update function that isn’t used much, it’s to let the “param variances” tags have a cumulative effect
The tag value is a dictionary of ranges
And a function randomly chooses values from the ranges
terse folio
I did see some of that,
Fixdict acts similar to dict.update(dict)
But works recursively.
dict.update wouldn't update subdicts
halcyon quarry
Then the dict update will put together matching param values
terse folio
Interesting, I don't think I've seen that
there's still code i'm seeing for the first time haha
halcyon quarry
Yeah I love the recursive updates
The param variances is a cool tag feature, allows you to randomize state params or img payload params, within specified ranges
Bools will be random too
terse folio
Sounds nice ^^
terse folio
I had some functions for creating more random randomness.
Ensuring that the same values wont be picked twice in a row
It was inspired by I think iTunes creating an algorithm to make randomness more deterministic because people felt it wasn't working picking similar songs back to back
halcyon quarry
The auto-select imgmodel thing won’t select the same model twice… thats the only similar thing I have going
Did you see the loractl thing?
It’s in config.yaml
terse folio
The auto-select imgmodel thing won’t select the same model twice… thats the only...
Yes, it's similar to that, but splitting the list into 2 larger parts than just the last current model
like 50/50
So the randomness happens on the first half, and 1 at a time items trickle in from the other being swapped out in sequence
terse folio
Did you see the loractl thing?
I haven't, mostly just been in the python code
halcyon quarry
Well, I haven’t been using loractl recently because Forge isn’t compatible, but the extension allows scaling Lora weights on a per-step scale
I made a function that will automatically apply predetermined scaling rules to LORAs in the prompt
It works very well
Assuming A1111 merges the recent enhancements (let me know if you haven’t heard of this) I’ll be able to enjoy that again
It yields much better results for 2+ Loras, crossfading them instead of static weights
terse folio
I haven't messed with SD too much recently, I popped in to try out the transparent image generation 😸
halcyon quarry
Super cool shit too
terse folio
It yields much better results for 2+ Loras, crossfading them instead of static w...
Interesting,
It's kind of like my idea of that style that activates after a certain amount of samples
with a lot of keywords on quality the generated images seem more the same to me.
So applying those keywords after 30% render lets SD be more creative at first, then start bringing in the quality
terse folio
What's prompt editing?
having an llm write a prompt?
no, this happens in SD while it's generating
halcyon quarry
Sytnax for SD Webui that lets you modify the prompt at specified step
terse folio
ah, yea
my style is setup like:
[{prompt}:keywords ... {prompt}:0.3]
But, SD api doesn't like this, it tries splitting on "{prompt}" and errors because it expects 2 values
So I would do this from the bot instead of as a style
halcyon quarry
Yes, using positive_prompt_suffix or prefix tag
Have you considered that SD Dynamic Prompting is causing your issue?
terse folio
This was the error
terse folio
Have you considered that SD Dynamic Prompting is causing your issue?
This is the feature built into the SD webui
the [x : y : 0 to 1] syntax
halcyon quarry
Try disabling dynamic prompting
terse folio
Sure, will note that down next time I start it up
halcyon quarry
I had issues with Forge Couple because it suggested using an alternate syntax for regions than new lines (default), suggested syntax was {SEP}
Figured out issue was due to dynamic prompting
terse folio
is that an extension?
halcyon quarry
Yeah, I think most people have it because it’s very good
I’m sure you have it mr, randomizer 🙂
terse folio
I see,
I might have it, I found that when installing some big extensions there's it can corrupt SD or break things
I get spammed with missing Dll errors on SD startup haha
need to reinstall it
halcyon quarry
speaking of which I also replicated the behavior of Dynamic Prompting but for TGWUI
in the bot
terse folio
^-^
halcyon quarry
If you sent a prompt with {{one thing|another thing}} it’ll pick one at random
It also has the wildcards file feature
And all this other crazy syntax, I worked with ChatGPT for quite some time to reproduce all the features
I did not like their code, did not take a single line - just duplicated everything described in their wiki
terse folio
👍 Awesome
terse folio
I do have a lot of features on my todo list, inspired by various other tools,
One similar to that is langchain's example selection.
Picking random examples and shuffling them for each multishot prompt
That can help the model not get stuck talking about something that was mentioned in an example
since it will be gone the next moment
halcyon quarry
I believe that the wildcards feature I mentioned can do this
terse folio
mhm! i'm sure
halcyon quarry
Read my wiki entry on dynamic prompting
I think you just need to stick a txt file in the wildcards folder, filled with examples as you’ve referred to them
Then just include the syntax in lrompt whcih is ##filename
Tell me about a ##animal that escaped the zoo last week
There’s this user named Reality that is really ##cooldescriptions
terse folio
That's some interesting randomness,
can't really think where I'd use that.
I loved it with SD though, made some really big wildcards through a python script.
That give more life to a simple prompt.
For example most characters generated will have a simple, plain background, unless specified otherwise
halcyon quarry
Yes that’s true
Have you tried my M1nty-SDXL character yet via the tags feature, “draw ____”?
Try it on my server
terse folio
Not yet, just have been busy to explore the features of the bot.
And I don't think I could easily load an LLM and sdxl on this gpu haha.
Might have to find a smaller model for tmp tests, like 1B if those exist anymore
halcyon quarry
My server is essentially dead, come try out the image gen
terse folio
wow some really complicated regex with forward/back looking capture groups :O
halcyon quarry
chatGPT is a genius at regex
I probably only had to regenerate 20 times until it gave the correct answer lol
terse folio
I see, cool usecase, didn't expect that to work so well from Chatgpt
I feel like the patterns could be shortened by specifying [\w\|]
just accept all characters and "|" inbetween {}
nvm, it's pretty well optimized it seems!
halcyon quarry
I have a lot of complicated code that chatgpt wrote… a lot
Essentially every bit of complicated looking code
Including the dict update functions
Well, I suppose they’re not that surprising
Conking out, gnight
terse folio
Cya!
Making an update where regex patterns on compiled on start
not a noticeable change, but faster!
Good to know, copy and deepcopy just pass the value back in the cases there is no need to copy
so it's not wasting resources!
halcyon quarry
Subtle change, I'm going to change this
if state:
data = load_file(self._fp, []) # convert list to dict
self.load(data=dict(messages=data))
to this
if state:
data = load_file(self._fp, []) # load old file as list
self.load(data=dict(messages=data)) # convert list to dict
Also, I don't think the load file change will work because it looks like data may be undefined
We'll initialize it as None
halcyon quarry
Another thing I want to mention, is that initially all image models had to be predefined in a .yaml file.
Then, I added API support making it the default and ".yaml method" being optional.
A month or so ago I abandoned the .yaml method, since I added filtering methods to identify specific models / model types and define custom settings to them
Still a few traces of outdated comments apparently
I need to get better educated with the def function(variable:list) -> list I don't know shit about this arrow lol
Aside from that everything looks good will have to test a bunch before this goes to main
halcyon quarry
@terse folio Just noticed something in the recent "models select menus" thing you did, that I misread.
There's a line menu_ii += 1 that I thought was at the end of the iteration so that it would increment for each menu.
But it's outside the loop...
I had stuck my num_menus thing right below that (for controlling Submit button) so I just noticed this will always be "1" 😛
I think the same for that menu_ii line... is this intentional?
in any case I'm commenting out the Submit button for now b/c it does feel better without it
halcyon quarry
Pushed updates, mainly optimizations
- Everything from Reality (Thanks!)
- I reworked the logic for initializing / processing SD extensions for efficiency.
keen palm
The SD API is working just fine in Silly Tavern, so I dunno WTF
halcyon quarry
Remind me what problem you are having, we'll solve it right now
keen palm
Error 404 Not Found
halcyon quarry
Please start up the bot and copy/paste CMD when it first occurs
Ok, I am also getting 404 Not Found when I use A1111
halcyon quarry
I'll get this figured out...
It is connecting to the API in general, because on startup it is successfully confirming the client via API
keen palm
Yuppers
halcyon quarry
I'm going to dump the payload as a .json file (I have a commented out bit of code in the script for this)
keen palm
And it's changing model correctly
halcyon quarry
Then try playing with that in the API UI
keen palm
I did manually execute the API, and that worked.
halcyon quarry
I'm on it
Yes the issue is definitely with the sampler
wonder how the fuck you're supposed to enter the value now...
for specific scheduler
halcyon quarry
I see
They have a separate param now for the 'scheduler'
So now it wants sampler_name: "DPM++ 2M" and scheduler: "karras"
keen palm
Would it be as easy as simply adding that into dict_Base_settings?
halcyon quarry
Need to think how to handle this elagently for A1111 and Forge compatibility
For you, for now, yes
keen palm
I'll give that a shit
halcyon quarry
yea I simply removed the karras from my sampler name (no scheduler) and that is valid
scheduler is an optional param. But the sampler name must be valid
keen palm
I already had sampler_name as DPM++ 2M, and that no worky
halcyon quarry
It's because my default dict_tags.yaml has payload mods for SDXL and SD15 that override defaults
If you also edit that, should have no issues
keen palm
So far so good
I expect that the output image will be shit, but still
Okay, it wasn't as horrifying as I thought
How do I throw LORAs into it?
halcyon quarry
I have some example tags with example lora values
just duplicate and make them in similar fashion
The name of the lora is the filename of the LORA
does not need to include subdirectory
Ok, so this isn't as shitty as I thought it could be
keen palm
So you just include SDXL Tags grimace in a prompt, and it will add that LORA?
halcyon quarry
Just 'grimace'
It is part of the 'SDXL Tags' Preset
Check out dict_imgmodels.yaml
At the bottom - there are settings there to apply settings to specific model names or model types, based on how closely the image model matches the criteria
My examples show only tag_preset_name values, but you can also just define normal tags there
for example if you wanted a special positive_prompt_prefix for a specific model, you could add a new definition here with the model name as exact_match and add that tag
When imgmodels change, if the name is equivelant to exact_match - that is the extra information that gets loaded in
keen palm
Oh sweet. If I ever find a model I like, I'll do that
halcyon quarry
Some models suggest specific positive / negative tags to enhance them
Note that in all cases of the bot, tag items that are higher have higher priority
Payload parameters such as height and width it will only use the first value it received
so if you have 2 matched tags with updated height or width, it is keeping only the values from the first higher priority one
Here is how I'm going to solve this Sampler thing:
- Both SD WebUIs are happy with a sampler name that has no scheduler in it such as
DPM++ 2M - Therefore, I will no longer suggest something like
DPM++ 2M Karraswhich would still be valid for Forge only. - I'm going to add a warning message if the client is detected as A1111 and the
sampler_nameends with known scheduler names
(I may just directly extract the scheduler value while warning)
keen palm
I said: "take a photo of [my RPG character's name]". Since the character name tag was triggered, it used that information to create a prompt for SD, and the image is remarkably close.
It gave him a tail when he doesn't have one, but that's an honest mistake
halcyon quarry
My goal is to bring as many advanced ideas into this while making them as easy to use as possible
You could add a negative_prompt: tail
to the tag definition
Man I'm pissed to learn that the bot has probably been failing for A1111 users for a month - and no one reports shit
So glad you found this
keen palm
I'm great at finding problems!
Hmm, I could add a LORA to the character tag definition, couldn't I?
keen palm
Well, until about ten minutes ago it couldn't properly use SD 😜
halcyon quarry
😮💨
keen palm
So, I'd just put positive_prompt: 'lora:Name' into the tag definition?
Would that mess with format_prompt?
halcyon quarry
format_prompt occurs in the pre-LLM phase
By default, it will try to insert the value immediately after the matched phrase
However, if it is not found it will just add it to the end of the prompt
Believe me that this was a slippery thing to code
Almost have this sampler business tied up...
keen palm
Hmm, the positive prompt lora thing isn't actually getting passed through to SD
halcyon quarry
# Fix SD Client compatibility for sampler names / schedulers
known_schedulers = [' uniform', ' karras', ' exponential', ' polyexponential', ' sgm uniform']
sampler_name = img_payload.get('sampler_name', '').lower()
for value in known_schedulers:
if sampler_name.endswith(value):
# Extract the value (without leading space) and set it to the 'scheduler' key
img_payload['scheduler'] = value.strip()
if SD_CLIENT == 'A1111 SD WebUI':
logging.warning(f'Img payload value "sampler_name": "{sampler_name}" is incompatible with current version of "{SD_CLIENT}".\n \
"{value}" must be omitted from "sampler_name", and instead used for the "scheduler" parameter.\n \
This is being corrected automatically. To avoid this warning, please update "sampler_name" parameter wherever present in your settings.')
# Remove the matched part from sampler_name
start_index = sampler_name.lower().rfind(value)
fixed_sampler_name = sampler_name[:start_index].strip()
img_payload['sampler_name'] = fixed_sampler_name
bot_settings.settings['imgmodel']['payload']['sampler_name'] = fixed_sampler_name
bot_settings.settings['imgmodel']['payload']['scheduler'] = value.strip()
else:
logging.warning(f'Img payload value "sampler_name": "{sampler_name}" may cause an error due to the scheduler ("{value}") being part of the value.\n \
The scheduler may be expected to be a separate parameter in current version of "{SD_CLIENT}".')
break
Oh nice
halcyon quarry
Big ol block of this shit because A1111 decided to... probably do things better 😛
keen palm
Those bastards
So yeah, the loras are working for sure, but they're not passed through the bot character prompt
halcyon quarry
Please copy/paste the tag definition
keen palm
trigger: 'Zeke'
search_mode: user
format_prompt: "(Information about Zeke:
- Model: BX Commando Droid
- Home Planet: Geonosis
- Occupation: Wandering Sheriff
- Appearance: Red-Brown painted battle droid wearing a Cowboy hat and poncho
- Gear: Twin blaster pistols, IQA-11 sniper rifle, vibrosword, magnacuffs
- Distinctions: Outer Rim Law Droid; Feared and Fearsome; Veteran of the Clone Wars
- Approaches: Swift, Sneaky, Clever
- Roles: Warrior, Scoundrel, Scout
- Specialties: Gunslinging, Covert Ops, Investigation, Intimidation
- Signature Assets: Dual Blaster Pistols; Deputy Dirk (DRK-1 probe droid); Trusty Steed (Swoop Bike)
- Talents: Quickdraw; Trick Shot; Disguise
- Personality: Uncompromising Lawman, Defender of the Innocent)
{prompt}"
positive_prompt: '<lora:Commando Droid v1:1.0>'
That's actually the wrong lora name, though....
The other one has the correct name, and it's not working, but lemme check again
halcyon quarry
What should happen is that, if the user's prompt includes "Zeke" (or "zeke"), that tag will trigger.
It will process the LLM related items (such as format_prompt).
Then the LLM will reply.
Then it will attempt to find "Zeke" (or "zeke") in the LLM's reponse (the image prompt) and insert the positive_prompt
keen palm
Yeah, I just verified it did that for the other character
halcyon quarry
If the LLMs reply does not have Zeke, that just gets added to the end
I had to do some clever stuff to make it work this way
keen palm
Yeah, I see the lora being called now
The outcome isn't great, but I think that's just because of all the extra crap the character is adding to the prompt.
keen palm
But would M1nty-SDXL be passed the RPG character information from the tag?
halcyon quarry
Yes of course 😛
keen palm
I have the tags in the Gamemaster character file
keen palm
Hmmm, M1nty-SDXL isn't showing up under the character list, even after restarting everything
halcyon quarry
I have a custom parameter that can omit characters from the character menu
bot_in_character_menu: false
It will still work for Tags
If you want the character to show up in the menu, just change to True
keen palm
Oh I see I see
halcyon quarry
This is so when you realize you can have unlimited specialized character contexts for the Tags feature, they're not all in the menu
Can confirm this sampler thing is good to go now
keen palm
Sweet sweet
It looks like the 'draw' tag doesn't work unless it's in the main channel
Because 'draw' has to be the first word, but if it's not the main channel, the bot has to be @'d to get a response
keen palm
If it's not main, when I @ the bot, I do get an LLM response, but nothing sent to SD
If it's main, problem solved
halcyon quarry
debugging
keen palm
halcyon quarry
You may be missing important updates
keen palm
I've been updating on the regs
halcyon quarry
If I remove my channel from main channels and @mention with draw - it works
It used to get the 'discord.user_display_name` and update the prompt to remove it.
Now, the value is captured when switching characters and stored in the database.
So if bot.py is updated but the modules are outdated, it could be the reason
Ctrl+F to def on_message and put a print statement like this to see if your character name matches the stored value
print("bot_database.last_character", bot_database.last_character)
if text.startswith(f"@{bot_database.last_character} "):
text = text.replace(f"@{bot_database.last_character} ", "", 1)
Maybe later. Gotta do something else for now
halcyon quarry
Pushed: Fix 404 error for A1111 due to Sampler Name
Latest version of A1111 requires the "scheduler" to be separate parameter from the "sampler_name". This update will identify and correct the error while while sending a warning.
I'm testing my idea for using discord server name as name1, including a user_bio with server description, and sending multiple "prompts" as one prompt, prefixed by usernames...
I think it works 😛
The automatic @mentioning won't work well for this though
Going to see if it works better by updating the context to have a similar multi user thing happening
halcyon quarry
With some example dialogue of multiple users, now the response isn't so confused
I think the feature will be worthwhile to try out
keen palm
The draw thing is just because I use correct commas....
halcyon quarry
such as?
keen palm
I did @gamemaster, draw....
halcyon quarry
ahhh
halcyon quarry
If that's comon, might add that as an alternate thing for it to look for
keen palm
Yeah I can do that
keen palm
Trying to get it to draw two seperate characters in one image is... Horrifying
halcyon quarry
On my to-do list is to figure out an automated way to use Forge Couple (wouldn't be compatible with A1111 though)
The extension is already integrated, but you'd currently have to do some extra work to format the prompt correctly
keen palm
Oh neat
I wish I had an easy way to create images based on 3d models, because I already have good models for all these characters
halcyon quarry
You could put them in user_images folder, then trigger them as ControlNet input such as Depth model
Maybe even Reference model could help
If you can find a use for them in the WebUI, then you can replicate it in the bot
keen palm
I've tried that before with very limited success, at least on SD1.5. The overall design of the character is accurate, but none of the details or color
halcyon quarry
You could do automatic outpainting around the character
You could trigger inpainting model with the image as controlnet input and include a controlnet mask
If you put like a light grey color instead of pure white on the mask, it will slightly denoise the subject of the input image
Or you could use global_harmonius preprocessor
which gives it more freedom to slightly modify the unmasked area
keen palm
At this point it's just easier to fully render a picture on my own
halcyon quarry
I guess. There may be ip-adaptors that better handle inserting subject
the inpainting thing, it would work to put your subject into any scene
Just they would be in the same pose every time 😛
alternatively you could train your own LORAs
keen palm
Yeah, training a LORA might be the way to go
Which models have you tested M1ntySDXL on?
halcyon quarry
For SFW stuff, works pretty good with Llama-3-8b
In any use case, works great with NeuralBeagle14-7B-8.0bpw-h8-exl2
Also works OK for Mistral-7B-OpenOrca-GPTQ and airoboros models
That character context is going to yield better results from any model, though.
(probably)
keen palm
Oh, I meant SD model
halcyon quarry
garbage hands but pretty cool - most recent image made on my channel
I gave you a number of suggested models, I think 😛
halcyon quarry
Here is what the new behaviors are going to look like here
Actually, I think I need to change that first setting...
and Ill rename it all to spontaneous_msg
keen palm
Yeah, with that, it'll have a chance (depending on _chance percentage of course) of creating a spontaneous message, if there are a number of messages sent without interacting with the bot?
halcyon quarry
As I'm writing the description to you the logic doesn't make sense with my settings 😛
Actually I guess it does
0.0 chance - it will never get randomly prompted from the list of prompts.
1.0 chance - if the bot was not triggered to send a random message by the time it reaches max_wait - it will send one
Anywhere between those values, it is more likely to send a message at the target time.
Or something.
keen palm
Yeah, I was a bit confused by the max_consecutive thing, but now I get it
halcyon quarry
Yeah in case the channel is completely dead and you don't want it to keep messaging dead server
Or, if you want it to do some sort of auto-prompting task that will be disabled
keen palm
Make it so it can @ the person who has not said anything the longest 😉
halcyon quarry
I don't need target_wait - losing that
the chance will drive the likelihood of sending closer to the min_wait
I am adding one new cryptic feature to config.yaml which is server_mode
# Server Mode: Experimental feature to dynamically factor the server name, channel name, and usernames into the LLM payload. May improve responses, especially in a busy server with many users.
# If using Server Mode, it is recommended to also: (1) describe your server in the 'user_bio' parameter; and (2) revise your context/prompt to have example dialogue from multiple users (fictitious or actual usernames), or give some explanation of this world scenario.
server_mode: true
That could certainly prove useful
halcyon quarry
What this will do is:
- Set the name1 as the server name.
- Add additional manipulation to the prompts if the character's
responsivenessbehavior is not like a computer
prefixing the prompts with usernames
grouping prompts together
potentially cancelling something it's writing to consider new messages for its response
I won't be itemizing these settings, either ur IN or ur OUT 😛
keen palm
Oh boy!
lol
Everyone else on the server keeps putting an extra space before draw, and it completely screws the whole thing up
halcyon quarry
You can create another trigger that has a leading space before draw
keen palm
I might have to, for these chuckleheads
halcyon quarry
{| }{draw|generate}
terse folio
I'm testing my idea for using discord server name as name1, including a user_bio...
How's that working?
name1/name2 are formatted for the whole prompt.
So you'd have to pick some neutral name.
I just tested to see if you could remove the name entirely so it's inserted as part of the content. Doesn't work that way, TGWI just puts the default "you/ai"
Also hahaha funny waking up and seeing my name in the screenshots 😸
Interested to see how you did multiple users through the chat format
best I could come up with.
Could get better results by changing the prompt template for the chat mode and removing the name entirely ^-^
halcyon quarry
I’m adding a small function before llm_gen() that sets user1 as i.guild
terse folio
Oh interesting, got to try that out too!
Could also try channel name since those have a shorter character limit usually.
General: Kat: Hello!
halcyon quarry
We could collect channel specific histories…?
terse folio
Of course!
create a dict in the bot class/globally
it will be {channel_id: {history, and other info}}
As you send messages, add to history.get(channel_id)....append() or whatever
dict.setdefault(key, {}) will create the key if not exists, or return it like .get
halcyon quarry
I’m on the road right now using the microphone input I’m very excited about this idea of channel specific histories I think it’s gonna be really cool especially the thought of auto loading the history for each channel on start up
We will just have to pre-fix the bedtime from text Jen web UI function with the channel ID
For stored Jason files
keen palm
wat
terse folio
We will just have to pre-fix the bedtime from text Jen web UI function with the ...
Your biological LLM temperature was set too high
I think you also added too much repetition penalty "text jen" 😸
keen palm
BTW, this whole train of thinking is something I thought about bringing up, but didn't want to
halcyon quarry
Too late I’ve trademarked it already it is the A.D. discord bot histories feature TM
keen palm
(as envisioned by Speedy)
halcyon quarry
We will make it you’ll break it
keen palm
Oh you're damn right I will
Is the loopback tag supposed to output an image at the end?
halcyon quarry
Yes
keen palm
That's what I thought
It goes through the four steps, but I receive neither an image nor the final prompt
halcyon quarry
It was bugged after I added your request to format_prompt multiple times
I just fixed Flows like 2 days ago I think
Update required 😎
keen palm
Pretty sure I updated after, but we'll see
halcyon quarry
I had same result you described until fixing it
keen palm
Hmm, now no models in my list
Yes, I copied the bot.py file over. Yes, I restarted discord
Hmm, none of the lists are populating
halcyon quarry
also replace modules folder
keen palm
I didn't make any changes to it
keen palm
Yes, those are all updated
They were all recent, but I went ahead and just grabbed all the files again anyway
Oh I see. The entire menu selection process is different
halcyon quarry
Lol
keen palm
It now doesn't pop up a list when you type /llmmodel
halcyon quarry
Yes, big improvement
keen palm
But also big confusion
People expect parameters to display when you use slash commands
halcyon quarry
Fortunately its a hybrid command 🤓
Main difference now is the lists don’t have to be populated and frozen on startup
halcyon quarry
Now they get current files when cmd is sent
number of menus will adjust dynamically to accomodate changes
Huuuuuge thanks to Reality
keen palm
Good job, Reality!
halcyon quarry
Well, I started messing with it then Reality made it 10x better
The update shaved 500 lines of code no exaggeration
keen palm
I love it when that happens
halcyon quarry
loopback working for ya now?
I spent 5 minutes total on thinking of that Flow and writing it, just needed to put some kind of example on paper
keen palm
I'm using XL
halcyon quarry
I’ll often see details from waaaay deep in the prompt show up
keen palm
And many of the more salient ones are just ignored
But also the prompts are too long. They get cut off midsentence.
halcyon quarry
if you want even longer increase the max new tokens
keen palm
Right right
Some LORAs are "_____ style" but that causes problems when you try to use the /image command. It forces you to select a style
halcyon quarry
I don’t follow
If you start typing style it forces the param?
Just add _ style as part of the positive prompt tag
keen palm
When I type the name of a Lora in positive prompt that has "style" in its name, the command forces a style type to be chosen
halcyon quarry
Ill see what I can do
keen palm
How does the trumps thing work with tags?
halcyon quarry
all trump params are collected to a “Set” (list)
All matched tags have their triggers compared to this Set
If any trigger matches, it’s trashed
keen palm
I see. Possibly.
halcyon quarry
So, you may have a more generalized 'catch all' tag that gets triggered easily.
Then, you may have a more secialized tag with more specific trigger and settings. You can trump the catch-all one
keen palm
Yes, I was planning to implement something like that. Just not entirely sure on the syntax
halcyon quarry
It's basically the same format as 'trigger' except without the special syntax {one|two}
trumps: 'one trigger,another trigger'
If another matched tag has a trigger value one trigger or another trigger it will be trumped
I'm not sure what you are talking about with the /image command... I was expecting the 'style' to force the parameter but it did not at all
also on mobile app, when I write 'style' nothing funky happens
keen palm
Let me try it on the computer. I was doing it on phone
Yeah, fine on computer. Still fucky on my phone
halcyon quarry
android?
keen palm
Yup
keen palm
Hey, micosoft has nothing to do with this
halcyon quarry
adding things 👀
keen palm
More??
halcyon quarry
Now capturing last_user_msg time per-channel to the database for the time checks.
Also capturing a few basic statistics that may factor into the bot behavior (last llm gen start time / (avg. tokens/sec)
halcyon quarry
keen palm
Can those stats be pulled up in discord?
halcyon quarry
Yeah I could have a command to return the statistics
Since I only have the LLM statistics for now, the cmd will be labeled for that exclusely
halcyon quarry
pushed my meager progress to dev branch
keen palm
Could total context (I guess from the last generation) be shown on statistics?
halcyon quarry
Is it so you can share it? Bc it should already report that in cmd window…
keen palm
It does. Just easier to see it through discord
I'd be running the bot from my home computer but would be using discord from work.
halcyon quarry
TGWUI has a simple function that I imported for counting tokens in a given text
They do not have available such a simple thing to get current context length...
terse folio
I'm pretty sure the Openai API returns the token count on completion, could take a look at the code there
halcyon quarry
ChatGPT is describing a means to capture a print statement from a function without modifying that function at all
terse folio
yea, reroute sdtout, but there should be a way to just access it
halcyon quarry
yep exactly
terse folio
Issue is the results could get messed up with multi threading
like one print statement happening inside another
halcyon quarry
Here is where TGWUI tells you the current context length https://github.com/oobabooga/text-generation-webui/blob/abe5ddc8833206381c43b002e95788d4cca0893a/modules/text_generation.py#L403C125-L403C140
ChatGPT conversation https://chatgpt.com/share/3d6c5e33-f12c-444a-b249-4898c9dc1383
Although yes this looks a bit slippery lol
This solution doesn't work in particular
terse folio
Well, it could work since you're not dealing with async requests to tgwi.
And as it works right now tgwi can't generate in parrallel.
But, this would be capturing bot logs and other logs
it's a bit too hacky
Reminds me of the time I tried parsing json myself from an API by doing splits and all because I didn't know about json!
There has to be a better way!
The way the openai extension does it is by calculating the tokens again
If they can do that every response, the performance hit probably isn't that bad
halcyon quarry
Just not sure 100% the text I need to count tokens on to get that value.
Maybe just context + len history visible (limited to truncation length)
and deduct max_new_tokens
too much effort for this statistic lol
terse folio
you'd just pass the full prompt to the encode function
I see, chat mode doesn't really return a prompt
prompt = apply_extensions('custom_generate_chat_prompt', text, state, **kwargs)
if prompt is None:
prompt = generate_chat_prompt(text, state, **kwargs)
did find that the chatbot_wrapper uses this to create the prompt
Nevermind, that might be a bit too much effort to rewrite considering it doesn't include the history part
I would rather call the "chat_prompt" openai api endpoint to generate the prompt and get the length of that
halcyon quarry
He made a simple function for superbooga called get_context_len(), its too bad he didn't do something similar in here
but that code isn't relavent to this
terse folio
passing history[internal] to chat_prompt() should get you the same prompt as chatbot_wrapper ^^
But, try out your idea with reading the print statements, see which works out better
halcyon quarry
I don't think I'm trying it 😛
terse folio
with the print statements, I would add a regex match ontop to ensure the correct info was captured
halcyon quarry
I had no main reason to get this statistic other than, nifty to know
terse folio
yea
hmmm, that makes me wonder if there's a way to hook into the logger from tgwi
if that data is being sent through the logger library and not prints
halcyon quarry
As for the other statistics I'm collecting - know the average tokens/sec could be helpful for timing LLM responsiveness
I have these logging to database.yaml but will have these go to a new statistics.yaml
terse folio
many fun files!
halcyon quarry
the llm_gen_time_last will go to database since it will just be used for calculations purposes
I plan to separately track there in database, the average tokens/sec for the current session.
This will reset on LLM model changes
terse folio
Could just keep that in ram since it resets on model change
terse folio
try to limit the writes to disk when not strictly necessary!
Like rewriting a database on every message is a bit over the top
It's just good practice later down the line when working with larger data
halcyon quarry
Well, that information will get lost otherwise and will then be false information 😛
terse folio
Well, that information will get lost otherwise and will then be false informatio...
The way I solve this is by dispatching a timer for a set amount of time.
maybe a few seconds, or a minute.
After this timer ends, it checks if the database has been saved within that time, if so, cancel.
if not, save now.
This lets me update values without triggering a write to disk each time, but still keeping the data safe.
A good example of this is I was working on an image library, doing embeddings and captions of thousands of files.
The bot would scan in all these images, get info like file path, other metadata, hashes... etc
And it would get saved to a SQL database.
But in the first tests it was really slow because I was commiting after every entry.
I learned to do larger batches of data and commit less often
Using the same timer idea ^
halcyon quarry
Currently, we're saving a file that is < 500 bytes with less than 30 lines
terse folio
Yup, that's fine for now :)
halcyon quarry
let's not worry too much about it 😛 yet
I'm such a dolt when it comes to managing separate branches
terse folio
Just need the 2, one for people to download, and another for testing ^-^
halcyon quarry
Ok well I just realized the one thing I didn't realize...
When using github desktop, did not realize that when you switched branches it actually swaps all the files in Explorer
I'm wondering why the fuck it thinks my local copy of dev is up to date when I clone a fresh main, switch to dev and click fetch origin
terse folio
Yup, same happens in VScode, it will swap your files so you can work on the branch
If it speeds up your workflow, many of those features are available in the vsc too, haven't used github desktop, what's it like?
terse folio
:)
terse folio
which code?
halcyon quarry
Adding statistics.yaml - and I believe I'm duplicating one of the other classes as a base such as Starboard
adding that path to utils_shared
terse folio
duplicate active_settings, should be the simplest one
halcyon quarry
right, seems simple enough
class Statistics(BaseFileMemory):
This means it's a subclass of BaseFileMemory yes?
Am I reading this correctly...
If I want to store the last_llm_gen_time in class Statistics, if the value is prefixed with _ like _last_llm_gen_time: float then it won't be included with the save function
terse folio
Yea
You can also set a variable with "save_now=False" and it will keep the variable in memory until something else triggers a save.
I did that with last_message_time
halcyon quarry
# Store time for statistics
bot_statistics.set('_llm_gen_time_start_last', value=time.time())
you don't have to use .set()
Just assign the attribute as bot_statistics._llm_gen_time_start_last = time.time()
.set() is used to trigger a save to file
you can also use syntax bot_statistics['key'] = value
this will not trigger a save on its own
halcyon quarry
I think I got this figgered out now, thanks
Yes I do - it initialized the file, saved only those statistics and not the _last_time one
terse folio
Woo!
halcyon quarry
I'm doing something wrong here though
class Statistics(BaseFileMemory):
def __init__(self) -> None:
super().__init__(shared_path.statistics, version=1)
self._llm_gen_time_start_last:float
self.llm_statistics:dict[str, float]
def get_statistics_dict(self, prefix):
attribute_name = f"{prefix}_statistics"
return getattr(self, attribute_name, {})
...
@client.hybrid_command(description="Display performance statistics")
async def statistics_llm_gen(ctx):
statistics_dict = bot_statistics.get_statistics_dict('llm')
it's giving me an empty dict
terse folio
can you push to dev
halcyon quarry
yeah
terse folio
i keep my discord a little small 😸
halcyon quarry
haha
terse folio
and helps with the context of other parts of the file to see what's going on
halcyon quarry
pushed
this is relatively new territory for me
But nothing new, no growth 😛
Note that the update_statistics is working correctly, using similar approach
oh I know the problem
derp
terse folio
I'm a little lost there
but I have an idea to make the code utilize the functions built into the class
Add your fix, and i'll merge my changes
halcyon quarry
I'm just not well versed with the getattr function.
I see I essentially asked it to give me an empty dict rather than the value
terse folio
yea, it returns an empty dict if the attribute is not found
is llm_statistics populated in your statistics file?
halcyon quarry
🤦♂️
terse folio
wait no
statistics gets passed down to statistics[flag]=val
after it's set to the class
so that should work 🤔
very interesting
halcyon quarry
adding more prints
terse folio
little issue here is you have a function named the same as a variable
this function will be deleted on load
halcyon quarry
it doesn't like this.
def get_statistics_dict(self, prefix):
print("self.llm_statistics", self.llm_statistics)
print the attribute_name, perhaps it's setting some other name
halcyon quarry
attribute_name llm_statistics
When I tried printing self.llm_statistics it does not print and the command hangs until timing out
there's probably something very basic here in the init I don't understand
terse folio
I'm not too sure either, im fixing up some other things and then will look into that
halcyon quarry
So we don't step on each others toes, are you screwing with the per-channel history management?
Might get into that now
terse folio
im only touching the database file
terse folio
fixing this
halcyon quarry
oopsies 🙂
terse folio
PyLance is useful there, it will highlight your variables based on if it's a function or not!
halcyon quarry
getting it now, no more delaying
terse folio
halcyon quarry
ah yes I see it highlighed in yellow now that I installed pylance
very nice
is there something to ignore the import errors?
terse folio
I think your issue is you're trying to get attribute for "llm_gen_statistics"
but the class only has "llm_statistics"
halcyon quarry
😂 bloody hell man
terse folio
I'd recommend not doing getattr stuff, because the IDE can't do checks on strings to ensure they make sense
I would just access the variable directly
bot_statistics.llm_stats.get()
halcyon quarry
thumbs up
terse folio
if I can figure out how to connect to multiple remotes in the same repository, i'd push directly to your branch, and we could work together more live
Pushed the fix for the function thing
halcyon quarry
The thing I need to adjust now, is collect them to _session... while tallying to totals
tyvm
pylance changed all the default code highlighting, this'll be great to get used to
terse folio
this can be shortened to if not flag_name in statistics:
making more updates to statistics
halcyon quarry
The main intention I have, is that I do like my idea of allowing a typing_speed behavior (changing that now from maximum...)
If I factor in the avg tokens/sec // session avg tokens sec - can set a reasonable delay before generating text
Well, can set a reasonable time to initiate "Typing..."
And delay to actually generate
The alternative is to set "Typing..." and generate immediately, then wait to send it
avg tokens per response may be a factor too... anyway, will be able to tinker with it once the data is there and the basics are there
terse folio
i'm confused, what's the goal of using tps for typing?
halcyon quarry
you probably should be confused 😛 lets see
Well it would have to be a combination of average tokens and avg tokens/sec - to take an educated guess to how long it will take to generate its next response
Let's say someone is running a 3b model on a 4090 or something at like 200 tokens/sec, but they want the character to behave like a normal person
halcyon quarry
yeesh
Anyway I think with these statistics we can take a rough estimate on delay timing to match the desired typing speed
If the character is supposed to be someone's old grandma, a few settings we have will simulate it
terse folio
Let's say someone is running a 3b model on a 4090 or something at like 200 token...
I process that differently.
-
Run generation in async task that will return a future.
-
I first calculate a reading time of the user's message.
-
await sleep that time.
-
Now start typing.
-
text = await wait_for(future)
-
Get time we typed so far
-
calculate how long it would have taken a human to write that text.
-
Subtract time we typed so far from calculated human typing time.
-
await sleep for human typing time.
-
send text reply
keen palm
Give it a 512 context limit
halcyon quarry
Give it a 512 context limit
will still spit out the response instantly lol
terse folio
will still spit out the response instantly lol
gave you a solution ^^
keen palm
Actually, it should ignore the prompt entirely and only make up stuff based upon its character card
halcyon quarry
I know - that was my first idea as well
But, I kind of don't like the thought of generating then waiting
Part of the new behaviors I had in mind, is the possibility that the bot may cancel its current response if a new message arrives - and then handle both as one prompt
How many users would use this, instead of default "Bot is a computer" - I don't know, but this would be pretty cool shit IMO
terse folio
But, I kind of don't like the thought of generating then waiting
this is how the streaming api currently works.
Bot generates as fast as possible, but your client may read the results at its own pace
if you cancel generation, it will stop further than what results you have because it's generating ahead
halcyon quarry
hum
Let's assume we're using the streaming API (we're not) - it started generating, a new message lands in discord - the character has a high responsiveness so it stops shortly after - and now we have half the generated text
Then what, resubmit with the updated prompt (has both messages) and have it continue
terse folio
I think it would be a better user experience to start generating ahead of time.
Even if you interrupt the bot.
This way the bot can start processing new responses faster, because it will already be done with the first before users have the chance to iterupt it
halcyon quarry
Yeah... ya know, it just seemed wasteful in terms of like, energy conservation 😛
But screw it, it will be in the user's hands
terse folio
You can simulate the typing yourself by taking the time the bot has typed, dividing that by X, and trimming the list by that amount and sending that
halcyon quarry
if they don't like watts going down the drain, then turn off these silly features
Alright I'm sold on generating then waiting
terse folio
Yeah... ya know, it just seemed wasteful in terms of like, energy conservation �...
yes, I think it's wasteful too,
But as pointed out, streaming works this way too.
if a user cancels the streaming request, it might have generated more than you can see
so if that works for people, I think this would be just as fine :)
pushed some changes
halcyon quarry
So in the end, these statistics are a novelty 🙂
But novelties can be nice
terse folio
when adding new statistics, you'll have to copy this line as well
this initializes the statistics class for it
halcyon quarry
I just got terrifying e-mail that we are going to have no internat for 1 hour
halcyon quarry
Also no Wifi
terse folio
anyway, I'm off for now
terse folio
ouch, that's no fun,
same here, have to walk down the street to get signal
halcyon quarry
thanks for the help! will grab that fast
terse folio
Merged
Btw, bot_statistics.llm_statistics
we could shorten the name here 😸
Maybe, bot_statistics.llm
halcyon quarry
OK - I got it 😛
terse folio
noticed a little issue,
should be bot_statistics.llm_statistics.data
as a quick patch
halcyon quarry
bot_statistics.llm.data
great, you made the code even mroe confusing to understand XD
keen palm
https://tenor.com/view/avengers-endgame-hulk-bruce-banner-i-see-this-as-an-absolute-win-gif-17006739
terse folio
I can explain it tomorrow, I tried to mimic the idea you were going for with the function that updates flags.
But converting that to not use getattr.
In your code I ended up not even using the update function and just did bot_stats.save()
So maybe it could have all just been a dict!
We'll see later, I'll write a note to see if that works better
terse folio
time.time() is in seconds?
yes
Theres also an option to get nanoseconds as an int, this will eliminate any floating point error
but that's for stuff like performance tracking
halcyon quarry
yeah, I think I'm actually going to use something like round(value, 3) for these
eh, 4
halcyon quarry
I'm tempted to add 'time_total_secs' but maybe at some point (number > X) it then stored in minutes
This is a really dumb distraction I got into here
terse folio
Should write a utils_misc function for that.
Formatting time rounded to the nearest hour/minute/second based on size
halcyon quarry
Ok I know... when using the command the label/value can just be manipulated, while we store it as seconds
terse folio
This is a really dumb distraction I got into here
I know I have something like that somewhere
Lots of distractions over the years!
keen palm
Haha, the fact that you're focusing on that kind of highlights the entire distraction issues 😉
terse folio
My workflow is like those zooming in on a fractal videos.
Focusing on more of the fine details while everything else gets forgotten
Though I have improved lately :)
terse folio
^-^
halcyon quarry
although Ill have it return the label separately
yes looking better now
Commited to main
keen palm
Is it now on to me to update and break things?
halcyon quarry
I need to modify the update instructions...
From now on it's mostly going to be:
replace bot.py
replace /modules/
I need to chat with Reality about whether we may want to just git.ignore the user settings files
and then recommend installing to textgen-webui and updating there
keen palm
I assume it's going to create the statistics.yaml file
Ahh yes, there we go
What do I need to break here?
halcyon quarry
@terse folio I think if we could sit down for like 5 mins, we could probably remove /internal/ from the main package
vestal python
I'll be back up and running to test things finally tomorrow x.x I've been away too long. Seems as though there's a lot of work being done.
halcyon quarry
Well this project has been blessed by a real and very capable coder 😛
Things are much more optimized now
keen palm
Where are the img resolution settings defined?
vestal python
Was there some talk about using the textgens openai api?
halcyon quarry
Where are the img resolution settings defined?
dict_imgmodels.yaml
vestal python
nvm I'll look, ty
halcyon quarry
I know how to make the bot work with the API - I'm just not interested yet
I don't have a flexible, practical means to keep the TTS support
keen palm
`dict_imgmodels.yaml`
That's only the 1:1 ratios. I'm wondering about the 1152*896 ones.
vestal python
I was looking at some ridiculous way to incorporate an api call to trace back to sillytavern to call back to ooba 
it was just a silly idea I've been working on with just a simple webapp.
halcyon quarry
@keen palm Thanks for the reminder - the controlnet dict has become obsolete
halcyon quarry
I was looking at some ridiculous way to incorporate an api call to trace back to...
The TGWUI API is also very annoying in that it requires specific parameters in the payload, no more, no less.
With current method of the bot, the minimum parameters just need to be required and it will ignore anything that is inapplicable
the API payload has all sorts of differences from the normal payload
vestal python
No I get it. You can add more endpoints to the api to match what you need, but then everyone who uses the bot would also need to add that into their openai configs for textgen.
halcyon quarry
Here are all the modifications that have to be made from current payload, for the call to succeed
But again, this is just a mild inconvenience - I really want to keep the TTS support
keen palm
`dict_cmdoptions.yaml`
This is just for the /image command, yes? What about when you ask for a prompt to be created through the LLM?
halcyon quarry
dict_tags.yaml
keen palm
Yeah, I updated those, but they're not being used
The active settings still has the previous resolutions
halcyon quarry
When you use /imgmodel command, it will try and identify what type of model it is using the presets thing I have in dict_imgmodels.yaml
It will then update activesettings.yaml with the information it finds to be the closest matching to the new model
If you want the LLM to output different sizes when saying something like "portrait" or "landscape", dict_tags.yaml already comes with some good examples
As we speak I'm adding a new tag parameter, aspect_ratio
keen palm
There are tags already present in active_settings. Are those not used at all?
halcyon quarry
Those are the character specific tags, and the imgmodel specific tags
activesettings.yaml gets updated automatically when you change characters or imgmodels
keen palm
I've got a setting hidden somewhere that keeps changing the resolutions back....
halcyon quarry
yes, guarantee you its the preset matcher in dict_imgmodels
Note that almost all XL models include 'xl' in it.
If the filter includes xl, it will score a point if the file includes xl
if the file does not include xl it will deduct a point
keen palm
dict_llmmodels?
keen palm
Yeah, that has the square aspect ratio resolutions, but I'm trying to do a landscape. Landscape keeps getting reverted back to 1152x896
halcyon quarry
Essentially:
- Use the main filter/exclude setting at the top of the file, to filter the models that actually load.
- Try including things that are universal to the models you want to have particular settings
halcyon quarry
Yeah, that has the square aspect ratio resolutions, but I'm trying to do a lands...
I'm trying to do a landscape. Landscape keeps getting reverted back to 1152x896
What is the problem exactly?
If you always want landscape sizes, then go ahead and just replace 1024/1024 with the size you want
otherwise, just apply a tag definition always
keen palm
There are already tags for landscape versus portrait, though. I see them in the active settings file.
halcyon quarry
Every image I ask for it uses a match tag value for the dimensions
yes
I know the problem
The priority of tags is user > llm > userllm
(search mode)
So if you have 2 different payload tags that affect Width and Height, the priority does matter
keen palm
I shouldn't. I haven't made any different ones.
halcyon quarry
Not too long ago I updated te dict_imgmodels.yaml dict to swap the order of the tag presets
I recommend:
SDXL Tags
SDXL Payload
keen palm
The ones in dict_tags?
halcyon quarry
No - just make sure the order in dict_imgmodels is as I showed
Tags, then Payload
This lets the Payload preset have lower priority
As the specific 'portrait' and 'landscape' type tags are in the SDXL Tags preset
keen palm
Yes, that's the order, and I've updated the resolution in the tags
halcyon quarry
And you see those tags in activesettings?
When the tags are written to activesettings, they are written in the same order as they appear in the tag presets (they do not get sorted alphabetically or anything)
keen palm
They appear in active settings, but not at the same resolutions
halcyon quarry
This is because the model is being matched more closely to other presets
It's parameters are not being guessed correctly
If the desired preset is for 'xl' models
then just make sure your model filenames have 'xl' somewhere in them
You can add some exclusion like '15' which will increase accuracy
keen palm
I actually just put that model in as an exact match
halcyon quarry
Note that if you have it in a subdirectory (not in top level models/Stable-diffusion/), then the 'exact value' is different
keen palm
It's in models/Stable-diffusion
halcyon quarry
At the top of the dict_imgmodels.yaml file is a comment which explains that the values are fetched via API and you can see the exact model names by going to the API and fetching them
OK
keen palm
It should be getting the values from the vertical and horizontal tags under SDXL Tags, right?
halcyon quarry
Yes
The "Base Tags" will not appear in activesettings
Only the ones associated with the current imgmodel, and the current character
Base tags have lower priority than imgmodel and character tags
keen palm
Hrmmm
So why are the horizontal and vertical tags getting loadedinto active settings with the wrong resolutions?
halcyon quarry
Search for those resolutions in base tags file
youll find they are probably under a different tag preset than you think should be matching, but is not actually matching
halcyon quarry
So if it says in there
tags:
...
payload:
width: 1024
...
And you go to dict base tags, you dont find 1024 after Ctrl+F?
keen palm
1152, but no, it's not anywhere in dict_tags
halcyon quarry
It sounds kind of impossible, so I can't really help unless you share the files
could DM me
keen palm
Which files do you need?
halcyon quarry
activesettings, dict_imgmodels, dict_base_tags
keen palm
dicts_tags, you mean?
halcyon quarry
yep
halcyon quarry
Think I got this custom aspect_ratio tag done
Will be more flexible than setting specific width/height
and easier to use with the tags-from-text feature
Ah shit it definitely will not be the way I did it... expecting value to include a colon will cause issues
(for tags-from-text)
Oh heck yeah, it actually works without issue
[[aspect_ratio:16:9]]
Pushed update adding aspect_ratio tag.
Pretty nice way to set image orientation without having to specify Width and Height
Uses the same calculation method as the image cmd size options
halcyon quarry
Realized I didn't actually merge it until now
keen palm
Next bit of weirdness: the SD command prompt window says "Can't find VAE named sdxl_vae; using None instead", but...but that's literally the name of the VAE....,
halcyon quarry
If you try the API endpoint it will tell you the value is
sdxl_vae.safetensors
btw, if you go into the Checkpoints tab in your WebUI - you can assign the VAE to any models that do not have baked vae
and save config.
If you then use vae: Automatic it will load the vae automatically
keen palm
Ah nice
halcyon quarry
yeah just take a few mins to figure out which ones need vae and config em
keen palm
Well, that was an easy fix
halcyon quarry
Im whipping up an aspect ratio selector character now
to use the new tag
an extra 2 seconds of generation time for the character to dyamically choose one
(probably less)
keen palm
2 seconds is all I need. Err...wait....
halcyon quarry
Sooooo.... about writing speed.... uhh just remembered this little param guy here
max_tokens_second
keen palm
Mmhmm?
halcyon quarry
Earlier we were talking about generating text and delaying to send it, or trying to predict how fast the response will be generated and delay generation... etc
But, theres a friggen TGWUI param to control token speed
well, throttle it
keen palm
Well, I'll be damned
halcyon quarry
The aspect ratio character is working, but again, I goddamned hate grammar
ChatGPT says this should capture the value:
root ::= aspect_ratio
aspect_ratio ::= [0-9]+ ":" [0-9]+
When the text is 16:9 Landscape it captures 16:933604976
terse folio
Earlier we were talking about generating text and delaying to send it, or trying...
if the bot is only talking to one person at a time in one channel, yes, sure slowing down generation would make it more human.
But consider what you might want to do in the future.
Like the flows/tags, you could have seemingly "background tasks" being generated, making decisions or internal monologues.
But TGWI only allows 1 generation at a time.
If that's slowed, there's less opportunities to do more is all I'm saying
halcyon quarry
Seemed like everyone forgot this param (I certainly did)
terse folio
What would be awesome is if we had real parallel streaming,
So slowing down one stream to let multiple happen at the same time at the same speed would be a good option
halcyon quarry
Do you have a few minutes? 🙂
terse folio
Sure
halcyon quarry
First item on the list - When the internal items are not there (we're aiming to have none required on startup) its currently making an error.
I know how to replace this with an info statement that says "not yet created, creating it now..."
But need to follow up with the error if it fails to create
The other thing is, I'm not sure how to initialize the activesettings file with just the 4 empty top level dicts, if the file is missing.
Also - following your lead I was able to split off some more functions into another module
terse folio
The other thing is, I'm not sure how to initialize the activesettings file with ...
this file gets overwritten all the time, i'd first look there
deleting internal and running the bot, it created the database
starboard didn't work, hmm interesting
halcyon quarry
You have it enabled with a channel and everything?
terse folio
yes
oh, good to know, discord.log has the errors
need to create an error handler for discord so it can display in console too
I think it's an event like on_error
fixed starboard, it was never initialised with a default value for messages
halcyon quarry
with pylance is there a means to just jump to this function?
terse folio
is that from tgwi or your bot?
halcyon quarry
the bot - I know where it is utils_files
halcyon quarry
I just double click a file in windows explorer
I don't have it loaded as a git repository either
terse folio
okay, that wont work for this feature.
Because it doesn't know where you are, it can't help you find code from other files
halcyon quarry
got it
terse folio
Right click the folder, open in vscode
or in vscode, right click in the files area and add a new folder
halcyon quarry
do you stick the whole TGWUI folder in there?
terse folio
I didn't do that this time, because I've had experiences with vsc getting slow when it has too many dirs to search
but tgwi isn't that big ^-^
terse folio
I have the ad-discord bot code stored in another folder.
imported to vsc
And the active bot code from tgwi also added to vsc!
So I can make my changes that go to git, and also see the active settings of the bot
terse folio
Now I can jump to function. Very cool, thanks
and use git!
you should have an option to select it in the source control page
ill be back in a little
halcyon quarry
yoooooooooo I did resolve the activesettings issue