#ad_discordbot (Fork of Fork of xNul's bot)

1 messages · Page 5 of 1

halcyon quarry
#

That lets you change shit that isn't rooted explicitly "the matched trigger"

keen palm
#

So:

  • trigger: 'Bob'
    search_mode: user
    format_prompt: 'Information about Bob. {prompt}'
halcyon quarry
#

exactly

keen palm
terse folio
#

In my own projects I split history into 2 parts, long term and short.

It's structured a bit like this:

system prompt
long term history

Instructions
Context/RAG/whatever other info
Short term history
Response guide

The short term holds messages since within the last few minutes for example.
The idea was to give the LLM a sense of time, like this is active now.

Response guide means telling the llm to continue a response this can be left empty.

halcyon quarry
#

In the tags system is a load_history tag that lets you restrict the number of history shown

#

load_history: 2 is only the last 2 exchanges

#

You have my blessing to add or modify the bot in any way including adding/modifying tag behaviors

visual dagger
terse folio
#

Meant long term memory, but not in the sense of a database or anything.
Like that it's pushed up higher and not in the "active work space"

I like to use instruct finetuned llms, so the "active" area is after the ### instruct heading.
The part that the llm is taking instructions from.

But it still could call apon the above information, it just doesn't influence it as much from my experience.

#

In terms of reliability to get a model to do something:

  1. (best) adding something to the "guide"
    In the response section having the LLM write "Here is your X:" and continue
    (this is similar to grammar in the way that it restricts what the llm can generate)

  2. Instruct
    This usually works, but the llm could get confused and generate some other reply related to some other part of the instruction and end up not doing what you want.

  3. context
    Like a character card, I would only put stuff here that I want to act like a suggestion to the LLM, it won't always act on it.

visual dagger
#

how do you approach weight? that concept? how do you give more weight to a chunk of data or less weight

weight or focus, whatever you call it.

#

like putting XYZ data in system prompt vs NOT etc..

halcyon quarry
#

Just chiming in that one of the tags is for state parameter’s so you could actually have a custom continue a ramp

visual dagger
#

putting it in the begining vs the end

halcyon quarry
#

I’m using microphone input by the way

terse folio
keen palm
#

Hey, it worked!

#
  • trigger: 'Bob'
    search_mode: user
    format_prompt: 'Bob is a sentient Altoid mint. {prompt}'
#

@Gamemaster, can you tell me about Bob?

halcyon quarry
#

Of course it worked everything in the butt works

keen palm
#

Bob, as a sentient Altoid mint, is indeed an unusual character. He stands approximately two inches tall and resembles a standard Altoids tin, complete with the iconic design featuring a tooth inside a circle. His exterior shell is made of durable metal, which serves as protection for his internal mechanisms. Inside, Bob houses advanced technology that allows him to move, think, and interact with the world around him.

#

Bob was originally created by a rogue droid engineer seeking to push the boundaries of artificial intelligence. The experiment was successful beyond expectations, resulting in a sentient life form with humanoid cognition within a tiny package. Due to his small size, Bob relies on various tools and devices to navigate the universe, including a miniature repulsorlift unit for mobility and a holographic projector to appear larger when needed.

Despite his humble beginnings, Bob has developed a quirky personality and quick wit. He enjoys engaging in philosophical debates and exploring new cultures throughout the galaxy. Being a sentient being, he craves companionship and understanding, though his unique appearance sometimes leads to misunderstandings or outright hostility from other beings. Nonetheless, Bob remains optimistic and resilient, determined to make the most of his extraordinary existence.

visual dagger
keen palm
#

Now clearly, the bot really ran with what limited information I gave it, but that's to be expected with such an open-ended prompt.

terse folio
visual dagger
#

about the example you mentioned, quoting the exact tokens verbatim, did you have 0 chat history?

halcyon quarry
#

There is also the cryptic negative prompt to consider

visual dagger
halcyon quarry
#

Can’t tell you it’s too cryptic

visual dagger
#

: /

halcyon quarry
#

It’s one of the parameters you can use

visual dagger
#

Can't use it it's too cryptic.

terse folio
# visual dagger what.s that

Similar to stable diffusion, the negative prompt can subtract concepts from the generation.

For example:
prompt: Write a story with a castle
Common theme with such stories is a dragon.
Negative prompt: Include a dragon in the story

#

try it out!

visual dagger
keen palm
#

How do you insert the negative prompt with the bot?

halcyon quarry
#

Using the state tag

terse folio
keen palm
#

Ahh yes of course

visual dagger
#

when you promot engineer it's clear what is wrong and you can change it, when you play with parameters you don't actually kmow

#

what's happening behind the scenes

#

complex math then

#

"hmm did that change anything?"

keen palm
#

For the text string in format_prompt, can you use newlines and other formatting?

halcyon quarry
#

This but also has a special syntax to create tags from text

#

[[state:{negative_prompt:include a dragon}]]

#

That would automatically set the negative prompt

visual dagger
halcyon quarry
#

Yes its a TGWUI param

terse folio
keen palm
#

So the prompt you type to the bot would be:

Tell me a story about a castle. [[state:{negative_prompt:include a dragon}]]

halcyon quarry
#

Yes

keen palm
#

Yes, this could be quite useful

halcyon quarry
#

It will delete that from your prompt too

#

You can create any tag on demand like that

terse folio
#

When you write a prompt, it's put into an embedding space while the model generates.

The negative prompt subtracts a 2nd embedding from your prompt.

So each token is generated as (next token = prompt-negative)

(message was red, not sure it went through)

visual dagger
#

@terse folio can I say "exclude anything that contridacts the mall description dumping the whole desc"

lol

halcyon quarry
#

I swear on me mum, didn’t just go find functions that do this I had the idea then worked with ChatGPT and debugging until I got it working

#

This tags from text thing

#

It works for any format value including dictionaries, lists, nested sublists, anything

terse folio
#

I'm not sure, you'd have to test that.
But both prompt and negative prompt still boil down to the same thing.

prompt = prompt - negative prompt
if the llm wants to hallucinate it probably will, but worth investigating!

visual dagger
#

we need a police layer or quality layer or alignment layer that checks things up and removes conridactions

terse folio
#

:)

visual dagger
#

but that needs more prompting and will also make more mistakes

#

: /

#

I tried doing it

#

but gailed

#

failed*

terse folio
halcyon quarry
#

We have it already its called the Tags system 🙂
Using Flows, you can have other character contexts review the response from the LLM

terse folio
#

that too ^^

visual dagger
visual dagger
#

tell me about the logic

halcyon quarry
#

Oh baby do I have a flow chart for you

visual dagger
#

why we need one?

halcyon quarry
#

click to expand

#

It may look intimidating but just try following the solid line

#

then look at the dotted line which shows how Flows works

visual dagger
halcyon quarry
#

You can trigger a Flow that just infinitely cycles through the message loop doing absolutely nothing

#

Or, have it change LLM model, change character, change parameters, replace text, draw an image, enable controlnet parameters, etc etc etc etc

#

super flexible once you understand how it works

#

I'm not smart enough to dream up the use cases it can enable

visual dagger
#

what is the message loop?

visual dagger
#

but still confused

visual dagger
halcyon quarry
#

You write a prompt and trigger a Flow

visual dagger
#

sorry for the gif but I'm just a bit confused

halcyon quarry
#

That Flow definition builds a flow queue

#

Lets say you have a format_prompt tag in the first flow step

visual dagger
#

hold on let's start from the begining, you prompt what or who and if you include those trigger tags do they get removed?

halcyon quarry
#

It can include variables such as your previous prompt, or the LLMs response

visual dagger
halcyon quarry
#

You have to look at my example tags and it will all make sense

visual dagger
#

is that another link?

halcyon quarry
# 
  - trigger: 'draw,generate'
    insert_text: ''               # Replace the matched phrase with an empty string
    insert_text_method: replace   # Other methods are 'before' and 'after', but 'replace' is correct for this...
    search_mode: user             # Only search user's prompt (will not trigger from LLM reply)
    on_prefix_only: true          # Only triggers if matched at very beginning
    should_gen_image: true        # Triggers an image generation (will send to channel afterwards by default - cann be suppressed with 'should_send_image' tag)
    swap_character: M1nty-SDXL    # Swap to character finetuned for writing more elaborate image prompts from simple prompts
    load_history: -1              # 0 = default (all history included) / -1 = prompt excludes chat history / > 1 = llm only sees this many recent exchanges.
    save_history: false           # Do not write this interaction to the main chat history
visual dagger
#

I checked the prev links

halcyon quarry
#

So see how it says insert_text - empty string

#

method: replace

#

it will replace the matched text with nothing

#

draw a cow
LLM gets: 'a cow'

#

If there is no 'trigger' all the stuff is guaranteed to happen

visual dagger
#

where it starts?

halcyon quarry
#

Look at the flow chart lol

visual dagger
#

I did

#

does it start when you promot a char?

#

it doesnt say where it starts, I read it

halcyon quarry
#

When you write a prompt it gathers all tags, checks for matches, applies stuff, then LLM gens

visual dagger
#

when you prompt what? anything in ooba promptable?

#

anything promptable in ooba

halcyon quarry
#

Anything man, write a message

#

Yes

visual dagger
#

so if my prompt have no special tags then nothing will happen?

halcyon quarry
#

“Hi” - thats a prompt

#

Correct

visual dagger
halcyon quarry
#

Well by default it will generate text and send a reaponse

visual dagger
halcyon quarry
#

But you can have tags suppress txt gen, or response sent

visual dagger
#

or "draw" will always get replaced by empty str

visual dagger
halcyon quarry
#

The tag definition I shared swaps in a character that has example dialogue, where user says something small and it replies with SD prompt

visual dagger
#

so you are agenting here?

halcyon quarry
#

by using “draw” you trigger the prompt to be processed with that char context

visual dagger
#

AI Agents

halcyon quarry
#

I’m an underpaid graphic designer

visual dagger
#

I mean

#

this tool, the whole logic is similar to AI agents?

#

right?

#

no?

halcyon quarry
#

It doesn’t execute code or search the net

#

Unless you have an ooba extension that does it

#

I can’t chat going afk

terse folio
#

what defines a system with agents?
I thought it was multiple characters interacting to solve a problem, so maybe it's close

visual dagger
#

that's agents

#

multiple characters interacting with each other to solve a problem

#

each with it's own system prompt and tools

terse folio
#

Sure you could accomplish that with this I believe

visual dagger
#

even if it's not exactly ai agents, you can call it AI Coding the next era of coding

#

instead of only code, you leverage llms + code + prompt engineering

visual dagger
#

Reality, how you time tests went?

#

any ground breaking discoveries?

terse folio
# visual dagger any ground breaking discoveries?

nothing significant that I can remember,
There was a suggestion to use more general times like "morning, midday, afternoon" instead of timestamps, that's worth checking out.

Just do things randomly sometimes haha, my last focus was making my own RAG system that uses various zoom levels so the model can find relevant data possibly better.

visual dagger
#

wdym zoom levels?

terse folio
#

Like, when you embed text for RAG, you might split it by sentences.

and the model could query "what is Kat's favorite food"
The issue here is, that my favorite food might not be mentioned in one sentence.
The conversation might flow like:

Their favorite food is X```

There might also be another entry:
`Someone else's favorite food is X`

So the embedding model has a chance of picking the wrong item.
This can be solved by embedding at different "zoom" levels as I'll call it.

Level 0, capturing as much as possible in the context.
Level 1: embed each paragraph.
Level 2: Embed each sentence.

So first we match at level 0.
There's a match for "Kat" and "food"
In the database we check the sub items, and search on those, narrowing down to the right result.

Then this small sentence is added to the bot's context instead of pasting a huge amount of text
#

i'll be back in a little, brb

visual dagger
#

maybe classifying data can help you, that what am thinking of doing

like multiple folders:
Locations
Characters/NPCs
etc

under those folders there will be text files for each location or character, each file will have the full description

and when xyz location or character is mentioned I retrieve that specific info

it might or might not work for you but yeah

visual dagger
terse folio
terse folio
#

Also another benefit of this structure is the data is embedded at query time.
Only the top zoom level is embedded and saved to the database.

As queries come in that use different branches, they will be processed and saved to DB.

So I won't have to wait hours to embed 100s of thousands of messages all at once at the start :)

#

Anyway, on the topic of grammar
https://pypi.org/project/gbnf-compiler/

I found this library that can process grammar on text and show the result.
This could be useful to debug why something isn't working

PyPI
gbnf-compiler

Simple plug and play GBNF compiler for llama.cpp

visual dagger
#

so it's a combination of embeding and summarizations?

terse folio
visual dagger
#

classifying those topics into folders is a bad idea for your use case?

terse folio
#

Embedding kind of works like summerization, taking all the context (usually 512 tokens max)
and compressing them to a 768 dimensional space

terse folio
#

the LLM could also write a summary that can be saved to the vector database

#

but it's a trade off of time

visual dagger
#

about false positives that you mentioned earlier, the solution you came up with is to keep on narrowing down the results, but is that enough?

#

will be 100% accurate?

terse folio
#

Embedding text can take miliseconds,
Text generation 10s of seconds possibly

terse folio
halcyon quarry
#

Maybe we should make a “user_scripts.py” that users can put functions in that can be triggered by tags

visual dagger
terse folio
halcyon quarry
#

I was using it at one point awhile back, haven’t been dealing with the webui much though

#

If it doesn’t rely on UI should work with the bot(?)

visual dagger
#

we should add a Gaslighter Agent in the process (it will make you believe that there is nothing wrong and you are the one hulisunating, you are the problem)

bingo 100% achieved

terse folio
# visual dagger but why nothing can be 100% I am sob now 😭:

What I mean is everything has it's ups and downs,
Like if we did a simple word matching text search it wouldn't be able to account for typos that aren't predefined.

It also wouldn't understand the context, the meaning of text.
Some words can have multiple meanings.

A vector database is better at meaning, but can have trouble matching exact words/phrases.

We could use a hybrid that uses both.
But just like with llms, there can be edge cases that are missunderstood

terse folio
terse folio
#

Because when you run import X
it will only import once

#

and cache the results in memory

#

i'm pretty sure
now i can't remember

#

Yea,
when I work with python notebooks, I have to restart the kernal before importing my changes from my library

visual dagger
# terse folio and cache the results in memory

Gaslighter Agent retrieving from memory: "you see we already talked about that yesterday, there is nothing wrong, we established thst yesterday so stop bring this up, let's move to the next"

terse folio
#

😸

visual dagger
#
  • justifying why there is nothing wrong
#

that will be good

terse folio
#

and it would be all set and benifit the bot as whole

#

that would take restructuring the whole thing, a big task

halcyon quarry
#

How about this-
All the possible matched search results are shown to a character context along with your trigger/prompt - and it is tailored to pick the correct data (in regards to your rag system thing)

terse folio
#

Sure, but could be computationally expensive

There are some ways to tell if your search results are confused or not

#

like if all the results are 50%, 50%, 49%...

there's an average difference of "1%"
and 50% is pretty bad, so we can tell the model is confused and found now good matches

#

ideally we want

99%, 99%... 1%, 1%...

#

that tells us there's confidence in the results
usually
still can hallucinate

#

So maybe there could be a threshold range

If the confidence is at least 75% average, then pass it to llm.
because anything below that may as well be random choices and not include a good answer in the search results to begin with, and this would be wasted on the llm

keen palm
#

Is there a way to include apostrophes in format_prompt without messing up the works? Backslash before it?

visual dagger
#

who does the scoring?

#

the llm?

terse folio
# visual dagger who does the scoring?

Searching by vectors is done through a dot product (or cossin) of your search query and all the vectors in the database.
This returns a list of similarities for each item.

#

and you usually grab the top K, maybe top 100

#

with those top 100 you can check how similar the scores are

#

The idea for that came from when I was playing around with older neural networks, like image classification

#

the model outputs each class with a score

#

but you need to process those scores to determin how confident the model is that anything is in the image

#

else your code would just pick the item with the highest score, which could be wrong

Cat - 60%
Dog - 50%
Bird - 50%
...

visual dagger
#

hmm

#

ah I get it

#

so you calc the score in code right?

#

u write a logic that calculates the score

terse folio
#

ML libraries usually have a function for this.
RMSE I believe it's called

#

Mean squared error

visual dagger
#

so now you just pick the highest score results? and those are most of the time the best matches?

terse folio
#

Yea, you'll pick the highest, but also calculate a confidence overall

#

so you have 2 datapoints to decide what you want to do

#

like if confidence is below 50%, then any result is probably garbage

#

So back to Embedding models

#

this can be a little confusing because each embedding model gives out different scores

#

Like for some, a similarity of 0.2 is good. (meaning very close)

but others can only get up to 0.4 and the range is much larger.
So it can be hard to hardcode numbers for those

#

I'm still figuring that out

#

here's an example, this function is expecting an array that looks like this:
[1, 0, 0, 0, 0]

#

the function matches how close the inputs are to that expected array

visual dagger
#

how much similar it is

#

right?

terse folio
#

I have 1-result to translate it to a percentage

#

so 0.98 - 98% similar

#

or confident

visual dagger
#

oh yeah

terse folio
#

The next complication is what if you expect multiple correct results in your search?

That will no longer match the pattern of 1, 0,0,0,0,0...

#

so needs some sort of dynamic threshold to decide which items should be 1s, and which not

#

Still figuring that out too!

visual dagger
#

btw how do you quote the llm, it doesn't seem to get it

llm: so you will keep walking
me: hmm "keep walking"?
llm: -unrelated reply or totally misundertood it-
terse folio
#

try things out! there isn't really a concrete answer

#

Also, since that's directly the next message.
I think asking "what do you mean keep walking?" would also work

visual dagger
#

I want to autonate it but the problem that might arrise is me not quoting the llm but the implemented logic/code will detect quotes anyways

#

edge cases problem --

terse folio
#

Yea, if you do something externally, you could detect quotes from the user, then format it differently with code.
Maybe injecting User Me is replying to your message <quote>, here is their reply:\nMe:

#

and this would be hidden from the user

visual dagger
#

yeah but edge cases

#

so let's head to "new york"

terse folio
#

yup, so many edge cases

visual dagger
#

wait I found a solutio, checking if the keyword is mentioned by llm in the previous reply

terse folio
#

why would you quote NewYork?
Unless it's a custom location the llm makes up, which you could still just call by it's name

visual dagger
terse folio
visual dagger
#

I think we should fix the user instead

#

😐

#

me

#

I have to quote differently in a clear manner that can be detected by my code

#

removing any possible edge cases

halcyon quarry
#

format_prompt: "this is 'stuff in apostrophes' and that's OK"

visual dagger
#

me: hmm ql"keep walking"?

ql means quoting the llm

terse folio
#

mhmm, mhmm,
Back before LLMs, I tried writing chatbots using fuzzy matching and patterns.

Would get stuck trying to overengineer it to take into account every possible user error.
I even tried using some libraries to deconstruct a sentence into a tree of inheritence (what word relates to what)

But I finally came to the conclusion that I should just tell people to write in proper english lmao
it was getting to be too much work

visual dagger
#

the users are the bug

terse folio
#

exactly haha

visual dagger
#

what about a live video call with an llm

#

you will make the thing with your fingers "quoting"

#

how do you detect that

#

openpose?

terse folio
#

Never used it, I've used Mediapipe, but I wouldn't recommend it right now.
They don't exactly have windows support, and you need to compile it yourself to use GPU with C++ on linux only.

#

I think it would be doable if openpose has a finger tracking model

#

You would want to detect if the 2 fingers on each hand are close to eachother in a raised position, and that could be your air quotes flag

Getting distance between end of index finger and middle.
And also checking that they are NOT close to the thumb/or palm to check if raised

halcyon quarry
#

I remember there were a few mediapipe controlnet preprocessors/models at one point, for SD 2.0

#

lost to history with the advent of XL

terse folio
#

^^ mhm

halcyon quarry
#

@terse folio the activesettings migration fails

terse folio
#

Uhoh, are there any errors?

halcyon quarry
#

It doesn't want to overwrite the one that ships with the bot in /internal/ now

terse folio
#

Ah, thanks, I wouldn't have caught that ^^

halcyon quarry
#

I didn't catch it earlier at work because I moved it manually

#

decided to let it migrate here at home

terse folio
#

I wonder what should be the logic there.
Should it overwrite the newer file?
because old likely has the settings you want

halcyon quarry
#

Yes, should overwrite

terse folio
#

or could also warn the user to delete it if they don't want the new file

halcyon quarry
#

or, could rename it like you did with the old database file

terse folio
#

true true

#

that's a good idea

halcyon quarry
#

...just in case

terse folio
#

rename + warning that you should merge

halcyon quarry
#

ehh

terse folio
#

lol there will be a lot of migration code running on start up.

Check if bot.db exists,
check if activesettings.yaml exists
check if starboard exists...

halcyon quarry
#

It's very unlikely that anyone is going to run the bot and have new settings there they don't want to lose

#

don't think its possible really

#

their existing activesettings will have current settings

#

I / you can just axe it in a month or so

terse folio
keen palm
halcyon quarry
#

Pushed that update to optionally keep history when changing characters

keen palm
#

Coolio

#

So, the tag system only triggers the first available, but doesn't do multiple in one prompt?

halcyon quarry
#

Certain tags values, it only keeps the first instance.
Other values can accumulate

#

Yeah... I should add a search_and_replace tag

#

Due to the way I process the text insertions, it couldn't work to replace all instances

#

that would have to be handled separately

keen palm
#

I'm using it right now as a sort of lorebook, and I've saved information on two characters (RPG characters, not bot characters). When I prompted the bot about X and Y characters, it only triggered X.

halcyon quarry
#

You may need to update something to update tags

#

reset char, load char, etc

keen palm
#

I did that

#

I can try again, but yeah

halcyon quarry
#

Two separate tag definitions?

visual dagger
keen palm
keen palm
visual dagger
#

and puts those info where? user prompt?

#

next user prompt?

keen palm
#

Only the first trigger is getting formatted into the prompt

#

It puts that information ahead of the prompt

halcyon quarry
#

format_prompt I think is one that is currently only allowed to use once...
...I'm thinking if there's any reason why I shouldn't change it to allow multiple...

visual dagger
#

oh before user prompt?

halcyon quarry
#

yes

keen palm
#

Right

#

So, let's say I wanted to tell the bot that character X jumps into the air John Woo style and shoots at someone. Then, a brief synopsis of character X's traits are injected prior to the action description.

halcyon quarry
#

I understand.
Uhm, you could have a separate tag that you have "X and Y" as a trigger

#

with a trumps parameter to trump the other tags

keen palm
#

But then I'd have to put their respective information again into that format

#

Which would mean I'd have to do it for every potential combination

halcyon quarry
#

thinking :p

visual dagger
#

Speedy you are making like a group chat?

#

or a text game with npcs

keen palm
#

Group RPG. The idea is to use the bot to handle things like interactions with NPCs, plot ideas, scene setting, etc.

halcyon quarry
#

I guess there's nothing against formatting repeatedly...

#

I would just need to collect it as a list and process each one

visual dagger
#

so the bot isn't a char, it's a world engine?

keen palm
#

Yes

visual dagger
#

that manages the world

#

like a narrator

keen palm
#

The bot character is called Gamemaster, and its personality revolves around storytelling

visual dagger
#

that's cool

keen palm
#

Yup! I've used it fairly well in the webui, but now I want to take the concept multiplayer with a discord bot

visual dagger
#

yeah you can go big with such a idea

#

you can even make it a video game, but ofc no one said it's easy

keen palm
#

At some point AI could be good enough to where an open world RPG could be made, with AI generated voices, images, plots, characters, etc., in any conceivable setting, but we're a way away from that.

halcyon quarry
#

Ill expand that format_prompt tag now...

keen palm
#

Ooohhhh

halcyon quarry
#

@keen palm

#

Looks good to me on paper - if it works for you I'm pushing

#

Note that it will format higher priority tags first (ones sorted higher in your tags dict)

keen palm
#

File "D:\Documents\text-generation-webui\bot.py", line 2089
formatting = ['format_prompt': value]
^
SyntaxError: invalid syntax

#

The ^ is under the colon

halcyon quarry
#

😁

#

Ok Ill have to test real quick to ensure I have this right...

keen palm
#

I probably should have saved the previous bot.py, but oh well

halcyon quarry
#

comment line 2089

#

nvm

#

2088 - 2090

keen palm
#

I got a module error: ad_discordbot.modules

#

I need to update

halcyon quarry
#

Don't delay!

#

Think I have this right now... testing

#

Works, pushed it

keen palm
#

Yeah, it works

#

Woot!

halcyon quarry
keen palm
#

I should probably stop testing this and go, like, eat dinner with my family

halcyon quarry
#

nahhhh

keen palm
#

But yes, it works! Thanks!

#

I might need to reformat the info so the bot knows it's from multiple characters

halcyon quarry
#

@keen palm you may benefit from the send_user_images tag, that lets you trigger an image send (does not require image generation)

#

I mention this because you had me thinking of other things I can do multiple of that I thought could just do once. Updated this to allow multiple image sends

keen palm
#

Haven't looked at that one.

halcyon quarry
#

You can put images into your user_images folder, and use the filename (or directory name to send random image)

keen palm
#

Hmmm, that could be interesting. I do 3d renders of the characters, but I usually put the pictures in their own channel

halcyon quarry
#

Good thing I am messing with this b/c I noticed its bugged

terse folio
#

working so far

image.png
halcyon quarry
#

Sending local images working

Screenshot_2024-05-16_211848.png
#

again, collecting multiple tag matches instead of just the one match - and sending all

#

Commit commit commit commit

terse folio
#

also decided to change the starboard to be a dict
messages:[message ids]

that way it also can be loaded into the FileMemory class

halcyon quarry
#

I had updated a few of the comments you updated

#

Well, at least one

terse folio
#

There's no reason to keep the database in the tgwi folder is there?
Could just move that to internal

halcyon quarry
#

Yep!

terse folio
#

You might get an error about failing to load the starboard,
just move it out of internal to the ad_discordbot dir and the bot will migrate it

halcyon quarry
#

Good looking code there

#

Migration successful 🕊️

Screenshot_2024-05-16_231423.png
terse folio
#

Dunno where the issue is, but some args don't pass through the bot to the webui.
Like setting the listening port.

halcyon quarry
#

Do you already have listening port in CMD_FLAGS.txt?

#

The bot does load ooba flags

terse folio
#

no, I use a custom run file with the args.
I update the tgwi often and it overwrites the cmd_flags file.

I used to have flags there, but when I was working on the PR for the openAI extension, I didn't want to push my personal commits by accident

halcyon quarry
#

Well you could stick a print statement in the tgwui loading code to see when it reads the args, it wasn’t all too long ago I combed through it carefully

#

Or Ill take a look tomorrow

terse folio
#

you create your own arg parser that only accepts --token and whatever the other one was.
I think this couldbe solved by importing the arg parser from shared.py and running it after

halcyon quarry
#

Be my guest 🙂

#

Idk though, when I pass args they are applied…

#

Such as —model and —extensions and —loader etc

terse folio
#

model and extensions work

#

Hmm, yea good point

#

mightbe the wrong port, because I tried 7860 too

#

whatever the gradio port is*

#

Ahh, bot loads but doesn't say the webui is online

halcyon quarry
#

I think there is a bit of code that prevents it from using the UI

terse folio
#

I pass the api flag, but timeout on :5000

#

ill look into that later probably

halcyon quarry
#

I mentioned this before but I have a version of bot.py with api working, I just like the versatility for the tts stuff

terse folio
#

lets work on a tts api!

#

that's something I want to do anyway

#

I like the idea of having all these tools as separate services

#

I'm working on a project to be able to organize a "render farm" type thing with LLMs, SD, tts, whisper... etc ^-^

halcyon quarry
#

Alltalk has an api

#

Looked too complicated to use though

terse folio
halcyon quarry
#

If you shoot down to process_speak() you can see there’s a few subtle differences between the extensions

#

I check the tts client and handle accordingly

terse folio
#

mhmm, I would do the same letting you access all the clients from one api

halcyon quarry
#

Well you may get a good head start there 😛 There’s been other promising projects emerging I haven’t paid much attention to

#

Again probably with their own subtleties

terse folio
#

we need a standard for tts too now

halcyon quarry
#

It’s been prob 2 months now I opened an issue aaying they should have both Internal and Visible responses returned from api

#

I forget which but the one is what has the filename for generated tts

#

The chatbot wrapper function returns both responses

terse folio
#

ahh,
well if tts was its own api you wouldn't need that, just pass the text you want to generate yourself and get the file back

halcyon quarry
#

Which is what this bot does

#

Well it’s easy enough to detect the file and find it. But yeah, setting the params would be impossible right now I think without something like the monkeypatch I have going on

terse folio
#

hmm hmm, will see what I can do when I get tts installed

halcyon quarry
#

Surprised you’re having so much problems with that

#

Tried alltalk yet?

terse folio
#

They all use the same XTTS lib on the backend

#

trying this idea, reinstalling Visual studio

#

But I have all the build tools and manually set the paths for pip to find the lib files and all..

#

Nope that failed, sad
wish there was a solution I could just use on windows without duel booting or anything

#

actually, new error: LINK : fatal error LNK1158: cannot run 'rc.exe'

halcyon quarry
#

Just try the install method for alltalk

#

I’m on w11 I never had any issue

keen palm
#

Hmm, where's the config option about starting as a particular character?

terse folio
#

I did try the alltalk extension with tgwi

halcyon quarry
#

It starts up from last character

terse folio
#

Amazing

#

the horrors that led me there

image.png
#

So, what changed was use buildtools from programfiles/visual studio not program files x86/visual studio 14

And to include a few more paths to some missing libraries

halcyon quarry
#

Welcome to semi realistic robot voice gowron1

#

You’ll enjoy how the bot handles tts I think

terse folio
#

woo,
Am hoping I can get xtts finetunning to work and make something semi realistic, I think it was this video that inspired me to try again https://www.youtube.com/watch?v=ds5LLIt5OLM

The idea of finetunning the model, then passing that generation through another model finetuned again on my voice.

halcyon quarry
#

I’m outta here, good night!

terse folio
halcyon quarry
#

pip install -r ad_discordbot\requirements.txt 🙌

keen palm
#

Did you update some things again?

halcyon quarry
#

Reality working their magic

keen palm
#

With that 'draw, generate' tag that switches to Minty-SDXL, do you have to manually switch back afterward?

halcyon quarry
#

No - swap_character affects only the current interaction
To set a new character you can use the change_character tag

keen palm
#

Sweet

halcyon quarry
#

similar effects for swap_imgmodel / change_imgmodel and swap_llmmodel / change_llmmodel

#

Better have a lightweight model or really good GPU for constant model swapping though

keen palm
#

Right, I was just looking at that possibility

#

Thinking about adding swap_llmmodel none to the img generation trigger, to unload that model and free up VRAM for the SD side

halcyon quarry
#

You could also have a tag like trigger: '{draw|image} mode' that will change the LLM model to None

#

and keep it that way, if you want to just do images for a bit

#

...or just use /llmmodel command of course

#

When the LLM model is unloaded, the users text is used directly as the image prompt

keen palm
#

I don't know if I'd be doing multiple image generations in a row like that, but we'll see. I haven't even really found any good SDXL models yet anyway. I just started playing around with A111 again after like a year.

halcyon quarry
#

SFW or NSFW stuff?

keen palm
#

SFW, since it's for a game on discord

halcyon quarry
keen palm
#

Star Wars stuff, so that might work

halcyon quarry
keen palm
#

Thanks, I'll check them out

halcyon quarry
#

Did you try NeuralBeagle llm model?

keen palm
#

I have SD set to just load into my second GPU (since last I checked it can't split properly anyway), so I might be able to get away with having an LLM loaded across both GPUs, and just unload it when necessary for SD generation. Since I think those models use very little VRAM when not doing anything, unlike LLMs.

#

I didn't, on account of the apparent fake context size

halcyon quarry
#

It doesn't have a huge ctx size I think just 8192

keen palm
#

Yeah, the one I've been testing recently has...supposedly...32k context

halcyon quarry
#

Do you often need it to remember things from 10 messages ago>?

keen palm
#

Yes, that is helpful

halcyon quarry
#

@terse folio colorama seems to be installed by TGWUI in the venv already

#

Although, that does have me thinking...
I claim that this bot can be used for image generation independently from TGWUI, but I'm a liar because this relies on many libraries in the TGWUI venv

keen palm
halcyon quarry
#

Beautiful logger though

#

In a few weeks if I remember, I'm switching it to INFO mode 🙂

#

DEBUG could be good for now with all these changes

#

alltalk goes absolutely bananas with debug info

halcyon quarry
#

I can't get away from it, every model I try doesn't compete

keen palm
#

I'll try it out. The one I've recently tested has been doing quite well.

#

Do you use the refiner with SDXL?

halcyon quarry
#

Nah

#

The Refiner works best with the base model, and the base model is great all around but the refiner isn't very impactful with the finetunes

#

When it is impactful, probably for the worse

keen palm
#

What's that Beagle model again?

halcyon quarry
keen palm
#

Thanks, broseph

halcyon quarry
#

There are some amazing performance improvements coming to A1111 to make it on the same level as Forge

#

If only Automatic1111 actually merges them

keen palm
#

Good to hear. I've ben rather consistently disappointed in that regard. Easier for me to just 3D render characters myself, since I have complete control.

halcyon quarry
keen palm
#

50% performance boost is insane

halcyon quarry
#

That's really how big of an edge Forge has on A1111

#

no lie

#

This is someone from the Forge team who did all the work A1111 should have been motivated to do himself when Forge was unveiled

#

idk how I could sleep at night knowing someone forked my project and made it 2x better

keen palm
#

People with no work ethic or sense of pride don't really have to worry about that

#

I've noticed that with some coworkers. The question "how do you sleep at night??" is met with "astonishingly well"

keen palm
#

My negative prompts are being ignored 😦

halcyon quarry
#

You could trigger the LORAs from keywords

keen palm
#

Yeah, I checked that one out the other day. Should get it.

#

But I meant negative prompts to the LLM. I think I just needed to word it differently.

halcyon quarry
#

I know 😛

keen palm
#

I was playing with the quick tags thing.

halcyon quarry
#

I just didn't respond on that

#

derp

#

yeah I also use negative prompts with the LLM and and they don't seem to have any effect

keen palm
#

It's been hit and miss so far

halcyon quarry
#

Wanted to sit down and add features but very busy today

keen palm
#

Don't you hate it when actual life gets in the way of fun things?

#

Have you considered integrating the bot into oobabooga as an extension, so you can change settings in a tab in the webui?

halcyon quarry
#

Gradio is a bitch

keen palm
#

Sigh, yes

#

Every extension I've installed has required fucking around with gradio versions until it works.

halcyon quarry
#

I'd probably have a standalone gradio interface for the settings

keen palm
#

I have that

halcyon quarry
#

my version?

keen palm
#

Oh actually maybe not

halcyon quarry
#

I researched all the versions that were out there and they suck

keen palm
#

I installed an extension for that quite a few months ago, so likely not yours

halcyon quarry
#

Trash it and get mine 🙂

keen palm
#

Haha, if I play around with SD more. It's still disappointing in general.

halcyon quarry
keen palm
#

No response, but one thumbs down!

halcyon quarry
#

Asshats. Asshats everywhere

keen palm
halcyon quarry
#

They have 3 versions of the same extension in their Available Extensions list and they all work technically, but suck

#

There are various extensions which have their own rounding precisions such as Controlnet and layerdiffuse.
Using default A1111 dimension increments or rounding to nearest 1-4px, you get controlnet maps that don't line up and shit like that

#

And, when you set aspect ratios by forcing in one dimension the total pixel count skyrockets

#

how useful is that?

#

Can I get an amen @calm rain

#

(mebbe not :P)

#

Solution: Switch to Stable Swarm 🫡

keen palm
#

Alternate Solution: learn to draw

halcyon quarry
#

@terse folio There's a pretty significant change I'd like made if you don't mind... if / when you are able.
Mainly, I don't think the internal directory really needs to exist in the bot package - the directory along with all current items (activesettings, database, and starboard_messages) could all be created on initial startup.
For the database and starboard files - there's probably a simple thing to add...
In regards to the activesettings, the file really just needs to initialize with a value like:

            'behavior': {},
            'imgmodel': {},
            'llmcontext': {},
            'llmstate': {}
#

When the settings are initialized in BotSettings(), just adding a .save in there should make the bot initialize with all the default inernal settings saved to the newly created file

    def update_settings(self):
        defaults = self.settings_to_dict()
        # Current user custom settings
        active_settings = copy.deepcopy(bot_active_settings.get_vars())
        behavior = active_settings.pop('behavior', {})
        # Add any missing required settings
        bot_active_settings = fix_dict(active_settings, defaults)
        bot_active_settings.save()
        self.settings = bot_active_settings.get_vars()
        self.bot_behavior.update_behavior(behavior)
#

(something like that)

visual dagger
#

hey fellas

halcyon quarry
keen palm
#

If I use this instant tag:

[[save_history:false]]
That particular exhange (prompt and response) shouldn't save to the log, right?

halcyon quarry
#

correct

#

good job 👏

#

Glad to see some of my crazy features being used out in the wild

keen palm
#

Yeah, though it didn't work

halcyon quarry
#

tf

#

Try it with a capital F while I look into it

keen palm
#

Okay

halcyon quarry
#

nah that shouldn't matter

keen palm
#

It shouldn't

#

And it didn't

halcyon quarry
#

:] I got to the code block before you could test

#

Ok I probably did not account for this tag when I added the new history crap...

#

Looking into it now

keen palm
#

I figured as much

halcyon quarry
#

I had same result in testing

keen palm
#

I thought it might be convenient to do such a thing, like asking for a list of ideas, that I don't want to then influence the context

halcyon quarry
#

Yes there's many times that saving to history has a negative effect

#

I suspect I know the problem...

#

nvm

#

I know the problem now

#

# Process the tag matches if flow or save_to_history or load_history or param_variances or state or change_character or swap_character or change_llmmodel or swap_llmmodel or send_user_image:

#

Its a boolean so False is not triggering this

#

heh

#

Fixing...

#

I never caught this myself because every time I use save_history: False I am also using something like change_character

#

I actually dont need that particular line at all.

#

@keen palm pushed update that fixes it

#

Also, /reset_character has been changed to /reset_conversation

#

You need to reload your client after running bot, to see the change

#

If you use save_history: false on a fresh chat, the cmd window will say that it saved chat history to new json file - however, what it saves is just an empty list

marsh harness
#

Wonderful gpt4-o access.

halcyon quarry
#

yep, it's slightly smarter than GPT3.5 so far 😛

#

Although I havent toyed with any of the extra cool stuff it can do

marsh harness
#

I should've planned things I wanted to test/try out. Maybe I'll try and build something simple with it and see how it goes.

#

Maybe I'll see if it can fix a bug in a game I was making that I couldn't be bothered to figure out.

#

Either way, it's nice that they gave access to it for free.

visual dagger
#

@keen palm how is the project going?

#

how can I encourage the character to make decisions on its own? without neither negativity bias or positivity bias?

#

when I do the "Alex is an independant thinker who makes decisons on her own etc" the character became suddenly kind of a feminist out of nowhere, lol, therefore the negativity bias rise up and if I do the opposite way of prompting it goes to the other extreme of positivity bias

halcyon quarry
#

Pushed an update that adds another config option

#

greeting_or_history: history

  • 'history' - Sends copy of most recent message exchange whenever history is loaded
  • 'greeting' - Only send character's greeting
#

guess I could've also put option for 'None"

visual dagger
halcyon quarry
#

Actually I'm going to change it real quick so - woops

visual dagger
#

take ur time lol

halcyon quarry
#

Changed my mind again... just leaving it as is

keen palm
#

Got work shit to deal with. I'll check on that update later

#

Crisis averted. Lemme update

#

It, umm, didn't seem to work

#

[[save_history:False]]

#

Correct syntax, yes?

keen palm
visual dagger
#

good luck Speedy

halcyon quarry
#

If the CMD window said that it saved history it might just mean that it created a new history file with an empty message history

keen palm
#

Nope. Added it to the existing history.

halcyon quarry
#

I’m on the road now I’ll test shortly but it worked for me when I tried it earlier

keen palm
#

Not like it's a yuge deal, as I can manually delete snippets from history, but it is certainly a matter of convenience

halcyon quarry
#

This might be a dumb question but just making sure did you replace the bot file and also reload it

keen palm
#

That is such a dumb question. There is no way that once again I would forget to move the bot file over into the main folder. I am insulted that you would suggest such a thing....
Don't mind me as I...do something....

#

Okay, for some reason that no one really knows or will ever know, it's working now

halcyon quarry
keen palm
#

Truly, one of the mysteries of our time

visual dagger
halcyon quarry
#

Try more models

#

Some models will have drastically different results

terse folio
# halcyon quarry

Thanks!
Also you can disable loggers for other libraries,

For example I set the log level of asyncio to warning only.
Because that spammed me with some stuff on other projects before.

So just need to find out what the logger name of alltalk is and change the levels

terse folio
#

I would change how activesettings works as well, so that it doesn't keep a copy of all the default params for all the systems on disk.
And only a difference of what the bot changed.

This would make it easier to read for the user i'm sure

halcyon quarry
#

heya, I'm here

terse folio
halcyon quarry
#

I don't think we need to get too crazy with the optimizations - unless you're willing to do it

#

I had tried loading the bot without activesettings present and it did not create a new file

terse folio
#

there are some lines of code where you do active_settings[some setting].get().get..

The [item getter] will throw an error if the attribute isn't found.
So if the file is empty on a fresh start those would be a problem.
Just have to change those lines of code to use .get instead.

terse folio
halcyon quarry
#

The first thing that happens when the bot loads, is it gets activesettings and copies it over the default settings - and those become the settings

#

So if activesettings is an empty dictionary, then the result is the default settings

#

When I do bot_activesettings['imgmodel'].get('some_setting', "") its because the imgmodel key is guaranteed to exist

halcyon quarry
terse folio
halcyon quarry
#

In any case, I'm still going to put the .save there 🙂

terse folio
#

i would put a save if file not exists

halcyon quarry
#

I want to save even if it does

terse folio
#

could probably put that in the init of the active settings class

#

at the end

#

where it says if migrated

#

actually, no not there,
after the super().init

halcyon quarry
#

My settings management is a little unconventional maybe.

So when the bot starts, like I said immediately it fixes anything broken with activesettings.

Then, it loads the character data and merges it with basesettings.

Finally... and I just added this the other day, it saves to activesettings

#

I was only saving the result after /character or /imgmodel but I think it's more candid to show the actual settings in play by saving to the file

#

You're probably correct that only saving the difference is more optimal, but this is probably like milliseconds we're talking

#

If you have any better idea - by all means 🙂 You've been a godsend thus far

#

also a pleasure to work with

terse folio
terse folio
halcyon quarry
#

if you search bot_active_settings theres only 18 instances (try comparing to embed lol)
It's not called often, when it is its to to dump or merge into

#

Im going to edit line 527 which should reference bot_settings instead... boom, 17 instances now

#

found 2 more...

# 
        # Save the updated bot_active_settings
        bot_active_settings['llmcontext'] = char_llmcontext
        bot_active_settings['behavior'] = char_behavior
        bot_active_settings['llmstate']['state'] = char_llmstate
        bot_active_settings.save()
terse folio
#

but confused what's being removed, but I'll read over it after you make a commit ^^

We could start a test branch for in development commits back and forth

halcyon quarry
#

Nvm what I said about moving the .save() into the update_settings() function - the 2 places it is right now are good

#

But, need to text how it runs without activesettings there at all...
Would be nice to have no internal or those 3 files

#

(until created)

#

oboy... good thing pushed that to dev

halcyon quarry
#

Rolled back one error I made, which has to do with the way the command options are currently built for /imgmodel

#

I’m halfway done reworking that command… will finish it tomorrow

#

Currently it has all those command options that all need to be defined at startup. Changing it to a regular slash command that sends a view containing the select menus

terse folio
#

internal is created on startup if not exists

#

I'm confused here

image.png
#

Ohh ohh, now need to find out what bot_settings.settings is

#

It's best practice to use .get() when you expect some values to be unset

#

this is weird

#

you're creating a copy of active_settings

#

which also gets it's own updates

#

bot_settings.settings gets updates.
and active_settings gets updates.

#

so, anything changed with bot_settings.settings probably wont be saved

#

I think it would make more sense to just load the defaults into active_settings, and use that on it's own

halcyon quarry
#

bot_settings.settings are the settings.
When settings are to be changed, they are merged with base settings, written to activesettings and also updates the bot_settings.settings

terse folio
#

I think a step could be skipped by just loading active_settings with the defaults at the start

#

if it's all the same

halcyon quarry
#

If activesettings.yaml doesn’t exist then yes, it should initialize with just defaults

terse folio
#

I see

halcyon quarry
#

Ideally, if the user says oops, and deletes activesettings and basesettings, it still runs on the internal defaults. It’s been awhile since I introduced base_settings.yaml but I think it isn’t required to exist. The system works pretty good, maybe not super efficient

terse folio
halcyon quarry
#

if the bots running and they delete those files, we need to code an Ascii animation

terse folio
halcyon quarry
#

Iknow 🙂

#

How are you doing this evening?

#

Was just watching SHOGUN

halcyon quarry
#

I'll eat my hat if this update works first try...

terse folio
# halcyon quarry How are you doing this evening?

Was up pretty late working on some fun projects forgetting about the passage of time,
So I actually woke up in the evening :)

Mostly going to chip away at my todo list with some music/videos in the background

terse folio
halcyon quarry
#

It didn't because I deleted this slightly important thingy 🙂

    @client.hybrid_command(description="Choose an imgmodel")
    async def imgmodel(ctx: discord.ext.commands.Context):
        await select_imgmodel(ctx)
#

When this works now as I suspect it will, I cut like 100 lines or something

#

that big chain of bullshit options

terse folio
# halcyon quarry that big chain of bullshit options

Nice nice!
I was thinking about how to shorten that using loops, just to see if it was possible (after you told me you'd be changing it)

And it turned out more complicated than I thought,
Because discord.py uses the typehints in the function to define behaviors as well.
So i couldn't simply generate the functions as needed

halcyon quarry
#

This will be much better solution

terse folio
#

😸

halcyon quarry
#

Well, once I debug this 😛
Maybe not deferring the response correctly...

Screenshot_2024-05-17_234248.png
#

nvm I know the problem

#

I got it working but obviously I'm overlooking something important with the hashing

Screenshot_2024-05-17_235252.png
terse folio
#

One's a string, other int

#

unless you're just printing it differently

#

but i'd do a type check

halcyon quarry
#

it needs to be an int right?

terse folio
#

Btw, you can print the repr of an object with python f'My object: {obj!r}'

halcyon quarry
#

It is an int - its just printing with double quotes

terse folio
#

it was originally a str because you used to have the model name there

#

[str] this tells the class the value is expected to be a str

halcyon quarry
#

Aha

terse folio
#

it doesn't affect the name, name will always be str

#

not sure if this means what I think it does

#

but maybe, you could set the choice to output multiple types

halcyon quarry
#

Got a sec?

terse folio
#

sure

halcyon quarry
#

You'll probably spot the error in a moment's time

terse folio
#

Ahaha, I have my discord pretty small off to the side, so this isn't easy to read vertically,
Could you push to a test branch maybe?

halcyon quarry
#

yep!

#

erm you committed to the branch so now I uhhh

#

Eh, pushed it

#

line 4064

terse folio
#

need more branches! haha

halcyon quarry
#

Don't really understand the hashing - I basically converted the original command options into selects

#

didn't change anything related to the hash

#

...Do those look like hash values?

#

Img model not found: 808230645446604537

terse folio
#

is this where you get the selected value back?

halcyon quarry
#

In the View class, when a value is selected from one of the menus the class attribute gets the selected value

#

When submit is pressed, the menu exits - and the value can be accessed

terse folio
#

made a merge

#

what was important was you missed this line of code

#

that got the name from the hash that was stored in a dict

halcyon quarry
#

Ahhhhhhhhhhhhhhhhhhhhhh

#

🙌

#

Moving too fast here 😛

terse folio
#

also there's ways to shorten your menu code, will merge that soon

halcyon quarry
#

I'm very much intrigued about this

terse folio
#

How many dropdowns can you have?

#

4?

halcyon quarry
#

Probably something to do with the iterations...
Yes, with the submit button it can only have 5 total items

#

in one view

#

If anyone has > 100 imgmodels, they are SOL

terse folio
#

Hmm, I worked with views before, I had like 12 buttons.

halcyon quarry
#

If I add another item it will fail

terse folio
#

Maybe it was a different type of view, but okay!

halcyon quarry
#

I went through the whole gambit when making the Controlnet options for /image command

#

The View can have 25 buttons

#

because you can put multiple buttons on a row

#

I don't think you can put multiple selects on a row...

terse folio
#

Ahh that's what it was, a limit of 5 rows

halcyon quarry
#

That certainly did the trick my friend

Screenshot_2024-05-18_001549.png
terse folio
#

👍

halcyon quarry
#

I could go repeat the same for /llmmodel and /character - but you said you have something cleaner?

terse folio
#

I would wait until I merge this

#

does this mean labels need to be unique?

halcyon quarry
#

Not anymore!

terse folio
#

okay

halcyon quarry
#

Now the value is just for the "Placeholder" - does not need to be unique

#

no more hack required

#

for imgmodels command it's almost always S-S 😛

Screenshot_2024-05-18_002036.png
terse folio
#

maybe renaming it to 0-24, 25-49 would work

halcyon quarry
#

It's because I have them in a subdir

#

and the subdir becomes part of the name

#

in this case they're either in 'SD15' or 'SDXL'

#

It is helpful though if unsorted

terse folio
#

I have quite the range of model names!

#

and yes, some 100char+ merges

#

what's response.defer do?
not familiar with this function

#

Hmm, looks like it exits and pushes the result to the class

#

the wait() function

halcyon quarry
#

When a user does a command, you have 3 seconds to send a response interaction to them - otherwise the interaction times out

#

ctx.defer() says relax discord, Ill get to it

terse folio
#

I see

halcyon quarry
#

Yes the submit button will call stop() allowing the view to exit

#

There seems to be many different ways to handle views, using classes, using callbacks, without either, etc

#

This is what I was able to get working with my limited knowledge and what information I could understand

terse folio
#

Mhmm, I used callbacks in my views

#

as part of a View class

halcyon quarry
#

Either you work fast AF, like super fast AF - or you don't sleep 😛

#

I'm outta here - get some sleep 🙂

terse folio
#

Cya!

terse folio
#

first test working so far

halcyon quarry
#

Submit at the top tho? 🤷‍♂️

#

Is that typical?

terse folio
#

no, a sideeffect of creating the menu items in __init__

because submit is created using a decorator before init runs

halcyon quarry
#

not a big deal if it works nice this way

terse folio
#

can move it later ^^

terse folio
#

updated

#

Use the SelectedItemMenuView for all the other commands

halcyon quarry
#

I know how to do it...

halcyon quarry
#

testing with imgmodel... and it is popping the first item as self.unload_value if the cmd_name == imgmodel

halcyon quarry
#

@terse folio I have it working, but you may have a cleaner solution.
The only problem I'm having right now, is hiding or disabling the button
Pushed what I have

halcyon quarry
#

I added a positional arg for “cmd_name” but should’ve just popped and send the unload value instead

halcyon quarry
#

Got it

#

Got it perfect

#

I've added an unload argument, for the value to be popped from the full list before sending to View function.
If unload = None the Unload button is removed from View.
Otherwise, added to get_selected()...

        if self.selected_item == self.unload_value:
            return self.unload_value
#

NOW we're talking... button style = DANGER

Screenshot_2024-05-18_134930.png
keen palm
#

There's no real way to cycle between regenerated text, is there?

halcyon quarry
#

I kind of forget exactly how it works... I know that the message you perform it on, the text is used as the input

#

need to look at it again... in a few minutes... if wifey permits 😛

keen palm
#

Stinkin' SOs always demanding attention

halcyon quarry
#

Submit button not required if only 1 menu

Screenshot_2024-05-18_142316.png
#

Reality did some very beautiful work with the new Select view handling

#

@keen palm This is kind of advanced, but you could do custom tailoring of Regen / Continue if you use the tags from text feature again

keen palm
#

I'm not quite following

halcyon quarry
#

[[state:{regenerate: True}]] and then you can try modifying the input in other ways... what exactly are you trying to accomplish?
Like I said, it currently takes the text from whatever message you selected and uses it as the input text

# 
    @client.tree.context_menu(name="regenerate")
    async def regen_llm_gen(i: discord.Interaction, message: discord.Message):
        text = message.content
        await i.response.defer(thinking=False)
        
        async with task_semaphore:
            async with i.channel.typing():
                # offload to ai_gen queue
                logging.info(f'{i.user.display_name} used "Regenerate"')
                await cont_regen_task(i, i.user.display_name, text, i.channel, 'regen', message.id)
                await run_flow_if_any(i.user.display_name, i.channel, 'regen', text)
#

mainly text = message.content and that is used for the regenerate

#

I may not understand how it's supposed to work... lol

#

Maybe text is supposed to be the prompt used in the original message...
And it uses the most recent history as the item to regenerate

keen palm
#

The idea, especially with storytelling, is to regenerate a response a few times until you land on the most interesting or entertaining option, then follow along with that.

#

Yes, you want to take the prompt and generate a new response to it

halcyon quarry
#

I'm open to suggestions on how you think would be a better way to handle it

#

Make a config option for Regen / Continue text to either replace original, or create another instance?

keen palm
#

I can't remember, but, in the webui, are you able to cycle back and forth between outputs?

halcyon quarry
#

So yes I see... the command is not working as I thought it did

keen palm
#

Is it using the first generated response as the prompt for the regeneration?

#

I know with ChatGPT, you're able to regenerate responses, cycle through until you get one you like best, then continue off that

halcyon quarry
#

It seems like when the payload is sent with regenerate: True, it takes the most recent LLM reply from the history parameter and regenerates it

#

The value of text (the prompt) doesn't seem to actually have too much bearing on the result

keen palm
#

Yes, that explains things

halcyon quarry
#

Actually

#

text seems to be ignored

#

it uses the most recent user msg from history

#

So it appears to be driven exclusively from the history parameter

keen palm
#

So it's not really deriving a new response from the prompt?

halcyon quarry
#

It's getting the prompt from the history variable

#

which is what you can see in your chat log

#

Regenerate seems to ignore the text variable, which is usually the prompt

keen palm
#

Interestingly enough, the regenerated response is not included in the history. Only the first one.

halcyon quarry
#

hmm

#

Oh I know why

keen palm
halcyon quarry
#

It's because I force save_history: False for Regen and Continue

keen palm
#

That makes sense

halcyon quarry
#

I'm willing to rethink all of this...
When I added these features, it was mainly because I thought they at least needed to be there and work... but I did not finetune it

#

Since you are someone who uses it frequently, I'll take all your feedback into consideration

#

meanwhile I very rarely use those

keen palm
#

At the simplest level, regenerate should just take the prompt and existing context and create a new response, without including anything from the first response

#

Ideally, you'd be able to cycle back and forth through responses, but I don't know how feasible that would be with Discord's UI

halcyon quarry
#

I'll do some more testing... it may actually ignore the most recent LLM reply

#

I mean, it probably does

#

I could maybe make new App Commands to edit history....

#

"Replace last reply"

#

You right click on a message and select it as "Replace last reply" and it will swap it for the most recent history item

keen palm
#

I might just need to mess around with parameters to get regeneration working a bit better, since right now I'm basically just getting variations on the same thing.

#

Replace last reply would be very useful

halcyon quarry
#

I need to run along, but since this isn't using the WebUI there is actually more freedom to customize te behavior of regenerate

#

We can modify history however the heck we want, if I just know the best way to do it usefully

keen palm
#

The most useful options would be:

  • Delete last reply
  • Replace last reply (ideally, where you could type out a revision to what the bot just spit out, and that becomes its last response)
halcyon quarry
#

btw the character, llmmodel, imgmodel cmds are going to be much nicer either tonight or tomorrow

keen palm
#

Ooooh

visual dagger
#

and sometimes diff models seems just copies of each other so... yeah

halcyon quarry
#

There’s a reason there’s thousands of different models

#

Best possible prompting won’t get the result without the correct model

halcyon quarry
#

The context I use for generating image prompts is good - but only a select few models work well for it, and I’ve tested somewhere around 50 models

#

Works very very good with Neuralbeagle model in particular, really amazing output

terse folio
#

by holding CTRL in vscode, you can jump to definitions of variables/functions/classes

#

saw a commit noting where to find the class ^-^

halcyon quarry
#

🗒️

#

Working on the other commands right now

terse folio
#

It's great for debugging,
Ctrl click on discord.ui.Button, and it takes me to the class so I can see how to create a callback!

halcyon quarry
#

There must be something you have extra that I don't have enabled

#

When I hold Ctl and click it doesn't do shit

terse folio
terse folio
#

lemme see

halcyon quarry
#

Fortunately the way you set it up makes updating super breezy

#

excellent stuff m8

terse folio
#

It should be Pylance

#

It takes it's time, but once it loads your file/initiates its server, some of the syntax highlighting will change
And then things become clickable

halcyon quarry
#

surprised you removed the hashing

terse folio
#

One less computation!

#

I then added a function to retrieve items

#

But you can pass in other lists if you wish

#

Maybe pass in the discord.Choices list
so you can access the .label and .description or whatever if you need for some reason

#

But it defaults to return the name which is what the SelectedItem class is expecting as list of

#

interesting, removing the submit button and just running load here.

halcyon quarry
#

I was thinking of maybe ditching the Submit button altogether lol

#

First item you click thats the item

terse folio
#

Definitely could, but I did like the confirmation of the submit button
since loading a model can take some time

#

Let's try it out, see which experience is best!

halcyon quarry
#

I weed out models constantly so mine are all just one menu 😛

terse folio
#

I could add a flag that you could pass to the class which changes the behavior between immediate return, and require submit button

halcyon quarry
#

Or, I use my model filter feature to limit it to just XL models, omit the inpaint, base, etc

#

The Submit button could be useful for the future... especially if I update the /speak function to use this

#

That... could be a can of worms

#

(maybe leave it as is)

terse folio
#

mhm, lots of voices

halcyon quarry
#

nvm

terse folio
#

Reuse functions when possible!
Write code thinking ahead about how it could be used in other places

halcyon quarry
#

Goddamn it is so easy to update these commands

#

I just did the LLM model one in like 5 mins

terse folio
#

:)

halcyon quarry
#

Characters will be done in minutes

terse folio
#

Also windows doesn't care if you use forward or backwards slashes

#

I think Linux too

halcyon quarry
#

Well its probably why most install instructions say cd extensions cd coqui_tts

terse folio
#

nvm, linux only wants slashes, at least ubuntu does

halcyon quarry
#

if only we could pip install -r os_join('extensions', 'coqui_tts')... 😛

terse folio
terse folio
#

I was looking through XTTS webui, and saw they create a python subprocess that uses your custom venv to install RVC into a new venv.

#

so it's very doable

halcyon quarry
#

Feels soooooooooo good to delete those command option abominations

#

Just like that - /characters is now also updated

#

All done

#

Pushed to dev

#

Now must cook dinner

halcyon quarry
#

Going to update one more thing with that before commit to main - need to get the models/characters when cmd is sent now, rather than at init and storing them as global variables

halcyon quarry
#

all done, all done

keen palm
halcyon quarry
#

Now, characters, llmmodel and imgmodel commands will get all current items in those locations

#

Not just building them on init

#

Done

halcyon quarry
#

Pushed the updates

#

oops 😁

Screenshot_2024-05-18_194845.png
keen palm
#

I'll check them out later. I'll try to remember to move the bot file over

keen palm
#

Select a Large Language Model Model

halcyon quarry
#

🤷‍♂️

#

I'll do a sweeping update for that because, yes, that is a bit dumb

halcyon quarry
#

Welp, just noticed the Flows feature is bugged after recent updates

#

time to figure out why...

keen palm
#

I don't think anyone understands how Flows work

halcyon quarry
#

The best way to understand it is just execute one of the examples

#

found one bug in it...

#

Fixed it

#

I screwed it up when I made the Tags system allow multiple format_prompt tags

keen palm
halcyon quarry
#

Very simple example flow I made

Screenshot_2024-05-18_205436.png
#

What it does is it first prompts the image prompt character, for a prompt.
In this case it provided:

Ernest Ranglin playing a guitar in a warm and cozy setting, with a fireplace and a bookshelf in the background, soft lighting, warm colors, intimate atmosphere
#

The 'Flow' took that response and prefixed it with: Tell me an alternate version of this image prompt:
Then, sent it to the normal chat character with load_history: 1 and save_history: false

#

Another example flow....

#

This one just feeds the LLM response back to the same image prompt character, except prefixes it with Provide a more detailed version of this prompt:

#

Once one of you brainiacs understands how it works, I expect greatness to ensue 😎

#

The bot's reply can be used to dynamically set values for pending Flow Steps

#

such as with my Image Model Selector flow example

#

Before generating the image it will share the LLM's prompt to a character tailored to choose the most appropriate image model.
Then, the image is generated

keen palm
#

Hmmm, hmmm. Pondering the possibilities

halcyon quarry
#

They are endless - especially if you take into consideration, I can just knock down any roadblocks to your amazing idea with this

#

Here, please

#

Try this example flow here... copy/paste it into your dict_tags.yaml

# 
  - trigger: 'flow test'
    flow:
      - flow_base:
          should_gen_text: false
          should_gen_image: false
      - flow_step: Do literally nothing
        flow_step_loops: 10
#

This will give an idea just how fast the message loop executes...

#

I did generate text each loop on this one… but used should_send_text: false. Tormenting the LLM

#

pretty nice short and sweet poem

keen palm
#

Hmm, I am pondering the possibilities. Is it possible to include a prompt alongside the trigger? So you can have the bot iterate a response to that prompt?

halcyon quarry
#

you can use variables for recent messages like {user_0} is the most recent user msg

keen palm
#

What's the code for the reimagine flow?

halcyon quarry
# 
  - trigger: 'followupreimagine,reimagine'                   # Intended to be used with a prompt like 'draw _______ check imgmodel'
    insert_text: ''
    insert_text_method: 'replace'
    search_mode: user                           # Only search user's prompt (will not trigger from LLM reply)
    should_send_text: true                     # Suppress text response (still generate text)
    should_gen_image: false                     # Suppress image generation (won't send an image by default)
    flow:                                       # The actual 'Flow' tag value
      - flow_base:                              # Flow_base are tags that apply to every flow_step
          save_history: False                   # Do not write these interactions to the main chat history
          load_history: 1                       # 0 = default (all history), -1 = excludes chat history, > 1 = LLM sees this many recent exchanges.
      - flow_step: Ask LLM to reimagine the prompt # text displayed in discord embed
        format_prompt: 'Tell me an alternate version of this image prompt: {llm_0}'
        should_gen_image: true
#

I put followupreimagine as a trigger so I could cancel it with this other tag I have

  - trigger: 'reimagine'
    format_prompt: 'Tell me an alternate version of this image prompt: {llm_0}'
    on_prefix_only: true
    search_mode: user
    should_send_text: true
    should_gen_image: true
    trumps: followupreimagine
#

So the one is a Flow - engaging the LLM twice, two different characters.
The other tag is just to ask the chat character to 'reimagine' the image prompting characters previous reply

halcyon quarry
#

Here’s an interesting theoretical use…

  • Have a context focused on choosing the best of two messages.
  • Trigger a flow that asks for a short story or whatever, but loop it so you get two versions. The Dynamic Context feature could be used in the prompt so that it uses wildcards.
  • 3rd flow step, show the two replies to the judging context
#

When I finally add the new behavior settings… coming soon… that will also allow automatic prompting, so something like this could be used autonomously

#

I mean technically you can already do that with some third party software that types shit into the box and hits enter

keen palm
#

Yeah I was thinking about something to that effect. Maybe take a prompt and iterate responses with different parameters, to quickly figure out what's best for that model

halcyon quarry
#

Had an idea for regen / continue…
maybe I can keep track of message IDs paired with history length

#

When regen is called it can check if message.id in [stored_messages] then temporarily move that prompt to front

#

Make a copy of history, move that exchange pair to front

#

Yes I think every time it writes history it should add the message id to a list. When regen or continue is called it can match the id and count how deep it is and grab the same depth history

keen palm
#

That certainly makes sense to me

halcyon quarry
#

The other idea I had is maybe adding small buttons to all LLM replies

#

small regen and continue icon

#

Good morning Reality

terse folio
# halcyon quarry Yes I think every time it writes history it should add the message id to a list....

Create a message class that contains a message_id attribute.

And have a function to render the history to text.
this function will take a list[Message] and output string.

You can use junp to a specific message by iterating backwards.
Create a copy of the list, and .pop until you reach the message with matching id.
Then feed this new list to the render function to create the temporary prompt!

I would help, but am on mobile right now

#

Morning ^^

#

The message class could also contain both internal and visible contents, and work as a replacement for the dict in the history manager

halcyon quarry
#

Do you mean like, it pops the message.id list as well as the history copy simultaneously? When it matches the message id, the history copy is now same length already?

#

TGWUI seems to ignore the text param (prompt) when using Regen (need more testing for Continue)

#

Seems to operate solely on history param

terse folio
#

But, when history is being sent off to tgwi, there can be a conversion to the format it expects
which is

dict {internal, visible}

#

I'll write this in a little bit

halcyon quarry
#

right… but prior history may have an effect as well (message before the message you want regenerated)

#

Can’t tell if what you are proposing slices actual history for lack of a better concise way of stating it (I know not literally slicing)

terse folio
#

Mhm, this is just a way of grouping related data together so you dont have to handle 3 lists.

history = dict{internal=[], visible=[]}
messageids = []

when adding more attributes someday this could get out of hand.
So one solution is using an object or dict.

storage = [item1, item2..]
where item1 contains everything you need.

This is useful because you'll only need to update code once if you want to do changes in position of history or remove the last message.

terse folio
halcyon quarry
#

so far everything you’ve suggested or done has been pure gold so I’m going to put my mind to something else 🤗

#

Must must must add those new behaviors

#

As with this endeavor, I jumped in with no prior experience with the application. I was occasionally putting out little signals inviting any veteran MUGEN dev to help which was never answered.

I am a hack when it comes to this python stuff.

It’s impossible to express the gratitude I have for your recent contributions @terse folio

#

Don’t want to make it weird or anything 🥹

keen palm
#

Too late!

halcyon quarry
#

That MUGEN project came out quite good but would have been top tier if another experienced dev jumped in for a swim

terse folio
terse folio
halcyon quarry
#

I started off by amassing as much reference materal I could, and was using actual screencaps to trace and whatnot

#

I came up with a 3D model where everything could be animated the flaps and everything

#

And a post-processing method to force the colors down to a limited specific palette index

#

That project required coding, graphics, and sound engineering

terse folio
#

That's awesome, shaders are cool!