ad_discordbot (Fork of Fork of xNul's bot)

[halcyon quarry]

then going from there

[keen palm]

I haven't been doing that, so we'll see if that has an effect

Also, are you making sure to use the command on responses that have been split into multiple?

[terse folio]

you gave something a uuid that was expected an HMessage

[halcyon quarry]

yes

[halcyon quarry]

@keen palm I'd be very surprised if you could break it at this point

[keen palm]

https://tenor.com/view/challenge-accepted-challenge-accepted-gif-21781157

Let me know when I should update

[halcyon quarry]

Right meow

The last thing I want to update before pushing to main, is that History Create improvement Reality and I were discussing...

Select from Menu

[keen palm]

I gotta do some work stuff first

[halcyon quarry]

@terse folio added is_continued.
Doesn't do anything special yet aside from set the message label

as in, it's not a separate HMessage.
The updated message is getting the attribute

[terse folio]

Yea, whenever it changes to a seperate message, I would make it continued_to: hmessage

[halcyon quarry]

hmm

I'll try taking a stab at that for shits n giggles

Maybe not 😛 I'm getting cold feet already

[terse folio]

mostly copypaste the reply_to attribute and rename it

personally I would add 2

so you can go forwards and backwards in the chain

and only save one of them to file

then it would need custom logic in the loader to set the attributes correctly

but that's something i'll work on later

[halcyon quarry]

Thumbs up to that last message

It shall remain is_continued for the moment 😛

I'll try f*cking around with the discord menu for regenerated messages

[keen palm]

I'm getting that same error when regenerating, on the newest version, buuut it could be due to a problem in the history file

[halcyon quarry]

I was using random combinations of all the commands on all different messages, everything consistently working without error

Be sure to have history.py updated too, and use reset conversation

[keen palm]

Will do

[halcyon quarry]

@terse folio I think I have a good idea.

Instead of attaching buttons to the outputs of regenerate create, to be pressed for selecting the preferred output - I think it should reflexively send an ephemeral message with the select menu, immediately after each regeneration

[keen palm]

Would that give the user enough time to decide?

[halcyon quarry]

yes I think it times out after like... a minute or more

(by default)

[keen palm]

Some people are slow readers

[halcyon quarry]

I'm trying to come up with something that doesn't involve buttons, because it's pretty complicated with the buttons...

[keen palm]

I'm assuming it can't work off of reactions

[halcyon quarry]

Maybe...

[keen palm]

If it could, doing a 👍 on the chosen one would be neat, but possibly difficult to manage, what with multiple people screwing around with reactions

[halcyon quarry]

The reaction doesn't hold any information, so to use that method (user clicks the reaction), we would have to iterate over all the message ID's that it could possibly belong to to find a match

which would be, the messages we've logged as being regenerated

[keen palm]

So probably not that

[halcyon quarry]

Possibly... I'll explore it a little further

I think it has merit

Kind of clashes with my Starboard feature...

[terse folio]

If you are able to do both, try them and get upset feedback ^^

I'm not sure a minute is enough, not just for reading the menu buttons but also the regenerated message.
Going back and checking what you want different maybe

[terse folio]

reactions is probably how I would have done it, (As someone who never really used the new interactions)

I like that idea and it's also possible!

On reaction to message:
get local history,
Get matched HMessage
If HMessage is part of a group of regenerations, use HMessage.replace()
(This function is not a thing yet, I'm just talking to myself)

That would swap out other matching messages with the same ID, to be it.

As for handling multiple regenerated messages getting a reaction, I dont know. Maybe the bot could remove reactions from the others acting like radio buttons.
(A reacted attribute would tell if it has one)

It's a bit more to wrap ones head around but doable!

Before interactions and slash commands were a thing people had to think pretty outside the box for these kind of things.
Dont limit yourself! ^-^

[halcyon quarry]

I'm limited in that whatever the solution is, it's going to take me a lot of trial and error

What I was thinking was the radio button behavior idea...
So complicated. Thinking a bit more on other possible options

Looking further into it I don't think there is actually a timeout for select menus, by default

[terse folio]

I dont remember if I set it manually or not, but one of my menus times out after 5 minutes iirc

but that's if you dont click anything in the meantime

[halcyon quarry]

here's an idea...

The regenerated message that has the most reactions, is the one that is kept in history

could be a bit frustrating to some people though 😛

[terse folio]

The radio reactions thing wont be hard to implement for me,
Just get the buttons working for now ^^

Also the default behavior will already kinda act like radio buttons, where each time you add a 👍 it will replace that group with it in history.

It's just the bot wont remove reactions from the others yet.

Also hmmm, a voting system seems overkill haha.
maybe a different command for that as you need to wait for votes to accumulate first and not swap every reaction

[halcyon quarry]

I'm adding a command now to set a message exchange as hidden

[halcyon quarry]

will be the crutch to publish the recent changes until better selection method for regenerate create

almost done

[halcyon quarry]

It is working 👍
It's good enough to play around with, but I still need to add logic to ensure it only makes the user message 'hidden' if all bot replies are hidden (if multiple bot replies for same message)

I also want to set something up so the bot will instantly add or remove reaction to user's message if it is a hidden message

I added a regex to find bot history labels to be able to easily relabel text

Added it to dev

[halcyon quarry]

Ok I'm adding another nifty little addition now

I added a regex to filter out leading @ mentions when doing these operations

(the ones automatically inserted by the bot)

Default content for edit history now stripped of the mention and label.

[keen palm]

Ohhh, I see the (hidden message) thing now. That just means it's not being saved to internal history?

[halcyon quarry]

Internal history is what we refer to as everything
hidden keeps it out of TGWUI history

[keen palm]

Gotcha

But because it's in internal history, it will still be in context for the bot, and will be loaded with each bot session

[halcyon quarry]

Reality had set it to "not save" by default, which means that after reloading the bot it would not load back into internal history.

Yes, context of the bot - but not context of your chatting

You could then regenerate hidden text, but it is separate from the chat history

[keen palm]

In this case, I don't think it matters, because the tag I'm using doesn't take history into account

[halcyon quarry]

Now that hidden is actually saved, you could close/open the bot and still work from those previous hidden generations

And!

You'll have the ability to unhide them

They will still be in chronological order

I want to refine a few last details of this toggle as hidden but work has been busy today 🙂

[keen palm]

S'all good.

I'm just now managing to fix this fucking thing I've been dealing with for weeks because the software has been bugged

[halcyon quarry]

with your game?

[keen palm]

No no

At work

I need to use this software to update a hardware config on a piece of equipment, and the software requires an online login. However, it doesn't have a way to change to online mode. It's supposed to detect the network and automatically change, but it hasn't been for weeks.

Finally got it to work this morning

[halcyon quarry]

Yikes

[halcyon quarry]

Pushed to dev... again... improved management of the history labels

@terse folio take a look at the get_labeled_history_text() function in history.py when you get a chance.
I think it's pretty solid, and able to return a variety of useful outputs dependent on few settings

[terse folio]

I think it should be save by default.
If you mean the config.yaml setting

[terse folio]

Will soon!

[halcyon quarry]

My comment # 🙂

Thanks for reminder, though, I need to test out what happens via /speak command, and address any issue there

[terse folio]

oh that one, I see

[halcyon quarry]

🙈 found the better 'hidden' emoji

hidden

[halcyon quarry]

I did successfully set it up so that, if you use toggle as hidden it will add/remove the label from bot's text AND add/remove reaction from the corresponding user message

[halcyon quarry]

Things are working so well now I am going to push this to Main

This toggle as hidden is super cool

Merged nice update to main

33 Commits!

[keen palm]

Wooooooo

[keen palm]

Quick clarification on this point:

regenerate create will make a new response instead of replacing the original one. Paired with toggle as hidden its easy to select your favorite output, or revert back if you change your mind.
The hidden responses, since they still appear on the context of the bot, could influence the bot's future responses, right?

[terse folio]

Hidden responses are not in context

Hidden is saved to file.
Unsavable Is not saved to file.
Unsavable is available while bot is running, only during that session.
The 2 arent related but can be mixed ^-^

Unsavable can be in context If not hidden

[halcyon quarry]

hidden means the message can still be used with variables like {llm_0}, {user_2} (mainly for Tags system), and can also be toggled back into TGWUI history now

[keen palm]

Ahhhhhhhhhhhhh

[halcyon quarry]

I kind of lied about this part Paired with toggle as hidden its easy to select your favorite output, or revert back if you change your mind.
But I'm fixing that lie now 🙂

[keen palm]

https://tenor.com/view/anakin-liar-star-wars-lying-gif-8634649

[halcyon quarry]

I think this will do it, but no time to test atm

            # Prevent user message from being automatically hidden while open bot replies
            num_bot_hidden_msgs = 0
            all_bot_replies = local_history.search(lambda m: m.reply_to == user_message.uuid)
            for bot_msg in all_bot_replies:
                if getattr(bot_msg, 'hidden', False):
                    num_bot_hidden_msgs += 1
            num_bot_open_msgs = len(all_bot_message_ids) - num_bot_hidden_msgs

            if num_bot_open_msgs > 1:
                user_message.hidden = False

[halcyon quarry]

OK I did crack the code, it is working brilliantly

[keen palm]

The hide response code?

[halcyon quarry]

Yes, now when you use regenerate create and hide one of the new bot messages, it will only hide the user's message if all of the created bot messages are hidden.

If you try revealing the user message, the most recent bot message is revealed with it

Seeing if I can automatically set older bot message to be hidden from regenerate create...

then, will see if I can automatically toggle it so only one remains revealed at all times...

@keen palm Have you updated yet?

[keen palm]

I did, yes

Only briefly tested things, but it's working so far

I haven't tried out regenerate create

[halcyon quarry]

It's also working for continued messages - treating a pair of continued messages as one message

with the toggling as hidden

[keen palm]

Hmmm, discord seems to be shitting the bed

[halcyon quarry]

I had some issues for a few minutes, but they resolved

Just pushed update that improves the toggling for hidden messages

(does not hide user message unless all replies are hidden)

[keen palm]

I haven't even gotten the opportunity to break the previous update!

[halcyon quarry]

snag this one, make some continued text randomly, use regen create randomly - then have fun toggling things as hidden or not

Be amazed as all the message labels update smoothly and predictably

mirroring their actual hidden status

[keen palm]

I think I broke it?

Continued some responses, toggled them hidden, then continued off them again. The response was a continuation of a different, non-hidden response, and now I can't toggle any of the hidden responses

[halcyon quarry]

Will try repeating tomorrow!

[terse folio]

I think there's a bug here

passing a discord message to the get_labeled_history_text function which is expecting HMessage (because of the getattr text)

Also!
HMessage.text will always be set, it will be '' by default

so if message.text: {do code} is plenty ^-^

[halcyon quarry]

I may have the hints reversed in that Modal

[terse folio]

I'll go around adding some assert statements to enforce types, in the case something is passed by mistake we'll know!

[halcyon quarry]

target message should be the discord message

[terse folio]

okay

[halcyon quarry]

Part of me is tempted to replace the labels with reactions. A very small part

[terse folio]

how about emojis in the text as a label

[halcyon quarry]

Most of the reason I want to use the labels is it’s clear what they mean, the reactions/emojis aren’t obvious

The hidden monkey on user messages has some nice context clues, in that it always has a bot measage attached to it with “hidden”

[terse folio]

if only there was a way to do hover cards or something with discord

[halcyon quarry]

Yea seriously

You should update and try it out 🤗 the commands are working great

[terse folio]

😸
soon, having some fun getting distracted making a node visualiser!

wanting to create a graph based database for RAG

[halcyon quarry]

Yeah you were right, I was passing the wrong message to the labeller function

I revised the labelling for the messages involved in this, too

[burnt patrol]

when i reset convo, the greeting message doesn't appear, is that because I'm using history?

[halcyon quarry]

I could update it to still send the greeting to the interaction channel (where the command was used), but I don’t think sending greeting to all channels is a good idea 😛 I had removed it after per-channel history

[burnt patrol]

Agreed

[halcyon quarry]

I had also made it send embed to either announcement channels or otherwise interaction channel - recently changed that to both

Will update greeting to send to interaction channel

[keen palm]

I haven't broken anything yet, but it was a busy day at work

[halcyon quarry]

@burnt patrol look what I did... was literally this one line only lol

Pushed updates to Main

[halcyon quarry]

Got a few improvements in mind that I'll try bangin' out today

[keen palm]

Oooh nice

Is delete response on the radar?

[halcyon quarry]

Would need help from Reality on that. toggle as hidden does almost that

[halcyon quarry]

ugh, thought I could toggle TTS model loading/unloading but it's not that simple

I'll see if I can at least allow toggling TTS generation on/off, but the model status will not change after init

[halcyon quarry]

yes do have the toggle working nicely

[keen palm]

I haven't updated to the latest version yet; does it change how the internal history is formatted? Just wondering if I'll have issues, since I have a somewhat long history going right now.

[halcyon quarry]

Nothing new should break old history

or change old history

Well, it will change history if user uses the new commands 😛

[keen palm]

Fo sho

[halcyon quarry]

Pushed smallish update to Main:

• Now a /toggle_tts command to disable/re-enable TTS.
• Edit History and Hide/Reveal history now also use the task queue - to prevent screwing things up while something is being generated

[halcyon quarry]

I'm just about done making this work via DM's - with an admin setting to disable bot usage via DMs

Surprisingly, DMs are considered their own channels so the per-channel history management actually works.
However, I set these as unsaved so the conversations will not be recorded to file.

[halcyon quarry]

@terse folio I haven't looked too deep into it yet, but do you know how to prevent it from creating a directory and saving only the channel details locally, when all messages are unsaved ?

Pushed the DM Support to dev branch for now, which is otherwise looking quite good.
I'll try to crack the code on that saving behavior if you don't crack it for me 😛

[terse folio]

Yea, I remember that issue.
The mkdirs function runs in the "start saving" area while it is deciding what name to create for the file.

Then it waits (If the last save happened recently)

Then finally saves if the setting is enabled

[terse folio]

moved mkdir to trigger_save so it creates the dir if not exists then.

pushed to dev

it was happening because when you create the history object, it also creates the unique id, and path on init.

[terse folio]

confused about this, there would never be an exception here

[terse folio]

Should discord.tts_settings be the default if tgwui is not set?

This will never be None,
HMessage.hidden will only be True or False
Just replace it ^-^

[terse folio]

made some random cleanups
added assertions so the history functions will error if the something wrong is passed to them like a discord.Message

[terse folio]

You can now use HMessage.set_continuation(HMessage)

This will assign HMessage.continue_next and HMessage.continue_prev

There is also a property
HMessage.is_root_of_continue_chain
that returns True if it's the first item in the continue chain.
it will also return True if there are no continuations on either side.

[terse folio]

HMessage.get_continued_chain will return an iterable of HMessages.
Passing start_from_root will determine if the chain starts from the real root, or from the current selected message.

But I haven't tested the functions, do report if something's broken ^^

Actually will hold onto this for now since one might one to create multiple continuations for a message. like regenerating them.
But that breaks the chain idea

guess I could do filtering, like pick the first non hidden one and it would be up to you to make sure to hide the previous continuations for which one you want to be active.
But could also add a function for that

[halcyon quarry]

Oof, I don’t like making these sorts of simple mistakes lol

Thanks for reviewing my recent changes

And for resolving that makedir, I’ll check it all out in a bit

[terse folio]

here's an interesting one

we have a list, that becomes a set.
and using dict methods on it

also don't define lists/dicts in function args, those are global and will create unintented behavior

use trump_params:list=None as the arg
and trump_params = trump_params or [] inside the function

[terse folio]

i am surprized so much of this code works!

line ~3697
check_cnet_online needs an endpoint url.
I dont know what to give it

[terse folio]

need to stop using try:catch everywhere.
Errors likely happen and we never know

it's fine to have errors outside of the main loop.
Command errors wont crash the bot

Just need an error handler and it can tell the user something went wrong!

[halcyon quarry]

I'm still learning, believe me 😛

[terse folio]

were you trying to sort message ids?

[halcyon quarry]

It does

[terse folio]

They should already be sorted by time

because that's when they get added

[terse folio]

Yea, idk why it's complaining, it has a default value of None

Anyway, pylance has a type checking setting that was off by default

it's proving very helpful in spotting mistakes that wouldn't be caught otherwise

[halcyon quarry]

I came in and have a lot of work work to do, want to dive right into the bot code but alas, needs to wait

[terse folio]

for example, pylance warned me ictx.author might not always exist
then I swapped it out to get_user_ctx_inter and things are good :)

[halcyon quarry]

Nifty, I need to enable that setting

pylance is so amazing, idk I went so far without it

[terse folio]

It has some annoying issues with discord.py though, constant redlines for most discord related things sadly

because not all channels have a .send attribute, and it doesn't really know what type of channel you're getting from client.fetch_channel

it's this setting, after searching pylance.
it's way too much to read for this large bot!
i was just chipping away at small bits

here's something that annoys me that's out of our control:

This should be correct

But client.get_channel says it requires an int.
not an optional int (can be None)

But then the next function call inside client.get_channel passes to _connection.get_channel
which is properly set to optional[int]...

[halcyon quarry]

eyyy

I think you broke something I fixed before

[terse folio]

we shouldn't be doing getattr(id) on nonetypes!

add an if statement beforehand that changes if message is not None

and do logic after that

[halcyon quarry]

This one was working previously, I'm sure of it 🙂

[terse folio]

line 28:
if user_message is not None and user_message.id:

[terse folio]

it was working, but its harder to debug because getattr isn't evaluated by VScode to check if things are working or not

[halcyon quarry]

updated, trying again

[terse folio]

yea, it doesn't know what "x" is in this case

[halcyon quarry]

working now 👍

[terse folio]

before there are too many changes, i'll push some more stuff

[halcyon quarry]

Had a chance to play around with the new features at all?

[terse folio]

also this is always something to look out for.

If one of these functions errors it will reach the end of the function and return None by default.

As this gets unpacked into other variables outside the function it will throw another error.

Need to match the same number of outputs everywhere ^^

[terse folio]

been busy with my own things

[halcyon quarry]

When I sent a DM, it still did create the directory with file/ channel info

[terse folio]

did you run save history?

[halcyon quarry]

Nope, simply sent a message then hit the X

[terse folio]

hmm, well it should all work

havent tried dms yet

[halcyon quarry]

I'm pretty excited about the DMs, and it was fairly easy to restrict majority of the commands from it

With 2 new config settings, admins can easily disable it

I'm thinking it could be nice to add another setting to rate limit DM usage

[terse folio]

I think dms should be "main" channels by default

the reason you mention the bot in the first place is so the bot doesn't spam all text channels when someone sends a message

but in dms there's no reason to

[halcyon quarry]

Ah, just automatically set it as a main channel

good idea

[terse folio]

don't save it to file, just do a check "if in main channels, or is a dm"

you have the user_message set to don't save with dms

[halcyon quarry]

Are you playing around with the code atm, or soon?
Like you said, don't want to be updating same stuff at same time 😛

[terse folio]

was checking that the bot runs before pushing

i got it to save in dms by commenting out those 2 instances of dontsave

also huhhh, since when did they give dms a seperate channel id from the user id

interesting

pushed

process_tag_trumps is still doing something I don't know what, should take a look at that

[halcyon quarry]

Will do

[halcyon quarry]

You've been quite the busy bee, so many changes

@terse folio it's still creating the directory for DMs 🙂

[terse folio]

do you have the save history setting disabled?

[halcyon quarry]

This is specifically for the internal directory - I suppose we dont have to care that it could make a buttload of essentially empty directories

[terse folio]

Yes there was a setting to disable internal history

[halcyon quarry]

right... it's a bit heavy handed though, as that disables the ability to load history from bot startup

I don't want to record DM messages though - which, it isn't, but will just make a basically useless directory for each user

[terse folio]

It's not writing a file?

Did you use my merge from the dev branch?

It should only make the folder on actual save now

Which means it has to first get past the dont save checks

[halcyon quarry]

I'm just booting up, sending the bot a DM, then hitting the X to close the cmd window

it's creating a directory with the json for channel details

Did revise the DM main channel handling

[terse folio]

okay so messages don't save to history, but since the history class exists, it does save to file since you don't disable saving

[halcyon quarry]

right - this was the extent to which it was happening when I pinged ya last night

[terse folio]

it's not going to be too easy to stop saving only for dms

would have to get the channel id, get the channel from the bot, then check if its a dm

and I don't know if that's a good solution

what's the reason to not save dm messages?

[halcyon quarry]

Mainly to try respecting privacy

[terse folio]

~~For example, if per channel history is disabled dms will be writen to history. ~~
hmmm maybe not because messages are marked as dont save
there's no way to filter that based on the function im about to write.

[halcyon quarry]

checking if the channel id is a DM channel isn't too intensive 🙂

[terse folio]

yea it should be fine, its more just adding confusion with the global channel state and all

maybe should just add an attribute to history

that marks it as don't save at all

[halcyon quarry]

Yes... could you?

either ignore or junk etc

I want to use such an attribute for output from the /speak command, possibly other interactions further down the line

[terse folio]

history.dont_save() same as HMessage

[halcyon quarry]

actually could probably bypass creating message objects for /speak...

[terse folio]

pushed it

[keen palm]

So is it not possible to have the bot use different characters on different servers?

[halcyon quarry]

Currently, correct

You have me thinking, though

[keen palm]

well that's good!

[halcyon quarry]

I'd always had it in my head how it wouldn't work for per-channel characters, but per-server characters is a more reasonable idea for the name management

[keen palm]

Yeah

[halcyon quarry]

I could do it, really, but if it’s an idea that Reality also likes - my way would work but probably not the correct way, then they’d do it the correct way 😛 like this per channel histories

[terse folio]

I dont know how to go about per channel characters from within tgwui.

That's something I would do from the Api

[halcyon quarry]

per guild characters

I already have a working system in mind

[keen palm]

Would be super convenient to have that option

[halcyon quarry]

Mainly, instead of activesettings, settings dicts would be dynamically created and loaded based on guild id

Which reminds me, I need to stop saving model name and vae

The way I would handle this would be identical to my approach for per channel histories

Well, except for separate files rather than all in one file

[terse folio]

do you have a way to seperate characters per guild that doesnt include switching them really fast each time?

[halcyon quarry]

It’s just the payload…

[keen palm]

share-the-load.gif

[halcyon quarry]

The way I would do it is bot_settings would be nested under a dictionary key for each guild id

[terse folio]

Oh perfect!
I was thinking tgwui did something weird by loading characters globally

you could just tie a character to the history class for it to use in each channel/guild

[halcyon quarry]

Nah, it’s why swap_character tag has instant effect

[terse folio]

good to know

Instead of writing the characters per guild/channel yourself.
You can implement a command like /main

Or maybe using change character would save the last used character in that <area>

[halcyon quarry]

Nah

Yes to the second

The bot settings will just initialize as defaults in all guilds

Then any settings changes will apply to that dictionary

And save to that guilds activesettings file

Much like how channel histories initialize for each channel

[terse folio]

I see I see

[halcyon quarry]

While we’re on the topic, I plan to make the bot operate exclusively on “per channel histories” mode, remove the setting for shared single history

Idk how you could possibly do anything productive with shared histories between channels

Other than, celebrate chaos

[keen palm]

Yeah it would be ugly

[terse folio]

Shared channels would be cool through a memory system, being able to call upon other's doings. but not sharing the same context

[halcyon quarry]

Shared channels could only be useful if the bot sent copies of the conversation to paired channels in real time

I think we’re already approaching too many interactions though… I’m sure people using this in medium servers get a lot of rate limited responses

[terse folio]

rate limits were like 5msgs per channel per second iirc

I think the generation time would slow down the bot before any rate limits happen

[halcyon quarry]

There’s a limit for api calls in general

[terse folio]

Yup, I wonder if that applies to sharded bots as well

as you reach a larger memeber count/guild count, you can have multiple bot connections that handle different groups of guilds.

I think they have their own ratelimits because there are bots that handle millions of people

[halcyon quarry]

This would be some other system we can tease on the backburner, wouldn’t matter if we obsolete the single history

[vestal python]

:x had to shut off one of my discord bots. Apparently running the AC, the 5GPU server, and the dual GPU AI bot was causing the lights to flicker.

I'd ask the bot a question earlier, and as it was processing you could hear this buzzing noise and flickering lights.

[keen palm]

How much power are your GPUs using?

[vestal python]

Not much.. Idk the server is the whole 5 rtx 3060 12GBs, and the one I shut off was a P40+GTX1080ti

[keen palm]

Undervolt them?

[vestal python]

I'm going to see if my brother can host it at his house.

[keen palm]

Undervolting is a good idea anyway. I have my 2x 3060s undervolted, and they draw ~75%% of the power they normally would

[terse folio]

5 gpus is getting close to the range of a stove/oven in watt usage.
At that point I would make sure the circuit you're on can handle that much power.

Assuming they're like 250w each

[keen palm]

3060s don't draw that much power though

[vestal python]

They're around 90w each (aphrodite tensor parallels) 4 running 1 isn't even in operation right now, with a xeon cpu that might be pulling 40~60.

Their limit is 170w, but they never go past 120w

[terse folio]

wow, wonder what's up with the flickering

maybe the overhead spike in power draw as it starts generating?

[vestal python]

It only happens when both cpus are on, and the main 5GPU server is generating yeah... Having my Ogma AI Discord bot shutoff stops the flickering. It's this damn high heat with the ACs and fans throughout the house keeping the place cool.

[terse folio]

With pylance, just found out how to hide certain rules across your whole vscode workspace ^^
In settings you can search diagnosticSeverityOverrides
Enter the config.json and add "yourrule":false

for example reportOptionalMemberAccess annoys me because we have a lot of code like dict.get(x, {}).get(y, {})
And because the first item could return None, it complains as it doesn't take args into account.

[halcyon quarry]

I'm having lots of fun cleaning up my imgmodel handling...

I'm adding last_imgmodel_name and last_imgmodel_checkpoint to database, for model swp tags / auto-select-imgmodels

Keeping model checkpoint unsaved from payload from now on

[halcyon quarry]

@terse folio bug which occurs if there is any existing history to load up. Happens on first message

[terse folio]

need to trace that back and find out why a str is being put in the replies list

self.assistant.reply_to if self.assistant.reply_to is not None else None

that's really unnecessary!

replied_to_user = self.assistant.reply_to
Would evaluate to the same thing, being HMessage or None

Hmm that's odd

I added assertions to the mark_reply function to make sure only an HMessage is passed

oh

you say on load

that must mean the messages aren't being converted to HMessage objects

[halcyon quarry]

Right, I was just starting the bot and sending a message request

[terse folio]

ill test that out

are you on the dev branch?

the code works for me

[halcyon quarry]

Yes - what happened is I loaded up, sent a message and received the error.
Used /reset conversation, then it went away.

[terse folio]

does it happen again if you restart the bot and get new history

also whats in the history file? maybe something broke

do the ids match?

[halcyon quarry]

Well, I deleted all the history 😛

[terse folio]

print(f'Marked reply: {message} for {replying_to}')
in history.py I tested this

around that line

[halcyon quarry]

Did you know, it's still creating the directory for DMs? 😛

[terse folio]

yea

i added the history.dont_save() function

just need to put that somewhere

when it detects youre in a dm

perhaps the on_message function

[halcyon quarry]

Finally finished this image models crap

It's... considerably cleaner now

@terse folio look about right?

[terse folio]

yep

[halcyon quarry]

didn't work 🤷‍♂️

[terse folio]

ohno! what's the error?

ohhh

silly me, I added the function and attribute but didnt use it

pushed

[halcyon quarry]

it works!

🫡

[terse folio]

[halcyon quarry]

Ready for main now

I'l have to fix trump params tomorrow, assuming something is broken there...

[halcyon quarry]

Need to update TGWUI tomorrow - probably a few keyerrors now that there are params for DRY sampling

[terse folio]

so that updates don't break the bot, you can use a difference of payloads.
only set the values you want to change, and leave everything else as default grabbed from tgwui

[halcyon quarry]

The bot’s method is different from API - there is no problem with sending invalid parameters

[terse folio]

okay

[halcyon quarry]

The problem is not sending all parameters 😛

[terse folio]

is there a way to access them all in tgwui?

and compare automaticly

[halcyon quarry]

I was thinking of adding a loop to collect the key names from the error, and rebuild the payload with defaults imported from chat.py - if not there then placeholder defaults

I’m in bed but I think most default values appear in chat.py, may be a different module

[terse folio]

modules.presets load_preset could help

[halcyon quarry]

Bloody hell, just realized that all private channels are triggered by the isinstance check I shared earlier

Not just direct messages

[terse folio]

what are private channels?

[halcyon quarry]

If you right click a channel and set it as private

[terse folio]

it's probably just applying a template of role permissions

I see

marking the everyone role as can't view

[halcyon quarry]

Ok I know how to differentiate them now. DMs do not have the guild attribute

[terse folio]

does discord.DMChannel apply to private channels?

that doesnt sound right

[halcyon quarry]

Apparently yes

[terse folio]

hmm

[halcyon quarry]

Well I have a private channel and it’s triggering the direct message handling lol

[terse folio]

just tested with my bot on a private channel, isistance discord.dm channel returns false

[terse folio]

looking at the code

i think it will always trigger in all channels

allowed_commands = config.get('discord', {}).get('direct_messages', {}).get('allowed_commands', [])

random public channel

[halcyon quarry]

Eek.

[terse folio]

this should only run if in a dm channel

also there are better ways to restrict commands!

[halcyon quarry]

Any chance you could fix that for me and push to main?

[terse folio]

use the check decorator

like @guild only or whatever that was

[halcyon quarry]

Only apllies to /image and /speak

[terse folio]

yup, just wrap the /image and /speak commands in @guild_only

[halcyon quarry]

I used @guild for most of the commands… I want image and speak to be configurable

[terse folio]

make another check that's configurable!

[halcyon quarry]

I’m a noob XD Almost got it right, I think

[terse folio]

Yea will add something

Need to take a look at the discordpy extension docs.
A lot of interesting stuff there you might never have thought about.

[halcyon quarry]

I’ve looked many times, there’s an overwhelming amount of stuff there

[terse folio]

https://discordpy.readthedocs.io/en/stable/interactions/api.html?highlight=default_permissions#discord.app_commands.default_permissions

Looks like app commands have their own version of checks that don't mix with normal commands

Not sure what to do about hybrid commands

nvm, the guild_only check seems to work fine

/reset_conversation allowed in dms by default?

[halcyon quarry]

Figured there’s no disruption to bot outside the DM by resetting your own conversation

Didn’t test that it worked after your recent change 😛 trying that now…

Woops… found an error, if there’s TTS

reset seems to work though

It’s not creating a directory

[halcyon quarry]

Added a little function in utils discord to check if it's a DM

def is_direct_message(ictx: CtxInteraction):
    return ictx and getattr(ictx, 'guild') is None \
        and hasattr(ictx, 'channel') and isinstance(ictx.channel, discord.DMChannel)

I know simply getattr(ictx, 'guild') is None may be all that's needed

[halcyon quarry]

Working great, so pushed it to main along with your command handling.
Thanks!

[keen palm]

So that's DM support?

[halcyon quarry]

I had set a condition that I thought would trigger for DM messages - and it was - except the condition also applied to private channels

Also, it would error when trying to process tts on voice channels from DMs (we don’t want to play on them anyway). Now, am properly checking that

[keen palm]

And per guild characters shouldn't be too hard to do?

[halcyon quarry]

It’s feasible

Last night I resolved the one setting that would cause issues with per guild settings (sd_model_checkpoint)

[keen palm]

Why was that one problematic?

[halcyon quarry]

If it’s included in the payload, the model will change

Saving it to the settings is a relic of my old model changing handling.

The commands / tags that trigger model changes already use API calls to perform the change

So if a model is saved to one settings, and another model saved to other settings - the model will be constantly changing

[keen palm]

Ohh I see

That would indeed be mildly problematic

[halcyon quarry]

Very problematic lol

[keen palm]

Teensy bit detrimental

[halcyon quarry]

Even until last night it was annoying because it would force model change on startup, if the last model you were using locally is different from the last one saved in settings

That’s fixed now though

[keen palm]

Excellent

[halcyon quarry]

👀

[terse folio]

was actually kinda curious because I didn't know 100%
But now know for sure!
When using sql dbs it does page from disk. not holding everything in ram ^-^

Changing a single letter in a row and committing it sent a few bytes, not rewriting the whole GB sized file, nice nice

probably just loads the indexes into ram to speed up queries

wow yep, if not using the browser, and just sending a query, it's only a small spike of traffic! yet still returns the same results

[halcyon quarry]

Big db!

[terse folio]

Yea!

I'm thinking if it would be a good idea to try making a version of the database class for using sql backend.
Of course abstracting away the need to run any queries yourself.
You would just need to know the methods provided.

but this is another good solution to handling settings across many servers/channels!
I could get internal history stored there too.

Not something that would happen any time soon though

and by going a little further and adding the setting for the ability to use a mysql server.
One could run multiple bots sharing the same database!
Like if a user could set a custom bio it would appear on all bots.

[halcyon quarry]

The settings don’t save very often

[terse folio]

yea just thinking to myself ^^

[halcyon quarry]

And neither does history due to your timer

[terse folio]

Hmm, so I think the discord.Interaction only happens from app interactions, like the menu buttons and such

so when using slash commands, you get the normal commands.Context object

noticed that while looking into the discord.py error handler

there's a seperate event for on_app_command_error which doesn't trigger from slash commands

but I think that might be due to them being hybrid?
Which adds them to the bot.
But I don't really know what the alternative would be.

[halcyon quarry]

To improve error handling? Or is there an outstanding issue?

[terse folio]

Was figuring out why the dm check wasnt working with slash commands yesterday

Turned out there was an error in the check, and it was failing silently

the on_app_command_error doesnt seem to be triggered by hybrid commands, maybe only those defined specifically as app_commands

it's all fine!

[halcyon quarry]

I think I know why

Nvm

[keen palm]

Is it possible to pass discord channel descriptions to the bot as context? I'm thinking of being able to use descriptions as a way to influence/guide the character

[halcyon quarry]

Didn’t know that channels could have a description property

Do they?

[keen palm]

It's the channel topic field, actually, not description

[halcyon quarry]

Ah. That’s a good idea for an addition to the in progress Server Mode feature

Been a bit sidetracked with all this per-channel stuff

[keen palm]

Server Mode?

[halcyon quarry]

Yes, plan on having a number of special handlings go into effect for one config setting, server_mode

I’ve simulated it already with good results

[terse folio]

you can set a prefix the bot looks for in the channel topic to know if it should add it or not to context

[keen palm]

That would be useful, so some channels could opt out of that, while still having a topic set

[halcyon quarry]

Next thing I’m doing is adjusting regenerate create, so that the previous output is automatically toggled as hidden

Also! Just found out that the bot can have a unique voice client for each server (can join one voice channel per server).
I'm also going to add the functionality to speak on multiple servers

I think what I'm going to do is remove the voice_channel config option, and instead add a new command called /set_server_voice_channel which will require a channel ID as input value. It would then update the channel ID in the database for the guild it is executed in

I wonder what kind of commands can be added here... (right clicking the bot)

[halcyon quarry]

Automatically hide original bot messages when creating… easy, and done

[keen palm]

Woooo

Did you push that, or waiting?

[halcyon quarry]

In a bit - need to test a bit more to endure there isn’t some unforseen issue

[halcyon quarry]

I think I’ll also make it so that if only one message pair is revealed from messages that were regen created, and you reveal another bot message, it will hide the one that was revealed

[keen palm]

That makes sense

[halcyon quarry]

To hide all, the command would be used on the user message

[terse folio]

reset convo command could go here

or does this apply to the entire server?

if so, maybe change character menu

[keen palm]

Oh, is it possible to do boolean logic in tags? For instance Trigger A AND Trigger B? Regexp somehow?

[terse folio]

do they only trigger one tag per message?
Like it will pick the one that triggers first or has a higher priority?

if so, maybe there's a way to execute the triggering of another tag within one.
Perhaps theres an example of this

[halcyon quarry]

I took a stab at it before but I ran into issues - mainly, when it came to performing text updates.
Idea is to allow any number of triggerX tags where all would need to be triggered.
I've been planning to try again, but only allow the first trigger to be used for tags like insert_text

Also planning to add a regex_text tag

So, you could still use the multiple triggers logic even for regex_text - except it would perform the regex as a result, instead of modifying the trigger match text

[terse folio]

yea, you could have some sort of flags object that is initliazed when the tags run on that message.
when using insert_text, set a flag that prevents/errors if other tags use it.

Maybe other tags could have the option for "optional_insert_text"
These will just ignore if the flag is set.
Others would send an error to the user.
Maybe another "break_if_insert_text"
that stops execution at that point but doesn't error.

[keen palm]

Neato. The reason I'm asking is because Novel AI has that with lorebooks (you can actually do regexp triggers), and that's useful for injecting context about the relationship between two or more story elements.

[halcyon quarry]

The first trigger (trigger or trigger0) would have to be unique for each tag for clean logic.
(If using text modifiers)
I could just add a check to ensure two tags do not have identical trigger, and send a warning message of why things are about to work unexpectedly 😛

[keen palm]

The prefix/suffix context things would work, though

[halcyon quarry]

yes

[terse folio]

about the trump_tags thing:
I am tracing back the code to figure out what kind of type it is as I type this.

sort_tags() can return a list/dict depending on if it errors or not.
If successful it returns a dict, so we could assume that's the intended behaviour? ⚠️ Bit confusing

That passes into get_tags(), which also returns dict/list (list on error)

That output is passed to match_tags() where the type hints are expecting a dict.
If an error occurs it will return the original tags (which could be a list)

The reason an error would occur is because you can't get items out of a list like you would a dict.
Then the tags['trump_params'] is passed to process_tag_trumps

tags['trump_params'] is a list according to sort_tags
That is also reflected in the type hint for process_tag_trumps ✅

trump_params is changed to a set.
And I didn't know this, but sets do have a .update method!
So everything works out ✅

Only issue is if the function errors it will return matches. (1 return)
but normally it will return untrumpted_matches + trump_params (2 returns) ✅ (added a default empty set output)

it all works fine, but just wanted to leave a breakdown if someone wants to revisit it and make it a little more stable!

[halcyon quarry]

Appreciated! I probably just got confused myself about the dicts / lists when creating the system, it was a big task and my head was constantly spinning

[terse folio]

You definitally should draw out a flowchart.
I started doing that for some of my huge projects and it really helps for those things where everything interacts with eachother in some tight complicated way

i've been using this

https://mermaid.live

can draft relationships as text, and the graph gets updated live

what I love is I can just add new things and it will find a way to fit them in.
unlike drawing on paper/digital where i'd have to move it all around myself or redo it if I get an idea

and llms can kinda output this syntax if you ask them to draw a graph ^-^

I slapped a grammar and post processing ontop and got results

[halcyon quarry]

Rewrote the code for toggling hidden messages... not working as expected.
Debugging...

[terse folio]

Oh!
if a 10b model could give decent results, i'm sure chatgpt could do even better for graphs!
Maybe you can ask it for a flowchart of some given code

[terse folio]

hmm, what's up?

[halcyon quarry]

I revised regenerate create to automatically toggle the previous bot reply as hidden
Now, I'm revising the toggle as hidden cmd to only hide the user message if:

• was executed on the user message, OR
• there is only one bot reply.

If there are multiple bot replies, and it is executed on one of them, will alternate with the next bot message

hiding one bot message will reveal the next one / vice versa

[terse folio]

I see

maybe an attribute for the HMessage, like "shown_reply"
That way later while iterating the replies, once you reach shown_reply, go to the next item and toggle to that one.

if shown_reply is None, just set it to the first item in replies that is not hidden,
or if all are hidden, pick a random one

[halcyon quarry]

Any chance you could take a look at this code? I feel like you'd spot whatever the flaw is pretty quickly

[terse folio]

is it pushed to dev?

[halcyon quarry]

Will push now - it is flawed though

function of interest is apply_hide_or_reveal_history()

I found I couldn't avoid a repetitive logic for toggling 2 messages, so I made a little helper function.

[terse folio]

it's okay to have some repetition!
expecially while testing out ideas

[halcyon quarry]

the regenerate create is working great though

...scratch that

works as intended if performed on the most recent reply or the user message, but not when performed on a prior reply

[terse folio]

what's stagger?

[halcyon quarry]

If we're toggling the user message and bot message, they will both be same value (True, True / False, False)

If we're toggling 2 bot messages, they'll be staggered (True, False / False, True)

[terse folio]

hmmmmm okay

[halcyon quarry]

Ok, already found one flaw with this version of apply_hide_or_reveal_history()

• when setting hiding_messages and revealing_messages its checking the most recent bot reply, instead of the selected bot reply

[terse folio]

in get_history_pair_from_msg_id it selects the last reply if there's multiple

[halcyon quarry]

yep

[terse folio]

can make a copy of the function for now to return a list for both sides

[halcyon quarry]

I think the better logic will be,

If the input ID matches a particular bot reply, return that reply

[terse folio]

added get_history_pair_from_msg_id_as_list for now

yea, get the list, then match the bot id

but how do you know which one is the bot reply you want?

[halcyon quarry]

We're sending one ID to the function.
If the ID is not found in an assistant message, we'll return the most recent bot message.
If the ID is for any matched assistant message, return that message

[terse folio]

but what if the id is for a user message?

if it's for an assistant message, it will return the correct bot message and user message

but if it's a user message it will return the user message and all the bot messages (or the last one in the case of the current function)

[halcyon quarry]

It already works as intended if the provided ID is for a user message

[terse folio]

oh so the issue is if you provide the bot message id

[halcyon quarry]

yeah

[terse folio]

it selects the correct 2 messages

But

later in the code, you get the replies from the user message

and then its confused again

over here you can use for ii, reply in enumerate(all_bot_replies)

discord.utils.find would work

discord.utils.find(lambda item: <condition with item>, <list>)

[halcyon quarry]

I think I fudged this part up too... I think this is correct,,,

            msg_ids_to_edit = []
            for target_msg in hmessages_to_relabel:
                msg_ids_to_edit.extend(target_msg.id)
                if target_msg.related_ids:
                    msg_ids_to_edit.extend(target_msg.related_ids)

I also have incorrected indent after the break

[terse folio]

whats it mean re-label?

[halcyon quarry]

The function will regex the original labels out, then process the text based on the current attributes of the HMessage

[terse folio]

I see

I think it would be simpler to have the labels and original message seperate.
No need to regex anything, just create the message when sending ^-^

but that's for a later time

[halcyon quarry]

Also, potentially just using reacts 😛

[terse folio]

sure, you could make it a setting at that point, to use text or reacts, or emojis
reacts would need a helper function to check message reactions though and update as needed

[halcyon quarry]

yep, similar to the labelling thing I got going on

I'm making some progress on this, now I feel bad for calling in the big dogs lol

I think this is the correct logic to check at start...

            if user_message is None or bot_message is None:
                await inter.followup.send("A valid message pair could not be found for the target message.", ephemeral=True, delete_after=5)
                return
            hiding_messages = not target_hmessage.hidden

[terse folio]

looks good

[halcyon quarry]

(or revealing_messages = target_hmessage.hidden)

[terse folio]

also you could raise Exception with a custom message.
And in the error handler it could tell you something went wrong on discord

But I think that uses inter.response.send

not followup

maybe would be worth testing both

[halcyon quarry]

My code is working now in this function - it seems to be that there must be a bug in my relabelling code

[terse folio]

possibly silently erroring?

[halcyon quarry]

It's just not labelling one of the messages correctly

I need to retire for the evening, thanks for the help. I knew this bit was going to be a PITA but proving to be a bit more than expected

[terse folio]

got to remember not to do this in function definitions!

this creates a global dict

If you use tags.update inside this function or any that use it's output, it can propagate back up all the functions

[terse folio]

uhoh, updating properties of the original tags

then adding that to the matches of the deepcopy

fixed arrows

I think tags need to be reworked a little bit, it's really really complicated

but I think I fixed the dict/list issue

perhaps a class to hold the tags, and another for sorted tags.
That way you know exactly what kind of objects are being passed down the chain at all times

that would support doing this

where that specific tag is now a tuple also specifying a slice of text where the tag matches

A tag class that contains optional text_start, text_end attributes would do

to make things consistent, Im also converting the trump_tags set back to a list.

[terse folio]

I have a todolist plugin for VSCode that lets you easily jump to #TODO comments

you could make use of this too

[halcyon quarry]

You’re fixing something that isn’t broken 😛

[halcyon quarry]

I had recently updated this bit of code here, and yes it is complicated

[terse folio]

yea, while it did work, it was confusing.
Mostly just went through it to make sure everything was correct, and removed the try:catch from sort_tags
because there's no way that would error afaik

that change simplified everything, making the sorted tags output only a dict

instead of the possibility of dict|list

[halcyon quarry]

I do my best work when just thinking about the code heh

Realized the last flaw in the code I was working on yesterday

Was using the labeller function wrong

Needed to use it within the loop when iterating over the Hmessages collected to list

Because it uses the values from the hmessage to update the list of discord messages. So it was setting all messages to reflect the status of one bot msg but not the other

Probably taking a break all day, found myself in the doghouse last night

[terse folio]

Same, have some projects I want to work on!

[halcyon quarry]

Ok I see that you didn't mess with the hide_or_reveal_history() after all - good 🙂

[halcyon quarry]

Did buy some time to mess with it.

I pushed an updte to dev where, from my testing, toggle as hidden is now working perfectly.

The only thing not working perfectly atm, is when using regenerate create on an already hidden bot message

[halcyon quarry]

Ok I've perfected the regenerate create now.

Yall minds are going to be blown how well this works

The way it works now is that there will always be just one user / bot pair revealed.
Unless toggle as hidden is performed on the User Message - in which case, the last remaining revealed bot message will toggle off with it.
If the user message is then revealed, the most recent bot reply will reveal with it. Otherwise, whatever bot message was targetted will reveal with user message.

It'll make perfect sense when using it

[halcyon quarry]

Will push to main tomorrow, need to run tests make sure Reality didn’t break anything 🤗

[terse folio]

how'd that go?

[halcyon quarry]

Needed to modify the regenerate function more than I thought I would need to, but now it's all good

Realized one little caveat for per-server characters... I think one server would have to be designated the "main server" to control the settings for the bot outside guilds (DMs)

[terse folio]

what kind of settings?

oh I see

since you can only dm bots if you share a server, you could pick settings that way too

[halcyon quarry]

Right, but the same oddity could appear that has me shy away from per-channel characters which is username / avatar

[terse folio]

Most bots don't do that,
people are usually fine with bots all having the same name/pfp

Especially large ones like Dyno that also have chatbots I think.

one day it could be possible to run multiple discord instances, one per server if needed

or maybe try webhooks

[halcyon quarry]

I had a setting to not change username/pfp at one point, but removed it. I suppose when per server/channel character support is a thing, could bring that setting back

What’s the benefit of using “Optional[]”? Seen you add it to a number of positional args

[terse folio]

It just means the variable can be something or None

mostly just a thing for pylance to check everything correctly

Optional[int] is the same as int|None

You can use the pipe character as "OR"

like list|dict is the same as Union[list, dict]

^-^

[halcyon quarry]

Cool I was also wondering about Union

[terse folio]

Some classes don't have a function defined for the OR operator so Union fixes that

[halcyon quarry]

TAG_LIST = list[dict]
TAG_LIST_DICT = dict[str, TAG_LIST]
SORTED_TAGS = dict[str, Union[TAG_LIST, TAG_LIST_DICT]]

I think this should be
SORTED_TAGS = dict[TAG_LIST_DICT]

[terse folio]

idealy yes, but that's not what it's like in the code

[halcyon quarry]

Ahh\

[terse folio]

this is why it's written that way

[halcyon quarry]

I initially had the same structure for matches (had the same dictionary lists), but due to the way things are processed there was no reason to categorize them

I know the smart thing to do it probably have separate dictionaries rather than everything in tags, but alas I like limiting the number of positional arguments when possible 😛

As you can also see with the params dict

[terse folio]

having all the tags as one object in a class and passing that around is also an option.

It wouldn't change any of the behavior of the bot.
that's something I could put on my todo list for later

[halcyon quarry]

There's a number of things that would probably benefit as being a class

[terse folio]

And if you ever worry about performance with creating classes
You can specify that a dataclass has "slots".
This makes the class only allow the attributes you define it with.
So python can only allocate that specific amount of memory instead of assuming the object could grow to any size later ^^

[halcyon quarry]

I can't comprehend this 😛

(yet)

[terse folio]

you'd usually use slots in classes that are use a huge amount, for example if streaming audio (an audio packet class) could benefit from that since so many get created per second

[terse folio]

In other languages you have to do memory management,
But python generally handles that for you

It's nothing to worry about now!
and easy switch to flip if it ever needs to be done :)

[halcyon quarry]

@terse folio Since I am now enforcing that only one regenerated message pair remains visible in history, I've duplicated the mark_as_reply_for and replies and reply_to attributes - and made a mark_as_regeneration_for with regenerations and regenerated_from attributes.
For regenerate task, I'm going to fetch regenerations instead of replies
Let me know if this sounds good to you

Wondering if I should also use the "unmark reply" function while toggling regenerated messages as hidden

        if mode == 'create':
            # Update the new user message with the original discord message ID
            new_bot_message.mark_as_regeneration_for(original_user_message)
            # Hide the appropriate bot message(s)
            target_bot_reply.unmark_reply(original_user_message)
            target_bot_reply.update(hidden=True)
            target_bot_message_ids = [target_bot_reply.id] + target_bot_reply.related_ids
            await apply_labels_to_msg_list(inter, local_history, target_bot_reply, target_bot_message_ids, target_discord_msg)

[terse folio]

yup, that's something I wanted to do ^-^

[halcyon quarry]

If we ever have a junk attribute, would probably set the unused regenerated items as junk items instead of hidden

[halcyon quarry]

Spent most of the day overhauling the labelling thing - now it uses reactions

[terse folio]

Awesome!

[halcyon quarry]

There’s still a few little bugs to work out but it’s working quite well now for the most part

I need a drink

[halcyon quarry]

Ok, now regenerate replace, regenerate create, and toggle as hidden are working picture perfect

Maybe even continue but I need to test that one further

[halcyon quarry]

@terse folio what's with the noqa: XXXX comments in the start imports?

[terse folio]

it tells pylance or other python linters to ignore certain warnings.

For example with the import custom logs, I want it to ignore that I run 3 lines of code on the same line.
because its convenient to copy paste anywhere

[halcyon quarry]

ah

[terse folio]

same with #type: ignore

[terse folio]

In bot.py there was a line at the top # pyright something=false

This was to disable a certain check entirely before I knew about how to do it globally across my entire vscode

pyright is what pylance is built on

[halcyon quarry]

Alright so my new system of handling regenerate create and toggle as hidden is working perfectly including with continue

And with the reactions system

Reality when you get a chance, I'd appreciate it if you could take a look at the new regenerations and regenerated_from
I did not fudge around with what you started on the continue - is_continued is still just slapped on the HMessage.

I revised get_history_pair_from_msg_id() to allow desired attribute IDs to be passed in, for what it uses to supply the HMessages

(such as regenerated_from and regenerations -- instead of reply_to and replies)

I also revised send_long_message() to accept a ref_message. If supplied, it will reference the message or first message chunk

[halcyon quarry]

...basically just want to ensure I didn't inadvertently screw something up 🤗

[halcyon quarry]

I think I just got per-server voice channels working for TTS...

[keen palm]

Nice, nice

I wonder if I should bother with TTS

[halcyon quarry]

You won't know until you try

I recently added a command to toggle it on and off on demand

[keen palm]

In other news, the context length on my RP campaign channel is now ~26k

[halcyon quarry]

Only thing is it will not unload the model. But it doesn't occupy that much memory really.

[keen palm]

How much do they usually take? I'm basically maxing my VRAM as is

[halcyon quarry]

I'll let you know shortly

There is also a lowvram mode for alltalk_tts that doesn't add very much generation time

@terse folio Also unsure about one thing...

When I add any new parameters to database, my existing database file gets wiped clean.

In this case, I added a dictionary voice_channels for storing guild.id: channel.id

yessss I do have the per-server TTS working

Unsure why I care as I only run it on one server lol

[keen palm]

I have it running on three servers right now, but only one is really active

[terse folio]

hmm, that's odd!
I'll check that out soon

[halcyon quarry]

@keen palm with TTS

Without TTS

didn't note how much GPU it used during inference...

[keen palm]

Ohh, that's system ram. I've got plenty of that left.

[terse folio]

yea, xtts uses a 1.5GB model at fp32

I tried loading the model as half precision but latents need to also be converted.
it will just be a lot of searching and replacing to make everything work I think.

I wonder why the original devs didnt go that route

[terse folio]

what actions lead to it being wiped clean?
just changing something in python?

oh

perhaps it's because you are saving discord.channels to json

which is invalid

and because we already opened the file

it write an empty file on close with the exception

this should be channel.id

[halcyon quarry]

Ok - I had accidentally passed the guild object to it

and it did error

But I revised it

[terse folio]

working now?

[halcyon quarry]

Yes, I need to revise the label there... I am indeed only sending the ID for both

it's working well 😛
I suppose thats what caused the file to reset then

[terse folio]

you can use asset to validate things

like assert isinstance(channel, int), f"oops you passed {type(channel)} to the voice channels setting"

[terse folio]

Looks all good, at first it looked like the code to import replies was replaced 😸
I just added a space between there so it's more clear

[halcyon quarry]

thanks!

[terse folio]

since regenerations is a thing, I think I don't have to worry about handling multiple continuations anymore

because the new continuation would count as a regeneration of the continued message

then I would just have to add logic to navigate through the chain of continues+regenerations to give you a list of messages leading up to a branch

[halcyon quarry]

Sounds good to me!

[halcyon quarry]

Pushed update to Main!

[keen palm]

https://tenor.com/view/what-have-you-done-eddie-murphy-prince-akeem-coming-to-america-what-did-you-do-gif-19294582

[halcyon quarry]

Main improvements are with the context commands, and how the bot will now react to messages if they are Hidden, regenerated, continued

[halcyon quarry]

I’d like to see you break it now

[keen palm]

https://tenor.com/view/waiting-cat-filing-nail-getting-ready-bring-it-on-im-listening-gif-17677371

[keen palm]

What am I supposed to be breaking?

Edited my prompt, then clicked regenerate:

16:31:37.520 #2339 ERROR [bot.main]: An error occurred while processing "Regenerate": cannot access local variable 'original_message' where it is not associated with a value

Same thing happened without editing a prompt first. Clicking regenerate on my own message gives that error in command, and in discord says "Original message was deleted"

I thought for a second it had to do with the response being too long (spread out over multiple discord messages) but no.

[keen palm]

Also, edit history doesn't work when the message is split

[halcyon quarry]

wtf

This was not happening earlier...

[keen palm]

Is it happening to you?

[halcyon quarry]

Yeah, just updated here and am getting same error

[keen palm]

What the actual fuck

[halcyon quarry]

ugh

I see the problem...

[keen palm]

You shouldn't have said this

[halcyon quarry]

😭

[halcyon quarry]

What do you mean by this?

from a continued message?

[keen palm]

No, when a message is split into two in discord

[halcyon quarry]

ahhh

chunked

[keen palm]

Right

It only lets you edit that chunk, but that's not ideal if you want to delete a large portion of the response

[halcyon quarry]

I can improve that, here's what is happening right now:
Regardless of what chunk you target, what is in the box when you submit will be the edited history for the full message

[keen palm]

Ohhhh

[halcyon quarry]

If the message exceeds 2000 characters it clips it in discord, but the full edit is in history

[keen palm]

But only that chunk shows up in edit

Gotcha

[halcyon quarry]

So - I could try collecting the full message as the default text in the box

[keen palm]

Would be a nice convenience thing, especially since that would allow the user to edit without having to copy paste everything first

[halcyon quarry]

I fixed the original message bug

its one line of code 🙂 update and things should be working very well

Ill see about that improvement now...

[keen palm]

I hadn't noticed that before because the bot responses had been short

[halcyon quarry]

Alright I think I have the edit history thing nailed down now

I'm pretty happy about it.... I previously made a special function to handle regenerate replace which I tried to dress up as a modular function I could use for other things- but didn't think I ever would. Is perfect for this, though.

For a long edited message, if the new length is < 2000 characters, it will delete all the chunks but keep the first one and edit it.
If > 2000 it will delete them all and resend

[terse folio]

might get some messages spread out with others in between by saving that first one and deleting others/resending
no really good solution to fix that on discord.

but yea, nice!

[halcyon quarry]

If only they had an insert message function

[terse folio]

mhm!

[halcyon quarry]

Pushed to MAIN. Now available @keen palm

[keen palm]

Breaking Updating in 3...2...1....

[halcyon quarry]

Updated the /image command so that it has an option to send the prompt to the LLM

This allows using all those nice things plus the LLM - controlnet, img2img, faceswap, size, style, etc.
It will not swap characters or anything, unless you include a trigger phrase to make that happen via tags

[halcyon quarry]

In dev - will push to main tomorrow

[halcyon quarry]

So I want to add a command for more advanced prompting. Open to ideas

[keen palm]

Well what do you mean by that?

[terse folio]

ask the Ai how to improve itself!

[halcyon quarry]

Like how the image command has extra options

One thing I’ll include is a text field Start Reply With

Could also put an option like include a memory (prefix context)

dropdown to customize character…

Could select mode (chat/chatinstruct/instruct)

Field for instruction

[keen palm]

Start reply with would be quite useful, for sure

[halcyon quarry]

Here’s an idea… dropdown list of most tags, multiple select. Bot follows up to ask for more info for any tags that need it

pushed that little update to /image cmd

Alright that's it, no more distractions, those behavior updates are coming

[terse folio]

Little random thing I learned,
asyncio.wait_for() actually stops execution of the async function you're waiting for if it doesn't return withing a set amount of time 😸

like if it's a whileloop streaming data forever, you can wait for the first 10 seconds and stop it automaticly

But if you wrap the function you're waiting for in asyncio.shield() it will continue after timeout

[terse folio]

~~pushed a file with a CoroHandler
this lets you create_task_safe which wraps the async function in an error handler so that there are no silent errors.

I'm not fully sure about where it's needed in discord.py,
But a lot of my code deals with floating tasks that aren't running from the main function.
This is needed so they don't fail silently

For example, I think when discord.py makes an event, it's a background task.
such as on_message
But this wont be caught by on_command_error because it's not a command~~

Yup it does work like that

I removed the try: catch from on_message
and made an error, and it fails silently

[terse folio]

Searching around the discord.py code to find out why it silently errors, I found the on_error function.
Not sure why that was so hard to find.

Now i'm curious what exactly used to cause silent errors in my own code.

[terse folio]

You should probably set up permissions for commands

I imagine any random user could enable/disable announcement channels or change settings like that

Server owners wouldn't be happy with a malicious person enabling /main in all the chats for example

@hybridcommand...
@app_commands.checks.has_permissions(manage_channels=True)
async def ...

Should do it so only admins (or whoever has manage channels perms) can set the main/announcement/tts channels

yup that'll do it

[halcyon quarry]

Server owners do have the ability to set their own permissions for each bot command

It couldn't hurt to set a few obvious ones as admin only - thanks 🙂
I'll add instructions on how to adjust permissions in the ReadMe

[terse folio]

Oh that's interesting!
wasn't aware of this menu 😸

[halcyon quarry]

Server Settings > Intergrations > Your Bot Name >

[terse folio]

I see, is there a way to set permissions for a whole group of commands at once?

hmm

no option to set pure permissions

I think having both as an option would work out better

A config for permission values, like requires (manage_channels=True)
or you can set it yourself to require a specific role if you want

[halcyon quarry]

Yes I suppose we could classify them, and add a config for each group

So with the new error handling, I'll not use try / except in commands?

[terse folio]

you shouldnt have to use it anywhere

except places that would crash the bot

like startup

some examples using on_command_error,
and on_error to catch things

you can create your own errors from commands like this

and it will handle the .reply, the embed and all for you

using AlertUserError as an exception

[terse folio]

While I was fixing that issue with the silent async errors I started working on something else.

A decorator that can assign a custom message to errors, add timeouts and all the other kwargs of coro_handler.run_safe()

And you just execute the function as normal ^-^

In this example it only counts up to 18, which is about 2 seconds of runtime

[halcyon quarry]

Pushed to Main!

• Reality's improved error handling
• I broke Starboard feature recently. Fixed it now.

[terse folio]

Using an asyncio Lock, and Condition

You can create a function that only lets you do the thing once, and all other requests to that thing will wait until that one finishes, then they cancel.

I'm currently using this to handle model load/unloading. (in my own projects)

For example, only load the model once on the first request to load it.
But other points in the code need to wait for the model to be ready before continuing.

If you have any other creative use for this lock/condition combo tell me ^-^

[halcyon quarry]

The Semaphore kind of does the trick already… for cases when the same model is selected and it’s the same as already loaded, it doesn’t reload it

(coded that way)

[terse folio]

The thing about the semaphore though is it queues up requests, not canceling all others to do it once and only once

[halcyon quarry]

I’ll keep that in mind

[halcyon quarry]

A1111 just added my fork of sd-ar-plus to their extensions list

sd-ar-plus-plus 🤓

[keen palm]

How feasible, at all, would it be to create new tags from within discord? Instead of needing to edit the json?

[terse folio]

writing json/yaml might mess up fancy formatting of files.
But yea, should be doable.
Create a command that uses a template to build some structure, then add that to the file

[halcyon quarry]

The thought has crossed my mind... a separate file specifically for ones created via discord

[halcyon quarry]

I've just been messing around with SD the past few days - the next thing I'm working on IS the behaviors.

[keen palm]

https://tenor.com/view/its-been84years-titanic-rose-dawson-gif-5372593

[vestal python]

I got both my llama 3 70B and the sdxl server running now on the same server. I have my Ecne Bot on a simple GTX1080 talking to the sdxl server, but there's some error I haven't figured out. I just got the forgeUI sdxl server running a min ago:

juggernaut v8 file is definitely there

It is working on the server with that juggernaut :/

Should I be using Forgeui still or do you recommend something else now?

[vestal python]

It's nice at least, using RTX 3080 10GB instead of the old GTX 1070, speeds went from 1min per generation to 16 seconds for sdxl 50 step

[halcyon quarry]

@vestal python heya

hmm...

Less than a week ago, I reworked the image model handling. You may need to update

Could you share your dict_imgmodels.yaml ?

I finally got around to inviting my bot to a second server to test the new TTS changes - glad to report they are working

@vestal python I think I see the issue. The model should show the hash up in the dropdown menu... I don't see it... try pressing the Refresh button, or manually loading it first

I recommend using Forge, I still am

[vestal python]

Some progress

It works, just the backend has some errors?

[halcyon quarry]

I'm pretty sure you have one or more tags formatted incorrectly

I think I could improve the handling here, though...

I just pushed an update that will save the "Imgmodel Tags" as an empty list [] instead of saving them null, when there is an error with changing imgmodels (like you had)

As a temporary fix for your problem, add this to your internal/activesettings.yaml under the imgmodel key (same indentation level as payload)
tags: []

[vestal python]

sorry had to put my boy to bed. He takes a while now hyperkid.

[halcyon quarry]

Yep

That should stop the errors

[keen palm]

Oh god, the GPUs have started pulling like 160W each during generation. They used to use like a third of that.

I wonder if it's because the context length is >40k

[vestal python]

mine does kinda too :/

It peaks 150w each during processing

Is this conext memory in discord chatting?

[keen palm]

yes

[vestal python]

Cause that's pretty boss

[keen palm]

Yup, though I'm certainly not a fan of that power draw

[vestal python]

No way around that I guess if that's what it's holding onto every prompt

Heading off, glad the image generator is working for my discord bot again ty. 4 times faster than it was going from GTX1070 to RTX3080, but omg 320w's during generation

[keen palm]

I can potentially lower the context limit to like 32k though

[vestal python]

Yeah what model?

I'm soo used to llama 3 now being stuck at 12k, and codestral at 30k before breaking

[keen palm]

The generation I just did barely used the GPUs at all, so I'm a bit confused....

Dracones/Merged-RP-Stew-V2-34B_exl2_4.0bpw

Yeah, I dunno what the hell was happening there. Now it's back down to normal power draw.

[keen palm]

Damn, it's using way too much power again. Probably need to lower the context.

And I'm getting <1 t/s

It's probably burning through a lot of power to parse all the context, and then the actual generation portion is using less power.

[keen palm]

Oh god. Now my history seems to be causing problems. I lowered trunc length to 32k, but now every time I try to generate something I get:

09:01:40.311 #1892  ERROR [bot.__main__]: An error occurred while processing "on_message" request: 'str' object has no attribute 'text'
09:01:40.312 #1719  ERROR [bot.__main__]: An error occurred in on_message: Type is not JSON serializable: AttributeError

When I try to save the conversation:

09:04:45.263 #5437   CRIT [bot.__main__]: 'str' object has no attribute 'uuid'

[keen palm]

I reset the conversation, saved it, then manually transferred all the old history back into that file. Seems to be working now.

I also lowered the context limit down to 32k, but it still uses a lot of power in the first phase, unfortunately. I think it's just a reality I have to deal with when the context gets so long.

[halcyon quarry]

Yes, I believe the model context length setting must be the same or lower than the truncation length value

[terse folio]

if you have a broken file, it would be beneficial to save it, maybe strip out anything personal and share it so it can be debugged.
I saw this error before about the HMessage for some reason becoming a string and im not sure what causes it.

is it consistent with high trunc?

[halcyon quarry]

That error is likely coming from TGWUI

[keen palm]

Actually, I got an error when the model context length was lower than the truncation length value

[terse folio]

'str' object has no attribute 'uuid' sounds like a HMessage issue

but yea, tgwui might be outputting something unexpected

that's not being parsed right

[keen palm]

The errors started happening with that file when I lowered the trunc length. I think when it saved and snipped off the early portions of the history (because I had set limit history to true), it caused errors.

Now, though, I turned off limit history

[terse folio]

oh interesting!

trimmed history doesn't get recursively removed, so messages that still point to the removed message have no idea what to do, and just keep the str reprosentation of the HMessage (as it is saved to file)

[keen palm]

Keep in mind I wasn't trying to do anything with those trimmed messages. I was just trying to regenerate the most recent response.

And then add new responses.

[terse folio]

hmm, no that shouldn't be the case.
If an id is not found it returns None, and the reply code is ignored.

While yes, you might not be regenerating old messages, they are still needed to create the context used for generation

[keen palm]

Yes, true

[terse folio]

heh, so glad I put the line number in the logs!

Actually

are you using the main branch or dev?

numbers dont line up for me

[halcyon quarry]

dev and main are caught up

[keen palm]

I'm on main

I thiiink the latest version

But possibly not

[terse folio]

do you have a traceback to this error?

[keen palm]

Nope, sorry

[terse folio]

okay, according to the log, it happened somewhere in message_task

It's also odd to see the error is trying to be saved to json.

[keen palm]

I kind of got distracted checking out this power/context thing. 3.46 t/s at 15468 context, versus 1.11 t/s at 44633 context. The initial context parsing period is much longer at higher context, naturally. It's interesting, though, to observe how my GPUs operate. The main one is pegged at 100% and 160W for the entire parsing period, while the second one flip-flops all over the place.

(To be clear here, I'm just regenerating the same response to test this out)

[terse folio]

yea, with gptq and lower contexts I had some long encode times, I wonder what Exllama did differently that lets it start generating almost immediately (at least at relatively low context in the 1k-3ks)

[terse folio]

regenerating will probably use cached encodings if using exllama, so need to change the prompt.
But even then it will still look for the largest shared chunk of text to use as cache

[halcyon quarry]

er

[keen palm]

There wouldn't be anything cached though, because I'm restarting the bot as I change context lengths.

[halcyon quarry]

I think this is normal - context length has a huge toll on resources

[keen palm]

Yeah, I'm seeing that now

I just am weighing the pros/cons of lowering context length

I'm not generating that often, so the power draw overall is still minimal. Like a few cents a month really. I just don't like the jet engine effect it has.

[terse folio]

not sure what's going on but it was really hard to get to the context limit for me,
Messages seem to disapear on their own without triggering the truncate history code

also hmm, that also deletes from the internal history files.
Should that be the intended behavior?

maybe marking truncated messages as hidden instead of removing them so they can be saved if someone wants to recover them

[halcyon quarry]

Are you talking about the limit_history option?

[terse folio]

yea

it saves like 4 minty exchanges haha, but i sent about 20

[halcyon quarry]

The idea behind that was just to put some sort of limit on the history in RAM - it falls out of history anyway

[keen palm]

laughs in 64GB RAM

It's nice that the bot remembers everything, but 1 t/s is painful

[halcyon quarry]

I stick to 8k context 😛

[keen palm]

But I know now that it can do so much more

Last generation was 45350 context, and it's not confused at all.

It'll pull out story bits from beginning of the campaign that I've forgotten about.

I know that I'll occasionally need to have it make a summary of the story, though, since I'm about to exceed my limit

[terse folio]

text is almost nothing,
But yea, with millions of messages you'll start getting slowdowns because I haven't implemented caching for the history rendering or any of that yet.
But the processing needed for that should also be almost nothing.

Limit history should be focused on keeping things in context without cutting off character cards or messages in half

[halcyon quarry]

It's to my understanding that the user message will not be truncated

ok I think I know what you mean - just keeping history small enough that it doesn't get truncated by TGWUI / cutting things off in the middle of a message

[terse folio]

everything does, it's just that it's only triggered on bot message

if user messages were kept it would eventually fill up

but one day we could weight what messages are more important to keep and what gets removed first

like considering user inputs more important, but will still be clipped if needed

[terse folio]

can't figure this one out yet.
It's supposed to print a traceback to where the error happened, that would be a useful start.
if it happens again please share!
ofc remove any info you don't want to share like names in a path if any.

[keen palm]

Even at 45k context, the last few generations have been much faster, so I'm not sure what's going on here entirely

[terse folio]

Is it being tested the same as before? Where you were doing a generation after model load.
That would definitely be interesting if it's faster all the sudden like that!

[halcyon quarry]

First generation is supposed to take longer

[keen palm]

Okay, now that I'm home I can look into the numbers for each:

1.11 t/s @ 44633 context (regenerate replace)
0.78 t/s @ 45350 context
1.13 t/s # 45225 context (regenerate replace)
5/46 t/s @ 45233 context (regenerate replace)
0.88 t/s @ 45616 context
5.45 t/s @ 45623 context (regenerate replace)
5.23 t/s @ 45905 context
5.44 t/s @ 45912 context (regenerate replace)
5.46 t/s @ 45932 context (regenerate replace)

This was a series of messages, some of which I regenerated.

And now 5.19 t/s @ 46224. New generation, at the usual power draw.

So yes, there's definitely a bit of burn-in time in the beginning to parse through all the context.

I decided to do a generation in another channel which has much lower context:
12.24 t/s @ 1002 context

Then went back to the high context channel. This time, it took longer and drew a boatload more power:
0.80 t/s @ 46407 context

This makes sense, but is interesting to see the difference.

[halcyon quarry]

Context is very taxing 😁

[keen palm]

I did another generation after that 0.80 t/s one, and it was back to normal again.
I think I just never noticed this difference until I hit high context lengths

If I had multiple channels on different servers with high context, then it might be a bit too taxing on the system

[halcyon quarry]

Well it only processes one request at a time, it's not like your context length goes higher because there's 2 separate requests

[keen palm]

That's not what I mean

[halcyon quarry]

According to Reality, the RAM from saving multiple histories is very negligible

[keen palm]

Based on that admittedly limited test above, when switching back and forth between channels, it needs to parse through context again

[halcyon quarry]

So even if the bot is active on like 100 channels, it shouldn't get bogged down due to the number of channels

Would be similar to just one channel hammering away at it

TGWUI is just receiving a payload - the only thing being parsed on the backend is just history in RAM, should be a drop in the bucket

[keen palm]

I'll test some more, but as above, the initial generation on the high context channel took longer and drew more power, then settled down for subsequent generations. But after I did a generation in another channel and then went back to the first channel, it had to go through that process again.