ad_discordbot (Fork of Fork of xNul's bot)

[halcyon quarry]

Ahhh... yeah.... okay, maybe TGWUI does cache the history

so when it receives a new payload of history, it re-caches

[keen palm]

Yup, so lots of channel switching could bog down that process, depending on context length

[keen palm]

I imagine a similar re-caching process happens every time you edit history then generate something

[halcyon quarry]

I see that llama-cpp has a cmd arg, --cache-capacity - you could try increasing this

There may be some value that will cut the re-caching in half or more?

[keen palm]

Hmm, is there any documentation for that?

[halcyon quarry]

On the main repo page, halfway down in a dropdown list for command line args

[keen palm]

Would that even work with ExLlamav2?

[halcyon quarry]

ExLlamav2 doesn't seem to have such cache control

I use ExLlamav2 personally... you?

[keen palm]

Yup

[halcyon quarry]

Ah ok, whoops. Assumed you were using llama-cp like most people 😛

[keen palm]

I'm not a filthy peasant!

The model config I'm using

ParasiticRogue_Merged-RP-Stew-V2-34B-exl2-4.0$:
  loader: ExLlamav2_HF
  trust_remote_code: false
  no_use_fast: false
  cfg_cache: false
  no_flash_attn: false
  num_experts_per_token: 2
  cache_8bit: false
  cache_4bit: true
  autosplit: false
  gpu_split: 9.5, 11
  max_seq_len: 50000
  compress_pos_emb: 1
  alpha_value: 1

[keen palm]

Hmm, it's back to doing that high power draw and really slow generation, on every response now. Maybe I should just drop down to 32k or something.

Although I wonder if I'm actually hitting the limit with previous history + character card + tag context, and it's having to re-cache every time to trim excess.

[halcyon quarry]

Yes, forfeit some of that context you greedy bastard 😺

[keen palm]

Never! That's my context!!

It's crazy how much power LLMs require of GPUs.
Right now, I'm rendering a 3d image with my GPUs. They're maxed out on VRAM, 100% usage, and they're drawing ~95 W.
But when they're at 100% usage with LLMs, they'll draw ~160 W.

[halcyon quarry]

TGWUI just got a nice update… tensor RT support

[keen palm]

Oooh

[halcyon quarry]

I’m pretty excited about this, there’s so many cons for using it with SD (having to convert each model, not being able to use LORAs), but with LLM not many people using LORAs or swapping dozens of modela

[keen palm]

Yeah, people tend to stick to one preferred model for their AI waifus

[terse folio]

tgwui tries to find the greatest common blob of text between the last context and new context so it doesnt have to reencode a large portion of the text.

So during conversations where you only add a message onto the end, this can be generated really fast

I found this out when I was digging through the code to figure out how to do parallel processing with exllama2

Caching would have to be implemented as a dict for that, cache for each parallel item?

[halcyon quarry]

I suppose this is basically the achilles heel of the per-server histories feature

[terse folio]

Yea, but if you structure your prompt right you could make it all shared.
Like <character card> <tags?> <chat>

Maybe the character and tags portion could be big enough and shared to where that will make a difference

[halcyon quarry]

I'm not sure you understand the payload going to TGWUI 😛

In order for TGWUI to do it's thing, we need to send all the parameters for it to work expectedly... current context, current history, current user prompt, etc. If we merge any of the channel histories together before sending (which I think is what you were suggesting), it will change the output

[terse folio]

I didnt mean merging

I mean if the character used across multiple channels.
and some other parts that take a large amount of context are also the same. it will be faster

That's just up to the users though and how they want to use the bot

[halcyon quarry]

Well the context is provided as a separate parameter from the history

So it should already be defaulting to that common text as the biggest shared item

[terse folio]

Hmmm I wonder how much vram/ram those encoded contexts use?
If it converts the text to the internal embeddings or whatever that could be in the size of GB.

I think it's doing more than tokenizing.

[halcyon quarry]

(Assuming this blobbing thing you mentioned factors it as part of "the prompt")

[terse folio]

The blobbing happens after the chat code converts history+context to a prompt (just context)

it's one of the last steps before passed to the llm

[halcyon quarry]

So then there should already be some caching effect happening for the context being the same across all channels

[terse folio]

yea

[halcyon quarry]

But with huge histories, it will drag.

[terse folio]

but I'm also saying, when constructing your prompt, this could also be considered.
Lumping things that are static and don't change together, and the variable history part more at the end

would give tgwui a better chance at reusing more

[halcyon quarry]

I'm having a very weird issue... maybe you have some insight

[terse folio]

what's that?

[halcyon quarry]

Upon launching the bot - if I use one of the context commands on a bot message such as regenerate create, it sends the message "You can only "Regenerate" from messages written by yourself or from the bot."

But here's the odd part...

In trying to debug it, I added these print statements. But nothing is printing - it is still reflexively sending the same message

    async def process_cont_regen_cmds(inter:discord.Interaction, message:discord.Message, cmd:str, mode:str=None):
        print("author:", message.author)
        print("user:", inter.user)
        print("bot:", client.user)
        if not (message.author == inter.user or message.author == client.user):
            await inter.response.send_message(f'You can only "{cmd}" from messages written by yourself or from the bot.', ephemeral=True, delete_after=7)
            return

[terse folio]

clear your __pycache__ folders and restart

using the search button you can search for them all

[halcyon quarry]

in /modules/? Just delete it?

[terse folio]

I don't think it matters in the installer files

yea, the cache can be deleted

it's complied parts of python to make it run faster

sometimes python can ignore your changes and use the older cached version of the code

[halcyon quarry]

ah

Well that didn't do it

Very bizzare...

[terse folio]

yea

[halcyon quarry]

ok I think my VS window is just bugged or something

Save never turns grey after saving. I'll bet these changes arent actually saving 😛

nvm. Hum

[terse folio]

not sure about grey, but the unsaved indicator for me is a * at the start of the file name

[halcyon quarry]

yeah... no its saving. So weird

its so wierd!!!! AHHH.

wtf

[terse folio]

regenerate on old messages after restart works for me.

strange

[halcyon quarry]

Ohhhhhh duh

man I'm blind

[terse folio]

issues with this though

[halcyon quarry]

I forgot that I invited my other bot to the server lol

I'm running the command on the wrong bot XD

[terse folio]

😸

[halcyon quarry]

The bug you see, I think I just fixed that a week or so ago. Are you up to date?

[terse folio]

yes, pulled everything yesterday

[halcyon quarry]

So what did you do that it yields that error? Just react to anything?

[terse folio]

used regenerate continue which adds emojis

or create*

huh that's weird, maybe I did something that reverted the change

just clicked discard changes, should work now

[halcyon quarry]

Please, see if you accidentally reverted anything else in that time

[terse folio]

there were 6 changes, like the [:2000] on some messages, and added [] to tags output

maybe something happened when i switch from dev to main with one file uncommited (logs)

its all up to date now

nvm, turns out I never switched to main

[keen palm]

https://tenor.com/view/mrs-doubtfire-the-whole-time-frustrated-angry-gif-5349807

[terse folio]

yea, I'm generally on dev so commits will go there first

also vscode has a problem with using ad_discordbot as a remote.
maybe some login/token issue.
It's all a big mess.
But I have to push to my own fork then merge it ;-;

[halcyon quarry]

Scaring me 😱

[halcyon quarry]

Idk if anyone frequenting this channel uses ReActor, layerdiffuse or Forge Couple extensions - but I noticed those were bugged after I updated something not too long ago.
Fixed that.
I also just made it so that when a user changes the Img Model, the auto-start imgmodels task is restarted (if it was running). This way, the model won't suddenly change shortly after user changes it manually.

[halcyon quarry]

Here’s a thought… offhand idk if this is possible with the Semaphore…
Maybe if there are requests queued, and one of them is for the same channel as currently being processed, that request can be arbitrarily prioritized. But limit this line cutting to like, one instance with a cooldown or something

[terse folio]

Semaphores are probably first enter first serve kinda thing, or random because of the asyncio loop juggling tasks.

You'd have to write your own queue system I think using events.
Have functions add their event to a queue list, then wait on it.
meanwhile in the list you pop items based on whatever logic you want and notify the waiting code.
But you need something else to handle the backwards notification where the function tells the queue it is finished.

For that the whole thing could be wrapped in something. will have to think about that more

[halcyon quarry]

Oh, like the one that I had that you replaced with a semaphore

[terse folio]

I don't remember exactly what that one did

[halcyon quarry]

Well at the time it had the same result as now

But it was probably set up in a way to manipulate the queue

[terse folio]

oh yea, I think I'm remembering now

[terse folio]

Yea, this is a problem I have to work on too for one of my projects,
Prioritising sending requests to the same server to reuse cached context instead of new ones.
But it's kind of the problem but backwards

[terse folio]

If you're thinking about re-implemented that, I have an idea to make it clean and fail proof.
You can write classes with an __aenter__ and __aexit__ method which lets you use

    your code that is in queue that might error```

using aexit will catch errors or successful exists and run code to remove it from queue cleanly

similar to async with semaphore

[halcyon quarry]

Ugh. The problem with TensorRT is that it seems to only support a limited number of models

It doesn't seem like you can just convert any ol GPTQ / AWQ / etc model

(at least, not on Windows)

idk, maybe it works for finetunes... ? 🤷‍♂️

[terse folio]

not too sure what that is yet, good speed boost?

[halcyon quarry]

Here's some results shared by ooba just yesterday https://github.com/oobabooga/text-generation-webui/pull/5715

It's really not drastic improvement...

Actually, seems like the TensorRT models process prompts much faster

Then, slightly faster t/sec

[terse folio]

interesting, i'd just wait for it to mature some more and get an implementation that supports more models if that's doable

[halcyon quarry]

Yeah the Windows implementation is beta, we're always last to the party

SO! I'm pondering the implementation of the bot cancelling message(s), in order to generate a new response based on two+ messages

[terse folio]

wait 0.5-1s to see if the user starts typing again, then you can wait for them to finish and process that

older messages can be held in a queue that get added to chat all at once after previous generation finishes

[halcyon quarry]

So, basically conocate the separately generated responses together

[terse folio]

cancel and use the response generated mid-way to continue if the user sends something new?

[halcyon quarry]

hmmm

[terse folio]

you'll have to enable streaming for that which will slow down inference

[halcyon quarry]

Yeah that's pretty good idea, use the Continue function

I like that.

We would just add the generated text along with the updated user message to the history variable and use Continue function

[terse folio]

sure ^^

[halcyon quarry]

can actually simulate how this could work quite easily right now, with edit history and continue

NOT what I wanted to run into

[terse folio]

need to figure out why nonetypes are being put into history

[halcyon quarry]

Could've sworn I wrote a condition for this...

[terse folio]

or is that discord messages?

[halcyon quarry]

Ahhhhhhhhhhhhhhhhhh

hmm

Should still be able to "edit history" for direct messages, although it won't ever save locally

...right?

[terse folio]

sure, the edit history command doesn't change the saveable value

[halcyon quarry]

Ok I think I did have a check for it, but I see that is completely gone. Maybe I'm mistaken with other context commands

Modifying edit history to prevent it trying to update discord message for non-client message...

ok I think I got it...

Alright, but still have the NoneType object id error

[terse folio]

maybe print some things that use .id

see which item it is

[halcyon quarry]

Ok it is specifically for Direct Messages

and btw, the continue method was a success

I added this part about basketball and used Continue

[terse folio]

nice nice!

[halcyon quarry]

For this "respond to multiple messages", I think I will also arbitrarily remove the last sentence from the previous response, because the LLM often likes to end with a question to the user

(if there are 2+ sentences found in the response)

I'll also run some tests with that, see if anything meaningful just gets lost completely when continuing after deleting whatever the last sentence was from its initial response

[terse folio]

id make that a setting, something people can play with

[halcyon quarry]

Maaaaaaybe

Well, the whole consolidating responses will be an optional setting

And the likelihood for it to happen more or less will be dependent on settings as well

I'm outta time, will have to check out that ID thing tomorrow

[terse folio]

You asked a while ago why use typing.Union:
this was why
TypeError: unsupported operand type(s) for |: 'type' and 'str'

Some of the basic types such as str don't support OR operator ^-^

[terse folio]

Can use the on_typing event to wait for the user to finish what they are saying too

typing has a timeout of ~4 seconds
if a message sends before then it stops
or it shows that the user stopped (but there's no event for stop typing)

[halcyon quarry]

Nice! Will try and also make that a factor

I think we need a class for "What the bot is currently doing" - which we haven't really needed but will be useful for the behaviors.
For instance, a condition that will need to be checked is, is the bot currently responding to a message request in X channel

I know the database could be utilized for this, but it's so temporary and not particularly of any use or interest after the request has been handled

[terse folio]

Should it be per channel?
There could be multiple layers of what the bot is doing

[halcyon quarry]

Certainly!

[terse folio]

Organsing prompts to utilize cache better - global

channel stuff to make it feel more human

[halcyon quarry]

Well mainly, should be able to check if the bot is currently in the message_task, and for what channel it is working

So that if another request comes in while it is doing that, from the same channel, then it may perform the behavior

[terse folio]

Also have you had any luck training a lora in 4bits?
I'm thinking of making a discord based model eventually!

[halcyon quarry]

Haven't messed with LORAs at all for text generation

[terse folio]

okay, my worry is training with monkey patch might lock me into using something more unstable for inference

I'm hoping to be able to either apply to lora then convert to an exl2 model or something like that

[halcyon quarry]

Loras for text gen seem wayyyyy more complicated than Stable Diffusion

And there's the whole, the Lora only works for the model you trained it on thing

[terse folio]

Yea, that's usually the same for SD too, but most models are based off the same roots

like Sd1.5 lora cant work on sdxl

[halcyon quarry]

Well yeah, but there's like hundreds of finetune models that the Lora will work for

[terse folio]

yep

I wont settle for training with 7b haha, they reguarly get confused about who's who and switch up in most tasks.

Solar10B has been good and avoids these mistakes similar to a 13B model.
Difference there is I can fully fit the 10B model with more than 2k context!

So that will be my target to train with, a good middle ground

Will make a small test dataset as soon as possible and try getting something working.
Then get together with some friends perhaps and makde more data!

[halcyon quarry]

From what I've read, the training itself is also extremely taxing and timely

May be better off renting inference for it

[terse folio]

yea, hope google collab would work

[keen palm]

Here's an odd question: is it possible to have per-guild tags?

[halcyon quarry]

Yes, it's possible. Not a bad idea, actually

The tags system doesn't have too many conditional tags yet, so far only trigger and random, really

[keen palm]

I'm assuming it would just be an additional parameter under the tag

[halcyon quarry]

Yep

[keen palm]

from_guild: <guild_id>

[halcyon quarry]

Added to the official TO-DO list

[halcyon quarry]

This is an error in current version?

[terse folio]

No, just came across it while writing my own things and was reminded of our conversation where I didn't have a clear example of when to use Union

[halcyon quarry]

I have not been using Union or Optional in anything I've been doing lately 😛

[halcyon quarry]

Had like the most boring day so far... my C Drive was almost full, with 400+ GB of AI stuff. Needed to reinstall all programs and migrate everything to 3TB drive

[keen palm]

Is that SD models and shit?

[halcyon quarry]

yeeeeep. Got so much stuff

[terse folio]

had been in the same situation, I moved all the AI stuff to a dedicated drive

it's great for archiving stuff, but as an HDD, not the fastest with loading models

[halcyon quarry]

I said 3TB - I forgot, this monster is a 4TB

https://www.amazon.com/WD_BLACK-SN850X-Internal-Gaming-Solid/dp/B0B7CQ2CHH

Hopefully these things aren't $300 because they explode just outside of warranty

[terse folio]

Yea, ssds can fail without a warning I believe

not sure about price, never went shopping for one!

[halcyon quarry]

I do also have a backup drive with backup software

FreeFileSync - really cool program

[keen palm]

I have everything backed up to my NAS, but I probably need another redundancy

That drive is $425 here....

[halcyon quarry]

Ouch

[keen palm]

USD to CAD is a bitch

[halcyon quarry]

Finally got around to looking at this ID issue in the Continue function, in DMs

It's because a line checks for voice client using guild.id. (No guild for DMs).

So just need to tweak that...

Yep, that resolved it

Pushed to main.
In DMs, can now edit your own messages in history, and can use Continue.

[burnt patrol]

is it even possable to use llava?

[halcyon quarry]

I'll assume not

[halcyon quarry]

Working on behaviors...

Doing the easier half first, feature I plan on using myself (spontaneous messaging)

[keen palm]

What's the prompt for spontaneous messaging?

[halcyon quarry]

Here is what the new Behavior settings are going to look like... the prompts will be a list which a prompt will be randomly selected from.
The prompts can include variables.
This example includes a new variable

What will happen is that, when the bot receives a message request for a channel, it will roll random to see if it should make a spontaneous message task.
If it rolls true, it will pick a random time from the range and create a task.
If it receives a message request from that channel in the meantime, it will cancel the task.

I'm personally going to use this feature mainly as an "auto-prompter"

combined with Dynamic Prompting, should often get surprising results

[keen palm]

Ahh that's interesting. However...needs per channel settings

[halcyon quarry]

I'll think about it 😛

That's a whole nother can of worms

[keen palm]

Haha, yeah, it would be.

[halcyon quarry]

It wouldn't be difficult to have a command to set the settings for current channel, and log them to the database, and fetch them as overrides

It's just managing those settings would be weird

[keen palm]

Yeah, I can see that. You'd need different versions of that block of parameters for each channel.

[halcyon quarry]

Well they wouldn't be required - if they existed, they'd override defaults.

But then it would be kind of odd looking for them, changing them all again... managing them

For now, it's going to be dictated by the character's behavior, which overides Base Settings

[keen palm]

Per channel characters!

[halcyon quarry]

This spontaneous messaging thing is working

[keen palm]

https://tenor.com/view/frankenstein-its-alive-gif-10618052

[halcyon quarry]

Well, need to rework a few things with it after all 😛

May not get it out there today...

[halcyon quarry]

The spontaneous messaging feature should be working well and as expected.

I pushed it to dev branch

altoiddealer: "'[SYSTEM] The conversation has been inactive for 2 minutes, so you should say something.'"

[burnt patrol]

Cool!

[halcyon quarry]

yep, polishing it up some more now

[halcyon quarry]

Update

Merged the new Spontaneous Messaging behaviors to MAIN

This can send one message after some inactivity, or can periodically send unlimited number of messages, and anything inbetween

[halcyon quarry]

Added a config setting to control whether the bot reacts to messages to indicate hidden/continued/regenerated/etc

[halcyon quarry]

@terse folio any thoughts on how I could access database > Config() from the utils_shared.py module?
I want to add config options to set the hidden, regen, and continue emojis - ideally, could set those in Shared.

if I just try this in utils_shared, I get circular import error

from modules.database import Config
config = Config()

Traceback (most recent call last):
  File "D:\0_AI\text-generation-webui\ad_discordbot\bot.py", line 40, in <module>
    from modules.database import Database, ActiveSettings, Config, StarBoard, Statistics
  File "D:\0_AI\text-generation-webui\ad_discordbot\modules\database.py", line 1, in <module>
    from modules.utils_files import load_file, save_yaml_file
  File "D:\0_AI\text-generation-webui\ad_discordbot\modules\utils_files.py", line 4, in <module>
    from modules.utils_shared import shared_path
  File "D:\0_AI\text-generation-webui\ad_discordbot\modules\utils_shared.py", line 6, in <module>
    from modules.database import Config
ImportError: cannot import name 'Config' from partially initialized module 'modules.database' (most likely due to a circular import) (D:\0_AI\text-generation-webui\ad_discordbot\modules\database.py)
bot.py execution failed

[halcyon quarry]

Oh!
I solved it myself 🙂

Idk if this is some sort of race condition or not...

In bot.py if I import utils_shared before database, and if I also import Config further down in utils_shared... it works

[terse folio]

you can use unicode values in yaml I think?

Also you could set the emojis to a variable in shared.py and use that variable, it doesnt have to be part of databases.py.

If it's a default variable for something the user can change in database, you could set a variable in shared.py and use that as the default value in the dict.get() for the emoji from the database.

config.get(regen_emoji, shared.default_regen_emoji)

This would fix any circular imports if one day we want to reference shared.py in the database.py

[halcyon quarry]

This is how I have it now in utils_shared.py

from modules.database import Config
config = Config()

class SharedBotEmojis:
    hidden_emoji = config.discord.get('history_reactions', {}).get('hidden_emoji', '🙈')
    regen_emoji = config.discord.get('history_reactions', {}).get('regen_emoji', '🔃')
    continue_emoji = config.discord.get('history_reactions', {}).get('continue_emoji', '⏩')

bot_emojis = SharedBotEmojis()

The thing is I also use the bot_emojis in the utils_discord module, so I would have the same issue with accessing config there

I'm not sure exactly why it is working now... but it is, so 🤷

[halcyon quarry]

Time to take a stab at these other new behaviors... I know this is going to be a little bit trickier than the Spontaneous messaging

Reminder to self: will use Continue function to expand responses if delays sending message

[keen palm]

What other behaviours are you implementing?

[halcyon quarry]

mainly the responsiveness and how it factors with max_reply_delay

Also adding a maximum_typing_speed param that is basically going to be a shortcut to the max_tokens_second param, but converted from WPM (words per minute)

The typing speed is definitely working

I'm calculating this by a simple:
round( (maximum_typing_speed * 4) / 60)
(According to ChatGPT, the average word is 4 tokens)

The result of this... it does feel more like it types the same speed as a normal person who can type well

uncapped for comparison... Output generated in 3.16 seconds (33.18 tokens/s, 105 tokens, context 1016, seed 2092933051)

[keen palm]

So it works, but is this something people desire?

[halcyon quarry]

I imagine there could be people that want the bot to behave more like a person

And it's a setting for character behavior, so you could have grandma character that writes slow AF, and another character that is just default computer

[keen palm]

Haha, that would be pretty funny

[halcyon quarry]

@terse folio I removed all the source positional arguments, in favor of the new current_task Class I added

Wherever the source would have been defined, its now for example:
current_task.set(ictx.channel, 'on_message')

checking... if current_task.name == 'on_message':

[terse folio]

it looks like current_task is a global variable that gets accessed from everywhere.

This could have issues with multiple people doing things at the same time.
like one person starting a tag to generate images, it sets the current_task to generate image

While another is chatting, current_task on_message

you're on the right track, what you should do is create an instance of a class with all needed information and pass it down your function chain

in place of the source variable you could pass the current_task instance

[halcyon quarry]

Currently, it works because the places that it has any effect are in a semaphore - so it can only have one valid value atm

[terse folio]

if it's only used for messages

then yes, it will not make an issue

But I believe there were semaphores for other thats too

like generating images can happen in parallel to text gen as they are separate

[halcyon quarry]

For instance, if someone uses /image I'm setting the task channel/name as it is queued in the semaphore.
Ditto for on_message, speak, flows, spontaneous_message

like generating images can happen in parallel to text gen as they are separate

Nope! They can't happen in parallel right now

[terse folio]

ill check it out when I get free time

[halcyon quarry]

Mainly wanted to give a heads up since it was in a bunch of positional arguments

[keen palm]

Random idea: verbose mode in discord that also outputs the generation time, tokens/sec, tokens, context, with each response.

[halcyon quarry]

Unfortunately, there's no simple way to hook into what TGWUI prints in the cmd window

Without modifying TGWUI files

the best that could be included would be statistics calculated on the bot end

Couldn't have an accurate context size

[keen palm]

Oh bummer

[halcyon quarry]

Have a pretty good calculation figured out to skew the probability for delaying more or less depending on responsiveness

So it will make a range to choose from, and assign weights to each choice based on responsiveness

I'm going to set it so 1.0 == 0 delay always, but any other value will be selected from this method.

[halcyon quarry]

I now have the msg_length_affects_delay setting also factoring in appropriately.
If enabled:

• For shorter messages, it will skew towards shorter response delays
• For longer messages, it will skew towards longer

It generates another layer of weights for the response delays, so there's one based on message length and one based on responsiveness.
It merges the weights together before picking the delay at random.

[halcyon quarry]

Next up... simulating the bot stopping to read new input and continuing its generated response before sending

[keen palm]

I imagine that would be far more useful in practice if a decently long response time is given for the bot

[halcyon quarry]

I may be lumping the feature into the "server_mode" setting. Not sure yet.
But yea, if active it would certainly trigger more often if delays are there

server mode is another thing in the works

mainly it will automatically prefix each message with username: and use the server name for name1 parameter (or be customizable).
I remember your suggestion to use the channel description for customization

[terse folio]

Something that can be done to further humanize the bot is make it send messages per sentence while typing.

Like stream text generation into a queue and do your typing/wait logic to extract from that queue.

Things like quotes and code blocks should ignore the sentence splitting.
And maybe a setting to how many sentences on average per chunk.

I have some code that does this somewhere

[halcyon quarry]

Basically introducing another setting, like “chance_to_split_reply”

After each sentence it would roll for it

High setting makes it one of those people that spam short messages

[terse folio]

I think i had a min time limit on the split

yea

wouldnt want to do spamming

[halcyon quarry]

Well if the setting is like 1.0 then yes spam away to users desire 😛

[terse folio]

Can add a configured minimum type time (I found len(text)/7 is pretty realistic to estimate typing times)

[halcyon quarry]

I have the maximum_typing_speed behavior incorporated already, which is basically a shortcut to TGWUIs max_tokens_second param

the difference is that the value is "words per minute" which it roughly converts to tokens /sec.

Easier for the average user to understand its effect since WPM is pretty standard

And after running a few tests, the conversion is accurate enough

[halcyon quarry]

So these optional response delays are only going to affect "normal message requests" and not any commands, flows, etc

What I'm doing for on_message() is instead of directly putting the request into the semaphore, I will instead put it into an on_message() queue.
Any delay time will be determined immediately, and the requests will process in the order that those delays are met.

If there are no delays, they just go straight to the semaphore.

I think I'll also use 'responsiveness' to factor how long it takes for the bot to be 'afk'.
All this would affect is, if the bot responds to a message in the channel, whether it will quickly respond to a new message.

[halcyon quarry]

Stability just finally acknowledged their issues with SD3, and revised their license, and gave an update that they plan to fix the issues with the model

[keen palm]

Is their license no longer complete dogshit?

[halcyon quarry]

Dogshit removed

https://stability.ai/news/license-update

[halcyon quarry]

Coded the new “Message Manager” that specifically handles normal discord message requests. Upon completing a message request, if there are more queued and they’re all delayed, it will immediately process one if from the same channel as the one it just finished

This behavior will probably be replaced with Continuing/merging output or whatever, but for now it’s a pretty good answer for the bot not seemingly AFK immediately after responding

Still more work to do on it…

[terse folio]

Not sure how you implemented this, but it's better to run generation at max tps.
Because the currently tgwui doesnt support parallelism.
so if you slow down the generation time for one message it's now holding up the entire queue.

(But this is probably okay for smaller communities)

What I recommended earlier was using streaming to generate text to a queue, then read from that queue at whatever speed you want. slow or fast.

That would allow the bot to generate text in other channel while still seemingly typing to you.

[halcyon quarry]

Ahhh yes

Well good, I didn’t spend much time at all with the TPS

[halcyon quarry]

Ok so I’ll flip the behavior so it processes messages then waits. I’ll have a separate task that schedules the ‘typing’

[halcyon quarry]

The typing speed thing is going to be a bit slippery

I'm just going to leave that out for the moment.

[halcyon quarry]

Made a bunch more updates... all on dev branch still.

If responsiveness < 1.0, I now schedule a time for the bot to "go AFK" in a guild if it has not received a message.
The time it takes to go AFK is based on the responsiveness. If responsive, it will be active for awhile.

If not AFK, it ignores the "responsiveness" weights on delay.
If msg_size_affects_delay is True, that will still be factored

[halcyon quarry]

So I think what I need to do is take the big message_task() function, and make it a Class… and queue instances of that class

The way I’ve coded that is not flexible enough to work with atm. To handle delays after LLM gen, would need to background it to wait after LLM gen (process something else), then complete img gen + send responses.

[terse folio]

Yes, it would be a background task.
In your class create a queue.
Dispatch a function to generate text into that queue.

In the main function of your class, iterate on waiting for items to come out of queue and use those to send to discord

you can use wait_for to wait on generating text from queue.
If you dont get a result within set timeout you can inform the user something went wrong

On the background task side, make sure to pass some sort of stop signal into the queue so your main function knows generation is complete

like
STOP = object()
queue.put(STOP)

if item is STOP:
break

[halcyon quarry]

I'll certainly try to get to that - but I think I need to rework the code a bit to make these kinds of pivots more manageable.

This is what I plan on doing...
https://chatgpt.com/share/7c099538-ec19-4cae-9635-e7869d4329b0

[halcyon quarry]

This is going to be a massive amount of code changes...

[halcyon quarry]

Still have quite a ways to go, many things left unresolved
https://github.com/altoiddealer/ad_discordbot/tree/class-everything

[terse folio]

ohboy

[halcyon quarry]

yeah oboy is right 😛

new branch in case I can't find my way out of this

[terse folio]

at some point should consider starting from the beginning,
building up the bot using cogs, use api, custom extension to hook into internals of tgwui if needed.
Draw out a plan

And load in the pieces from before

if i had more time, i'd work on that, slowly reimplmenting features into a new bot

[halcyon quarry]

Cogs wouldn't be too difficult to implement

[terse folio]

I've recently revisited making discord bots, after getting typehints into my workflow

using that if TYPING import thing to import cogs avoiding circular imports is amazing!

discord.py lets you get other cogs by name

the database thing could be a cog, message handling a cog... etc

and they reference eachother where needed like db: DBCog = bot.get_cog('db')

with things split up like that, they then can be easily reloaded

only thing to look out for is how you store data in cogs.
Because when you reload the cog it will re-init the class.

So if you're storing data there, it will be wiped on reload.
To avoid this you can write data to the bot object which is global.

the drawback of doing that is it could be harder to make internal changes to how information is stored.

reloading the cog could lead to the bot continuing to use the old version of a database.
a full restart would be needed for that kind of development

[halcyon quarry]

Before the BotSettings() class I was storing all the settings in the bot variable 🙂

[terse folio]

I do this for loading models,

    bot.model = load_model()...```

[terse folio]

yes, close

I mean if the bot_settings was defined in a cog so that you had the ability to reload it.
You can do something like bot.botsettings = BotSettings()
so that reloading the cog wont reset the settings to default

[halcyon quarry]

Every variable regarding hmessages is now 'user_hmessage' or 'user_hmsg', 'hmessages', etc.
Wherever 'message' (discord Message) was used is now 'discord_message', or 'discord_msg', 'target_discord_msg', etc

I'm using message for the new Message() class

[terse folio]

what about meta_message?

as a variable name idea

[halcyon quarry]

made some more progress but still a long ways off

Nah, makes me think of facebook 😛

[terse folio]

hahaha okay

[halcyon quarry]

ad_message 🤓

I'm working on Tags() class too

now has self.matches, self.unmatched, self.trump_params

[terse folio]

that's instanciated each time tags are processed?

[halcyon quarry]

When on_message happens, it will create a Message() and queue it to the MessageManager()
Which will then run a MessageTask().
Tags() will be instanciated at start of MessageTask()
Will probably also need to instanciate Tags() for other things

[terse folio]

sounds good, a good way to keep data organized for moving things around

[halcyon quarry]

I'm always happy about being able to add new features easier,,, which will be possible after this not easy task

It is very satisfying to wipe out so many positional arguments

[terse folio]

yup!

[halcyon quarry]

With this rework, should then be able to finish up the new behaviors very nicely

I'm outtie, have a nice night 😄

[terse folio]

Cya, goodnight

[halcyon quarry]

I'm in pretty deep, I think the new structure will pay off big though

[halcyon quarry]

I've discovered Protocol
I'm defining the main variables commonly shared among tasks under TaskAttributes(Protocol)

For now, I'm lumping damn near every operational function under a TaskProcessing Class where each positional arg begins with self:TaskAttributes.

Then in a Tasks class, there will only be functions which encompass primary tasks

TaskProcessing could then be divided into smaller groups at some point but it would just be for better organization

[terse folio]

yea, remember you can subclass things too
like changing the behavior of a function in the class if it's part of subclass A or B

[halcyon quarry]

Is this an example of what you mean? Essentially duplicating the function, but with different handling?
https://chatgpt.com/share/1f1660ce-fdfc-4db6-b27e-c9273b73deef

This example doesn't really make it seem appealing...

[vestal python]

I've been wondering. Is there a way to stop these additional notes and details from popping up, or is this more in line of the type of model to use?

[halcyon quarry]

That's the LLM / prompting

If there's a common occurance you could add stopping strings like
"Note:\n"

[vestal python]

I'll keep an eye on how it does it. Someone else is interested in getting to understand creating a discord bot. I was interested in seeing how Hermes-2-Theta-Llama-3-8B-Q5_K_M.gguf works with RP and coding

If all is good he can just copy my notes and make his own character yaml

resetting conversation and greeting seems to fix it. It's probablky one of hose snowball effects. Once it starts, it's just stuc in its head until wiped

oh wow nope it happened again. Same thing "Note:\n"

Let me see how to add that to stopping

[halcyon quarry]

You could also try editing history instead of resetting

[vestal python]

Thanks I think adding it in the character yaml fixed it

I'll add in any other key phrases he likes to use that cause immersion breaking

[vestal python]

Ok so RP it's good enough the past 45 minutes with those stop strings in place, and asking it to write me some different python scripts comes out great. I haven't had the discord bot be this coherent feeling since it was set with Mixtral on my larger server.

[halcyon quarry]

Starting to get somewhere with this overhaul... probably another 2 days or so

[halcyon quarry]

Then that’s when I learn ChatGPT hallucinated the whole strategy and it doesn’t work 😛

[terse folio]

ohno!

[halcyon quarry]

Finally got around to checking out Live Portrait and holy shit

It's so so easy to animate any face quite convincingly, and fast

[keen palm]

I should finish watching that video explanation of it

[halcyon quarry]

I followed this brief tutorial by Sebastian Kamph and had it running in like 15 minutes, including installing ComfyUI and the node manager, etc
https://www.youtube.com/watch?v=8-IcDDmiUMM

Fire up the example workflow > add an image and a driving video and hit Queue Prompt, and voila

[keen palm]

Yes, that's the video I was referring to

I started but didn't finish it

[halcyon quarry]

super easy

[keen palm]

https://tenor.com/view/supereasy-barely-an-inconvenience-screenrant-pitch-meeting-ryan-george-gif-13356160

[halcyon quarry]

after a little more chatting with ChatGPT, seems like I really don't need "Protocol" to achieve what I'm aiming for with this structure.
In fact Protocol was screwing up the typehinting, especially for shared methods.

I think this has also given me a better understanding of inheritence

ChatGPT states that when a class inherits from another class, if they share any attribute names, those become one.
I'm beginning to understand why some of the classes have only names with typehints but no values

[terse folio]

Mhm!

I did some interesting stuff with typehints and inheritance recently

I have this base class that acts as a table for SQL.
It has a function to import data from a discord object.
Such as a message, member, channel...

Each subclass inherits the ability to save that data to it's own table.
But the thing was typehints were screwed up because I can't specify things that the subclasses will use in the parent.

For example:

class Base:
    def load(self, object):
        self.inner_load(object)

class Message(Base):
    def inner_load(self, object: discord.Message):
        ...

In this example, I would want Message.load() to typehint that it requires a discord.Message type.
But that function is defined in the parent

The way you can solve this variable typehint, is using typevars

T = TypeVar('T')
class Base(Generic[T]):
    def load(self, object: T):
        self.inner_load(object)

class Message(Base[discord.Message]):
    def inner_load(self, object: discord.Message):
        ...

In this example on the otherhand, we define typevar T,
that lets us change typehints later on for subclasses whereever T is used

[terse folio]

never heard of protocols before, but after a short read that looks really interesting.
it's like typehint matching based on the methods/attributes a class has without having to be that exact class

[halcyon quarry]

I had that issue setting up this new structure, then ChatGPT suggested Protocol which resolved the attribute hinting but it makes it so shared methods in the parent class are no longer hinted

[terse folio]

I need to read back about your plan, but I think protocols are not nessecary

[halcyon quarry]

After some more questions, prompting etc, it suggested to just define ClassAttributes() as just attribute names. Inherit those for ClassProcessing(ClassAttributes). Then define all the attributes in ClassAttributes as self attributes for Tasks(ClassProcessing)

All hints lit ip everywhere which is promising

[terse folio]

that seems like a lot of steps

is this all to create one class? or is it for multiple subclasses?

[halcyon quarry]

I’ve split out a few things to separate classes: Tags, Params, Flows

[terse folio]

okay

[halcyon quarry]

SD

TTS

VoiceClients

[terse folio]

I would create a baseclass with shared methods,
then for each SD, TTS, Params, Flows...
they can have their own attributes/extra methods defined in the subclass

[halcyon quarry]

The structure I set up is to handle the various tasks that are currently queued to the task Summers for summer for

This structure will eliminate almost all of the positional arguments for most of the tasks that the bot performs in terms of text generation and image generation and model swapping

[terse folio]

I see

[halcyon quarry]

Almost all of the attributes for a task can be pre-defined with quarks

kwargs

[terse folio]

the way I would aproach this is creating a wrapper for TGWUI/others
like await tg.load_model(...)
that would create a queue to load a specific model and handle all that stuff

[halcyon quarry]

If you check out the class everything branch and just Ctrl+f down to “class Tasks” it will make sense

[terse folio]

will take a look in a bit ^^

[halcyon quarry]

It’ll make even more sense when I define the TasksQueue

To replace the semaphore

Messages will be processed by MessageManager first before going to the TasksQueue

This is either genius or complete dumb-assery

When I get this all actually working again, I had an idea to use my fix_dict() function for the user settings files - to apply any missing default variables from the settings templates (but Not saving them - just upating bot variables)

The I don’t need the huge .get chains for new features

[terse folio]

The queues were a good idea, but I was worried about edge case errors and didn't fully understand all the benifits of what you had.

Yes the semaphore has some drawbacks like not being able to reorder the queue.

We can recreate the queue using an async context manager which will make sure to properly exit the queue in the case it errors, that way you don't have to worry about forgetting a try: except!

the async with X() as Y:
syntax catches errors and runs a exit() function in X that deconstructs anything needed

[halcyon quarry]

Well the queue manager main method is going to call those main tasks after getting tasks, so as I had before, should be able to just wrap that with try except, which will trip if anything errors in a task… right?

We’ll cross that bridge when we get there 😜

The TasksQueue will be easy to set up I think

[halcyon quarry]

Btw I started some new thing with Embeds management but had an even better idea

Almost everywhere I check the condition for the existing embed, etc, I’ll skip that and just call something like embeds.update() with the name string, and it will manage the returned discord messages automatically

[terse folio]

embeds can be created on the fly

[halcyon quarry]

Yep

But I like editing them as much as possible

So the recreated ones don’t move up in discords chat history

[terse folio]

oh that's what you mean

you can edit an embed in a message too

yea

but you need to resend the whole embed iirc

[halcyon quarry]

I’ll have a method like “edit_or_send()”

[terse folio]

mhmm, can pass a message channel object + Optional[message id]

if message id is None, send
if set, try fetching and editing,

[halcyon quarry]

Yep! That’s the idea 😛

I have some wrongly defined embeds atm

[valid crypt]

how can i disable commands for dms?

[halcyon quarry]

Heya

They almost all are already, by default.

The ones which are configurable are in the config.py
Check the settings_templates directory to see the latest version of config.py, to ensure you have those settings

In the very near future, I plan on adding a notification on startup if new user settings are missing from the user config files

@terse folio Here's how the new embeds methods are working 😄

        except Exception as e:
            print(traceback.format_exc())
            log.error(f'An error occurred while processing "{current_task.name}" request: {e}')
            await self.embeds.edit_or_send('img_gen', f'An error occurred while processing "{current_task.name}" request', e)
            await self.embeds.delete('change')

Previously:

        except Exception as e:
            print(traceback.format_exc())
            log.error(f'An error occurred while processing "{current_task.name}" request: {e}')
            if self.embeds.enabled('img_gen'):
                title = f'An error occurred while processing "{current_task.name}" request'
                if self.embeds.img_gen:
                    await self.embeds.img_gen.edit(embed = self.embeds.img_gen, title=title, description=e)
                else:
                    await self.channel.send(embed = self.embeds.update('img_gen', title, e))
            if self.embeds.change:
                await self.embeds.change.delete()

[halcyon quarry]

Enjoying the bot? Any feedback? 😄

Oh and welcome to the server

[valid crypt]

thx, and the bot is really good, but i dont have config.py

[halcyon quarry]

When you first run the bot, it should automatically copy into the main directory.
OR, you can manually copy/paste it

from the settings_templates directory

Oh derp

I keep calling it config.py even though it was changed to config.yaml like 5 months ago 😄

[valid crypt]

maybe it is not .py?

[halcyon quarry]

I've been working on this thing for awhile, and for awhile it was config.py

config.yaml 😄

[valid crypt]

also im having problem with long character card

when the text-gen-webui has no problem, but the bot responds with " "

[halcyon quarry]

Your settings may be slightly different... especially your context length / truncation

double check if you need to include any CMD Flags for your model loader

Although, if you save the model config via TGWUI window, those should be honored by the bot

[valid crypt]

i left that on default

[halcyon quarry]

Try loading with the default but press save config. For some reason, defaults via the UI and defaults on the back end may be different

Beyond that... the UI default settings (not the model settings, the main settings) may also be a little different than the defaults in the bot

You can check those in dict_base_settings.yaml or you can override any of those settings in your character (see M1nty example character)

...Orrrrr there could be a bug in the code 😄

You're welcome to share the character and I could check it out, if all else fails

[valid crypt]

Failed to build the chat prompt. The input is too long for the available context length.

                     Truncation length: 2048
                     max_new_tokens: 512 (is it too high?)
                     Available context length: 1536

helpful?

ERROR [bot.main]: An error occurred in llm_gen():

[halcyon quarry]

You could try increasing your truncation to 4096

Most models are trained to handle at least 4096 these days

[valid crypt]

this one right?

changed but didnt change

[halcyon quarry]

Thats for the model - there's another setting in the chat parameters tab

for the Truncation length

For most cases, you should make them identical

I'd recommend setting both to 4096, or if you have good vram and performance, set both to 8196

@Speedy would laugh at these numbers 😄

[valid crypt]

if it is what i think...

is that 10000 by default?

[halcyon quarry]

hmm

clearly it shows 2048 in your cmd window

[halcyon quarry]

Is that with the bot, or with TGWUI?

[valid crypt]

bot

[halcyon quarry]

Oh duh definitely the bot LLM Model

[valid crypt]

tgwui has no problem with the charater and they use the same model

[halcyon quarry]

The bot does not use the TGWUI API

[valid crypt]

?

[halcyon quarry]

The bot imports a number of TGWUI functions and just runs them directly

The payload it sends has nothing to do with what is in the UI

the settings are managed independently

[valid crypt]

that was answer of one of the questions in my mind

[halcyon quarry]

Mainly, I still do this because it allows very flexible TTS management

I love being able to change TTS voices and such instantly

[valid crypt]

how do i change to 4096?

[halcyon quarry]

Like I said, either in dict_base_settings.yaml (its near the bottom), or add a custom state to your character (see M1nty example character)

After changing the value, use /change_character and pick your character again - the settings will update

You could also test using a neat little feature in the bot... advanced feature

[valid crypt]

i dont even know how to code, i just use my feeling...

[halcyon quarry]

Just include this anywhere in your prompt 😛 would update for that prompt only
[[state:{truncation_length:4096}]]

It's daunting without the good ol' webUI that everything else has

but once you tweak a setting or two, you'll get it

it looks like code but it's about the same as a text box

[valid crypt]

🤯

[halcyon quarry]

The code is all on the back end 😄

You can trigger all kinds of things on demand with the bot via the [[ settings_key: settings_value ]] syntax

There's a number of interesting new Character behaviors coming soon, but I got a bit sidetracked with overhauling like, the whole code...

[valid crypt]

it works but doesnt work, bot replies with just emojis

[halcyon quarry]

try /reset_conversation

it may have logged a few empty responses and now thinks that's the new normal 😛

You may need to customize more settings...
If you really want a true 1:1 experience, just have dict_base_settings.yaml on one side, and your UI window on the other.
Check the values that are visible in the UI window

[valid crypt]

can i modify in real time?

[halcyon quarry]

You can but the settings won't go into effect unless you:

• reload the bot, OR
• use /character

I gotta run for now, but if you have further trouble you could DM me a copy of your character and I could see if I can reproduce the issue

[valid crypt]

pretty much fixed

[halcyon quarry]

Ahh nice

[valid crypt]

can i turn on the api somewhere

[halcyon quarry]

Nah 😛

[valid crypt]

also whats the difference except the tts part

[halcyon quarry]

internally, there's a number of code changes

the API is super picky about parameters - but the method I'm using you can send everything and it ignores everything except what it needs for the current model loader

[terse folio]

Hopefully some day,
I might make the fork of a fork of a fork of a fork bot XD
that supports api and has it's own custom extension to access anything needed

[halcyon quarry]

Make the custom extension and I'll update to accomodate 😄

gotta step away...

I don’t remember if I tried monkeypatching load_extensions() while using the bot via API , but I doubt it would work…

@valid crypt btw the regenerate and continue functions are very powerful in the bot - can be used anywhere in chat history

Along with edit history, toggle as hidden (right click on a bot message, Apps > )

[valid crypt]

dont know code, 5am, dont understand

[halcyon quarry]

Haha

No code required. If on desktop, you can right click on a message in the discord channel and choose something like “Apps > Regenerate Create”

On mobile, long press the message

If the message is found in the bot’s internal history, it can use the feature

[valid crypt]

did not understand load_extensions(), got confused with "using the bot via API", when you said no API support?

[halcyon quarry]

The reason my bot is able to change extension settings on-the-fly (including TTS voices) is because I replace TGWUI's load_extensions() function which unlocks the sealed gate

[halcyon quarry]

I don't think I can replace the function while using the API

[valid crypt]

i havent tried extensions yet, but now i know what i wanted, how can i make the bot talk in a specific server and channel

[halcyon quarry]

You need to have discord developer mode enabled for your account in order to easily get IDs via right-clicking.

1. Create a voice channel in your server(s)
2. Use the /set_server_voice_channel command, and paste the ID (you can do this for each server, if you have multiple)
3. Use M1nty character as an example for adding TTS settings to your character
4. Add the tts client (best IMO is alltalk_tts) in config.yaml and adjust anything else you want to adjust

You'll need to reload the bot for it to load the TTS extension.
If you include specific TTS settings in your characters, they'll automatically go into effect as they are loaded

[valid crypt]

i meant write bcs i dont want others add my bot to another server and use it

[valid crypt]

that is helpful for me to add tts tho

[halcyon quarry]

Ah yes, I've been meaning to add this information to the Wiki... so much to do 😛

If you go to your main server settings > Integrations > Your bot

You can manage permissions for each individual command

[valid crypt]

the thing is, others can dm or add the bot to another server and use it and use commands
so i want my bot only work in my server

[halcyon quarry]

If you change the command permissions to (Only your role) they can't do that

idk if you tested out commands via DM yet, but like I said they are almost all disabled by default

You can run through all of them, you'll see it error for 95%

[valid crypt]

yeah but i still can add it to another server and use commands so...

[halcyon quarry]

As in, you want to prevent someone who has access to your account, to be able to manage the bot?

[valid crypt]

wait, only i can add the bot?

[halcyon quarry]

Whoever has admin access to your server can invite bots to it

People that can add channels, delete the server entirely, etc.
By default it is just the user acct who created the server

[valid crypt]

so to bring the bot to another server i must be an admin of the server where the bot stays at

[halcyon quarry]

You could share the invite code with another admin - which you generate via your Developer Portal

If you want them to invite the bot to their server.

I see that this is actually a valid scenario to have local permission settings

(Let someone invite the bot to their server, while retaining admin rights over the bot's behavior there)

No one can simply invite it though, you need to generate the URL and share it

[valid crypt]

problem solved

ill try the tts

wait the dm, i dont want the bot respond in dms 😓

[halcyon quarry]

In config.yaml:

  direct_messages:
    allow_chatting: true                  # Allows the bot to interact in direct messages (DMs). Most commands are disabled by default.
    allowed_commands: ['image', 'speak']  # Exclude commands by removing them from list. [] = all disabled

allow_chatting: false
allowed_commands: []

[valid crypt]

last time i tried it disabled the bot

tried again, bot does nothing

literally nothing

[terse folio]

I changed /set voice channel to just bring up a channel picker ^-^

[halcyon quarry]

...did you then proceed to try DM-ing the bot?

[halcyon quarry]

DERP - why didn't I think of this before 😱

[valid crypt]

bro, that option disables the bot, sure it does not respond in dms, but i can't use it either

[terse folio]

Discord.py comes with many converters.
You usually can plug in most discord objects into a command arg's typehint for it to use.

For example here I did channel: discord.VoiceChannel

[halcyon quarry]

You may be testing in a channel that is not "a main channel"

Try @mentioning the bot, or using /main in the channel

[valid crypt]

didnt work, no error, just nothing

[halcyon quarry]

Ill try reproducing...

Alrighty then, you are correct 😛
I'm going to fix this right now

[valid crypt]

^_^

[halcyon quarry]

hehe... pretty big mistake in this code

[keen palm]

Whoopsy!

[halcyon quarry]

        if not config.get('discord', {}).get('direct_messages', {}).get('allow_chatting', True):
            return False

fixed:

        if is_direct_message(message) and not config.get('discord', {}).get('direct_messages', {}).get('allow_chatting', True):
            return False

@valid crypt Fixed it - run the updater and youll be good to go

[valid crypt]

🫡

[halcyon quarry]

It pains me so bad when a bug this shitty is just out there for like 1 month

But I won't get anything done if I work slower 😛

[valid crypt]

help

i opened a file called atsetup.bat and feels like it does everything for me and...

[halcyon quarry]

You may not have followed their install instructions to a "T"

[halcyon quarry]

https://github.com/erew123/alltalk_tts/#-quick-setup-text-generation-webui--standalone-installation

[gilded granite]

hey altoiddealer how do I retrieve name of loaded character in my extension?

[halcyon quarry]

Be sure to activate the TGWUI venv first, then cd to extensions/alltalk_tts then run it

[halcyon quarry]

I don't quite understand the question...

Your extension could read the /ad_discordbot/internal/database.yaml and get the value for last_character

It's also saved to activesettings.yaml under the key llmcontext: { 'name': '' } (although this is the "name" value and not the filename string)

[gilded granite]

I'm writting extension for TGW, I need to retrieve loaded/selected character. I need that on extension init, because I want to load certain data before from file specifically for character, and then for detecting character change to remake/reinit db

[halcyon quarry]

hmm

IF TGWUI does save a variable that tracks what the name of the current character is or whatever (not in the payload) - I could probably set that variable even though I don't need it in the bot

You would probably know of such a variable if you are getting it for normal TGWUI usage (via the UI)

[gilded granite]

I can get it from state, but I don'ŧ know how to retrieve state outside of outlined functions

[gilded granite]

Yeah but I need it in script to load or create db for specific character

@halcyon quarry

[halcyon quarry]

The bot doesn't use the API

The name is certainly in the payload, but the payload is used directly in chatbot_wrapper() function-

[valid crypt]

https://tenor.com/view/goat-gif-22057358

[gilded granite]

modules.shared.settings.get("character")

def get_character_name():
return modules.shared.settings.get('character')

[halcyon quarry]

Ahh, I should be able to set that

[halcyon quarry]

I just realized I lied - forgot to push it to the Main branch 😄

Going to sneak in this character thing real quick before I push to main

[valid crypt]

._ .

[halcyon quarry]

confirming I can set this via the bot, and will

[gilded granite]

are you a bot or human?

[halcyon quarry]

beep boop

[gilded granite]

no way

btw don't forget to:
import modules.shared

[halcyon quarry]

Have a few more changes merging to main that I forgot about

[halcyon quarry]

I already do 🤓

[gilded granite]

I'm fixing up the superboogav2, to have persistant chromaddb and chromadb per character

[halcyon quarry]

Pushed the change to Main, along with fixing the option for DMs @valid crypt

[valid crypt]

can i use any tts?

[halcyon quarry]

You can try -

youll need to see what params the extension expects, and ensure those are included in the character's extension settings

[valid crypt]

😱

[halcyon quarry]

Let me know how it works out

Really starting to close the gap on this huge code overhaul

[valid crypt]

have you tried edge_tts

[halcyon quarry]

Nope but I’ll check it out

[valid crypt]

ill explain why i think it is good, edge tts makes a really realistic voice and if what in the img is what im thinking, rvc refers to https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI which imo is the best on converting voices

but edge is not local...

[halcyon quarry]

I won’t try it 😄

[valid crypt]

wait how can i forget

what about stt

does it have it?

[halcyon quarry]

Maybe soon

Heard the code for that was just revamped - probably a good time to look into it

Will add it to the official To-Do™️ list

[gilded granite]

Ok it's pretty bad, it only retrieves the setting or the character thats gonna be loaded upon start. It doesnt change when user switches characters.

[halcyon quarry]

In your extension, correct?

This bot assigns the variable every time a character change occurs

[gilded granite]

How?

[gilded granite]

Yes

[gilded granite]

I've beeb looking for like 5h through the script and I havent found solution 💀💀💀

[halcyon quarry]

from modules import shared

😛

[terse folio]

According to their other forum post I think they're looking for the character in gradio and not necessarily what's in shared.py

Thing is, gradio just makes web requests to the webui.
It is likely just passing the character name in the payload used to generate the next response and not actually storing a variable anywhere.

[halcyon quarry]

then in my character loader shared.settings['character'] = char_name

[terse folio]

So, I would say, have your extension hook into the generation pipeline/whatever its called
And check the payload for the name2 key, using shared.py for default if name2 is not specified.

[gilded granite]

I have tried both

[gilded granite]

This works but it doesnt update on my end

I have:
def get_character_name()
return modules.shared.settings.get('character')

[terse folio]

I'm not really sure why TGWUI has a shared state for things like character because gradio uses threading.
So what happens if 2 people request generation using 2 different characters?
is there a race condition?
Wonder how they handle that

[gilded granite]

I dunno an doing persistance for superboogav2

[halcyon quarry]

this code update is never ending

[gilded granite]

I'm just making code for friend

could I copy your function

[halcyon quarry]

I'm just complaining out loud about a deep code hole I dug myself into

Copy anything you want!

Well, the History manager you'd have to ask Reality - the genius behind that module 👏

[gilded granite]

I just wanna see what you have different

[halcyon quarry]

The way this bot interacts with TGWUI is probably not ideal

[gilded granite]

cause I used the same syntax and:
[n3ko@n3kobox ~]$ /home/n3ko/text-generation-webui/start_linux.sh
05:01:06-332058 INFO Starting Text generation web UI
05:01:06-333857 INFO Loading settings from "settings.yaml"
05:01:06-335454 INFO Loading the extension "superboogav2"
Character: Silica is detected
Collection 'ChromaCollector_Silica' created or loaded successfully.
05:01:07-207349 DEBUG Loading hyperparameters...

Running on local URL: http://127.0.0.1:7860

Character: Silica is detected
Silica
Max existing id:13
05:02:29-134464 INFO Adding 10 new embeddings.
05:02:29-213251 INFO Data successfully saved to JSON file.

[halcyon quarry]

The main TGWUI stuff is near the beginning of the main script

[gilded granite]

Is there way to ask them question? Directly?

[halcyon quarry]

who? Ooba?

[gilded granite]

Prob. who else could know the TGW best?

[halcyon quarry]

You could just toss a question out there in #api-dev-help or #other-dev

maybe #extension-dev

[gilded granite]

yeah that's dead

[halcyon quarry]

but when it comes to this coding stuff it's pretty hit or miss

better off copy/pasting some concise code and describing the situation to ChatGPT

[gilded granite]

Yeah chatgpt sucked for this... tbh, its nice tools but way too dumb...

[halcyon quarry]

works great for me when I'm stuck

Just need to make sure to have a clean prompt without extra stuff to cloud it up.
And maybe generalize a few things to make the question simpler.

Or start with some lowball questions and build up to the more difficult one, while it reasons its way along

[gilded granite]

Yeah basic python stuff yeah but the helish chromadb library, with poor documantation... ChatGPT knew as much as I did....

But it works now! except getting correct current name.

btw sorry I bothered you I thought you were some bot running on knowledge of TGW.. 💀

[halcyon quarry]

Lmao

I know a few things but there’s a few people in the server that know wayyyy more about TGWUI code than I do

[halcyon quarry]

I'm too burnt out to try running the new class-everything branch, it is destined to have bugs with how much I changed... but should be pretty damn near close to wrapped up

This should be much more flexible for new features

[halcyon quarry]

message_task is working with the new code structure… just need to debug everything else 🤗

[valid crypt]

what is that?

[halcyon quarry]

I’ve put maybe 20-30 hours or so into overhauling most of the bot code. I’m just excited things are working. message task is just the code block that processes typical message received from discord

[halcyon quarry]

@Reality is going to have mixed feelings about the changes for sure, mainly that since 75%+ of positional arguments are now self attributes the functions have no hinting. I’ll need to go through and add little comment blocks to describe what each function does

[terse folio]

If you use data classes you can speed up your class making process (The code chatgpt gave you was showing how to create classes the normal way)

[halcyon quarry]

When I have everything functioning I welcome you to improve it (I’m hoping you will 😆)

[halcyon quarry]

I’m starting to recognize the places that Optional[] is suited

[halcyon quarry]

Pretty much everything is working with the new code except something weird going on with Tag matching

[keen palm]

Weird how?

[halcyon quarry]

Been having fun trying to pinpoint the issue… the issue is with text insertions. The matching is good, trumping is good, it’s adding multiple instances where it was only adding once in the past

[keen palm]

So same tag triggering multiple times?

[halcyon quarry]

Yeah… or something. I’m blaming Reality 😆 I haven’t tested if the bug is in Main

[halcyon quarry]

Actually, I think I was mistaken

Maybe I just wasn't managing my trump tags well enough 😛

Pretty cool pic

[keen palm]

https://tenor.com/view/high-home-alone-gif-13019295

[halcyon quarry]

Anyone subscribed to all my commits are surely shaking their heads

[halcyon quarry]

semaphore fully replaced by TaskManager() queue

[terse folio]

task thingy!

[halcyon quarry]

Everything is working.
Now, finally enhancing the new message behaviors....

[keen palm]

https://tenor.com/view/enhance-photoshop-gif-4844535

[keen palm]

So basically a complete rewrite

[halcyon quarry]

Pretty much lol

[halcyon quarry]

Automatic settings fixing with warning 👀

Before you say it!

I'm updating it...

[terse folio]

using repr will remove the quotes around "True" but keep quotes for strings
I know that's meant to be True because True isn't valid for json

[halcyon quarry]

to include the key

[terse folio]

confused what's going on here, but looks better 👍

ah, must be a json string, or something tgwui will decode

[halcyon quarry]

Added some 'ignored' keys to fix_dict() (ones intentionally omitted from user settings files)

[halcyon quarry]

Making more progress... here's the new logic to handle the new maximum_typing_speed behavior

It goes to a priority queue (self sorts by time_to_run) which will send the message.
Will be adding a check to see if it should merge messages or something.
Brain hurts trying to sort this out 😛 But it's certainly coming along

[halcyon quarry]

@terse folio Maybe you can help clarify something...

I want the warned_once to always initialize as an empty dict, so that as things are warned they do not warn again during runtime.
However, I want them to be reset on next launch.

What is very odd to me is that if I simply put self.warned_once = {} it is still retaining old values.
But if I call this little function, it actually resets it.

[terse folio]

you're probably resusing the same instance of the class

add a temporary id or something to the class, and paying print that ID when you use it so you can track when new ones are created/destroyed

oh, what do you mean retains?

between multiple commands?

between bot restart?

[halcyon quarry]

        self.voice_channels = data.pop('voice_channels', {})
        self.warned_once = {}
      #  self.reset_was_warned()

The bot will initialize with whatever the values were for warned_once on last runtime

        self.voice_channels = data.pop('voice_channels', {})
        self.warned_once = data.pop('warned_once', {})
        self.reset_was_warned()

The bot will initialize with empty dict

[terse folio]

don't have to bother using .pop() then

just set it to an empty dict

[halcyon quarry]

That's what Im doing!

But it doesn't work lol

[terse folio]

push a commit, i'll see what I can find

[halcyon quarry]

It's in the class-everything branch

I broke something in messages last night that needs to be fixed

heh... I apparently Ctrl+X (cut) a method to move it, then never pasted it 😛

[terse folio]

ohno!

[halcyon quarry]

pretty sure I have the function in prior commit tho

Yep, got it

[terse folio]

if warned once is reset on start, is there a need to save it to file?

[halcyon quarry]

Nope

ok that's a nice way to fix the issue... except, would like to go ahead and reset all users existing warnings 😛

[terse folio]

self.save_pre_process(data) is called on save of the BaseFileMemory class

    def save_pre_process(self, data):
        return data

you can define this and add
data.pop('warned_once', None)

and it will remove that item before saving

data is a shallow copy of the class's attrs

so removing the key wont affect your database, but modifying the dict in 'warned_once' will

[halcyon quarry]

So since my dumb little function actually does the trick... I'm going to keep it that way, unless you have another solution

[terse folio]

your behavior is on load, this is on save

[halcyon quarry]

We can modify the on save - but I also want to modify on load for this, to clear user's existing warnings that I never wanted to persist

[terse folio]

it should act the same, but is worth a shot

[terse folio]

yea, that's what the load_defaults is for ^^

[halcyon quarry]

Yes well it doesn't seem to have a means to easily load a default attribute as an empty dict

[terse folio]

what you had should work

[halcyon quarry]

    def load_defaults(self, data: dict):
        self.first_run = data.pop('first_run', True)
        self.last_character = data.pop('last_character', None)
        self.last_change = data.pop('last_change', (time.time() - timedelta(minutes=10).seconds))
        self.last_imgmodel_name = data.pop('last_imgmodel_name', '')
        self.last_imgmodel_checkpoint = data.pop('last_imgmodel_checkpoint', '')
        self.last_user_msg = data.pop('last_user_msg', {})
        self.announce_channels = data.pop('announce_channels', [])
        self.main_channels = data.pop('main_channels', [])
        self.voice_channels = data.pop('voice_channels', {})
        self.warned_once = {}

Actually does not work

[terse folio]

hmm hmm, good to know

oh, i know why

the reason .pop is used is because we don't want to process the same item twice

so after load_defaults runs it loops through the remaining data.items()

and sets the class up with those items

self.warned_once = data.pop('warned_once', {})
was should be correct

[halcyon quarry]

This is working lol

        self.warned_once = data.pop('warned_once', {})
        self.warned_once = {}

[terse folio]

print the output of data.get('warned_once')

surely it's not None

[keen palm]

https://tenor.com/view/-gif-5526730

[halcyon quarry]

        print("0:", data.get('warned_once'))
        self.warned_once = data.pop('warned_once', {})
        print("1:", data.get('warned_once'))
        self.warned_once = {}
        print("2:", data.get('warned_once'))

0: {'fixed_base_settings': True, 'char_tts': True}
1: None
2: None

🧐

[terse folio]

yea,

.pop only sets default IF it is not set

(is None)

not if the item is empty

like {}

[terse folio]

that's a valid solution

or, use data['warned_once'] = {}

and don't bother with using self.warned once

since it will be picked up by setattr loop later

[halcyon quarry]

data['warned_once'] = {} it is!

works

[terse folio]

👍

[terse folio]

that defines it globally for all subclasses

you should just put that code inside self.save_preprocess()

this is just a template function for you to define in your subclasses

to do something before the final steps of saving

[halcyon quarry]

ah

Ok so put it in Database

[terse folio]

a setup I like to have with many parent classes is defining things like load/save in the parent

and using _internal_save and _internal_load in subclasses

[terse folio]

yes

[halcyon quarry]

Also removed the save_now arg for update_was_warned

[terse folio]

that works ^-^

i'd go back and search the function in the folder

as some functions might try calling the old "save_now" arg

[halcyon quarry]

Yep, already got em

I had created the MessageManager() to queue messages to factor responsiveness, to delay message tasks before processing them.

I'm now just going to process everything in order of occurrance, and if there are delays they'll just be applying to the sending.
So the MessageManager() new role will be to simply send parked messages as they are scheduled to be sent.

(in addition to managing all the logic behind the delays, "bot is afk", etc)

One thing I can say is that if you use a setting like maximum_typing_speed: 80, when the bot writes back to you the timing of it certainly feels much more like a human

[terse folio]

yea, adding delays to sending is the way to go.

What I do is spawn a task to generate text, then pull out sentences one at a time, calculating how long it would have taken to type them (subtracting the initial generation time) and sending them off.

That gives you the most immediate results, but also doesn't lock up resources, so the bot can go generating text for other users/tags

[halcyon quarry]

I've been thinking about that

Due to how many different vram intensive features the bot has, it would add some additional complexity to juggle tasks while streaming text

[terse folio]

could have a low vram mode that dedicates windows of time for certain models to run tasks before switching to others

but I imagine most would be running all the models they can fit in vram at the same time

(and because some models can be accessed via api they could be on different machines)

so that can't really be tracked for the lowvram mode, maybe some configuration option to add/remove models from the calculation

[halcyon quarry]

Would also be interesting for someone to use /reset_conversation or /character while it is streaming a response 😄

[terse folio]

tgwui keeps the context in memory while streaming so this wouldn't have an effect until the next message

but it might change the bot's profile/nick while it's streaming messages

if you really wanted to, active streams could be added to a dict[channel: list[streams]]

and you could detect changes like character change for that channel and cancel active streams for that location

(but because of how tgwui works at the moment, only one stream at a time anyway)

[halcyon quarry]

For the features I have currently planned, I don't think I would need to set up anything in advance to incorporate the text streaming - that could just be worked in later

For now, I'm teasing that idea 😛 We'll see

When I get these other new features tamed... should be soon... well, the text streaming may be something you need to get your hands in

(or not 🤗 )

[terse folio]

I don't think its really needed, streaming is a bit slower than running generation normally.
And most chat messages should be shorter, like a few sentences per reply

the code that handles the fake typing wont care if the replies were streamed or not, it will just type while generating text, then continue typing for a calculated time minus the gen time.

[halcyon quarry]

Yep, I'll be doing that 👍

In the class-everything branch there are very few global variables now, got basically everything bundled up in a class now

[halcyon quarry]

I have the new delay mechanisms working mostly 😄

Just need a few more little tweaks

There is now a TypingTask() that can be assigned in the Task() object

it is now processing the LLM gen immediately, then delaying the typing and sending according to user config

[halcyon quarry]

So close to done with these updates...

[keen palm]

https://tenor.com/view/jamie-oliver-yayya-schnayyy-schnayy-gif-8626877149086045636

[halcyon quarry]

When ready, these new behaviors will be worth checking out - actually pretty cool

It only factors delays and special handling for regular message requests, not any commands

The bot can also “go AFK” if it has any sort of responsiveness setting.

While not AFK, it will only factor “text length delay” if enabled

The bot status will change in discord user list to reflect AFK /online

[halcyon quarry]

may even have a quick llm_gen task with limited tokens, to set a custom idle activity status

[valid crypt]

👁️ 👄 👁️

[halcyon quarry]

I'm putting the last tiny finishing touches on this code then pushing this absolutely massive update

[halcyon quarry]

PUSHED THE BIG UPDATE

[keen palm]

https://tenor.com/view/it-crowd-chris-o-dowd-roy-trenneman-oh-god-embarrased-gif-25416062

[halcyon quarry]

Some nice features:

• Will now warn for missing user config values on startup.
• When a message triggers both a text and image response, the image response will be offloaded as a separate task (it will "reference" the initial text response). This will prevent a single interaction from taking too much time.
• All the new behavior settings to play with

If there is a model swap triggered by tags the swap back will now be handled as a separate task as well

[halcyon quarry]

Most of the new behaviors will not affect anything if responsiveness = 1.0 (as default)
msg_size_affects_delay will add some delay based on the length of the text (longer text = longer wait).
If responsiveness is anything but 1.0, then the bot will eventually "go idle".
While idle, there is a response delay for message requests.
It actually generates the text immediately and parks the response... and schedules "come online" early enough to seem like it is reading your message, before is typing... and finally sending response.

I'll definitely be writing a new Wiki page for this

[halcyon quarry]

Pushed an update called Fix Bugs 😄

[keen palm]

Oh snap

[keen palm]

I'm on vacation, so I can't update and test things out

[halcyon quarry]

There’s a flaw in “response delays” logic I need to resolve. It’s currently possible for the bot to reply to a later request first 🤡

[valid crypt]

👌

[halcyon quarry]

@terse folio I'm getting this error when using the regen context command (right click cmd) on a fresh run

After a little chat with ChatGPT, it says this could be due to the HMessage class not having an__eq__ method.

[terse folio]

is it currently pushed?

ill take a look

[halcyon quarry]

It suggested adding this, which I did and the error is no longer occurring:

    def __eq__(self, other):
        if not isinstance(other, HMessage):
            return NotImplemented
        return self.uuid == other.uuid

But I'm just doing this blindly, I really don't understand this code 😛

[terse folio]

I think the error is caused by something happening by accident

like putting the wrong object into history

[halcyon quarry]

hmm

That's likely

[terse folio]

at line 360 in history.py
i'd print what self and new_history._items are

[halcyon quarry]

I think the error was a result of something not being saved to history correctly on previous runtime

[terse folio]

when copying history, it doesn't actually create a new internal message list

so maybe when the messages are being popped from the new history (trimmed one)

it is also affecting others by accident

but the "if message in history" check should still work..

[terse folio]

add a log statement there just incase it happens again, it would be insightful

[terse folio]

what branch should I push the change to?

[halcyon quarry]

I'm pushing a few updates to Main right now - class everything is going to be deleted - I need to reset dev after this update

[terse folio]

ill push to main

wonder if that will fix it,
still want to figure out how to reproduce the error before the change to make sure.

there are plenty of assert statements all over history.py to ensure the correct types are passed into it.
so it can't be that something was accidently put in history._items

[halcyon quarry]

When you have a little time, I'm very interested in your thoughts on this code revamp

Now is a good time to check it out 🤓

Everything is working fantastic except I have two things on my plate atm:

• Something slightly wonky happening with my is typing... method for Context commands. Can't quite put my finger on it.
• As I mentioned earlier, need to revise my logic when adding delays to messages (particularly in same channel), just need to ensure newer messages don't get qeued ahead of older ones

When it is processing the queued responses, the first one will cause the bot to 'no longer be idle'.
When it is un-queueing delayed responses and is "not idle", it negates the response delay

[terse folio]

I think sorting the queue by channel would work.

channel_queue_item class contains list of messages that are queued. but it will only return the first item for sorting which one hte bot should focus on first

[halcyon quarry]

What I was thinking was to have a dictionary of asyncio.PriorityQueues under key name channel.id

[terse folio]

the queue is to sort which items are going to generation first

no need for multiple queues (unless you have a TGWUI instance for each)

i think

[halcyon quarry]

The current priority queue system I'm using is sorting them on send_time

[terse folio]

it would be nice to have flexibility to enable parallel processing with multiple instances.
but that's some wild logic in itself.
my last idea there was creating a queue per instance.

[halcyon quarry]

    async def queue_delayed_message(self, task:Task, send_time:time):
        # Schedule sending this message if it sends soonest
        if self.next_send_time is None or send_time < self.next_send_time:
            await self.schedule_send_message(send_time)
        # Add to self-sorted queue
        await self.send_msg_queue.put((send_time, task))

This particular aspect doesn't have much strain on it as all the heavy lifting already happened - these are just messages ready to be sent out

The parallel processing - Yes, that is definitely possible for the new task manager

Just need to plot out which tasks are compatible in parallel

[terse folio]

set that up at a lower level

where something like "await gen_text()" has logic to pick the least active instance for generation

this would speed up tags, messages, everything without breaking logic

[halcyon quarry]

Think you'll take a look at the new code a bit soon? 😄

[terse folio]

will try to find a good time!
lately i've been slacking off on my own things, need to catch up

[halcyon quarry]

The huge TaskProcessing() class doesn't have anything too interesting there, just all positional arguments replaced with self

I believe I correctly used Union in all its methods for self hinting
ex: async def send_responses(self:Union["Task","Tasks"]):

[terse folio]

you usually don't have to typehint self as python understands it's referring to itself

do you have a case of using a subclass's methods in the parent?

[halcyon quarry]

Yes

[terse folio]

interesting, okay

[halcyon quarry]

A number of occurrences where using methods from Task() and Tasks()
I'm still a pretty amateur python coder so I could be doing some seriously dumb shit XD

[terse folio]

I do that too sometimes, back before I knew about the import TYPE_CHECKING it was a lot riskier because it wouldnt be immediately clear that something changed in another file

but yes, there is probably a better way to do it

because you would have to set the typehint for self in every method to make sure it auto completes correctly

[halcyon quarry]

Which indeed, I did XD

[terse folio]

how come they are named so similar?

[halcyon quarry]

Well, there's no instances of Tasks and TaskProcessing, just inherited by Task()
Tasks is just an organized collection of the main task methods.
Task() is the object to get passed around and collect attributes and such

[terse folio]

so technically it could be a single class?

[halcyon quarry]

You'll know in a matter of minutes what I did there and if it's pretty good idea or complete lunacy

[halcyon quarry]

Yeah probably

[terse folio]

I would name it Task, and TaskBase to make it obvious

anyway, that's fine, I understand the want to split things up so they are easier to digest

but maybe reverse your idea

having the attributes on the parent (base)

and the methods added to the Task() class

that way you don't need to do the typehint trick

[halcyon quarry]

I'm interested in knowing what you think about this strategy I made for queueing new tasks while running a task

Made a method to "clone" the current task, with some simple arguments to control the result

[terse folio]

like the params for the Task()?

so you're creating a subtask, that inherits kinda

interesting

[halcyon quarry]

when triggering a text/image response, it would always run both together.
But I made it less greedy, now it creates a separate image task that it will reference the text response from.

    async def message_img_gen(self:Union["Task","TaskProcessing"]):
        await self.tags.match_img_tags(self.img_prompt)
        self.params.update_bot_should_do(self.tags) # check for updates from tags
        if self.params.should_gen_image:
            # CLONE CURRENT TASK AND QUEUE IT
            if hasattr(self.ictx, 'reply'):
                await self.ictx.reply('(**An image task was triggered, created and queued.**)', delete_after=5)
            img_gen_task = self.clone('img_gen', self.ictx, ignore_list=['llm_payload'])
            await task_manager.task_queue.put(img_gen_task)

It's kind of a lazy way to just send the current state as a new task

[terse folio]

I think I get it

[halcyon quarry]

In this case, the new task is sending a number of things not really necessary for the img_gen_task, but because I've divided the workflow it's the same info I already would have collected by now

I made the creation of new Task objects really flexible too

It has all None defaults, accepts kwargs - an init method to set a lot of default values for anything that is still None

[terse folio]

nice nice!

[halcyon quarry]

I'll be revising the delayed message response logic as follows:

• They will be sorted in the queue based on queue number instead of send time, ensuring messages are always responded to in the order they were received.
• While the bot is scheduled to go idle, only the first message will get a "Response delay". Messages queued after that will only have a "read" and/or "write" delay based on those settings.
• Just need to ensure I track everything well.

[halcyon quarry]

Revised on dev branch, now just need to find time to test

[halcyon quarry]

Yes, things are looking pretty dang good now

[halcyon quarry]

It's close! Seem to have a bug after the bot goes idle.

But all this timing and typing code is otherwise working quite logically

pretty sure I know the problem as well

I was having an issue where it didn't seem to actually cancel an asyncio task.
ChatGPT suggested adding an 'ensure_future' which actually seemed to do the trick.

    def cancel_go_idle_task(self):
        # Cancel any prior go idle task
        if self.go_idle_task and not self.go_idle_task.done():
            self.go_idle_task.cancel()
            try:
                asyncio.ensure_future(self.go_idle_task)  # Wait for the cancellation to complete
            except asyncio.CancelledError:
                pass

I think I just need to apply this to the come_online task management as well

[terse folio]

use await asyncio.wait_for(go_idle.., timeout=???)

I think

wait, I'm not sure what the code is doing

why wont using await normally work?
okay i see

[halcyon quarry]

I defined a new class you might want to check out

in the dev branch

BotStatus()

https://github.com/altoiddealer/ad_discordbot/blob/dev/bot.py#L3676

[terse folio]

instead of using cancel checks

you can just use a shared variable in that class

a flag

await sleep...
if flag is True:
return # (because we canceled)

but that comes with an edge case

nevermind

I think cancel_go_idle_task could be converted to async

i'll have to test this, but I think you can use asyncio.wait_for to wait for a task to finish/cancel/error

[halcyon quarry]

Oh OK I’ll have to check that out

[terse folio]

import asyncio

async def idle():
    try:
        for i in range(10):
            await asyncio.sleep(1)
            print(f'idle countdown {10-i}')
        return 'Done'
    
    except asyncio.CancelledError:
        print('Canceled idle countdown')
        await asyncio.sleep(3)
        print('Done sleeping extra after cancel')
        return 'Canceled'
        
        
task = asyncio.create_task(idle())


async def cancel():
    await asyncio.sleep(5)
    task.cancel()
    
    x = await asyncio.shield(asyncio.wait_for(task, timeout=None))
    print(f'Output from wait_for: {x}')
    
    
await cancel()

print('Now running extra 10 seconds to clean up running tasks')
await asyncio.sleep(10)

this seems to be the behavior you're after

But you don't need this

since your code doesn't run any more async functions after cancelation

since it's synchronous, python will run it before running the next iteration of the async loop

The example above does a 3 second wait after canceling the idle task to test that idea

[halcyon quarry]

ChatGPT seems to think the shield / wait for combo is the better solution

time to see if this actually resolves that issue

[terse folio]

The previous solution wouldnt have waited (If your idle task had async code in the canceled portion)

Because ensure_future tries to run the task later

it takes it out of the sync code into the async loop

it just coincidently worked because your idle function code immediately cancels when it gets the exception

[halcyon quarry]

Maybe? I had repeated the same tests but after adding that code it seemed to be actually cancelling while before it wasn't

[terse folio]

task.cancel will run in the future on the next iteration of the async loop

Because the loop needs to get to the function to know that is has been canceled

Because of this, async code can seem to go out of order

like that example I posted above

it cancels the count down task after 5 seconds

but because the count down task is pushed to the future, it starts slightly after the 5 second timer

And when the timer goes off, it cancels, so it stops the current iteration

so the output is 10, 9, 8, 7

it wont reach 6 or 5 as you might expect

[terse folio]

an illustration

Knowing this, you can reorganise the code a little, like putting the sleep statement after the "print number"

which will give you this result

which is more expected, canceling the task after 5 seconds counts down to 6

[halcyon quarry]

I didn't understand the purpose of the i in range, I thought it was just to make the pretty printing.
But I see maybe that helps with the cancellation

[terse folio]

yes it was for pretty printing

i wanted to demonstrate a task getting cut off mid execution

But also yes, you're right

if you want more granual control over where the cancel happens

you can use await asyncio.sleep(0)

this just gives it a point at which the function can be canceled/start processing something else in the async loop

For example if you run a while loop in an async function like this:

async def test():
    while True:
        print('test')

that prints a value.
that will block anything else from happening.

But if you use await asyncio.sleep(0) after every print, you can run other things at the same time

async def test():
    while True:
        print('test')
        await asyncio.sleep(0) 

making the function actually async

[halcyon quarry]

There's an error in this screenshot that has been causing me a headache for the past 30 minutes or so

It's like Where's Waldo to me but maybe it's a giant blinking sign to you XD

I found it and fixed it, just sharing my grief

[terse folio]

looked over it a few times, couldn't spot anything obvious, what were the symptoms of the error?

[halcyon quarry]

send_message() not executing 🤓

[terse folio]

ahhh, I see I see now

i feel like there should be some syntax warning about awaiting the function without calling it.
But perhaps python has no way of knowing if it's a variable or whatever because variables are dynamically typed