#ad_discordbot (Fork of Fork of xNul's bot)

1 messages · Page 8 of 1

terse folio
#

I see I see

halcyon quarry
#

But, I have not come close to that on very long responses

terse folio
#

Also, not sure about pydub.
But with ffmpeg you can specify a size limit and it will pick the bitrate

#

I think pydub uses ffmpeg in the backend

halcyon quarry
#

This is working pretty good

terse folio
#

^-^

#

Also there is a token limit on tts, about 1000 in/out

halcyon quarry
#

Getting the TTS up and running was fun

halcyon quarry
terse folio
#

just split it into paragraphs, or sentences

#

I was bringing this up more to say that I bet it would be hard to reach the discord file limit with what Xtts can output

halcyon quarry
#

Ah

#

Yeah, definitely

#

Remove the limit if you’re there 🙂

#

It’s probably 25MB like you said anyway

terse folio
#

the limit is probably a good idea on the wav version

#

But may as well always convert to mp3, the quality loss is minimal compared to the already not the best tts outputs

#

and saves users on data if that's ever a worry! / faster upload

halcyon quarry
#

Looked it up, ur right about 25mb

terse folio
#

😸
wow, 18 minutes of music comes out to 16mb mp3

#

yup yup, can remove it!

#

wrote the mp3 to a buffer, so don't have to worry about the deleting mp3 code that was commented out ^^

#

Discord accepts bytesIO types 😸

#

hmm, looking at the play in voice channel function, I wonder how one would achieve audio streaming

#

there must be a function to add to the voice channel

halcyon quarry
#

You mean merging audio sources?

terse folio
#

yes, xtts streaming sends out little chunks that might cut off words

#

so it needs to be added to some queue

#

i'll look more into how music discord bots work soon

#

might get some answers there

halcyon quarry
#

Currently, the bot puts the audio to queue and moves along, and it plays on VC. If another TTS resp is generated right away it plays immediately after the first finishes

terse folio
#

I see I see

halcyon quarry
#

idk if you saw what I wrote yesterday but there’s a guy with this bot on 3 servers at once lol

terse folio
#

I did read that, nice!
some distributed stuff going on

halcyon quarry
#

he said it has some odd responses due to the shared history/context but generally works well

terse folio
#

oh 3 discord servers?

halcyon quarry
#

Ya

terse folio
#

yes the chat should be seperate

#

interesting that happens

#

maybe creating a bot_history per guild

halcyon quarry
#

Channel specific history will handle that

terse folio
#

mhm

halcyon quarry
#

It’s not too complicated I don’t think

#

It’ll be easy 😛 Had too many distractions these past few days to make it happen

terse folio
#

actually would rather write that as an if statement,
I think python will evaluate bot_history() and create a class each time regardless.

#

This could be costly if you do something expensive in the __init__ of that class, like loading files or other tasks
just to keep in mind for the future

halcyon quarry
#

‘Unique_id’ is the datetime when the conversation began - for file saving. If TWGUI supports subdirs in char logs that would be nice

#

For the people who care about loading this history from the webui

#

(Make subdir for each channel)

terse folio
#

why not use the unique_id + channel_id in the file name?

#

if you can set the names

halcyon quarry
#

Certainly can

terse folio
#

also you could add an os.sep into the name, it probably would create a subdir

#

maybe throw an error dir not found. hmm

halcyon quarry
#

if subdir would be the os.path.exists channel OK

#

or whatever 😛

terse folio
#

looks like this would support streaming to vc

#

just write packets as they come in, and specify the correct encoder

halcyon quarry
#

Essentially, to start playing TTS sooner than waiting to complete the full generation yes?

#

Gnight 😛 get some sleep!

terse folio
visual dagger
#

hello

#

any anti repetition prompts that worked for you guys?

#

system prompts or one shot pompts or whatever

halcyon quarry
#

The bot has a Dynamic Prompting feature.
You can put txt files in the wildcards directory and use them like
##instruct me in a ##mindset manner, just ##examples. Please ##final_instructions

#

The sd-dynamic-prompts extension ships with a buttload of wildcard txt files, maybe you can come up with one effective Dynamic Prompt with varying results

#

While testing you can use tags to freeze history

#

You could also use the llm_param_variances tag feature to randomize params and look out for improvements

#

You could also make a Flow where it simply loops the dynamic prompt X times such as flow_loops: 10 and the LLM will be prompted 10 times with the randomized prompt/params

terse folio
#

if my live STT ever survives,
we can call it "BooginSTT" in spirit of Oogabooga

image.png
#

I got livestreaming audio over websockets into whisper working!
sadly I can't run whisper+TTS+llm on the same machine to do some cooler tests right now

#

I'm going to implement silence tracking so it doesn't have to process constantly.

#

then im sure it could be adapted to a discord bot

visual dagger
#

like there is multiple versions of the system prompt

halcyon quarry
#

The feature can select a random line from the txt file and swap it in

visual dagger
halcyon quarry
#

I have a wiki article on the feature

visual dagger
#

so hmm.. adding a relatively medium to big section of text or removing it from the system prompt will help?

#

I mean like example

sometimes we will include a "more info" section and sometimes we don't include it (we remove it)

halcyon quarry
#

Currently, the ‘custom_system_prompt’ parameter cannot use dynamic prompting.

#

But your actual prompt can

visual dagger
#

I mean... does it help? like will it work to reduce the repetition

visual dagger
#

thanks for help

halcyon quarry
#

You’re here in our project channel so I’m offering answers in terms of how this project could help

visual dagger
#

btw thank you guys @terse folio @halcyon quarry

it's so wonderful and fun discussing these stuff with you guys and trying it

visual dagger
#

what I got from it, is including or removing sections

#

or better say a chunk of text

visual dagger
halcyon quarry
#

Just blasting it out there in case it gives you any ideas

visual dagger
#

so I undertood how it works correctly?

halcyon quarry
#

Yeah it can replace your ##wildcard-syntax (representing wildcards/wildcard-syntax.txt) with a randomly selected value

visual dagger
#

ok cool 🔥

halcyon quarry
#

The value could be a single character or a book of text

visual dagger
#

got it got it, thanks

halcyon quarry
#

Well, the values is randomly selects from must be separated by new lines in the txt file.

visual dagger
#

it chooses a random line

#

from the text file then inject it

halcyon quarry
#

Alternatively you can use {{this syntax|this or that syntax|another syntax like this}} to pick something at random

#

Yes, try it

#

It returns a copy of the updated text to the channel

#

So you can see what random values were applied (also visible in cmd)

#

Good thing we're discussing this because just noticed the wildcards has a bug due to directory not being initialized correctly with the filesystem...

#

fixed that

vestal python
#

On the road for the next 2 days and all the new servers wiped and setup now... I didn't take the time to add in a new wireguard vpn x.x I'll at least grab the example character.yaml and get that done. After all that my 3 discord bots should be working as intended.. Or 2 plus an alltalk server with rtx 2070 maxq

halcyon quarry
#

The session_history variable is currently just one dict.

If autoload_history: True, the value is set by TGWUI load_latest_history() based on current state['character_menu'] and state['mode']

I think the correct course of action will be to just snag those two values, create a loop for all channels the bot is currently a member of, copy then update the value of character_menu to prefix it with the value of those channels. Then assign each history by channel id in a chan_session_history variable

#

We'll collect a list of channel IDs the bot has message permissions for on startup.

# 
    def collect_all_bot_channels(self):
        for guild in client.guilds:
            for channel in guild.text_channels:  # Only consider text channels
                # Check if the bot has send msg permissions for the channel
                if channel.permissions_for(guild.me).send_messages:
                    self.bot_channels.append(channel.id)
halcyon quarry
# 
    def load_history(self):
        current_state = bot_settings.settings['llmstate']['state']
        values_for_history = {'character_menu': current_state['character_menu'],
                              'mode': current_state['mode']}
        for channel_id in self.bot_channels:
            chan_values_for_history = copy.copy(values_for_history)
            chan_values_for_history['character_menu'] = f'({channel_id}_{chan_values_for_history["character_menu"]})'
            channel_history = load_latest_history(chan_values_for_history)
            self.session_history[channel_id] = channel_history
#

Pushed WIP to new branch per_chan_histories - done for probably the day

halcyon quarry
#

More settings.
If any channels are specified in history_channels then it will not dynamically use all available channels

Screenshot_2024-05-25_124010.png
halcyon quarry
#

To use TGWUI's save/load history functions with per-channel histories - the chatlogs will have to be saved into directories suffixed with the channel ID.
Meaning, there will be a 'Character_xxxxxxxxx' directory for each channel with its own history management.
These will not appear in TGWUI chatlog dropdown without renaming the directory to match the characters name

#

Making a lot of progress on this

halcyon quarry
#

calling it quits for the night, but per-channel histories are probably coming tomorrow/Monday. Very close.

visual dagger
#

heh

#

hey

#

@halcyon quarry thank you for yesterday, I think it's working, I'm noticing less repetitions

#

but I still need to do more testing

terse folio
#

"Note: Can't delete outputs uploaded to channel. (../extensions/coqui_tts/outputs)"
interesting, why's that?
Os error? like file in use?

#

if so, you could put the files you want to delete in a yaml file, then check if they exist on the next load and purge them at that time.

halcyon quarry
terse folio
halcyon quarry
#

You nailed it

terse folio
#

put alltalk in tgwi to test my update to the file sending

halcyon quarry
#

If my error reads like that, then I never got around to making it use the tts_client variable (originally only supported coqui

terse folio
#

hmm I wonder if it has to do with tgwi maybe using that outputs folder like a "static" folder on a webserver.
So it can link to those files in the webui

terse folio
halcyon quarry
#

They can be deleted after new file generated though if memory serves me right

#

Nah haven’t touched tts code in a loooong time

terse folio
#

oh nvm, i could install coqui pretty easily, it's already part of the extensions
and its only requirement is "tts" which alltalk also uses

#

Lol thought something was wrong with the api.
That's an odd thing for the LLM to reply to for "test"

image.png
#

anyway, file upload works as normal!

halcyon quarry
#

Lol

#

Too much ChatGPT response in that training

terse folio
#

Yea, it's an older llama2 model for the fast load times during testing

halcyon quarry
#

Definitely makes sense

#

Models quality has come such a long way since then

#

Assuming you mean like, when llama2 was new

#

I have mixed feelings on llama3

terse folio
#

at the latest I've been using Solar10b, a mix of Mistral, and Llama3-8b.

they have been pretty good for the random tests I do

halcyon quarry
#

Sounds good need to check that out

terse folio
#

Mistral was the first above 2k context model I tried, so that opened up a lot of possibilities!
Then llama3 came around at 8k context by default which matched the settings I used for mistral anyway.

#

Llama3 performed pretty okay with tool selection (shown in that screenshot yesterday)

visual dagger
#

any useful 32k llama3 8b finetune?

#

all extended context ones I tried break at some point

terse folio
#

and yes, in many people's experiences after around 8k tokens things start to break down.

visual dagger
#

what's that rope thing? I keep hearing about it

terse folio
#

It's in the model load tab, 1 sec

visual dagger
#

even for 200k finetune or 64k

#

all of them break after 16k

#

I couldnt get any finetune to survive until 32k at least

terse folio
#

#general message

alpha 4 for 4x
2.5 for 2x

visual dagger
#

4x the context?

terse folio
#

yea

#

this slider

#

of course, this will degrade the quality the more you expand it

#

even if you don't use all the context

#

Kind of like quantizing the model to fit into a smaller space I assume.
This compresses tokens to fit in a smaller space

halcyon quarry
#

Even though I’m not going to use it personally I’m very excited about the per channel histories

#

This will be a feature that definitely sets us apart from the rest

terse folio
#

😸

halcyon quarry
#

I realized that model and character changes could be mysterious happenings, but what I’m going to do is just use the history_channels list to chanel.send the embed. The message for each embed will be dynamic for if the channel is in i.guild.channels such as ‘User in another server changed character’

#

Or if in same guild, ‘{user} in {channel_name}} changed character “

#

I’ll offload this to a background task and maybe add asyncio.sleep if there’s a buttload of channels

terse folio
#

it would be best to handle characters/system prompts per conversation

visual dagger
#

if yes how does it not affect the pc's performance,, I mean it will use double or more resources than usual I assume

halcyon quarry
#

The bot is going to handle one request at a time for now

#

But it is going to have an option to manage history separately per channel

#

Which means multiple history variabls so some additional ram usage but it’s just text so probably nothing significant

#

It’s almost done really

halcyon quarry
#

The only way this bot could allow multiple characters in a per-channel discord context, is disable avatar/display_name change.

#

On a per-guild basis, those could be updated

terse folio
#

webhoooooks

#

someday™️

halcyon quarry
#

The multiple character thing I think is too much effort / too specialized, when the user could have multiple bot instances. I think I’d rather improve support for that concept

terse folio
#

at some point I want to build a new bot base where all these tools can be plugged in as hot swappable modules 😸

Tts support, images, textgen,..

halcyon quarry
#

Really just need to put like, root_dir as a variable in config.yaml

terse folio
halcyon quarry
#

I meant so there could be a ad_bot1 folder and ad_bot2 folder

terse folio
#

a lot of settings will be the same

#

like if users want tts, so that could be shared

halcyon quarry
#

Well they will literally be the same

terse folio
#

just the "active settings" needs to be serperate per bot

#

if using different characters

halcyon quarry
#

If separate bot instances could be completely independent of each other, it could be easier for user to share the settings they want to share

terse folio
#

oh I see

#

well then you have the problem of vram

#

running multipel copies of tgwi

halcyon quarry
#

It would be for someone with 2+ gpu using one for each instance

#

I have no clue I’m spitballing here lol

#

Multiple character support sounds like a nightmare

vestal python
#

.> I could test it out on my 5GPU server if you want.

terse folio
#

Was talking about pattern matching in another channel.
This could be of benefit to you too!

Letting users build patterns that can pre/post match on user/llm text

#

based on this list of patterns in the database.

image.png
#

The difference here is this runs through a recursive tree a token at a time to find the first best match
instead of running a match for every entry.

halcyon quarry
#

It’s your canvas to paint upon my friend

#

Just make sure potatoes is spelled correctly

keen palm
#

Reality is Dan Quayle confirmed

halcyon quarry
#

You can make your first tag 😮

terse folio
#

Lol, I thought it was interchangeable I've seen people spell them both ways

halcyon quarry
#

Instead of sending Change embeds to history_channels (from /character, /llmmodel, /imgmodel, Tags), I'll send them to main_channels. So this will be an enhancement now, separate from the per-channel history thing

#

Are there better alternatives? Yep, but I want to get a pretty good solution published for now

terse folio
#

an /announcements possibly

halcyon quarry
#

Excellent solution

#

Now that I already coded this thing lol

terse folio
#

it's just copypaste the main command and switch what database entry it uses

#

database.main_channels -> update_channels

#

you can select a piece of text and use Ctrl D to duplicate cursor

#

easy rename :)

halcyon quarry
#

How about we just add an announcement_channel to config.yaml under the discord key

terse folio
halcyon quarry
#

Ahhh yes

#

How did I make it this far without you? Sheesh

#

Now that is a nifty trick

#

thanks for that advice

terse folio
#

😸

#

one of my favorite keybinds

halcyon quarry
#

One of my new favs already

#

Beautiful. I like this solution a lot

terse folio
#

Mhm!
gives people the option to opt in

halcyon quarry
#

Especially b/c we don't have to do too much

#

Now it's on them 😎

#

🥂

halcyon quarry
#

Time to start testing the finished per-channel histories code...

terse folio
#

hope that works!

halcyon quarry
#

Didn't 😛 need to see what the dealio is...

#

I probably can't do this

Screenshot_2024-05-26_222554.png
terse folio
#

i don't use setdefault too often, not sure

#

it's worth using an ipynb file (python notebook)
for live testing ideas like that.

Just those lines about setdefault and see if they work the way you think they do

halcyon quarry
#

another thing I did not take into consideration, is that instruct mode doesn't save logs to any subdirectories

#

ok my problem was simple

#

I was initialzing with placeholder lists in the session history variable. So these were just adding new lists to the existing placeholders.
Derp

terse folio
#

interesting

#

also to save you some code:

#

just set the default channel to "internal", or 0 if not using per_channel

#

maybe you reference session_history in other parts of the code, and this could remove some possible bugs if an if statement is forgotten

halcyon quarry
#

progress

Screenshot_2024-05-26_225729.png
terse folio
#

No worries about the channel ids
They have no personal information ^-^
All someone could extract from one is the date/time the channel was created.

#

Also awesome

#

Actually all the discord IDs are built like this, users, roles, channels, servers.

Only user ids can be resolved to a username.
This allows bots to ban users from joining your server if they never joined.

iu.png
#

this inspired me to make my own snowflake (unique id) thing 😸

halcyon quarry
#

I like to exercise precautions like this

terse folio
#

Absolutely!

#

perfectly reasonable

viral lagoon
#

Fancy JS to convert snowflake (id) to a date js new Date((Number(id) / 4194304) + 1420070400000)

#

I've had this as a command on my bot, it's super handy

halcyon quarry
#

New announce channel thing that sends embeds to all announce channels for character, imdmodel and llmmodel changes - is working

#

Per channel histories is still a bit bugged, I'm outta time though

terse folio
#

yea that's a big change to make

halcyon quarry
#

Hopefully have that all buttoned up tomorrow

#

it's close - probably one or two lines wrong 😛
It's saving a new log file on every message. Otherwise it seems to be working

terse folio
#

hmm that is an odd sounding bug

halcyon quarry
#

I wrote a lot of lines, just overlooked something. Head's spinning at the moment.
It is managing the history seperately for both channels I'm testing in

terse folio
#

that's good progress!

halcyon quarry
#

loading the correct one on message, continuing along, saving to correct thing, etc

#

For chat mode, it is creating a new subdirectory for each channel

Screenshot_2024-05-26_232208.png
#

I may not need to do this... need to rethink

terse folio
#

interesting, if tgwi can't read them, you may as well save them somewhere else
and in the structure you want

halcyon quarry
#

Rather than trying to dissect the code I'm just seeing how I can tweak the data to continue using imported history managing functions

#

Just need to doublecheck my logic to see if I need to do these separate folders...
Anywa, calling it a night

#

pretty damn close

terse folio
#

a weird hack would be to copy the bot profile for each channel (when sending messages)
so tgwi can import it.
But also name them like TMP_<name>_<channel>
so the user knows they can delete the characters if they want to change something in the base character file

halcyon quarry
#

I'll figger it out tomorrow 😛 Pushing what I have in case you want to look at it

terse folio
#

maybe, maybe, I feel like I'll be too distracted by my project haha

vestal python
#

You use forge. Is this correct to use the branch instead? I'm about to get back to working on getting my bots and sdxl working properly, and then see about adding in some features from textgen I was thinking.

IMG_20240527_101732.jpg
halcyon quarry
keen palm
#

Per channel history hasn't been pushed yet, has it?

halcyon quarry
#

It's almost done, hoping to push that tonight or at latest tomorrow

#

As a family man, difficult to say when I'll have time 😛

keen palm
#

I know the feeling

#

I get approximately -20 minutes of free time daily

halcyon quarry
#

I think I've got this per channel histories thing nailed down

#

If the feature is enabled, it will intialize the history variable / log file with all available text channels

Screenshot_2024-05-27_150743.png
#

If /reset_conversation is used in a channel, only that chanels history gets reset

#

This will be exclusive for the bot, cant just load these logs up into TGWUI. Although maybe could make a function to split it to separate files if needed

keen palm
#

Do you know what happens if you try to load that log in TGWUI?

halcyon quarry
#

Probably error

#

Expected file structure is a dictionary with two lists. This log will be a dictionary of dictionaries with 2 lists each

#

I need to do a bit more testing later on to ensure all is working completely well

#

It should correctly autoload history, as well as honor the keep/reset setting for characted changes

#

I also have a feeling that ‘visible’ list has no purpose for the bot purposes… will likely keep it empty from now on

#

(For per-channel history mode)

#

May add a utility script to convert the multichannel log into multiple files, which could duplicate the internal lists for visible lists

#

Definitely will - chatgpt will tell me how to do a little bat file or something to drag n drop onto

keen palm
#

Regenerate is still broken?

halcyon quarry
#

One more thing… probably going to change it from “channel id” to “guild - channel name”

halcyon quarry
keen palm
#

I get "IndexError: list index out of range" when I try to regenerate

halcyon quarry
#

Did you write anything in current session?

keen palm
#

Oh, you know what, that'd be it

halcyon quarry
#

Requires at least 1 history entry to work

#

When I fix it there will be no error… it’s on the official todo list 🙂 pinned msgs

keen palm
#

I'm also trying to regenerate a response that I specifically had it not save

#

Hwhoops!

halcyon quarry
#

I may need to have 2 versions of Regenerate

#

One that uses history and one that uses recent messages (includes messages not in history)

keen palm
#

Do you know of any way to suppress the LLM response to a prompt? I am just looking for a way to add new information without the AI responding and clogging up history.

halcyon quarry
#

[[should_gen_text:False]]

#

erm

#

that probably won't work...

#

I need to run atm

#

So you basically want to incrementally prepare your prompt?

keen palm
#

I already tried that one. Doesn't work.

#

Basically yeah

halcyon quarry
#

You could do something like this for the moment…
[[state:{max_new_tokens:1}]]

keen palm
#

I did use [[state:{max_new_toke...yeah

halcyon quarry
#

I think 0 = error

keen palm
#

Correct

halcyon quarry
#

Adding to my TODO list before I forget:
user variable assignments
I plan on adding a feature for users to be able to assign variables on demand for whatever purposes.
This would solve it

#

Idk the syntax yet but you could just set each chunk as a variable, then when your ready send your prompt prefixed with the variables

#

May have syntax to add to existing variablr

#

I have it now only creating the channel dictionaries in the history on-demand.
It is now also using the server + channel name as the dictionary key

Screenshot_2024-05-27_215711.png
#

Also not collecting Visible for multi-channel mode

#

Also suffixing filenames with multiple-history

Screenshot_2024-05-27_215912.png
terse folio
#

the reason you originally collected visible was to get the audio file url from the html tag.

I'm not sure if theres another way to get it when running the generate text pipeline?

halcyon quarry
#

It still gets it - it iust doesn’t hang onto it now

terse folio
#

I see, yea good point, it's only needed the one time

halcyon quarry
#

I think the only reason to store it, is to be able to load the log file in TGWUI

#

But that won’t be possible in this logging format.

#

Well it will be possible after post-processing log file

#

@terse folio do you know if server/channel names can have any wacky values that could mess up this dict structure?

terse folio
#

Also, json doesn't like int keys iirc.
so guild/channel ids should be converted to str before querying the json data.

halcyon quarry
#

The way I have it set up, when a user sends a message it uses the guild name and channel name from the interaction as the positional arg to “get history”. If the key doesn’t exist it returns the 2 empty lists

#

But yes I see your point - the history would suddenly be reset after channel name change

#

Ok what I’ll do is put everything under the channel.id key

#

I’ll just add 2 dict keys inder thay

#

Server name and channel name

terse folio
#

you dont really need the guild Id

channel id should be unique .
The message fetch function uses the channel id, and message id for example

halcyon quarry
#

Correct- I just want to include guild name / channel name for easy reference

terse folio
#

for the folders?

halcyon quarry
#

for logging

terse folio
#

okay, yea

#

because if writing to a folder you need to filter the characters as some are invalid for a file path.

#

<>/'" are some invalid ones I think on windows

halcyon quarry
#

Good points

#

I’ll be sure to mention it to my pal ChatGPT when I request the utility 😛

#

I don’t speak with him much these days I know most of what I’m doing now

visual dagger
#

does it go and collect all metioned keywords's facts?

#

like this

the only trigger is this word 'potatos'
the input/user msg is I don't like potatos btw

the backend work collects those

{potatos: John don't like potatos}

which will trigger 'John' fact because 'John' is mentioned here

{John: John is 32 years old and works as a teacher}

{work: John is a teacher at LLM University}

{LLM University: LLM University is a university specialised in everything AI/ML}

So a user input I doesn't like potatos btw

Caused this much facts to get retrived even if these following facts' keywords aren't mentioned, but they got triggered because they are mentioned in other facts.

+ John don't like potatos
+ John is 32 years old and works as a teacher
+ John is a teacher at LLM University
+ LLM University is a university specialised in everything AI/ML
halcyon quarry
#

From what you are saying, you can already do something with the tags system like

trigger: 'John'
format_prompt: |
+ John is 32 years old and works as a teacher\n
+ John is a teacher at LLM University\n
+ LLM University is a university specialised in everything AI/ML\n
{prompt}

trigger: 'potatos'
format_prompt: |
+ John don't like potatos\n
{prompt}
#

When using the format_prompt tag, your actual prompt gets inserted anywhere you include {prompt} in the value

visual dagger
#

but what if it got too much, like an overload of info

#

tokens inflation

halcyon quarry
#

Maybe I don't understand.

What I'm suggesting is that you put the facts into dict_tags.yaml in this format

visual dagger
#

they don't get triggered automatically

halcyon quarry
#

Alright, so you are talking about capturing facts automatically - not so much triggering them automatically

visual dagger
#

uhm.. my wording is bad I guess

halcyon quarry
#

As it stands, the bot is great at triggering things 😛

Capturing things - can still be improved

visual dagger
#

but I'm debating if that's a good idea or not

#

since if I don't stop it it might litetally collect all the info

#

assming the db is huge

#

ALL will be triggered

halcyon quarry
#

Well, if whatever the feature is uses the Tags system - I have a trumps tag that allows a matched trigger to be cancelled by another matched trigger

visual dagger
#

maybe setting a max 'depth' can help

#

like a max 5 spiralling facts

halcyon quarry
# 
- trigger: some trigger

-trigger: another trigger
 trumps: some trigger

text: Let's talk about some trigger and another trigger

Only another trigger has effect

visual dagger
#

let me explain what am talking about, I think I made thigs confusing

the user msg is only this hey, Alex is calling

this user msg will trigger (by trigger I mean will cause a fact to be retreived)

(keyword: fact pairs)

Alex: Alex and John are chidlhood friends

^ this fact have the keyword John

which will cause this to be retrieved

John: John is a 32 years old teacher and have one kid called Charlie

^ this fact have the keyword Charlie

which will cause this to be retrieved

Charlie:Charlie is the son of John, Charlie is a 2 year old boy, Charlie have a sister called Martha

and you know the rest, Martha info will get retrived and it goes on and on

halcyon quarry
#

This is what you want? Cascading effect?

visual dagger
#

still debating if it is a good idea or not

#

bcz it will cause an inflation of tokens

#

so in theory it might even retrive the whole db

halcyon quarry
#

There could be another user setting like max_depth so like max_depth: 5 it could only look for 4 additional facts before stopping

visual dagger
#

and parallel facts, lol

#

hey, Alex is calling, I will sit with Charlie while you answer

#

you see, we got multiple paths here, Alex is mentioned and also Charlie

#

so yeah

halcyon quarry
#

facts could be collected in a Set() which omits duplicates

visual dagger
#

that's an idea to remove repetitive facts

halcyon quarry
#

before being appended to the prompt

visual dagger
#

but you misght end up with totally different topics triggered

#

which means you won't have duplicates

#

it's possible to summarise the similar facts I guess

#

instead of John is a teacher and John works at LLM university

that can be summarised to John is a teacher at LLM university

#

summarising or collecting repretitve/too similar facts , collecting them together

#

maybe there have to be a mechanism to score the retrieved facts and get rid of the less relavent ones

and keeping like the top best highest quality relavant 5 ones or something

halcyon quarry
#

Quite frankly, this is all above me, so let's wait for Reality - and I wouldn't count too hard on them implementing such a feature

visual dagger
#

you don't have to

halcyon quarry
#

We're in brainstorming phase 🙂

visual dagger
#

I'm just

#

not sure about it

#

if it is a good idea or not

#

bcz as I said I might end up retrieving the whole db lol

#

and this excesive retrival might cause excecive repetition

#

bcz the llm sees the same facts again and again

#

and the llm sees a lot of them, so... they have more weight than the user prompt

#

it will be like 70% attention to facts and 30% attention to actual user prompt

halcyon quarry
#

Just finished up revising all the history code

terse folio
terse folio
terse folio
halcyon quarry
#

New logging format for per-channel histories

Screenshot_2024-05-28_110558.png
terse folio
halcyon quarry
#

No, not me 😛

terse folio
terse folio
visual dagger
halcyon quarry
#

Actually, just rearranged it so that the guild and channel are assigned before the history list

#

so, slight difference 😛

visual dagger
terse folio
terse folio
# halcyon quarry Actually, just rearranged it so that the guild and channel are assigned before t...

Dicts actually have no order, it's just a table of keys:values.

Back in python3.5 if you put items in a dict they would not stay in the same location 😸
Free to move around as they wanted.

But yes, more recent versions keep them sorted based on how you input the data.

So don't expect dicts to always come out exactly the same way you put them in when translating them to other languages, such as exporting a json/yaml file.
(but from my experience they usually respect order when importing to python)

halcyon quarry
#

nbd 😛

terse folio
#

Yup, just little bugs I ran into in the past ^^

visual dagger
halcyon quarry
#

Hopefully TGWUI load_latest_history() isn't too complex to reconstruct / patch because it is not happy when it finds this new structure

terse folio
terse folio
visual dagger
terse folio
#

Yea, this is where we'd need an llm!

#

do some preprocessing on the text.
like Convert the text to use 3rd person pronouns instead of 1st, 2nd person (eg: you, i)

#

And then the complex ways that sentences work sometimes

#

like "Get the groceries" has an implied "you" at the start.

#

"Mark, go get the groceries"

#

Then tags could be run on these processed texts

visual dagger
# terse folio Pattern matching doesn't know who Mark is talking to, How would it know to quer...

it's recrusive as you said

Alex will be set as a triggering word, that will trigger this fact to be retrieved Alex and John are childhood friends

#Pairs on this format
TriggerWord: Fact

it's a dict of this ^

so because John is in this fact, another fact will be triggered which is John works as a teacher and have a sone called Charlie

and this fact will trigger the retrival of another fact which is Charlie is a 2 year old

#

not because John is in this fact, no, but because John is set as a triggering word

{John: John is a teacher,

Alex: Alex and John are childhood friends

Charlie: Charlie is a 2 year old}
terse folio
#

but why should Alex trigger Alex+John.
what if the conversation is only between Alex and Mark?

#

it would bring up John

visual dagger
#

so the fact can actually not mention anything about the triggerword

#

it's a trigger to fact relationship

#

like a var to value

#

the value can be anything

#

and can be even any random stuff

#

but when the var is called the value will get printed

#

like that

terse folio
#

Is see, but there would be multiple entries for the relationships
Alex: Alex+John, Alex+Mark, Alex+Charlie..

visual dagger
terse folio
#

you can detect loops by checking if a fact has already been visited
Just add them to a set and check if the current recursion item is in that set.

And cut it off there because it would do a circular import of the data again

#

Then as AltoidDealer mentioned, using a set to keep only the unique items

visual dagger
terse folio
visual dagger
#

you can make it stop by doing a max_depth concept

#

but still idk how that will work, it might ruin llm responses

#

confusing it

terse folio
visual dagger
# terse folio It's a start to use one word, just make sure to keep your code open to changes, ...

there is a problem, one word trigger can be needed sometimes, like assuming we did only this

Food+Potatos: John hates potatos (if both those words are present , this fact will be retrived)

but what if down the line in the story, this happend

John looks around and he finds out someone selling potatos

bcz we limited the fact of potatos to two words and not one, it won't be mentioned here, thist fact won't be mentioned here

#

we will wait someone to mention food before this fact is retrived

#

and by that time John might already be eaten the potatos, poor John

terse folio
#

create multiple!

#

create a hierarchy

#

"food+potatoes" will run before "potatoes"

#

The pattern matching thing works the same way.
It first checks if an exact match for your text exists, then it starts testing wildcards until it finds a match.

visual dagger
#

it will be the same outcome

#

so just do potatos only

terse folio
#

you could have different information mentioned

#

John+potato not the same as potato

visual dagger
terse folio
#

i'm not sure why "food+potato" was an example, what that was

visual dagger
#

food is not a topic

#

potatos also not a topic

#

they are both literal words

#

strictly

keen palm
#

Psst: potatoes

terse folio
#

O:

visual dagger
#

o:

terse folio
#

"John is getting potatos"

visual dagger
#

I run out of examples

#

: (

#

yeah food + potatos seems weird but can't think of any other example

terse folio
#

I see, In that case need to create a file of good examples that you think would break/cause bugs!
Use those to finetune your code!

visual dagger
#

I'm trying to generalise as much as possible, making things autonomos and general means working for any scenario

#

without me fixing llm mistakes every now and then

terse folio
#

mhm, what I mean is while you figure that out, you should have test cases that you can quickly run to see how the logic works out

visual dagger
#

but it seems like the system should be very robust and self sufficient

#

fixing it's own bugs somehow and updates false info

#

and facts etc..

terse folio
#

Here's an example

John+Potato(s) - John hates potatoes
Potato(s) - The town you all live in is actually known for potatoes!

These are 2 different peices of information that can be triggered in different contexts

visual dagger
terse folio
#

benchmarking your code progress to make sure you're bug free 😸

#

like that if you change something, it doesn't break previous tests

terse folio
visual dagger
#

how do you deal with the llm forcing including all facts in one response?

#

instead of being more natural

terse folio
#

I usually think about how my mind would handle it.

#

Maybe using embedding to match how related the tag contents are to the conversation

#

if it goes too far off track into unrelated territory it would return lower scores

visual dagger
#

like there gotta be a strict logic

#

that handles or decides what is relevant or not

#

based on A B C D factors

#

taking mutliple stuff into consideration

#

like how a human being make a decsion.. yk?

you rely on your undertanding of space, time, self, current situation, if there is other intities (people) around you etc

#

you process all those different categories of things before making a decision

#

of let's say screaming

terse folio
#

time and space, yes I agree,
But we are more limited depending on where we talk

visual dagger
#

it's more of a game that I am working on

#

rather than only chat

terse folio
#

if you're doing roleplay with moving around like your game, then yes

#

in that case you could have the tags look in different locations

#

some in chat, some based on location, some based on time

#

like if your current location matches "the town" it can mention it's know for potatoes

visual dagger
#

do you have any idea on how to craft such a scoring system?

#

that takes many factors into consideration

#

or maybe just use another llm?

#

it feels like I should have a phd in phsychology sometimes lol😭

#

or study the humans

terse folio
#

It's not something I tried.
But I was going for a similar idea where I planned to use Embedding models.

I think the idea I had was retrival of information, then highlighting the senteces that are most related (that's why I created the zoom levels thing)

visual dagger
#

who is up for a scientific expirement 😐

visual dagger
terse folio
# visual dagger it feels like I should have a phd in phsychology sometimes lol😭

yup yup I know.

Using negative words such as "not" "do not generate this"
Gives the LLM a reason to try doing exactly what you don't want.
(At least smaller ones)

Similarly if you tell a person to not imagine a chicken, they have to do so to understand what you mean.

Maybe we should do things in multiple steps?
Like how the person first calculates the meaning of what you said, then decides to act on it.
I have no idea what that would look like though.

Haha, it reminds me of people saying "be careful with that", and then you are suddenly more likely to slip up and fall with the item.

LLM agents seem to do that, working on a problem in multiple steps.

keen palm
visual dagger
#

dc lagging

#

i got a half baked idea, scoring facts based on priorities

#

but priorities of who? the dev? the characters?

#

idk

terse folio
# visual dagger what decides the most relavent?

When trying to do something completely automated, don't expect perfect results every time.
Anyway, the idea was to search based on the current conversation, this would give you the topic embedding.

From there search the database subset on the same conversation and find the topic item that best answers the question/adds context/whatever.

Once this context is added, generate the reply.
Embed that reply, then compare to the sentences of the context and try to find a quote you could cite.

If the score is too low, we could assume the bot hallucinated an answer.
if there's a good match, it can be confident the answer has truth

terse folio
#

that's a bit beyond me right now, but i'd look into Keras, they had some simple examples, like classification

visual dagger
terse folio
#

for embedding

#

tag matching uses if statements, so it's more accurate in making sure the tag exists

visual dagger
#

what is important? and how do you know that?

#

like how do you extract that from the chat history

#

conclusions

terse folio
#

I don't!
would have to build a test program and figure that out

visual dagger
#

and you might need to go short term vs long term, like this topic is imortant currently but might not be in the following weeks

#

topic let's say a temporary sickness

#

after some days-weeks it will go down on the list of priorities

#

until it is completely removed

terse folio
#

Lets say you have a home assistant.
And you're having a conversation with someone about what you want to eat.

During this conversation you raise your voice slightly and say "search for restaurants serving X"

We need some logic for the home assistant to figure out you're talking to it, and not the other person.

Most solve this with a trigger word "okay google"
Others might listen to a change in tone of your voice (like it being raised to project across the house)

How do we do this in text?
for the bot to understand you want to search the web while talking.
That's one of my goals

visual dagger
#

hmm umm...

#

text only?

terse folio
terse folio
#

if i say, search the web about it.
You would understand i'm talking about "search the web about remembering things that is sorted by time"

visual dagger
#

I was thinking of the assistant having a camera so when you do point to it with your finger and say search for xyz, it's a clear sign you are talking to your oc

#

pc/assistant

terse folio
#

😸

visual dagger
#

lol

visual dagger
visual dagger
terse folio
#

if you do a search online, you might value more recent information over decades old

#

because the tools might have changed if researching a code problem

#

BUT, there is still plenty good information over 12 years old that has helped me tons!

#

so we can't just set a cut off date on search results

#

This is related to the "feeling sick" temporary memory

#

how do we decide what should be kept longterm, and what short

visual dagger
#

and memory at play here

terse folio
#

Yea, that's right,
that's kind of what I was saying with fine tuning the model.

Baking that knowledge into the model.
The reason I mentioned fine tuning instead of dumping all the context was because consumer gpus have limited context

visual dagger
#

how do you decide?

#

and if you assume all info are important then how do you store it in a nice way to be retrieve it easily later?

terse folio
# visual dagger exactly, how?

I don't know yet, maybe I will do some sort of weighting?

Like the further ago the result is in time, the less it is worth.
But also consider the similarity scores.

Score(Score bias) * time(Time bias) = final score

and we fine tune these biases

So maybe, some item with really good score from 10 years ago would have enough weight to show up as a result.

#

I think that would work

#

because if there are no new results about that topic that are good matches, the old would would be the next best one to show

visual dagger
#

how wondeful is this?

halcyon quarry
#

I like how this channel is the most productive channel on the server lately

visual dagger
#

I don't have the gpu for it, and even if I did, finetuning is not that easy to get right

#

@halcyon quarry 🫡

terse folio
#

with all previous chats

terse folio
visual dagger
#

am not an ASI

terse folio
#

Mhmm,
Should look into how hybrid vector databases work.
That would give me better insight into how to do the weighted by time thing efficiently

visual dagger
#

I hope that it worked

terse folio
#

A database like Chroma supports putting tags in your items to search on.
But I don't think they optimised this for high speed.
So it's doing a comparison on every row in the db?
I could be wrong!

visual dagger
visual dagger
terse folio
#

after I get my other project done, i'll put some time into the pattern matching!
That gives us a logical way of extracting info

visual dagger
#

about similarity, the assistant should bring up your medicine up if you forgot it, so... similarity won't help, since you forgot

but that thing "medicine" is one of the top priorities

but there is a problen, when you are not sick anymore, the llm/system/logic should drop that priority and make deduction on its own

that

John not sick = no need for medicine

no need for medicine = drop it from the priorities list

terse folio
# visual dagger not sure, but ty for the info, I didn't know that a vector db can include also t...

Hmm, speaking of hybrid databases.
I read about them briefly once.

They work by doing the embedding vector search.
then another search using one-hot-vectors which contains actual tokens instead of being compressed.
This lets you search for exact words unlike embedding models that are more about meaning.

Anyway, what I'm thinking is:
Create a sql database, and query the top N rows that match certain tags, or time ranges...
This will return a list of IDs

In the embedding database, query another top N matches based on meaning similarity.

Do a set intersection to find the items that are returned by both sides.
Then score them by going through each result at a time.

Around 1000 items is still fast enough for millisecond response!
so we could use around 10k results and have a decent trade off of accuracy and speed.

visual dagger
#

so things gotta be dynamic somehow, if the enitity (llm+system/logic) heard you saying no I didn't forget my medicine for like 7 days on a row, the entity should conclude that there is no point in reminding you since it's getting annoying

#

so... it should go down on the priorities list a bit

#

but not much

terse folio
terse folio
visual dagger
terse folio
#

But consider each database would return 10k/20k+ items

visual dagger
halcyon quarry
#

Quick report that I did have to copy/paste 2 loading functions and modify them, to get per channel histories working correctly

#

And it is working correctly

terse folio
terse folio
visual dagger
#

then integrate the result/decision into the main chat

#

hmm.. so by the enitity having priorities it should have values... and one of the values are your well being

#

that. ... is interesting it won't base it's decisons on randomness but on actual values and priorities

#

not stupid censorship that the big AI players implement

#

but kind of a natural version of that

#

that is based on your best interest in mind

terse folio
#

Maybe add tweaks ontop of that which the LLM can't do on its own

visual dagger
#

[character card]
"you are an AI assistant that cares about the user. ... etc"

but bcz of this ^ it will happily agree that you drop your medicine (which might be dangerous), it's not helping in this case, it's doing more harm than good

halcyon quarry
#

Reality I'm having a little issue that maybe you have an idea how to fix...

terse folio
terse folio
terse folio
halcyon quarry
#

This is failing to find the channel ID key in the dictionary even though it is there:

    async def get_channel_history(self, i=None):
        # If per-channel history
        if self.per_channel_history_enabled:
            if not self.session_history.get(i.channel.id):

This gives me an error when trying to convert the channel ID to a string.
I believe the string version of the channel ID would find the value in the dictionary.

    async def get_channel_history(self, i=None):
        # If per-channel history
        if self.per_channel_history_enabled:
            chankey = str(i.channel.id)
            if not self.session_history.get(chankey):
terse folio
#

ahh, a new branch, missed that

terse folio
halcyon quarry
#

Yes - help

#

😛

terse folio
#

Json will convert int keys to strings.
It doesn't support non string keys in dictionaries

visual dagger
#

per my testings no matter how much you change the system prompt it's not enough, the llm gets lost, I tried a lot adjusting system prompt but I didn't get results like implementing a logic or/and adding facts etc.. < that helped more than adjusting the system prompt

making a system/logic is the way to go, the llm is just a cog/part of the overall system

#

system or entity

terse folio
#

what's the error?

halcyon quarry
#

The error is simply the channel ID value

Screenshot_2024-05-28_130900.png
terse folio
#

we need better tracebacks

#

add a print(traceback.format_exc()) before that error message line

#

and import traceback if not already

#

because the code looks fine to me

#

it could be getting the channel history, but another part of the code is causing issues

halcyon quarry
#

Nah - shortly after, it adds a duplicate copy of the channel key to the history

#

then starts adding to it

terse folio
#

just downloaded the code

halcyon quarry
#

This issue only occurs if I load previous hsitory

#

Thats outdated

#

1 sec

#

Pushed update

#

The problem only occurs if autoload_history: true and you had a previous conversation while per_channel_history: true

vestal python
#

Imma end up setting up my discord bots up, but on hold working further for them :x about to lose my current job, but they're looking to hire me maybe for AI stuff. So I need to finish my webapp design as a proof-of-concept for them and then back onto finishing discord bot and back end textgen webui implementations..

halcyon quarry
#

If you see load_bot_history() (line 4802) - it is correctly getting the latest history as either a single format or as a per-channel format

#

self.session_history is being assigned the dictionary value of the history file - if you print it, you will see the prior channel ID as one of the top level keys

#

When a message is sent init_llm_payload() function is supposed to get the correct existing channel history via get_channel_history()

But it is not matching that key value

#

Pretty sure it will match so long as the channel.id can be converted to a string...

terse folio
#

what happens when you run get_channel_history with multi channel disabled?

#

oh

#

nvm

halcyon quarry
#

😛

#

I have literally everything else working...

terse folio
#

well ill test the bot and see what happens!

halcyon quarry
#

I'm not getting an error if I copy it first like this...

# 
        if self.per_channel_history_enabled:
            chankey = copy.copy(i.channel.id)
            print("chankey", chankey)
            chankey = str(chankey)
            print("chankey", chankey)
#

It's also not printing it with '' around it though...

terse folio
#

print repr(chankey)

#

yes, that's an int

halcyon quarry
#

Ill try to fstring it

terse folio
#

printing a string won't show the quotes

#

using repr() will show them though

halcyon quarry
#

ah thats good to know

terse folio
#

in an fstring you can use f'{var!r}'

#

as a shortcut to repr()

#

fun fact, you can define your own __str__ and __repr__ functions in a class
so when printing it, or using repr on it, they can give custom results!

halcyon quarry
#

This doesn't seem to be the issue - I'm missing something in my logic heh

terse folio
#

Does it return the correct history?

halcyon quarry
#

It's just the first exchange after loading, is not handling correctly

#

This is weird.

#

See:

    async def get_channel_history(self, i=None):
        # If per-channel history
        if self.per_channel_history_enabled:
            print("history:", self.session_history)
            chankey = copy.copy(i.channel.id)
            print("chankey", chankey)
            chankey = str(chankey)
            print(repr(chankey))
            if not self.session_history.get(i.channel.id):
                print("Not matched")
#

Here is the printed history:

#

It should 'get' that key but it is not

#

ok... maybe this is because it is nested down another level?

#

no - idk

terse folio
#

you're not using the string channel id, you create the string version, but forgot to swap it in the .get()

#

if not self.session_history.get(str(i.channel.id)) should do the trick ^^

halcyon quarry
#

Yeah well,,, lets see

#

😛

#

been back and forth with things... think I already tried it correct...

#

Now I'm back to that error. yikes

#

Alright.

#

I got it

#

the error was because later I tried returning the value with the i.channel.id key so that was a keyerror

terse folio
#

something's wrong up here, I think history didn't get loaded

#

not sure if related

#

but there might be some recursion in the history

#

or this is it just failing over and over

halcyon quarry
#

I am not caught up with Dev

#

which could be the issue

terse folio
#

Where is history saved btw?

#

oh, logs?

halcyon quarry
#

It is kept in the session_history variable, which gets written to and saved after every LLM response to yes, logs

#

I think I have this business worked out...

#

At the start of manage_history() I also just need to convert the channel.id to str

#

YES - Did resolve this mess

#

Now it is correctly getting the dict.

terse folio
#

👍

#

cleared the logs and still getting issues

halcyon quarry
#

hum

#

Did you merge this with dev?

terse folio
#

no, just using the history branch

#

hmm I think I unintentionally fixed it

halcyon quarry
#

Did find another error which had to do with the imported start_new_chat()

#

I need to just skip this and instead init history as an empty dict

terse folio
#

another part of the code was not running because a forgotten await
So that's returning a future object.

And llm_gen is complaining that history contains a future object when trying to copy it.

halcyon quarry
#

...where? 😄

terse folio
#

ill push it,
the reset_history function

#

huh, how do I get it to import the previous history?

#

on restart

halcyon quarry
#

In config.yaml need to enable autoload_history

#

await bot_history.reset_session_history(ctx)
Ahh yes this was missing await
Woops!

terse folio
#

Pushed

#

have them all true

halcyon quarry
#

When you start up and send a message / get a response, it should save a new log file.

If you close and open the bot again, send message, that previous log should reflect 2 messages

terse folio
#

the history is empty on start

#

it creates a new log

halcyon quarry
#

I'm going to fix this one tiny thing and push what I have - maybe you still have the channel.id trying to match the string

#

(which I did not push the fix)

terse folio
#

ah right, yes
I undid that for the commit!

halcyon quarry
#

Checking your modifications before I push...

#

load_bot_history() does not need to be awaited

terse folio
#

I converted it to an async function for consistency so it's easier to remember

halcyon quarry
#

Before I screw something up

#

Here's my version
It does not have your merges in it

📎 bot.py
terse folio
#

wish discord had a copy text from file button

#

The cool thing about git is you can undo commits!

#

it's fine

#

how come per channel history trims the history shorter? for flows

halcyon quarry
#

I have a separate copy of history that includes all messages including those ignored by history

#

Which you are probably going to cringe about 🙂

terse folio
#

interesting interesting,
Also noticed get_channel_history is called once and put to llm payload
But bot_history.session_history is accessed many times with [internal]

#

Am I missing something how does that work?

halcyon quarry
#

If the key does not exist when get_channel_history is called, the key is initialized.
Otherwise, the history for that key is returned

#

After llm_gen, the prompt/reply and channel_id are sent.
It again finds the correct key and manages everything accordingly

terse folio
#

okay, I see there's som if statements around about "single" or "multichan"
This is a little complicated to follow

halcyon quarry
#

I added that bit because it will load the latest history file it finds... which may not be structured correctly for the current mode

#

If the mode doesn't match the history that is loaded, it basically resets...

terse folio
#

Got it

halcyon quarry
#

could be possible to try harder to find the most recent history that does match

#

but seems complicated

terse folio
#

interesting, with everything in str() it still makes a new history file

#

i'll check it out later after you push the commit

halcyon quarry
#

Ok give me a minute to try merging your changes into this 😛

terse folio
#

just await reset_session_history 3 times was really important

halcyon quarry
#

OK I see that was it

#

Pushed

#

reverted load_bot_history to normal function 🙂

terse folio
#

on another note, I was working with some complicated history stuff for one of my own projects!
I did the idea I talked about, using a Message class for each history item.

That contains attributes like is the message finished, when was it created and so on.

Because i'm trying to work with real time messages through voice, handling interruptions and continuing messages that are not at the front of the history!
I'll share when things are more solid.
It had some really tough bugs at first

halcyon quarry
#

For consistency, I'm going to change the str version of channel_id to chankey

terse folio
#

I saw, was updating some other spots where it was missed as well

halcyon quarry
#

Well I have it convert to string, but kept the variable name as channel_id

#

and yes will update the hints to str

terse folio
#

should update typehints too on that, maybe trace it back to the source

halcyon quarry
#

OK will convert channel to str before sending to manage_history

terse folio
#

yup just did that :P

halcyon quarry
#

We're probably changing literally exactly the same as the other...

terse folio
#

ill undo my changes and focus on some other things I noted

halcyon quarry
#

Thanks for all your help, again

#

Pushed

#

Are you messing with send_char_greeting_or_history() at all?

terse folio
#

no, trying to figure out why guild_name comes up as a guild object in history

#

this happened while I was testing earlier

#

oh, that might have been my mistake

halcyon quarry
terse folio
#

I removed the str() from channel.id earlier to test what the bot does without code modifications

#

and when usign ctrlD I also selected the str of guild and channel name by accident without realizing

#

all good

halcyon quarry
#

I'm updating send_char_greeting_or_history() to work with the per chan history

#

er actually...

#

yeah, we're not sending anything if its per-channel 😛

terse folio
#

to make things easy to read,
I would set a default channel id like "0" if it's using single channel mode.
This way you only need to write the code once!

halcyon quarry
#

I think the main reason anyone would willingly keep it all as one, is for immediate compatibility with TGWUI

#

If I reformat it under a key then that won't work either 😛

terse folio
#

I see

#

wonder why tgwi doesn't support custom names for logs

halcyon quarry
#

It does - it just does not support this dictionary structure

#

I add the custom name to the logs just for easy identification

#

So at a glance you can tell which are incompatible'

terse folio
#

i would create 2 logs

halcyon quarry
#

On that note - going to see what ChatGPT has to say about writing a utility function

terse folio
#

one for the custom stuff (guild/channel name), and one for the messages per channel

halcyon quarry
#

If I delete the guild name and channel name, its still incompatible

#

It's because the visible and internal are nested under a key

#

The alternative is to have separate log files for every channel

#

I'm going to see about a utility function now to extract these logs into individual ones

terse folio
terse folio
#

to make it readable for the user, you could save it like:

channel.id_character_guildname[:10]_channelname[:20]

#

so you don't have infinitely long file names haha

halcyon quarry
#

At this point, it is staying as is

terse folio
#

but make sure to clean up the guild/channel names

#

yea

halcyon quarry
#

Just thinking of that approach is giving me a headache - and I like having it all bundled up nicely into one log file

terse folio
#

Yea, I can imagine

terse folio
#

here's a few things that will take a while to fix

#

discord.Interaction uses i.user
commands.Context uses i.author

#

you can get the correct version like this

#

or maybe just pass the username down as a different variable

#

maybe ill add a function to get username

halcyon quarry
#

Got the utility script working...

terse folio
#

you have way too many types feeding into these functions!

#

Message, interaction, context

#

All with their slight little changes

halcyon quarry
#

some extra details in the screenshot than intended

#

About the author / user thing

#

I don't think I actually have any errors atm

#

but, I know could be confusing 🙂

terse folio
#

I can't believe it works 😸

#

just in a constant state of almost errors

halcyon quarry
#

The thing that is odd is that i.author isn't documented... but seems to work universally EXCEPT for the Regen and Cont App Commands

terse folio
#

discord.Interaction doesn't have an author attribute

#

but hybrid commands send out commands.Context which does

#

only your menus won't work with i.author

halcyon quarry
#

I'll repeat- everything is working with i.author except for the Cont and Regen commands

terse folio
#

ill check if that's still true, my discord.py is a bit older

halcyon quarry
#

Those are the only 2 places I replaced it with i.user.display_name

#

The documentation doesn't seem to reflect it - was just a happy accident I found when I simply tried replacing all instances with i.author.display_name

#

on_message, all the commands, etc

terse folio
#

I think it's just a near miss,
I see nothing in the code about an author attribute

image.png
#

impressed it has worked so long!

halcyon quarry
#

yeah like, 2 weeks (that's when I made the sweeping update)

#

Like I said, I accidentally found that it worked pretty universally

#

They have so many alias and crap for things, it's no wonder they overlooked something like this

#

Pushed the commit including the utility if you want to check it out

terse folio
#

ill be pushing my changes to :dev

halcyon quarry
#

I know you just recently pushed a number of things to dev, a few days ago

terse folio
#

put in a little warning log to see if it ever triggers for i.user

image.png
halcyon quarry
#

I need help merging that and the per-channel stuff 🤯

#

This per channel stuff is good to go I think

terse folio
#

😸

#

nvm, forget menus are defined differently

halcyon quarry
#

yeah, this is still a bit bugged unfortunately. Dammit

#

It seems like I do need to collect visible.

#

maybe...

#

it errors when using extensions that want to modify history... such as TTS

terse folio
#

interesting, they shouldnt be related

halcyon quarry
#

When I turn on alltalk_tts I'm getting this error

terse folio
#

it wants to grab the last message and I guess replace it with the html tag

halcyon quarry
#

Ill try collecting visible and see what happens

#

I think I just need to initialize with 2 empty sublists

#

yep

#

no error now

#

derp

#

Yep, no error if I initialize with 2 empty sublists

terse folio
#

To explain some of my changes:

Typehints are classes (types), python classes use Pascal Case for example ExampleClass
So you can expect the last item of a typehint to start with a capital letter.
discord.message - was a file
discord.Message - is the class you wanted 😸

I think the same rule could apply to earlier with using discord.User.mention as a typehint.
If User.mention was a class, this would have worked ^-^

Also discord.py is structured weirdly,
discord.ext.commands is like a separate module from discord.

So discord.ext.commands.Context didn't work as a typehint.
But
from discord.ext import commands
commands.Context does.

#

pushed some cleaning!

halcyon quarry
#

Alright I'll see if I can figure out how to merge the channel stuff into this

terse folio
#

That may be a big merge!

halcyon quarry
#

think I just need to open the two files in split view

#

then just be very careful

terse folio
#

you should be able to create a pull request, it will merge as much as it can, and then give you a diff view to combine

#

create a new branch ofc to do the merge

halcyon quarry
#

Yeah, good idea. Will duplicate branch

terse folio
#

Also found out aiohttp has a .request method!
That just lets you pass the request_method like get/post as a variable ^-^

halcyon quarry
#

I see that you did change the labelling to ictx yes?

terse folio
#

For the mixed ones

#

i was changed to message where it was only coming from on_message

#

i was changed to inter in interaction only functions

#

but there's some cases where all 3 is possible,
so I called that ictx

#

and gave that its own typehint alias CtxInteraction

halcyon quarry
#

I need to run, but I duplicated dev to dev2, then tried merging per_chan_histories - and went through and resolved the few conflicts

#

If you want to try out dev2 that would be great - I'll be checking it out later as well

halcyon quarry
#

Ya missed one!

        if bot_will_do['should_gen_text']:
            # build llm_payload with defaults
            llm_payload = await init_llm_payload(i, user_name, text)
terse folio
#

it created some duplicate functions

#

should do a merge side by side

halcyon quarry
#

I see,,, announce changes

terse folio
#

and the function below it

halcyon quarry
#

and yep

#

change char task

terse folio
#

oh nvm

#

that's one I edited haha

#

it might have been added in the merge

#

because I would have caught that with error checking

#

this is helpful before doing a push 😸

halcyon quarry
#

I got that bit straightened out...

#

(announce, change char task, etc)

#

Pushed fixes to dev2 🙄

#

Looking pretty good now...

terse folio
#

happy that worked, I was messing around trying to figure out how to merge 2 branches without automaticly merging.
Like making everything count as a conflict for manual resolution

#

Can compare 2 branches and get the diff view I want, just not to select left/right or edit them in the previews

halcyon quarry
#

There is the compare files thing in VS Code

#

It’s wha I should’ve done really

terse folio
#

File compare seems to only work to edit the latest, which is great.
But you need 2 files to compare in the first place

Maybe if you cloned the repo into 2 folders running different branches it could be done that way allowing edits

halcyon quarry
#

Yeah that’s what I meant

halcyon quarry
#

So I had changed visible to have 2 empty sublists and it worked

#

What I didn’t realize was that it fails on the next gen lol

#

I’m going to test adding empty strings so the internal and visible lists have matching lengths

halcyon quarry
#

Yep - appending empty strings resolves any error.

#

I suppose at all times, the internal and visible lists are expected to have matching lengths

halcyon quarry
#

doing one last run of vigorous testing then pushing this bad boy to main

halcyon quarry
#

Yeah we are a go

#

New setting in config.yaml enabling each channel to have its own separate chat history.

Compatible with the other history settings. The only downside really is that the log file cannot be directly loaded into text-generation-webui. However, a utility can be found in new directory /utils/ where users can Drag n Drop one of the _multiple-history.json files and it will split it into compatible logs.

New /announce command

The command allows users to set channels as "announce channels". If any announce channels are defined, Model changes and character changes will be announced there instead of in interaction channels. This makes it easier to inform users in all channels about things changing with the bot.

keen palm
#

How's regenerate coming along? 🙂

halcyon quarry
#

Might be next on the list

halcyon quarry
#

Ok I thought of a very good idea for handling Regenerate/Continue - which is also going to simplify my “Recent messages” handling (different from History)

#

Im going to collect message.ids into lists matching same structure as History and account for ones not in history. Will be able to get the corresponding messages needed / edit history for the commands

visual dagger
#

hey again fellas

#

@keen palm how is it going with the game?

keen palm
#

Not going anywhere right now. Regeneration is kind of the last piece of the puzzle needed for the bot

halcyon quarry
#

I’ll be working on that today

keen palm
#

Do you know of a way to put information into context without adding it to the ongoing history? The Complex Memory extension (which seems to be b0rked now) did that with keywords, where the information would be injected into the beginning of the context. Silly Tavern does a similar thing with world books. Same with Novel AI and their lorebooks.
The issue with using tags to inject information via trigger words is the information remains in the history, so it can rather quickly overwhelm the available context, especially if it's more commonly triggered information.

halcyon quarry
#

Unsure if I mentioned this to you or someone else, but I plan on added User Variable assignments.

So you could assign something to a variable, then use the variable in your prompt

#

It's on the TODO List™️

keen palm
#

You did mention that, but I don't know if that would fix the inherent problem

halcyon quarry
#

Ah yeah... you're right about that

#

I think what I need to do is add a specific handling for what the state tag is used - if it includes custom_system_prompt, to add them together rather than update existing value

#

Or, make yet another tag like add_system_prompt which will add to whatever the current custom_system_prompt is

#

To my understanding, the system prompt handles the behavior you're seeking

keen palm
#

Yeah, there has to be something for that

#

Maybe need to dig into the code for complex memory and see how that does it

halcyon quarry
#

You could now use custom_system_prompt but it will set it rather than combine with others

#

[[state:{custom_system_prompt:Bob is a raging alcoholic}]] What's Bob been up to lately?

keen palm
#

I'll give that a try and see how it works

halcyon quarry
#

Adding to to-do list: Append system prompts

keen palm
#

That does say in the command window that the state has changed, but the system prompt is ignored

halcyon quarry
#

Chat mode or instruct mode?

#

Shouldn't matter... hum, seems like the default chat_template_str defines how the system prompt should be handled.

#

If it's having no effect then that's a bit puzzling, unless the chat template is being ignored or something

#

I had to fix a few things but now I'm onto Regenerate

keen palm
#

This is the context modification code in complex_memory, BTW:

context_injection_string = ('\n'.join(context_injection)).strip()

if memory_settings["position"] == "Before Context":
    state["context"] = f"{context_injection_string}\n{state['context']}\n"
elif memory_settings["position"] == "After Context":
    state["context"] = f"{state['context']}\n{context_injection_string}\n"

return generate_chat_prompt(user_input, state, **kwargs)
halcyon quarry
#

The difference between this and custom_system_prompt (if it were working), is that custom_system_prompt would vanish from history

#

actually, nvm

#

obviously context does not get written to history 😛

#

Instead of dicking around with system prompt I'll just copy paste this as inject_context

#

open to suggestions on how to name it

keen palm
#

For tags purposes?

halcyon quarry
#

I'd need to create two tags in any case such as
inject_context_text: The thing you want injected
inject_context_mode: before (or after)

Instead I'll just do this
prefix_context: your text
suffix_context: your text

#

Yes, will just add params to the Tags system

keen palm
#

That would be the easiest, yes

halcyon quarry
#

Quite easy

keen palm
#

I don't know whether prefix or suffix is better in use, but whatever

halcyon quarry
#

Just gives it more or less priority than your actual context

keen palm
#

Yeah, I know. Just not sure which is preferable. Maybe important character information should be weighted higher than current context

halcyon quarry
#

May need to guess and check that - I'll add both

keen palm
#

Right, I'd play around with it a bit

visual dagger
#

but you have to include the memory info in a way that's easy to remove later

#

something like sections?

# 
+ info 1....
+ info 2....
etc ....

###John response
I will be right back

This is the full prompt. So after the generation you can easily remove/edit the memory info using .split()

keen palm
#

It would be far better to have that info not ever added to history, though

visual dagger
#

I think there is a post processing thingy in ooba, you gotta check the source code

#

I think you can clean things up before adding the response to the history

halcyon quarry
#

I'm adding this feature today.

visual dagger
#

you are DA GOAT

#

@keen palm NPCs... did you try this in any way shape or form?

keen palm
#

In what way do you mean?

visual dagger
#

like hmm.... interacting with a random new character, a one off interaction?

#

and also maybe saving that NPC to interact with later

#

or any other way

keen palm
#

I haven't done that so much with this bot, but I have certainly done so previously when using complex_memory.
With the bot, when the context injection thing gets up and running, saving an NPC for later would be as simple as creating a new tag for it.

#

But interacting with a random new character temporarily I have absolutely done countless times.

visual dagger
#

generate that NPC on the fly?

#

on real time?

halcyon quarry
#

Predefined

visual dagger
visual dagger
#

ouch

halcyon quarry
#

You know by now that the bot can't currently save information locally - don't hold your breathe as if this feature may only come if Reality figures it out, or I finish my todo list and start looking for new things to try

visual dagger
#

saving things is easy, one .py script away

#

or functino

halcyon quarry
#

The most it can save and call back atm is recent messages, and eventually user variables

keen palm
visual dagger
#

the problem is the logic, saving things for the sake of saving won't result in good results

#

but there gotta be a reason for it to exist (the saving feature)

#

like how it will be used in an effective way?

visual dagger
keen palm
#

Unless the NPC information is stored in a lorebook or something similar, then it gets pushed back in context until it's forgotten, basically

visual dagger
#

@halcyon quarry do you think of making an LLM conditionned function in the bot?

#

ifLLM("John is a teacher")

#

it will return true or false

halcyon quarry
#

No