#ad_discordbot (Fork of Fork of xNul's bot)

1 messages · Page 9 of 1

visual dagger
#

it will be neat to be able to use an if condition powered by an LLM in code

#

it will be truly insane

halcyon quarry
#

With Variable assignment, you probably could

#

in combination with the Flows system

keen palm
visual dagger
#

so you are automatically deleting the NPC and all the info about it aftewards?

visual dagger
keen palm
#

If I don't store the NPC information in a lorebook, then information in history about it would be gone as soon as it's no longer within the context window.

halcyon quarry
#

You can already use responses from the LLM as input for tag parameters.

The LLMs response can also trigger Tag triggers.

visual dagger
#

assigning what to what ?

halcyon quarry
#

You can also use the LLMs response already as a variable such as {llm_0} is most recent response

visual dagger
halcyon quarry
#

A fact is something you predefined, yes?

visual dagger
halcyon quarry
#

Or is it something the LLM defined that we're just going to assume is a fact

visual dagger
visual dagger
halcyon quarry
#

Yes, you can make a tag definition that can be triggered only by the LLMs response

visual dagger
#

if I would extract facts from LLM responses, I will count them as facts

halcyon quarry
#

search_mode: llm means it only searches the LLM's response for matching triggers

visual dagger
#

and update the db

#

what do you think is a better approach llm only, user only, or both

halcyon quarry
#

Depends entirely on the situation / use case

#

I need to stop chatting about this or I'll never finish enhancing Regenerate and Continue 😛

visual dagger
#

hold up, hands up

#

what are you working on?

halcyon quarry
#

Currently, Regen / Continue are only working one time before failing - thats a bug.
Second, they are currently based entirely on the most recent interaction without any ability to manipulate

#

What I am doing is making is so that the message you right click on to use the App Command, will be found in the History and used as the input

#

And also, not error when using it repeatedly

visual dagger
#

you mean inseting a predefined message when clicking regenerate?

keen palm
#

Stop asking questions and let the man cook! 😄

halcyon quarry
#

currently, you right click on any message and hit Regenerate - what happens is that it uses the current value of History as input.
The message content isn't doing anything to sway it

visual dagger
halcyon quarry
#

What needs to happen is to jump back in History (enhancement)
But importantly, also not break after one use (fix the bug)

visual dagger
#

you mean editing the history?

halcyon quarry
#

With this enhancement, it will also have the framework to edit history

visual dagger
#

wait, Regenerate doesn't go back in history by default?

halcyon quarry
#

Regenerate will not edit the history, it will temporarily jump in history without modifying

#

It goes back in history by default - to the most recent exchange

visual dagger
#

go cook, and make it delicious chef

halcyon quarry
#

What I am going to do is simulatneously collect message.id in lists at the same time as History

#

When the appcommand is used, the message.id is accessible.
I can match the message.id and use it's index to fetch the appropriate history

visual dagger
#

retrieving another msg from history?

halcyon quarry
#

The message

#

This will be much more effective than trying to match the full text string in the history

visual dagger
keen palm
halcyon quarry
#

You're here to sabotage me 😛

#

You ask the LLM a question and it says Yes.

10 messages later you ask a question and it says Yes.

You decide to regenerate a response which was Yes.

Where is it going to find it in history eh?

#

And it's going to take more computer to match a 1500 character text in history than by a 13 digit number

visual dagger
halcyon quarry
#

Hey there 👀

visual dagger
#

OI!

terse folio
#

That's a lot of processing for possibly random results (in my experiences 7b can't make such decisions very well)

if your only scope is characters, then creating a character object that can store attributes like "teacher" flag.
then you could use character.get_is('teacher') -> bool

These tags could be extracted by an llm, or through more strict logic.
I think extracting would be more reliable because asking "is john a teacher" and john is not a teacher, the llm might say yes, because you mentioned teacher in the first place giving it a bias.

terse folio
visual dagger
terse folio
#

sure

#

my suggestion was having an llm extract data it observes rather than asking if the llm thinks something

visual dagger
#

I didn't get it 🙂

terse folio
#

Conversation:
John: okay classroom, today we're discussing my starchy nemesis, the tuber!

based on the following conversion, describe john:

  • Possibly a teacher
  • doesn't like potatos.
#

saving these facts might work better than asking an llm each time

keen palm
halcyon quarry
#

🥔

terse folio
#

asking llm: do you think john is a teacher?
I feel like the result could be pretty random.
because the question is kind of guiding it towards saying yes by mentioning he could be a teacher.

#

on the other hand, if the llm is just asked to collect info, it's not being guided by the question, and Might™️ give more truthful information

#

Should try that out with the logit probabilities in tgwi

#

see what it thinks

visual dagger
#

Possibly a teacher doesn't like potatos.
so this is the llm response?

visual dagger
terse folio
visual dagger
#

I mean you meant llm response?

terse folio
#

yes, ideally

visual dagger
#

yup yup

terse folio
#

Even multishot prompting can risk contaminating your data

#

giving it examples, the llm might think those examples directly apply to John and repeat them

#

I've run into this problem with the tools experiments

halcyon quarry
#

send_long_message() function will now return a list of message IDs it sent.
Then, the message ID from user message and the LLM Response(s) will be appended to lists.
Simultaneously, a 1 or 0 will be appended to another list representing whether it was saved to history or not

Screenshot_2024-05-29_134239.png
visual dagger
#

yes I run into those problems too

terse folio
visual dagger
#

providing examples will cause those examples to bleed in the actual answer

#

and it confuses the LLM

#

specially if there is words that are the same in an example and also in the prompt

terse folio
visual dagger
# visual dagger and it confuses the LLM

for example

one example having classroom and the prompt having classroom too, the llm will link the example's classroom to another topic which will lead the llm to reference or put focus on that unrelated topic and the llm will get confused and the reply wlll be weird

terse folio
#

I can take a stab at that if you push it

halcyon quarry
#

If I have one accidental pop somewhere - just need to patch it 🙂

#

I have faith in this

visual dagger
#

who doesn't know that John hates potatos

terse folio
visual dagger
#

: |

terse folio
visual dagger
#

how come an AI not know that John hates potatos

#

I know it, and am a human

terse folio
visual dagger
terse folio
#

had to mention evil for it to recognise that

visual dagger
#

so you used the tool right

terse folio
#

Solar10B is pretty confident John is a teacher

visual dagger
#

Solar10B is THE AI

terse folio
#

And a random llama2-7B model also is pretty confident.
Great!
That might be more viable than I was thinking!

image.png
#

But he could also be a student for the same reasons.

#

like I said, be careful what you ask 😸

#

that's a good answer

image.png
visual dagger
#

any idea on planing? it will be good to mix facts with priorities and make aplan then execute on it somehow without the llm getting confused

this is not the llm answer per say, but the llm answer will make you imagine that the llm is thinking

are we going to school or the mall what this human is talking about? I am tired of dummy humans that can't talk LLM language

visual dagger
terse folio
#

Maybe the students switch roles and teach him how to properly prepare a dish with potatoes and change his outlook on life

visual dagger
terse folio
visual dagger
terse folio
visual dagger
#

why turtoring the llm, where is your empathy

#

: (

terse folio
#

it was for science!

visual dagger
#

in that case...

terse folio
visual dagger
#

one way to solve that is to NOT include internal thoughsts

#

I mean yes prompting them BUT not saving them

#

they will be like a temporary cache

#

but not sure how to optimally do that

terse folio
terse folio
visual dagger
#

oh, it doesnt mix it up with the character reply?

terse folio
# visual dagger oh, it doesnt mix it up with the character reply?

Yea,
originally, it was x: reply then run another llm generation x's thoughts: reply

Sometimes through random chance it would generate thoughts before reply.

And there was a periodic loop that just generated thoughts randomly after nothing has been sent in a while.

maybe to help start a new conversation

visual dagger
#

maybe this can work

  1. first making a todo list
  2. the LLM will think about the todo list, it will generate ###Thinking and ###Reply
  3. remove the ###Thinking section and saving only ###Reply
terse folio
#

what would the point of ### thinking be if it doesnt influence the next generation?

visual dagger
terse folio
#

my goal was to give the bot it's own unspoken ideas/emotions that would stay present throughout and change as things go

visual dagger
terse folio
visual dagger
#

maybe getting rid of stuff in the todo list one by one?

#

after having done them

terse folio
#

sure, this wasn't setup as a todo list

#

but that's a cool idea

visual dagger
#

making things coherant is tricky

#

imagine it like going back in time, You are interferring with history, that event of history should change, but not change to a point of being confusing or glitchy

#

the event still happens even that You are interfering by being a time traveler

#

but the event and everything still happens in a way that makes sense

#

that how I think about it, the plan is a foreign thing to the current scene, so it might mess things up

terse folio
#

Also I found a screenshot of the internal monologue being used by the LLM properly.

I think it's funny it has it's own different opinion than what it's saying.
Which then influences later replies to take on a sarcasm like tone I believe is the best way to put it.
"of course not! im just a robot."

image.png
visual dagger
visual dagger
terse folio
#

but it got it coherent by generating slow and taking it's time to let the user input messages

visual dagger
terse folio
#

I love seeing it start updating it's reply, steering it in a new direction as new information comes into the past

terse folio
#

would have to revisit that with more modern models and see how it goes.

#

This one often got confused who it was

visual dagger
visual dagger
#

don't let it suffer 🥹

terse folio
terse folio
visual dagger
#

and updating on real time

terse folio
#

this is running at 6tps on purpose

terse folio
visual dagger
#

how could you do that

#

making it coherant

terse folio
#

The text you see on the right is the live results from the llm.
That is the actual prompt being fed in

#

The llm is able to predict what you're saying and start forming a reply ahead of time

#

like me starting to say "why did it"
The llm already understood there's some technical issue, because I talked about voice chats

#

Also, I mark incomplete messages with --

#

and in the system prompt tell if that this means the user/bot is still typing

# 
You are talking to me in a voice channel.
The user will end unfinished sentences end with: --
This indicates they are still talking while you reply.
Keep it short.

is pretty much it

visual dagger
#

but do you regenerate each time or how?

#

how you mark the LLM response as permanant

#

by LLM response I mean as far as it went, since it's live

terse folio
visual dagger
#

what were you doing before?

terse folio
terse folio
visual dagger
#

the llm makes the decision of flagging it as finished?

#

you set a special token for that?

terse folio
#

What I mean is,
because chat from the user is broken up, the llm might overlook something.
So i should also hide the llm responses and treat the last few user messages as a new input and add that to the history.

terse folio
#

most importantly:
user, text, typing:bool, spoken:bool, trimmed:bool

#

typing indicates the message is still being added to.
Spoken means the AI ran it through TTS already

#

Trimmed means the message will be in a permanent state of typing, because the bot cut off the user midway.

#

and this influences how the string is formated

visual dagger
terse folio
# 
Message:
    user: str
    text: str
    typing: bool
    trimmed: bool
    spoken: bool

History:
    msgs: list[Message]
    current_typing: dict[username: Message]

When user input comes in, it checks if there is an existing message in history.current_typing[user]

if so, Message.text += chunk.
If that chunk ends the sentence, Message.mark_complete()

if more sentences are present in the chunk, add new messages to history.msgs.
And if last message is incomplete, add it to current_typing!

visual dagger
terse folio
#

This way of storing messages is something I want to eventually implement with Ad_discordbot

the history class would have a render method that outputs the message dict for tgwi.
and easier manipulation of messages/sorting!

terse folio
visual dagger
#

nani!

terse folio
#

was using a smaller prompt so I could ensure everything was being formatted correctly with verbose mode

# 
<s> Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.


### Instruction:
You are talking to me in a voice channel.
The user will end unfinished sentences end with: --
This indicates they are still talking while you reply.
Keep it short.

### Response:
# Voice chat:
User: "Hello this is a test!"
AI: "Hey there!"
AI: " How can I help?"
User: " I am a second message."
AI: "No problem, let's see--"




AI: "--if we can<continue here>
#

I added this spacing in for you

#

the messages above are the ones in memory, the last one is a guide so the LLM tries to continue its last

visual dagger
#

AI: "figure out a solution!"

llama3 8b

#

response/continuation

terse folio
#

remove the <continue ofc>

#

yup that sounds about right

visual dagger
#

AI: "What's the plan if we can, though?"

terse folio
#

this was the final output for me with the llama2-7b model I was expirementing with earlier

image.png
visual dagger
#

with <continue here> was better

terse folio
#

huh interesting, are you using chat mode or default?

visual dagger
visual dagger
terse folio
halcyon quarry
#

I'm such a slow worker

#

this is coming along though

keen palm
visual dagger
#

so idk what they set it on

#

oh the model is llama-3-8b-instruct

#

I didn't pay attention

terse folio
#

yes, for instruct, you probably want to tell it <continue here>

else it will start creating a new response like the issue I was having.

#

I am sending the prompt to v1/completions

all it does is continue the next token ^^

visual dagger
#

you can see that on one response it uses capital while the other not

#

lower case/upper case as a start

terse folio
#

mhm, because it's getting a prompt like this internally:

### instruction
continue this:

AI: "No problem, let's see--"
AI: "--if we can

### Response:
visual dagger
#

prompt inseption?

terse folio
#

compared to doing it yourself as:

### instruction
continue this:

AI: "No problem, let's see--"

### Response:
AI: "--if we can
terse folio
#

you don't have to think about prompting for text completion, rather you can talk to it like a person!

#

Do you know of the setting in the TGWUI for "always start your reply with X"

visual dagger
terse folio
#

if the model is censored, it might be beneficial to make the ### response section always start with "Sure!"

#

that's all :)

visual dagger
#

I force the LLM to start with something when using llama.cpp

terse folio
#

here, I force the llm to start with its last reply

visual dagger
#

the cli

terse folio
#

mhm mhm

visual dagger
#

some models do that , lol

terse folio
#

they trained it really hard to do that

visual dagger
#

meta, they gotta cover what left of their reputation

terse folio
#

mhm, it makes sense, censor the instruct model that businesses would use.
But still release the base version without censoring (at least they did with llama2)
because now it's the people's fault if they finetune it

visual dagger
#

why did they require the name TOS thing?

#

for llama3

#

what if someone finetuned to do harm

terse folio
#

I imagine so people don't steal credit or maybe use that to bypass tos?
By saying "well you can't prove this is your model"

visual dagger
#

probab they can by slicing it

#

an operation!!

terse folio
visual dagger
#

a bloody surgery

#

yeah but people will do people stuff

#

and one of people stuff is not understanding AI

terse folio
#

mhms

visual dagger
#

google seems done (AI wise), they mess things up lately a lot

terse folio
#

I've been using DDG for the past few years, it works perfectly fine for all my research needs!

People just need to know of alternatives.

visual dagger
terse folio
#

Actually a few days ago, they were having network issues possibly?
and I had to use Google a few times, was surprized to see the Ai feature

#

I havent been following that much

visual dagger
#

yes bcz of bing

terse folio
#

I see I see

visual dagger
#

microsoft bing went down

#

and brung down DDG with it

terse folio
#

Interesting, didn't know they were so closely tied together

visual dagger
#

DDG is based on bing search engine/API or something

visual dagger
#

and it gave me some uncomfortable feelings : /

terse folio
#

mhm, odd for a search engine meant to not spy on you

visual dagger
#

DDG and microsoft,,, what ? how that works?

#

my internal thoughts was back then ^

#

and I find it weird why mc would allow that.. maybe solely for financial reasons?

#

I assume they pay mc millions if not billions to keep search up and runing

terse folio
#

:O hmmm

#

I'll be back in a bit!

visual dagger
#

🫡

halcyon quarry
#

@terse folio Easter egg found

Screenshot_2024-05-29_150943.png
keen palm
#

Lol

terse folio
#

Ahh I see, that's for the regenerate command, yup yup

#

You can remove that line 😸

halcyon quarry
#

Making more progress... there are things I did not take into consideration with this approach 😐

terse folio
#

Like what?
i'm working on some new classes for history, if you have any ideas that might make it easier

halcyon quarry
#

Please don't make me merge them 😭

terse folio
#

it's in its own file, i would take my time

halcyon quarry
#

Before I started, I tried moving History into a separate module but couldn't figure out how to also access bot_settings from there

terse folio
#

initialize the class in the main file, just define the class somewhere else

halcyon quarry
#

Anyway - I'll have to reconsider my approach to assigning variables for the Flows system, because with this approach (linking message IDs to history prompts / LLM responses), there are internal messages that obviously do not have message.ids

#

Furthermore

#

I can't just initialize the message.id lists as empty while loading history

terse folio
#

maybe start it with None, None for now

halcyon quarry
#

I'm going to use "I" on all this because I am making a mess of things 😛

terse folio
#

ohno

halcyon quarry
#

So - I could additionally write and load these lists to the log files.

#

Which I think will be ideal solution.
The floating problem would then be Flows system

terse folio
#

adding some features like to create a temporary history up to any message

halcyon quarry
#

I'm excited about this 🙂

#

I pushed my progress to a new branch, id_pairing

halcyon quarry
#

I kinda ran outta time but what’s wierd is that I’m fairly certain I sliced the history and updated the payload param… but it still regenerated based on most recent exchange

#

the Regenerate param may be hardcoded, maybe TGWUI remembers or something

#

May need to manipulate history then just send it like a normal payload rather than with regenerate

terse folio
halcyon quarry
#

The logic of trying to match the message IDs to the history even though the history could persist between sessions wireless message IDs resets this may still work out if searching for matches in reverse and then slicing the index in reverse or something

#

I think the idea has merit but I may be biting something too big to swallow

terse folio
#

how would you match the ids to what's in history?

halcyon quarry
#

The way I have it there’s basically 3 lists.
History (user, llm)
Message IDs (user, llm)
History tracker (if message was stored to history or not)

If a message is not saved to history, it appends 1 to the tracker.

#

I made a function that will filter the Message IDs list based on the tracker list

#

It is correctly slicing the history based on where it matches the current message ID to the index in the stored message ID list

#

But this is on a fresh history. Anyway, it’s not possible for the filtered message.id list to be longer than the history list. But history will be longer if bot loads up with existing history

#

I think I can resolve the issue of Flows by just inserting dummy IDs when it won’t be sending a message to channel

#

Idk. Too complicated

#

I think what I’ll just fix Regen and Continue for now without trying to enhance them, need a mental break from this idea

terse folio
#

I see, yea that is complicated

#

hope my update will make things easier to work on

#

I just need to figure out saving with recursive references

halcyon quarry
#

I think I can make something happen if I just dial it down a bit

#
  • I could abandon the idea of caring about messages not saved in history.
  • instead of collecting message.id for both user and llm resp after the send_long_message() I could instead just capture the user msg ID
#

the point of this idea is to be able to edit history by right clicking a message in the discord window and using an AppCommand

terse folio
#

will that update a message in the middle or start a new branch at that point?

halcyon quarry
#

I could have it send ephemeral msg if it’s used on a bot reply, and say it’s intended only for user msgs

#

Well the first step is finding the corresponding msg in history

terse folio
#

I can do that ^^

#

also would per channel saves be fine? I think it would be easier to implement

halcyon quarry
#

I was thinking of an AppCommand like edit history - after you do it if it finds the match in history, it asks a followup on what to relace it with

terse folio
#

you could also listen for the on_edit event

halcyon quarry
#

I’m trying to make this less complicated lol

terse folio
#

trust me, it will be easy!

history = historymanager.get_for(channel.id)
entry = history.get(message.id)
if entry:
    entry.update_text(message.content)```
would be something like this
#

Also I think I have a solution for importing history to tgwi

#

just save 2 files

#

if the user opens the history in tgwui and continues chat there, it won't match the other log.

Maybe a converter could be written.
Or the new history could be ignored and only use what the bot knows.

halcyon quarry
#

The multi chan history logs are all saved with a _multiple-channels suffix

#

If loading in multi can mode could be a good idea to try filtering on that

terse folio
#

what I mean is, there could be a normal version of the logs that only contain internal/visible for the user to view in tgwi if they wish

halcyon quarry
#

Or filtering it out if boooting up in single chan mode

terse folio
#

so no translation is needed if the bot will be the only one really using those files

halcyon quarry
#

Even if there’s just 2 channels having conversations, the merged version would be all over the place

terse folio
#

hmm hmmm

#

new idea!
Store uuids for each object,
Each message, each history branch.
And save the references to eachother as uuids that will be loaded to the real objects on load.

#

this isn't a json save or anything, just printed the objects converted to dicts.

image.png
terse folio
#

interesting, everything is stored as a pair

#

how does chat mode handle multiple users talking to the bot?

halcyon quarry
#

On the todo list is that experimental feature I discussed before

#

Currently, name1 is set as the user. So for each interaction the message log appears to the character as they’re chatting with that user

terse folio
#

I see I see,
when I do chat with bots, I create the prompt myself which lets me insert my own names

#

i can add a history renderer for the different modes if you ever want to try that

halcyon quarry
#

I think it could be beneficial to use the server name as name1, and prefix each message with the username

#

I tested it briefly and it seemed to help

#

I plan to have a few things like that wrapped into a “server_mode” setting thats true/false

#

I’m going to avoid toying with History any further until you finish up what you’re doing 😛

#

In the meantime, will just fix Regen and Continue to actually work

keen palm
#

How difficult will it be to implement the context injection idea? Looking forward to trying that out

terse folio
#

using that you could delete all instances of context, store them in another list or do whatever

halcyon quarry
#

Will feel good to accomplish something instead of spinning my wheels

keen palm
terse folio
#

since visible does this to the text i'll stick to using internal for custom rendering!

halcyon quarry
#

almost done with it - had to take a break

#

testing

#

Works.

Note that this does not do do anything for instruct mode

Screenshot_2024-05-29_204640.png
#

@keen palm Those tags are now available for use

keen palm
#

Ooohhhh!

#

Can't test yet. Dinner time

halcyon quarry
#

The beauty of the tags system

#

took 5 minutes to add this

terse folio
#

😸

halcyon quarry
#

well, it probably wont work. Guy in video says 24gb vram minimum >:\

#

Welp, so far I can at least generate one of those snazzy openpose videos lol.
Now to see if I can infer the actual video gen

keen palm
halcyon quarry
#

Yes, below state in the LLM section

#

I also added them in the wiki

#

So I'm able to render musepose video except only @ 200 x 350 it seems 😛

keen palm
halcyon quarry
#

Output was also not good

#

Bah

keen palm
#

[[prefix_context: 'Bob is a piece of toast']] -- Correct?

halcyon quarry
#

yes, the quotes are optional though

#

it can detect strings / floats / bool (true/false), lists, dictionaries pretty reliably

keen palm
halcyon quarry
#

I had no errors... let me try

#

sec...

#

I'm going to blame chatgpt for this one

keen palm
#

lol

halcyon quarry
#

I wrote it correct and then it told me a different solution that borked it.

#

My test succeeded because I had 2 things. It is failing on just one.

keen palm
#

Ahh yeah, it works if both are included

#

Stupid chatgpt

halcyon quarry
#

Pushed the bug fix

keen palm
#

Oh yeah, push it.
Ahem, 'scuse me

#

Doesn't look like /reset_conversation is getting rid of history

halcyon quarry
#

per-channel history?

keen palm
#

Right

halcyon quarry
#

Ur right

#

wtf

#

Now I'm blaming Reality

keen palm
halcyon quarry
#

Had to merge into his code revamp

#

I have the fix, pretty sure.

#

Pushed the bug fix

#

I noticed something didn't look right today when I was working on the other thing - but I did not commit that change.

#

I think I inadvertently rolled back something when merging our 2 branches together. The channel ID was not matching the dictionary value unless it was first converted to a string. This was addressed but apparently was reverted

keen palm
#

Alright, reloading. LEt's see if I can break something else

#

So far so good

#

Now to redo all the tags in my gamemaster character file

#

Oh, actually, continue might not be working. The bot repeated the same paragraph like 8 times, and nothing new was saved in history

halcyon quarry
#

Regenerate is actually working now right?

#

I only had a moment to test it earlier and it seemed to be working, repeatedly. Continue looked like it was not.

keen palm
#

Haven't tried yet

#

It is working slightly, though history still isn't changed to reflect the new generation. Also, there's a bit of an issue with regeneration whenever the initial message was split across multiple responses in discord

halcyon quarry
#

If I can finish what I started and make it work, it should handle the split messages as well

halcyon quarry
#

I’m going to wait for Reality to wrap up what they’re working on regarding History, though

terse folio
halcyon quarry
#

Ok! I’ve had some time to think about it and I think all issues with the id pairing can be resolved.

#

There’s just likely to be a few headaches between now and then is all

terse folio
#

i'd keep the dev branch up to date.
It seems when the same things are worked on, it causes the most issues.
when merging branches on Git, it likes to merge code that doesn't conflict, but that doesn't necessarily make it correct

halcyon quarry
#

It’s deleted at the moment ao if/when either of us has commits just open again

#

If we have another big merge situation I’ll do a more thorough file comparison rather than just reviewing conflicts

terse folio
#

I think those merges need to be done manually.
Because a few lines of code got added from the old branch that used outdated variables that were removed in the new

#

yup

keen palm
#

Would incorporating delete/replace last response commands be easier than sprucing up regenerate?

halcyon quarry
#

This system would make it easy to manipulate history in any way

#

May add those commands beforehand though

terse folio
#

history saving/loading works ^^

# 
History(
manager='9b6e9b7f1fde4988b1bdc8cfc9c9e5b1', 
id=100, 
_last={}, 
_items=[<Message (user) by 'Kat': 'Hey!', Replies(1)>, 
<Message (assistant) by 'Skye': 'Hello Kat'>], 
uuid='3e14fea7f35f478e821fb3f438b092f2')
halcyon quarry
#

Is this a new logging format?

terse folio
#

this is an internal reprosentation of the classes used for history

halcyon quarry
#

Got it

terse folio
#

you can query different formats through render functions

#

{'internal': ['Hey!', 'Hello Kat'], 'visible': ['Hey!', 'Hello Kat']}
( I know this isn't correct yet)

# 
Kat: Hey!
Skye: Hello Kat
halcyon quarry
#

Looking forward to playing with that

terse folio
#

Looks promising

terse folio
#

Is this just user inputs?

#

those are some confusing lines!

#

ohno
Does this mean you were switching the whole self.session_history every time someone sends a message in a new channel!

visual dagger
#

I am having some success with this method I tried lately:

  1. Already having a structured kinda big yaml data about characters, places, and other details etc...
  2. the python script checks what places and characters has been mentioned in the past like 6 chats and puts them into a characters = [] and places = [] lists
  3. those two lists > characters and places gets put to a prompt that says something along the lines of This is the yaml data #I put yaml data here. Extract all informations about the following characters #characters-list and the following places #places-list, all the yaml data is included in prompt too, the prompt also instructs and foces the llm to make connections between characters and other characters & places and other places & places and characters like a how you Reality doing the similarity search using verctor dbs, but I used an llm for that, so the prompt have like this instruction You should make connect a character with other relavent characters, places to places, and characters to places and put the results into sections, in this format ##Characters, ##Places
  4. the result of step 3. is each section have "sub sections" of character A B and Places X Y Z etc..
  5. that result/analysis will be included in the user prompt, pre prompt, so when the LLM (main chat) answers it will have quite a lot of juiced, narrowed down and clear informations about the relavent characters/places to the current discussion
  6. the user prompt gets cleaned, so ONLY the user prompt gets saved to the chat history, not all that bloted amount of info, if all that info is saved to the history chat it will cause an insane amount of repetitions in the future and it will cause the llm to be dumb or confused
#

@terse folio thank you, for the similarity/relavency concepts you explained to me when we were discussing about the vector db

visual dagger
#

if you have noticed any loopholes or bugs in this method please tell me, I am still testing things out

terse folio
#

Btw!
you might benefit from the history code i'm working on too.
I'll put it on github as it's own module with some examples of how to use it.
And would be happy to get collaboration from others there to improve it!

visual dagger
terse folio
#

Yup

#

we need to follow the typehints, otherwise they'll confuse us even more!
@halcyon quarry

image.png image.png
#

@halcyon quarry question about this:
This is used in regeneration/continuation.

Was this meant to trim from the right to regenerate a specific message in the middle of the history?

image.png
#

test for sanity check

image.png
#

i'll keep the original behavior for now as im unsure what it does

halcyon quarry
terse folio
#

Yea, I remember now ^^
it dumps the channel ids + internals to the file

halcyon quarry
#

The cp_list / collected prompts is not implemented yet

terse folio
#

You can implement it later because it gets saved anyway with the new history 😸

halcyon quarry
#

Its for the event that manage_history is called before LLM GEN rather than after, with only a user prompt and no last resp

#

Once it finally does receive both items, if multiple user msgs accumulated, join them as one prompt

terse folio
#

I see

halcyon quarry
#

For the new behaviors where it may delay responding, or cancel a response to handle multiple user prompts at once

#

Hopefully we can do something similar with the new History 🙂

terse folio
#

I would setup an event for "wait_to_gather_msgs" in each history so the bot can wait maybe a second before starting to generate.
If no one starts typing in that time, generate

#

All you need to do is watch the on_typing event.

Grab the history for that channel and set the flag

halcyon quarry
#

The cancelling part had not yet been worked out quite yet

#

Didn’t try 😛 (not yet)

#

I just wanted to get that cp list and function started to remind me to make whatever changes I do work for it

#

Got it in there right before you started blessing this project I think

terse folio
#

i've kinda redone everything over the past few hours :P

halcyon quarry
#

I love it

#

You are a breathe of fresh air my friend

#

Don’t overwork yourself though lol

#

You were on when I thought I was on too late and now I’m up too early bc son didn’t sleep well, and here you are both times 😛 And your in Maryland?

terse folio
#

Yea, sometimes just in that mood to get something done

halcyon quarry
#

Been there. Still there but now burdened with a few other responsibilities lol

visual dagger
#

why would you cancel a response?

halcyon quarry
#

I think some people want to have the bot behave like a human instead of a computer

#

I want to offer that option

#

A setting will make such an action more or less likely to happen

terse folio
#

does findall histories return the newest file first in the [0] position?

halcyon quarry
#

the filenames are timestamps by default so 0 will be sorted as the most recent

#

unique_id is datetime now when it is assigned

#

I had to modify tgwui load_history function bc it errored on the per channel log format

#

It also made sense to modify the get latest history function

#

Im afk forget exactly what the other function was called

terse folio
#

last function to convert

#

not too sure what it does

halcyon quarry
#

Nothing anyone is actually using I don’t think

#

Er

#

This function replaces {user_2} variable with 3rd most recent user message, {llm_1} with second most recent LLM resp, etc

#

Those 2 variables are very useful for the Flows

#

The last one {history_X} I added in case anyone thought of something useful to do with it

#

It works similarly but returns the pair

terse folio
#

makes sense, okay

halcyon quarry
#

This is something that needs to be reworked a bit if using the id pairing

terse folio
#

Also the history no longer is stored in pairs, but it can be converted using the renderer

or history.last_exchange()
which I could add more of to query different indexes.

But the way that works is taking the last message (assumed to be assistant) and getting it's parent message, which is usually the one before it.
The message that trigged the llm

#

for now ill comment that function out and start getting the bot running

halcyon quarry
#

How I was planning to address the issue of when it generates from a Flow (prompt is internal and not from on_message) was to store that text value instead of the message.id, while adding another number type to the history tracker like 2 = internal prompt

#

If the function tris to get a message of type 2, it gets that text string instead of expecting a message.if

terse folio
#

after a few fixes such as circular imports, the bot has awoken!

halcyon quarry
terse folio
#

should history change with character changes?

#

I think so right?

#

Since tgwi saves logs to different files per characters, I will follow that

halcyon quarry
#

There is a setting in the config file to do one or the other

terse folio
#

what's it called? how does it change how the tgwi save function works?

halcyon quarry
#

Min….

#

In change char task, if its per chan history it will reset history everywhere every time. If single history mode it will check the config setting and honor it

#

Its the setting with values “keep” and “new”

terse folio
halcyon quarry
#

Until if when we figure out per channel characters, it doesn’t make any sense to keep history I don’t think

#

Although I suppose we could leave that decision to the users

#

How about we just print a warning in the CMD window if it is per channel histories and the mode is to keep history

terse folio
#

i thought that was the intended behavior

#

to keep history across multiple channels

#

so it picks up where it left off for everyone

halcyon quarry
#

I’m on the road but I think I may have arbitrarily just forced history to reset but could just as easily keep it

#

The condition is in change care task

#

One nice thing about having a jailbroken phone is being able to use Discord with Apple CarPlay

visual dagger
halcyon quarry
#

It may generate, then wait to show "typing..." then wait to send message

#

depending on how loose the responsiveness setting is

#

responsiveness is going to be something like
1.0 default (bot is computer, like now)
0.0 is busy human that may be afk for a bit depending on the max delay setting

terse folio
#

Woo history works over reloads now!
had a small bug

image.png
halcyon quarry
#

Red line = character change yes?

terse folio
#

and switching characters, it remembers last conversations before the restart

#

for "new" + character change, i'm thinking of adding a function to create a new history file and unload the old ones instead of deleting them

halcyon quarry
#

er

terse folio
#

because when you run clear that triggers it to clear the file as well

#

because I'm sure people would want to keep old chats

terse folio
halcyon quarry
#

I'm not aware that history was ever being deleted

#

(from files)

keen palm
#

Yes

visual dagger
terse folio
#

it's with the new history manager
when you update the classes it will push updates to the files.
If you clear the history in the bot it will clear them on disk

visual dagger
halcyon quarry
#

I'm excited to get around to the new behaviors, but I think we need to polish up our history management first

terse folio
#

can solve that later!

#

it's safe/stable as is right now

halcyon quarry
#

Understand that in current version (Main), history from files is never deleted

terse folio
#

yes, i'll have to look at the original code, what it was clearing

#

I think it just deleted the whole self.session_history

halcyon quarry
#

It clears the variables - however, the history is written to file on every interaction

#

So clearing the variables does not delete anything

terse folio
#

okay I see, i'll replace .clear() with .fresh()
this will create a new instance of history and start a new file
after I write that function

halcyon quarry
#

One thing the bot lacks but I may never get around to, is better settings management via slash commands.
Using command groups, etc

#

that .clear() function could be nice if we do like a /history command with a number of other things within it

#

Or, yet another config option

terse folio
#

this is where stuff used to get reset

#

but it doesn't change the "unique_id"

#

so what does it do with conflicting file names?

#

i tested out the branch

halcyon quarry
#

bot_history.load_bot_history()

terse folio
#

When switching characters, it's okay to unload the internal history and not change the ID because you're switching to another folder anyway and it creates a new file

#

But if I run /reset_conversation
it forgets temporarly, but still writes to the same logs

#

and if I had load history "keep" it would probably load everything that was reset the next run

halcyon quarry
#

I mentioned earlier that I had arbitrarily forced reset on per-channel mode, on character changes, regardless of the setting

#

But that we should change this to instead honor it - but print a warning

terse folio
#

im just trying to figure out the desired behavior

halcyon quarry
#

Current

        # Set history
        if bot_history.autoload_history and (bot_history.change_char_history_method == 'keep' and source != 'reset'):
            bot_history.load_bot_history()
        else:
            if source == 'reset':
                await bot_history.reset_session_history(ictx)
            else:
                await bot_history.reset_session_history()

How we should do it...

#

erm... hmm...

#

Ok it is more confusing than I thought lol

visual dagger
#

you are trying to manage history in a away without deleting it?

halcyon quarry
#

thought I had put something there

#

It is already managed without deleting it

visual dagger
#

what's the current issue?

halcyon quarry
#

There's a setting in config.yaml to keep or refresh the chat history on character changes

#

some people want to keep the history even though new character is coming

#

By refresh I mean it starts a new log, does not delete the old

visual dagger
#

using the old char's history with a new char?

keen palm
#

Using /reset_conversation causes the previous history to be deleted, actually

visual dagger
halcyon quarry
#

It is a setting 😛

visual dagger
#

but either ways the history shoudnlt be deletwd

halcyon quarry
#

We're discussing how it isn't working for per-channel history

visual dagger
#

a char/bot that interact with mutilple channels?

halcyon quarry
#

Where have you been?

#

We're all popping champagne and throwing confetti here the past few days because the bot now maintains chat history for all separate channels

visual dagger
#

I'm trying to understand what's the issue

#

you are facing

halcyon quarry
#

I could say it again maybe

terse folio
#

I think this will make it work as expected

image.png
#

in the method for getting history

visual dagger
#

so the problem is that's hard to manage the reset history feature for multiple channels ?

terse folio
halcyon quarry
#

It's not a big problem, this is small potatoes

visual dagger
#

let John at ease man

keen palm
#

Thank you for spelling "potatoes" correctly!

halcyon quarry
#

Ideally:
On character change:

  • regardless of per_channel_history settings:
    • change_char_history_method: keep - the chat log / variables will not change in any way.
    • change_char_history_method: new - a new chat log is initialized (empty), all variables are cleared, etc.

On reset_conversation:
per_channel_history: true

  • a new chat log is initialized, but only the history for that channel is cleared in the new log file.
  • variables pertaining to that channel are cleared.

per_channel_history: false

  • new chat log initialized, all variables cleared
keen palm
#

Right. Currently, /reset_conversation will overwrite the previous chat log, as soon as new history is put in.

halcyon quarry
#

understood - definitely an oversight

#

I could work it out but hopefully Reality will just do it so I don't have to 🙏

#

They've been pretty good at dissecting other spaghetti

terse folio
#

got reset_conversation working. had an issue with how I defined __copy__

halcyon quarry
#

Maybe you should push what you have to some branch so I can get a sneak peek

terse folio
#

time to do some merging

#

woo almost 1000 additions :P

#

pushed to dev

#

per channel, and single works fine

halcyon quarry
#

Amazing

#

Will check out very shortly

#

Like now

terse folio
#

👍

halcyon quarry
#

I can say one thing for sure - you are using so much code that is a foreign language to me.
But that's not too big of a problem

terse folio
#

haha, my original plan was to write as little as possible.
Hence why I used libraries like data class and data class json to automate so much.

But had to make little workarounds.
I plan to make this it's own library that will get some cleaning and updates to be more generalized.
But that would only touch the internals.

#

For example it's common in python to mark a function/variable as private by prefixing it with "_"

So by only exposing certain tools outside the class, theres less ways to break it

halcyon quarry
#

I did not try running it yet - if dataclasses_json is not installed, what happens?

terse folio
#

use the requirements.txt

#

there should be an update script that runs pip install -r requirements.txt

#

as well as pip install -U pip (before)

halcyon quarry
#

Is Ctrl+C a standard shutdown combination that you are just adding awareness for?

#

Thats typically the shortcut for Copy (obviously) and I could imagine folks trying to copy text from the window using that

terse folio
#

Ctrl C is used to interrupt terminals, yes it's common

halcyon quarry
#

ok, just checking

terse folio
#

If you have text selected on windows terminal and use ctrl c, it will copy it

#

but on linux it wont, just kills it

#

So I got in the habit of using ctrl x (cut) to copy out of terminals

halcyon quarry
#

Are there some things that happen when pressing Ctl+C verses hitting X?
As in, functions that save data etc?

terse folio
#

yea, it runs save history incase you want to do it immediately.

I added a save interval to the settings, which you could set to 0 if you wish.

But it will always have your file saved within 30 seconds, even if you dont add any new inputs

#

It does so by creating an event for the future and checking if it has been saved between that time, if not, do it now and move the time forward

#

With the larger classes, a lot more data is being stored compared to before

#

SQL would be great for chat logs, but I'm not up for reorganizing that haha.

halcyon quarry
#

The data is all going to new files in /internal/ ?

#

(aside from history log in /logs/

terse folio
#

Anyway, there are some functions similar to sql.

History.search(lambda m: m.name == Reality)
Would return all messages by me for example

terse folio
terse folio
halcyon quarry
#

How big of a PITA would it be to save the extra data somewhere in internal?
I think the 'per-channel' history logs are about as invasive as I want to get to native TGWUI files

terse folio
#

If you're using keep history, it will use the same amount of files as there are channels multiplied by the characters you use ofc

#

(And x2 for the class storage)

#

But I dont imagine most people to be running the bot on 100s of channels

#

But if they do, saving data will be more efficient by saving only the active channels!

#

in smaller chunks

halcyon quarry
#

Are you saying that there is now a separate chat log file for each channel?

terse folio
terse folio
#

and I think it's more worth it in the long run

#

You could load up any channel/character chat into tgwui and view it without the bot.

halcyon quarry
#

I see that the channel ID is not being converted to a string.
With the new handling, there's no issues 'getting' it?

terse folio
#

It is being converted to a string internally,
At first I had encoders and decoders convert it to string and back to int when saving to json.

But then I was thinking about generalizing the code to other uses, so make it a str.

And that worked out for the best anyway, because now the channel Id is (channelid+charactername+chatmode)

Same matching the log files so that you get a new history class and new tgwi history

halcyon quarry
#

when per_channel_history: False, is it still collecting both internal/visible?
Is it adding empty strings to visible in both cases? In any?

terse folio
#

it collects whatever llm_gen outputs

#

which currently is filtering visible for the tts audio file. I didnt look too much into it

terse folio
halcyon quarry
#

last_user_message, last_assistant_message

Dya think last_bot_message is more suitable?

#

yeah its only used 2x Ill update that to be uniform with the other variables using 'bot'

terse folio
#

Sure,

Also I use bot_message and user_mesaage when referencing the history items.

I named them this way not to be confused with 'message' of the discord.Message class

#

could also call them hmessages, that's what I renamed the class to, to make it more clear whats what

halcyon quarry
# 
        else:
            if source == 'reset':
                # create a clone with same settings but empty, and replace it in the manager
                bot_history.get_history_for(ictx.channel.id).fresh().replace()
            else:
                logging.warning('This originally cleared all history. No need, just unload maybe?')
                # bot_history.clear_all_history()
                bot_history.unload_history()

What does unload_history() do?

terse folio
#

it just runs .clear on the internal channels dict.
This would clean up the classes from memory.

it's not important since the way channels are handled already deals with "resetting" things

#

oh that reminds me, I need to test character switching on single channel mode.
I think since it ignores the channel input, it will still be the same history

#

easy fix though

halcyon quarry
#

I'll remove this warning since I think things are working expectedly

#

Yes I think the logic checks out for the settings combos...

#

Tell me about this 🙂
self._bot_id = 0

#

For future use, or in use here?

halcyon quarry
#

Trying to make an update script but it's not quite so simple since we rely on using the cmd_X.bat scripts in textgenwebui folder

#

to activate the environment

terse folio
terse folio
halcyon quarry
#

OK - I got some suggestions from ChatGPT that didn't quite work

# 
@echo off

echo Updating from Git...
git pull

rem Set the directory one level up
set PARENT_DIR=..\

rem Check if the script is running on Windows
if "%OS%"=="Windows_NT" (
    call %PARENT_DIR%cmd_windows.bat && pip install -r %PARENT_DIR%ad_discordbot\requirements.txt
) else (
    rem Check if the script is running on WSL (Windows Subsystem for Linux)
    if exist %SYSTEMROOT%\System32\wsl.exe (
        call %PARENT_DIR%cmd_wsl.bat && pip install -r %PARENT_DIR%ad_discordbot/requirements.txt
    ) else (
        rem Check if the script is running on MacOS or Linux
        if exist /usr/bin/uname (
            for /f "tokens=*" %%i in ('uname') do set UNAME=%%i
            if "%UNAME%"=="Darwin" (
                sh -c "%PARENT_DIR%cmd_macos.sh && pip install -r %PARENT_DIR%ad_discordbot/requirements.txt"
            ) else (
                sh -c "%PARENT_DIR%cmd_linux.sh && pip install -r %PARENT_DIR%ad_discordbot/requirements.txt"
            )
        )
    )
)

echo Update complete.
pause
#

This does activate the venv but does not execute the pip install -r requirements

terse folio
#

that's a lot of complications wew

#

I see, you're trying to do this all in sh?

halcyon quarry
#

This is 100% outside my realm of knowledge

#

Just told chatgpt that I want an update script that does git pull, that can activate one of the files depending on OS, and to run the pip install ir requirements.txt

terse folio
# 
@echo off
cd text-generation-webui
set INSTALL_ENV_DIR=%cd%\installer_files\env
call installer_files\conda\condabin\conda.bat activate "%INSTALL_ENV_DIR%" || ( echo. && echo Miniconda hook not found. && goto end )

cd ad_discordbot

@REM git switch dev
git pull

cp bot.py ../bot.py


python -m pip install -U pip
python -m pip install -r requirements.txt
cd ..

@REM call python bot.py... -flags...

:end
pause
#

I use something like this, you'd have to adjust the paths to match your environment

#

whereever you put the script

#

like if you put in in a utils folder under ad_discord bot

#

the first line could be cd ../.. instead of cd textgenwebui

halcyon quarry
#

I think ideally we would not only have UPDATE.bat in the main folder, but also bot.py with like a LAUNCH.bat

#

(so no file has to be manually moved)

#

But this is a pipe dream - nothing I can do at all

terse folio
#

take out all the manual tgwi startup functions and turn it into an extension!
never have to move another file

#

instead of client.run to start the bot
you'd have a script.py file with setup()
that runs client.run

#

in a different thread so tgwi could run

#

that's something to worry about later, it works fine as is

halcyon quarry
#

If you find yourself so dedicated to this project at any point, I would not feel bad if you forked it and I became the contributor lol

#

There's a lot of big brain ideas that are restricted by my amateur knowledge

#

I pushed a quick hotfix for /speak command, which was broke by recent updates

terse folio
halcyon quarry
#

Exactly

#

I added 'source' as a positional arg for llm_gen so only run manage_history() if source != 'speak'

#

did not check dev to address the issue yet

terse folio
#

yea, it doesn't manage the history for speak, well it shouldn't

#

It creates the bot_message, but wont commit it to history

halcyon quarry
#

I accidentally ran pip install r -requirements.txt instead of pip install -r ad_discordbot\requirements.txt

terse folio
#

it's possible to also use pip from within python

#

using sys.executable as the path to the python exe

#

but im not sure this works properly with activated conda environments

#

only tested with venvs

halcyon quarry
#

You might have to fork it at this point because man, I am really not a fan of the logging method here

#

I did not like the idea of having separate channel log files.
TWO log files for each channel is bananas

terse folio
#

i'll move them to internal

halcyon quarry
#

It's quite obvious how much work you put into there's an overwhelming amoutn of data being captured

#

This isn't necessarily a bad thing but yes, this data is excessive for the TGWUI log folder

#

I was very thrilled to accidentally start logging all chanells to a single file, I was initially trying to do something similar

terse folio
#

the log files for tgwi are completely unnecessary, just there if the user wants to load them in the webui without the bot.
That could be a toggleable setting.
Or have a command to convert a conversation to a tgwi file.

halcyon quarry
#

This amount of data would be staggering in one file, for sure

terse folio
#

but I also understand the appeal of that, I just can't imagine saving all those threads of history every time a message is sent!

#

This would be perfect for sql

#

that's one file, and it does loads in chunks and handles caching internally

#

And it fits perfectly with the data structure of history

manager: [channels]
channel: [messages]
message: {attributes}

#

in _internal_post_save you can comment out that line and it will cut the files saved in half.

#

it's not needed by anything else

#

actually no, it is needed sort of

#

I'm relying on tgwi's "all_history" function to get the unique ids

halcyon quarry
#

We were saving:
Guild name, channel name, the text exchange. Mostly ignoring 'visible'
I was working on also capturing: message IDs, and a separate list of 0's and 1's to map an ID to a message.

Now it's saving assloads of information for everything

terse folio
#

but I use some of the attributes, like "spoken"

#

so maybe have to dig around and check what to comment out

#

replies and reply_to is going to be useful for multi user chat

halcyon quarry
#

I'd say whatever we can comment out is probably best, it's just so much

#

until we need to uncheck them one at a time as we open new features

#

we'll be writing 50% data being unused

terse folio
#

yea, wish there was a way to set "don't output if default"

#

i need to read more about dataclass_json and do expirements

halcyon quarry
#

The individual log files are not too shabby if you could get them to save into a subdirectory, and ideally those log files would include the guild name and channel name

terse folio
#

they're not meant to be explored by the user, it's all internal data

halcyon quarry
#

For anyone bopping back and forth between the WebUI and this bot, it will be quite a shock for these to start amassing there

terse folio
#

the webui doesn't see them :)

#

only the normal history files

#

I noticed then when there was a bug saving the normal history files, that tgwui didn't output the "unique_id" for the save i was trying to load

halcyon quarry
#

Assuming it will be reduced to one file for channel, this still would be absolutely stuffed with files if its a popular server

Screenshot_2024-05-30_133539.png
terse folio
#

moving them somewhere internal shouldn't be too hard

#

will look at that tomorrow

halcyon quarry
#

I'm taking a break for a little, so no rush

#

Need a 1 or 2 day vacation lol

terse folio
#

oh maybe my tests with dataclass_json exclude failed because my environment is set to py3.9
but tgwi uses 3.11 or 10

#

I need to get around to update all my code to 3.11!

#

okay, I guess it wasn't bugged

#

odd that it needs a function input, and "True" doesn't work on its own

#

Oh!
maybe it takes the class as an input?
and I could decide to exclude a value on a condition.

#

yes!
it passes the value

halcyon quarry
#

This update is a lot to digest.
There's two things that will be holding this back from my consent on comitting to main:

  • The logging, as I've mentioned. Our logs are going to default TGWUI location so we just can't overwhelm this directory with potentially dozens/hundres of new files per day which could easily happen with 1 file per channel, and not organizing them to subdirectory based on unique_id.
  • I need to come up with a solution for format_prompt_with_recent_output() which is one of the driving forces for the Flows feature.
terse folio
#

think sorting into folders based off channel id would work?

halcyon quarry
#

Ideally... to me... something like how I split the multiple channel logs
{unique_id}_{server_name}_{channel_name}

#

or, server_name, channel name - and use unique ID in the subdir

vestal python
#

Got one of my 2 working again.

fixed.PNG
terse folio
halcyon quarry
#

I think they should go in a subdirectory, without zipping

terse folio
#

Also, it wouldn't be easy creating a file system that uses guild/channel names for folders/file names.
Because those names could change, and the files would no longer be found.
Then we'd have to have a database of the channels and their names when the folders were created?

#

maybe at the bottom level for the files, they could use guild/channel names in them

#

because the bot would be looping over all the files to get the latest time anyway

halcyon quarry
#

Thats a good point

terse folio
#

but the path leading to that needs to stay the same

halcyon quarry
#

I am looking forward to understanding the new system as it seems very versatile

#

Does the tts play more efficiently now? I didn’t look into it much but saw that you did change it

halcyon quarry
#

Let me know if you have a better idea for skipping history management from /speak command (could be more reasons in the future) other than if source != 'speak':

halcyon quarry
#

Can't seem to figure out how to avoid adding a /speak request to message history without also changing upload_tts_file() worst case, just add it to history 🤷‍♂️

terse folio
#

in your speak task function

#

in llm_gen, i get the save_to_history value

halcyon quarry
#

Yo

terse folio
#

and pass that to the "new_message" function

halcyon quarry
#

So there's essentially 3 types of messages that we need to track in order for the current features to work expectedly

#
  1. the ones that have message IDs that will be written to history
  2. the ones that have message IDs that will not be written to history
  3. the ones without message IDs (internal prompts)
terse folio
#

you can assign roles to messages and filter them

image.png
halcyon quarry
#

Well, this is a moving forward thing. Currently in main we don't care about message ID's - all messages are captured in text form

terse folio
halcyon quarry
#

Got a few mins to just discuss the new classes?

terse folio
#

mhm

halcyon quarry
#

Trying to see where is the best part to start off...

#

Let's start with local_history

#

(I know, not a class 😛 )

#

What does the fp variable stand for? in get_history_for()?

terse folio
#

file path

#

I could write a custom typehint for it ^-^

#

similar to what I did with ChannelID

halcyon quarry
#

I see, sort of 😛

#

Seems like it goes out of the way to avoid including the type hint

#

you skip writing ChannelID:str by importing ChannelID from typing

terse folio
#

the interpreter would evaluate these for you.
Yes I see what you mean, you don't know what it's expecting as input now

#

Hmm, I wonder how I could solve that while also keeping the name

halcyon quarry
#

Not a particular big deal

Screenshot_2024-05-31_111349.png
terse folio
#

i know some programs have fancy typehints that take types as inputs

halcyon quarry
#

How about, could you explain how the typing module helps

terse folio
#

Wanted to organise types into there own place that could be used by other modules

#

it ultimately could go into utils_shared

#

just update the imports ^^

halcyon quarry
#

Lets check out get_history_for()

#

What does search do?

terse folio
#

you'll notice what seems to be copies of functions sometimes.
they are inheriting from eachother

#

search (in the bot)
will search the logs folder for the latest log that matches character/mode/channel

#

so you don't have to pass your own file path every time you want to import history

#

huh, also for me, channelID shows the type

#

wonder if I have some other addon

halcyon quarry
#

hmm... I did open the folder in Explorer

terse folio
#

I'll write up some notes in the internals

halcyon quarry
#

So at the end of get_history_for it does a recursive call after setting a few params depending on the initial input

#

Yes?

terse folio
#

this is the internal get_history_for

If there's no history matched for the given id, check if the user passed a file path.

if not, check if search enabled.

if still no matches, create new history

image.png
terse folio
halcyon quarry
#

Thats updated code you're showing? Or are there two functions

terse folio
#

Sorry it's a little confusing to follow, here's an explination:

#

my original design was to make an external library you could import through pip for example

#

So I couldn't include the internals of the bot and I want to keep it as general as possible

#

So in bot.py, I import the history and history manager classes

#

and create subclasses

#

I overwrite a few functions

#

for example get_history_for

#

which adds code to get the bot's state, character name and all

#

then at the end of the overwritten get_history_for
it does super() (which is a reference to the base class version)

super().get_history_for(...) calls the original version of the function with the new paramaters that have already been processed

#

so if you shift click this, it takes you to the actual logic

#

this approach lets you change how the classes behave to an extent without needing to modify internal code! (as if it were a library)

#

the point of the subclass version is to create the custom channel_id

#

that includes character/mode information as well

Then it just works as normally 😸

#

i'm going to cleanup my config mess with the dataclasses now that I know how to exclude data to export

halcyon quarry
#

The reason things werent looking correct were because I opened the repo in explorer, without moving bot.py

terse folio
#

using top level imports like

from ad_discordbot.modules.history helps with that ambiguity

halcyon quarry
#

Yes, same - my work space is still Main version

terse folio
#

ahh

halcyon quarry
#

Just packed it up and set up Dev in working space

terse folio
#

do you use an external tool to switch branches?

halcyon quarry
#

Alright so after asking ChatGPT what super() is, let me know if I understand this right...

get_history_for() has all the attributes etc from the parent class, and can modify some specifics before getting into the whole package deal

terse folio
#

mhm

#

ask it for a simple example too

#

it's really cool

halcyon quarry
#

Overall, super() is used here to build on the existing functionality of the parent class's get_history_for method while incorporating additional logic specific to the subclass, such as modifying the search parameter and formatting the id_ parameter.

terse folio
#

yea, that's exactly what the function does: only modify the id, and update search based on the bot's state.
then pass it to the parent

#

something that might slightly confuse the typehints is this

#

I'm also replacing the class used for History, with Custom history

#

because the history manager can create history classes

halcyon quarry
#

If we're going to work together in harmony - I recommend trying to limit the complexity a bit if possible.

In this case, it seems like the code in the child class could be done in a previous step (skipping the need for the child class), except if self.change_char_history_method == 'new': would be I think if bot_history.change_char_history_method == 'new':

terse folio
#

it's still the same class, we're just extending it

#

so self is correct

halcyon quarry
#

Im saying this could all be passed to the parent function, skipping the need for the child one
state_dict = bot_settings.settings['llmstate']['state']
mode = state_dict['mode']
character_menu = state_dict["character_menu"]

#

Seems like the only line the child is inherting is the change_char_history_method

terse folio
#

i'll draw it out

halcyon quarry
#

I don't mean to complain, I just want to be real with you that I don't have any hand on knowledge with subclasses, and have trouble tracing where things are going

#

(currently)

#

Much of what you've set up previously I don't really need to understand all too much, or it's just not too complex that I get it 'enough'

terse folio
#

Yea, the plan was to separate it from the bot
Like how you don't change the internals of discord.py, but rather use it's tools or subclass it (using a bot class)

keen palm
#

I don't have a god damned clue what you two are talking about like 90% of the time

terse folio
#

i think this drawing could help, brb

halcyon quarry
#

I'm finger painting and they're Rembrant

terse folio
#

You're learning 😸 , I didn't get there in one day either

halcyon quarry
#

You'll be stuck doing everything that has anything to do with History if I can't get this 😛

#

And we're at that point where History is starting to be the focus on planned features

terse folio
#

it's still the same class.
it's the same bot_history object.

it just has extra modifications to work with your bot instead of general purpose

#

what I actually should have done in the beginning is this:

image.png
#

move those custom attributes to the custom version, but at the time I was testing in a standalone python file so wanted it all at once.

#

the great thing is, I could make this change and it wouldn't change how it works at all!

#

the classes could also be combined into history.py if you wish.
I just wouldn't be able to update them as easily when I continue it as it's own standalone project

halcyon quarry
#

Assuming that you are making some more changes - after you do, we need to regroup and go over it real quick

#

Currently, I'm a bit paralyzed

terse folio
terse folio
halcyon quarry
#

Or I need to read Python: For Dummies

terse folio
#

this is what you were originally doing

image.png
#

this can get really out of hand as you add more and more attributes

#

this is more cleanly recreated using classes

#

instead of all the attributes being held seperately, put them into one object!

#

Now that it's one variable, you don't need to change many lines of code if you want to add a new attribute!
And you will never have mismatch index errors if you accidentally forget to add to one of the lists.

#

Classes are just another form of structuing data, focused on "related things stay together"

halcyon quarry
#

I understand this 🙂

terse folio
#

now for a subclass example

#

the subclass extends the behavior of the previous Item
but now includes the added "custom_attribute"

#

you can do the same with adding functions or anything you want

#

CustomItem('pqr', 57196, False, 'Hello!')
this works too, I just used keywordArg assignment to make it easier to see the change ^^

halcyon quarry
#

In your illustrations,

#

"CustomHistoryManager" seems to not be the Subclass - correct?

#

(the parent class)

terse folio
#

I draw it this way to show that "CustomHistoryManager" is inheriting ALL the contents of HistoryManager plus its own

#

so it wraps it and is bigger

#

but yes, it's called a subclass

#

Class animal

  • legs ? unknown
  • noise ? None
  • has function "make noise" that prints noise attribute.

Class dog (type animal) is a subclass

  • it can auto-fill that the dog has 4 legs.
  • auto-fill that the noise is "bark"
  • can add a custom function: play fetch.

The parent of Dog is Animal, because dog takes the functions/attributes from animal and modifies them.

#

in this example, even though "dog" class doesn't define a "make noise" function.
You can still run dog.make_noise and it will print bark
since 'animal' had it

#

It can be useful to use a common base class like this.
For example if you are sorting through a list of items and want to extract which ones are animals.
You could do if isinstance(item, animal)

All dogs, all cats, all birds... would return True.
But the other items would not.

halcyon quarry
#

I appreciate your holding my hand here and walking me along

#

So I do understand that by making the Class, this is an easy way to make a broad definition for all items that will be processed as History.
The CustomHistory subclass inherits those broad definitions, and adds more specialized attributes

terse folio
halcyon quarry
#

So rather than getting a return from the sublass, we update the subclass which passes to the parent class, then we get a return from the parent class

terse folio
#

no classes are being updated when you call the get_history_for function.

It's just HistoryManager.get_history_for takes an ID for which history to get.

But in your bot, we need more than just the ID, we need character and mode info to decide which history we use.

CustomHistoryManager.get_history_for
Also uses ID
But it customises it, then continues as normally with
HistoryManager.get_history_for(New Custom ID)

#

this solution uses less code because
you would have to pass character and mode information each time you get history without it.
But here we do so automatically!

#

interesting, parents are also called superclasses.
that's where the super() function comes from

#

also little bit about why I try to use dataclasses as much as possible now!

#

When creating subclasses, you often need to super().__init__(all the args!)

#

to run init with the parent's code

#

But this is tedious to keep adding attributes

#

and having to remember where they all are.

#

Dataclasses kinda automate this nicely 😸

#

PS: a dataclass is just someone's library, it's still the same classes in the back, just a tool to make them easier

halcyon quarry
#

Let's talk about this...
bot_history = CustomHistoryManager(class_builder_history=CustomHistory, **config.get('textgenwebui', {}).get('chat_history', {}))

terse folio
#

the ** part?

halcyon quarry
#

That's key word args, yes?

terse folio
#

yes, i'm unpacking a dict into autosave=true, autoload=true, change_char_method='new'...etc

halcyon quarry
#

So it will basically dump whatever is in that dictionary as individual params

terse folio
#

mhm!

#

you can do the same with one *
to unpack a list to single args.

halcyon quarry
#

Now about the rest of that line...

terse folio
#

class_builder we talked about that,
the manager doesn't know about the new history class we created for the bot, so lets tell it to use it

halcyon quarry
#

I'm not sure that we talked enough about class_builder

terse folio
#

there we just pass a type
in the code it uses

new_history = self.class_builder_history()

terse folio
halcyon quarry
#

Are we using class_builder for anything else besides history atm?

terse folio
halcyon quarry
#

But we may expand

terse folio
terse folio
#

the history class is what's created for each channel, holding the messages and other info

halcyon quarry
#

I meant like, we may start making a similar variable for another element of the bot like class_builder_behavior

terse folio
#

remember, this is only for bot_history

halcyon quarry
#

hmmm

terse folio
#

hmm, this is complicated to explain

halcyon quarry
#

ok so the name is supposed to be a hint that this is building the class

terse folio
#

yes

#

example:

#

i'll call history manager HM for short

#

when you call function:
HM.get_history_for(id)
and the ID doesn't exist, it creates a new history class.

#

by default, that new history class is History which is found in history.py.

#

But I modified the history class (CustomHistory)

To change how saving works for tgwi

#

But HM is still creating the old histoy type, because it's programmed to do so.

#

So I gave HM an attribute, called class_builder_history = History by default.

#

when initialising HM, i can set it to a custom class

#

and HM will use that custom class every time you need a new history from HM.get_history_for

halcyon quarry
#

and the ID doesn't exist, it creates a new history class.

Do you mean it creates a new instance of the history class?

terse folio
#

a class is just a blueprint to objects.

halcyon quarry
#

OK

#

Got it

terse folio
#

that's what I meant 😸

halcyon quarry
#

Ok - that word helps a lot (object)

terse folio
# halcyon quarry Ok - that word helps a lot (object)

so, that should make more sense that multiple Objects aren't there when you create a subclass.

that it's actually just making a larger blueprint that combines all the information like a merge, then creates your object out of that.

halcyon quarry
#

Alright so it creates a new history object, and without customizing it it would just be called "History" but it gets a custom value assigned to it from the character menu, mode and uniqueID

terse folio
#

you can't do

bot.py: 
import history

history.py: 
import bot
#

to access eachother

halcyon quarry
#

I was wondering how to get around that

terse folio
#

interestingly, you are allowed to do that sometimes!

#

but for typehints

halcyon quarry
#

I could ask chatgpt this but maybe its a quick one, what is field?

limit_history: bool = field(default=True)
terse folio
#

here in utils_discord.py
I import history.py only for typehints.

in history.py
I import utils_discord.py for something I forgot.

terse folio
#

remember we are unpacking this dict into the HM class with **

#

I need to reorganise some of those variables because they are from when I was getting the base code working.

Stuff like limit_history should be defined in CustomHistoryManager.
not the parent.
Because the parent never uses it anyway.

terse folio
#

maybe some custom per channel stuff like: don't save history to file for this channel

#

oh yea, this absolutely could be per channel!

#

lets do that in a future update!

halcyon quarry
#

It just didn't jump out at me what the reason for the variable name was.
As I think I understand now, you used that label to signify that CustomHistory is building the Class

terse folio
# terse folio lets do that in a future update!

actually would have to sort out the multiple files thing here.
bot behavior could be saved inside the history json files.

And i would write it in a way where it doesn't save default values.
That might be a little bigger of a change than I thought

terse folio
#

added a hidden attribute to HMessage.

When a message is hidden it wont be rendered.
This could be useful for regenerating a reply, but also keeping it in the history save just hidden.

#

internal history is now significantly smaller
still a few more changes I could make

image.png
halcyon quarry
#

Is there currently any benefit/use for uuid?

terse folio
#

that's for it to load which message is being replied to.
i'll change the uuid to use an incrementing number soon.

#

these are reply pairs

image.png
#

in the future, when handling multiple messages from different people.
you might want to be able to select which message the bot is replying to

#

as other messages would find their way inbetween the history

#

I could do this by storing the index on save.
But that means I need to overwrite the .to_dict() function and figure out how that all works.

#

it's doable!

halcyon quarry
#

hmm

terse folio
#

if i "dont save" the replies in messages.
But rather as a dict in the history class.

i could write an encoder function that reduces everything to indexes as they currently are and uses those.
perhaps.

#

would look like
{
"1":"0",
"3":"2"
}
for the above screenshot

#

Commented out the hidden code.
A better solution for cycling possible regeneration would be to hold the messages in another list.
And use something like HMessage.replace() not implemented yet.
to save the one you want to in its place in history.
Could even have the bot edit its message!

But it would act the same as History.replace() swapping it in place of the old history in the manager.

halcyon quarry
#

Sorry for lack of response - I'm not quite going to understand it too much until I can get hands on with it after you make your adjustments.

I like how there is much less information written to file now.

As I said earlier, we just need to be sure that we can get the text for the 3 different types of messages.

For the 3rd type (internal message) - where there is no message.id, the user author is internal, the text is not written to history - we need to capture that text in one place or another, and be able to recall it

terse folio
#

you already can!

halcyon quarry
#

For instance with the Flows feature, on Flow Step 5 we may want to use the output from Flow Step 1

#

currently, this would be done with either {llm_4} or {user_4}

terse folio
#

set name=None (so it's not saved)
set role='internal'
id = None

You can use this to autofill that name='internal' if you need.

and that's it!

It will not appear in chat history because the tgwi renderer will only output pairs of user/assistant.

halcyon quarry
#

Where are such text strings being stored?

terse folio
#

with .text

#

as normal

halcyon quarry
#

Are we retaining all text to RAM then?

terse folio
#

if you want to collect all the internal messages, you can do

History.role_messages('internal') -> list

terse folio
halcyon quarry
#

The point of the message ID's I think was mainly to not retain the text to RAM - be able to just go fetch it from discord

#

Well, I mean that's how it was in my head

terse folio
#

because calling channel.fetch_message makes discord do database lookups

#

also, your discord bot already caches messages,
if you keep the bot running forever, it will keep up to a week of messages in memory.