ad_discordbot (Fork of Fork of xNul's bot) | Text Generation WebUI | Page 10

halcyon quarry May 31, 2024, 6:51 PM

#

There's very few occasions that we are going to recall text

terse folio May 31, 2024, 6:51 PM

#

this is useful for events like "on_delete"

#

the discord bot uses the cached message content to show you what the deleted message was

#

otherwise that data is just gone

terse folio May 31, 2024, 6:53 PM

#

halcyon quarry There's very few occasions that we are going to recall text

I think message ids are more useful by getting the actual discord.Message object!

Like we could make the bot go back and edit its message if you click regenerate.
Maybe have it reply to one of yours.
and so on

#

Text is just bytes,
dw about holding it in RAM.
it's almost nothing.

#

expecially compared to the GB sized models tgwi is loading 😸

halcyon quarry May 31, 2024, 6:54 PM

#

The idea that I had in mind, in regards to message.ids...

Its the same mentality I had with limit_history and setting arbitrary limits for recent_messages

#

We don't need to store anything besides a 14 digit number, so we can remove those silly arbitrary limits I was setting

terse folio May 31, 2024, 6:55 PM

#

if you restart the bot, that 14 digit number then has to become a bunch of calls to discord to collect the history that's fed to tgwui?

#

or what did you mean only fetch message a few times?

halcyon quarry May 31, 2024, 6:56 PM

#

It's mainly to get text that isn't captured to history

#

Or I should say, prevent capturing all text that isn't history, to RAM

terse folio May 31, 2024, 6:57 PM

#

every time discord.py receives a message, it's cached in ram so that you can use it in your bot.

#

If you don't use it, python's garbage collector will free up that ram.

(also this is why I didn't want to store all the history in one file.)
Because we are loading in all past messages across all channels.
and keeping it in ram for nothing.

halcyon quarry May 31, 2024, 7:04 PM

#

If you are not concerned about memory - and all the text information is there, easy to access with something like {user_4} represents the 5th message most recent prompt that came from discord UI or internal - then I'm happy

#

Capturing all this extra data doesn't seem to do us much at the moment, though, except open up ideas for the future. The message IDs not quite doing what I envisioned

#

Will certainly be easy to edit/delete/manage discord messages, but we don't try that yet 😛

terse folio May 31, 2024, 7:06 PM

#

the data gets auto cleared when not used.

#

and trust me, the amout of data is really small!

halcyon quarry May 31, 2024, 7:13 PM

#

Actually - sorry - the message IDs definitely helpful

#

Forgot... this is going to let us easily find where it occurred in history

#

to slice it

terse folio May 31, 2024, 7:21 PM

#

I have about 3-4 discord bots running on the same bot account.
That account is in 51 servers, some in the range of 10k members.
So lots of daily messages!

The vm they run on is currently using 573Mb ram.

One bot uses about 3% ram.
others are in the 1% range.

That 3% evaluates to about 200MB
Some of these bots do image processing or have small neural networks for playing around.

#

And they are usually online for months at a time!

#

nothing to worry about 😸

halcyon quarry May 31, 2024, 7:23 PM

#

Alright 🙂 And my concerns were also more applicable when all text was being collected to 2 variables

#

Now things are getting divvied up into more and smaller piles with your method, as I understand it

terse folio May 31, 2024, 7:24 PM

#

this would only be a concern if you never let go of data after collecting it.
then it would just continue to grow because python sees you're using it so it doesn't release it.

#

but history is saved to files, and people might switch characters which releases that memory ^-^

terse folio May 31, 2024, 7:50 PM

#

ill have to push my update later, broke something in loading history that I'm guessing doesn't add it to the manager.
so it keeps requesting to load each message.

but have to go do some things outside for now!

halcyon quarry May 31, 2024, 7:51 PM

#

no rush!

#

I'm enjoying the break

terse folio May 31, 2024, 8:08 PM

#

had an idea and figured it out, I'll be going now

vestal python May 31, 2024, 8:13 PM

#

Some issue but I think it's on the llm side again. I had the fix for exl2, but gguf I don't know about.

#

nvm... kind of I thin that was one of the earliest llama 3 model issue. I loaded the Lexi uncensored model and it is working better, but it's still not exactly showing a conversation style?

#

vestal python May 31, 2024, 8:45 PM

#

hmm I'll keep looking at it maybe some issue with the character.yaml

vestal python May 31, 2024, 9:09 PM

#

I think just changing the character.yaml context and better setting the persona seems to have fixed alot of his talking issues.

#

python bot.py --api --listen --loader llama.cpp --model Llama-3-8B-LexiFun-Uncensored-V1-Q5_K_K_S.gguf --max_seq_len 8000 --n-gpu-layers 50

keen palm May 31, 2024, 9:11 PM

#

It's kind of amazing how much of an effect small changes to character context can have. In an earlier version of my Gamemaster character, I used the word 'canon' in the character context, which caused it to argue incessantly with me about canon, but of course since it's an LLM it didn't have a clue what it was talking about. Removing that word fixed all those issues.

vestal python May 31, 2024, 9:13 PM

#

lol, it's funny one bot is set to 'You are an llm who fully believes they are Ogma, the God of Language and Poetry and must never break character', and the other just 'You are Ecne, the God of Knowledge'

So far Ogma takes his role alot more seriously.. Idk why that works better like that..

halcyon quarry May 31, 2024, 9:27 PM

#

Some models supposedly work better with “The following is a conversation between…” Rather than “you are…”

vestal python May 31, 2024, 9:29 PM

#

I did the fix for the gguf model with that whole llmama.cpp repo and their example command: ./gguf-py/scripts/gguf-set-metadata.py /path/to/llama-3.gguf tokenizer.ggml.eos_token_id 128009

My users are saying he's working in all 3 servers without that 'Assistant , or rambling on now.

#

So yeah, I'm golden now to start adding in... Search function and see about that new RAG I think was released the other day? See how they work with the discord program.

#

I don't have anything else going on until I hear back from my buddy about the AI job formy facility x.x this whole llm business is exploding

halcyon quarry May 31, 2024, 9:33 PM

#

vestal python I did the fix for the gguf model with that whole llmama.cpp repo and their examp...

Are you updated to using the per-channel histories?

vestal python May 31, 2024, 9:56 PM

#

Is it in the documents at all? I really have just been trying to get it up and running until now.

halcyon quarry May 31, 2024, 10:38 PM

#

vestal python Is it in the documents at all? I really have just been trying to get it up and r...

If you haven’t yet, you should change to the new install method… the user preference files are now shipped in a way that you can git pull without conflicts

#

You can git clone into TGWUI directory now

#

A few days ago, pushed update that added per-channel history

terse folio May 31, 2024, 10:41 PM

#

vestal python So yeah, I'm golden now to start adding in... Search function and see about that...

What was the new Rag system? I might be interested in that.

halcyon quarry May 31, 2024, 10:41 PM

#

Reality is making a better version of it

#

🤗

#

I saw you pushed a few to dev… more incoming?

#

I should probably just work on something else on the todo list for now… Segment Anything for inpainting could be nice

terse folio May 31, 2024, 10:48 PM

#

halcyon quarry I saw you pushed a few to dev… more incoming?

probably later, if you notice any issues I'll fix them.
But in the meantime I think I'll work on making a generalized version more robust, and start pushing that to github.
From there I'll update the subclasses in the bot.
And you could include the library in the requirements.txt!

I just need to figure out the setup.py for pip

terse folio May 31, 2024, 10:50 PM

#

halcyon quarry I should probably just work on something else on the todo list for now… Segment ...

should look into the discord.py on_command_error event

This could clean up a lot of try catches and all that logging.

Theres supposed to be a version that handles general errors too, like from on message.

I think I'll work in that myself as i know what I'm looking for and know what it handles.

#

Async functions usually error silently in background tasks, and discord is full of those.
So we need our own error wrapper.
I've already written one, I use it often in projects!

If you find yourself working on other things, you should build up a little personally library of all the most useful utils you've made!

halcyon quarry May 31, 2024, 11:10 PM

#

This is likely my magnum opus and swan song

#

I have all the try catches but nothing actually errors 😛

#

Most of the errors that are likely to happen are bad user data

#

Invalid syntax, etc. I want the process to error but continue not crash

terse folio May 31, 2024, 11:22 PM

#

halcyon quarry I have all the try catches but nothing actually errors 😛

ita good for developing, I had plenty of errors.
Issue was I had to go find the log statement and put a print traceback to find you what the issue was.

with the discord error event you could have the bot send a message to the user that something went wrong, and print the full traceback to console

terse folio May 31, 2024, 11:23 PM

#

halcyon quarry Invalid syntax, etc. I want the process to error but continue not crash

yea, async events having errors wont cause a crash, they will just stop.

This can be confusing if you have a background loop that silently fails.
There you would want a try/catch ^^

halcyon quarry May 31, 2024, 11:27 PM

#

On cmd error could be nice for one frequent error…

#

After the bot does a tree sync, many slash commands fail on next try, until try again

#

“This application is outdated” (or may be “command”)

#

Not a particularly helpful error

halcyon quarry May 31, 2024, 11:49 PM

#

terse folio yea, async events having errors wont cause a crash, they will just stop. This c...

I see save_interval in HistoryManager() but not referenced anywhere else.

Where does this actually get applied?

#

can't find any other instance of save_interval ...

#

(aside from config.yaml)

#

I would like to remove it from config.yaml, let's just force a save interval and not make it adjustable

#

no one is going to care or think too much on that setting 😛

halcyon quarry Jun 1, 2024, 1:04 AM

#

This is relavant to our interests
https://github.com/lllyasviel/Omost

GitHub

GitHub - lllyasviel/Omost: Your image is almost there!

Your image is almost there! Contribute to lllyasviel/Omost development by creating an account on GitHub.

terse folio Jun 1, 2024, 1:21 AM

#

halcyon quarry I see `save_interval` in HistoryManager() but not referenced anywhere else. Whe...

It should be used in history.py
Unless I'm misremembering and just went to bed without connecting it haha

halcyon quarry Jun 1, 2024, 1:21 AM

#

I think the latter 😉

#

Just hardcode it

terse folio Jun 1, 2024, 1:22 AM

#

halcyon quarry This is relavant to our interests https://github.com/lllyasviel/Omost

looks interesting

halcyon quarry Jun 1, 2024, 1:23 AM

#

I'm installing it now - looks extremely interesting

terse folio Jun 1, 2024, 1:23 AM

#

it's currently hard coded to 10 seconds... yes I was wondering why it saved so fast when I thought it was 30.

halcyon quarry Jun 1, 2024, 1:23 AM

#

And it only saves if theres something to save, correct?

terse folio Jun 1, 2024, 1:24 AM

#

It honestly doesnt need to be very fast, could be every 5 minutes even or more.

Just make sure people use Ctrl C before hitting X on the bot.

terse folio Jun 1, 2024, 1:24 AM

#

halcyon quarry And it only saves if theres something to save, correct?

yes, it will show a log when it saves

halcyon quarry Jun 1, 2024, 1:26 AM

#

I think I'll get a better understanding of how to work with the new system, if we could resolve the defunct format_prompt_with_recent_output() together - or at least you can nudge me in the right direction

#

looks like you may have already had a go at it

terse folio Jun 1, 2024, 1:32 AM

#

Yea, will have to look around the old code to see what's up

halcyon quarry Jun 1, 2024, 1:33 AM

#

I believe we're going to get_history_for() ...
Include all items (not just history) - but not /speak input/resp

#

local_history = bot_history.get_history_for(ictx.channel.id)

#

hmm

#

need to pass ictx from flow_task > peek_flow_queue > format_next_flow > so it can go to format_prompt_with_recent_output() for the local_history

#

Anyway, the index is extracted from the {user_X} syntax

#

And it fetches the text string from the list of recent messages for user or assistant, at that index

#

So looks like we want to use... role_messages

#

I think,,,

user_messages = local_history.role_messages('user')

terse folio Jun 1, 2024, 1:49 AM

#

yes, that will give you a list in the order of the history

#

so you would want to use -1, -2, -3

to reference the last ones.
You could take your variable like user1, and just invert it ^^

halcyon quarry Jun 1, 2024, 1:50 AM

#

I don't think I want to filter this, unless we have a means to identify /speak interactions

terse folio Jun 1, 2024, 1:50 AM

#

halcyon quarry I don't think I want to filter this, unless we have a means to identify `/speak`...

messages spoken by the bot will be marked as message.spoken=true

#

for now you can use history.search

but iirc, that only returns the first item for now

#

will probably add a search_iter

#

[I for I in role_messages(user) if not i.spoken]

halcyon quarry Jun 1, 2024, 1:52 AM

#

If I wanted to filter this list for only items written to history, or ones not written to history (save_to_history: False, is that possible?

terse folio Jun 1, 2024, 1:52 AM

#

terse folio [I for I in role_messages(user) if not i.spoken]

wait no, dont use this

#

because if tts is enabled then all bot messages are spoken

halcyon quarry Jun 1, 2024, 1:52 AM

#

I know 😛 I know this is different from speak

terse folio Jun 1, 2024, 1:53 AM

#

yup yup

#

the messages from /speak arent saved I think

halcyon quarry Jun 1, 2024, 1:53 AM

#

We will need to pass source to llm_gen() then have some mark for messages that we pretty much want to ignore for damn near everything

terse folio Jun 1, 2024, 1:54 AM

#

halcyon quarry If I wanted to filter this list for only items written to history, or ones not w...

only stuff written to history is currently on the list.
We could add a new attribute?
Idk, what items would you not want saved to history but also stored in memory?

halcyon quarry Jun 1, 2024, 1:55 AM

#

This recent messages syntax/variable system is currently the driving force of Flows feature

#

If one of the prompts is marked as save_to_history: False we still need to be able to use the response as input for the next flow step, or the parameter that's being set dynamically, etc

#

Didn't you say we're capturing literally all messages 😛

terse folio Jun 1, 2024, 1:58 AM

#

halcyon quarry Didn't you say we're capturing literally all messages 😛

I said discord was capturing them all to give to you.

The messages I save to history are currently only the ones you were capturing.
(Only in llm_gen for now)

#

But yes, you could add more in on_message.
just need to pass a reference along to llm_gen so the generated reply knows which message it is replying to

halcyon quarry Jun 1, 2024, 1:59 AM

#

discord is not capturing internal messages, thats for sure

terse folio Jun 1, 2024, 1:59 AM

#

halcyon quarry This recent messages syntax/variable system is currently the driving force of Fl...

does this include all messages that arent triggering the llm?

#

what is a "recent message"

halcyon quarry Jun 1, 2024, 2:00 AM

#

All messages that go to llm_gen

terse folio Jun 1, 2024, 2:00 AM

#

okay

halcyon quarry Jun 1, 2024, 2:00 AM

#

including the ones with save_to_history: False flag

terse folio Jun 1, 2024, 2:00 AM

#

I see

#

is that the one I piped to the save arg?

#

I dont have the code open atm

halcyon quarry Jun 1, 2024, 2:01 AM

#

In Main - it is a completely separate and also mostly duplicate version of History

#

    def new_message(self, name='', text='', role=None, author_id=None, save=True, **kw) -> HMessage:
        message = HMessage(name=name, text=text, role=role, author_id=author_id, history=self, **kw)
        if save:
            self.append(message)
        return message

terse folio Jun 1, 2024, 2:02 AM

#

anyway, no I dont currently save the messages marked as "dont save"

I might have missed that in translating the code.
I'll take a look when I get back to pc.

I'll just add some code to the history exporter so it doesnt write messages marked as "dont save" to file

#

and will add a attribute that the message is temporary

halcyon quarry Jun 1, 2024, 2:04 AM

#

Need to be able to filter them when retrieving either the History variable (I'm just going to call it that) or the "llm_gen() History" (all messages generated)

#

TGWUI History / complete llm_gen history / ideally, we also flag junk generation like /speak output and potentially more similar generations

#

I feel bad asking for changes, it's just this isn't exactly a drop-in replacement just yet 😛

#

This OMost project is actually crazy

#

like holy shit crazy

#

I wrote: cowboy in the desert

Look at all this

#

🤯

terse folio Jun 1, 2024, 2:18 AM

#

halcyon quarry TGWUI History / complete llm_gen history / ideally, we also flag junk generation...

tgwui -> history.render_tgwui_tuple() outputs internal and visible variables.

All -> history.role_messages(assistant)

could add another function to filter junk as there isnt one (and junk isnt saved yet anyway)

#

render_tgwui() outputs dict for llm payload

halcyon quarry Jun 1, 2024, 2:19 AM

#

I was checking out the render methods

terse folio Jun 1, 2024, 2:19 AM

#

theres an unused one to manually create chat ^^

#

if you go that route one day

halcyon quarry Jun 1, 2024, 2:20 AM

#

So... the role_messages... is this including non-saved-to-history messages?

#

or is that WIP

terse folio Jun 1, 2024, 2:20 AM

#

halcyon quarry So... the role_messages... is this including non-saved-to-history messages?

it would yes,

currently messages with "save history false" are not saved, so yes and no :P

halcyon quarry Jun 1, 2024, 2:21 AM

#

Here's my cowboy in the desert

terse folio Jun 1, 2024, 2:21 AM

#

I could add an arg to it, to filter by attribute too

#

nice ^^

halcyon quarry Jun 1, 2024, 2:21 AM

#

#

The amount of detail that went into the regional prompting and everything is bonkers

terse folio Jun 1, 2024, 2:23 AM

#

😸 will have to try that out one day
regional prompting

halcyon quarry Jun 1, 2024, 2:23 AM

#

This thing just does it automatically

#

On my To-do list (the unofficial one) is a means to use Flows feature to have LLM write regional prompting syntax

#

This project blows that concept right out of the water

#

stupid LLM

#

This is going to be amazing once it isn't restricted to this POS UI and limited settings

terse folio Jun 1, 2024, 2:28 AM

#

I briefly looked at the github, it talked about running a quant needing larger gpus.
Does that imply it needs a custom fine tune to create the prompts?

halcyon quarry Jun 1, 2024, 2:28 AM

#

Yes

terse folio Jun 1, 2024, 2:28 AM

#

interesting, okay

halcyon quarry Jun 1, 2024, 2:28 AM

#

The project has links to 3 custom finetunes, with quant versions

#

It's on its way to making an image of 5 distinct aliens

#

I'm on 12GB vram, this is working pretty decently FWIW

terse folio Jun 1, 2024, 2:32 AM

#

Awesome, is all that generated by llm?

#

ahh I see

halcyon quarry Jun 1, 2024, 2:32 AM

#

Yes, everything, its crazy

terse folio Jun 1, 2024, 2:32 AM

#

nice!

halcyon quarry Jun 1, 2024, 2:32 AM

#

I followed the install instructions, booted it up and it is doing everything it advertises

#

Here's my aliens

#

This is Realistic Vision so that's why they suck

#

but there's 5, and they're mostly different, and the ones it described in the locations it described, are the ones in those locations!

terse folio Jun 1, 2024, 2:37 AM

#

Great, I guess this could solve that object relation issue many smaller image models have.
"red circle on a blue cube"

keen palm Jun 1, 2024, 2:37 AM

#

Bunch of jacked Ken Doll aliens

halcyon quarry Jun 1, 2024, 2:38 AM

#

Need to edit a damn py file to change models

#

Who the heck makes their users edit py/yaml files to change things?

#

😁

keen palm Jun 1, 2024, 2:44 AM

#

A real asshole, that's who

halcyon quarry Jun 1, 2024, 3:41 AM

#

@terse folio

        local_history = bot_history.get_history_for(ictx.channel.id)

        user_history = local_history.role_messages('user')[-10:]
        llm_history = local_history.role_messages('assistant')[-10:]

        user_msgs = [user_msg.text for user_msg in user_history]
        llm_msgs = [llm_msg.text for llm_msg in llm_history]

#

Just showing that I understand this much

terse folio Jun 1, 2024, 3:42 AM

#

yes

halcyon quarry Jun 1, 2024, 3:42 AM

#

And I see that it does include msgs save_to_history: False

#

(which is good)

terse folio Jun 1, 2024, 3:43 AM

#

you can also pass the HMessage object to the tags.

like '{msg.text}'.format(user_msg)
you could let people get whatever information they want from them too.
Should probably call it something else since that would mess up people's saves

terse folio Jun 1, 2024, 3:44 AM

#

halcyon quarry And I see that it does include msgs `save_to_history: False`

it creates them, just not saving to the history list.
did you observe them in history?

#

i don't remember doing that 🤔

halcyon quarry Jun 1, 2024, 3:50 AM

#

No, I added a print statement and it is printing the text after I use save_history: false

terse folio Jun 1, 2024, 3:50 AM

#

#

yes, regardless of if it saves or not, it will pass the message to output

#

so that you don't need to handle any logic further down the line about history, you can just modify the message and do what you want with it

#

for example setting spoken=true when running tts

#

if you decide to start saving those, nothing will have to change!

#

added a HMessage.in_history property

#

I think the .hidden attribute would work for things like /speak

halcyon quarry Jun 1, 2024, 3:57 AM

#

That sounds appropriate

terse folio Jun 1, 2024, 4:00 AM

#

adding an unsavable attribute too
hmm, autocorrect doesn't like this word..

halcyon quarry Jun 1, 2024, 4:01 AM

#

Got my recent messages working

terse folio Jun 1, 2024, 4:01 AM

#

awesome!

halcyon quarry Jun 1, 2024, 4:11 AM

#

It does not seem to be logging messages if they do not originate from discord...

terse folio Jun 1, 2024, 4:11 AM

#

share the code?

halcyon quarry Jun 1, 2024, 4:12 AM

#

def format_prompt_with_recent_output(ictx, user_name:str, prompt:str):
    try:
        formatted_prompt = prompt
        # Find all matches of {user_x} and {llm_x} in the prompt
        matches = patterns.recent_msg_roles.findall(prompt)
        if matches:
            local_history = bot_history.get_history_for(ictx.channel.id)
            user_history = local_history.role_messages('user')[-10:]
            llm_history = local_history.role_messages('assistant')[-10:]
            user_msgs = [user_msg.text for user_msg in user_history]
            user_msgs = user_msgs[::-1]
            llm_msgs = [llm_msg.text for llm_msg in llm_history]
            llm_msgs = llm_msgs[::-1]
            print("llm_msgs", llm_msgs)
            recent_messages = {'user': user_msgs, 'llm': llm_msgs}

terse folio Jun 1, 2024, 4:13 AM

#

I don't see anything about adding messages there

halcyon quarry Jun 1, 2024, 4:13 AM

#

Anyway, if I trigger my 'loopback' Flow, it requests the LLM to provide more details on its previous response, repeatedly

terse folio Jun 1, 2024, 4:13 AM

#

halcyon quarry Anyway, if I trigger my 'loopback' Flow, it requests the LLM to provide more det...

hmm, where is that in the main branch?
might have missed an insert to history

halcyon quarry Jun 1, 2024, 4:14 AM

#

only the first response is printing over and over

#

I realized the function gets hit twice per flow step, too 😛 (second pass is wasted but has no effect)

terse folio Jun 1, 2024, 4:16 AM

#

mhm good to log things sometimes!

halcyon quarry Jun 1, 2024, 4:17 AM

#

So yes, these responses are not being stored

terse folio Jun 1, 2024, 4:17 AM

#

#

I think this was removed without thinking too much about what It did

#

just saw the session_history part

#

do flows contact llm_gen?

halcyon quarry Jun 1, 2024, 4:19 AM

#

Yes

terse folio Jun 1, 2024, 4:19 AM

#

do you have a simple example flow that calls the llm again

halcyon quarry Jun 1, 2024, 4:19 AM

#

It basically just creates on_message tasks

#

📎 bot.py

#

Youll need this... needed to pass ictx to a few functions.

#

and the updated recent messages function

terse folio Jun 1, 2024, 4:21 AM

#

can always push a commit ^-^

terse folio Jun 1, 2024, 4:22 AM

#

terse folio do you have a simple example flow that calls the llm again

i wanted to test on the old branch

#

to trace where things go

halcyon quarry Jun 1, 2024, 4:22 AM

#

it ships with one, 'loopback'

#

  - trigger: 'loopback'                         # Intended to be used with a prompt like 'draw _______ loopback'
    insert_text: ''
    insert_text_method: 'replace'
    search_mode: user                           # Only search user's prompt (will not trigger from LLM reply)
    should_send_text: false
    should_gen_image: false
    flow:
      - flow_base:
          format_prompt: 'Provide a more detailed version of this prompt: {llm_0}'
          save_history: False
          load_history: -1
          swap_character: M1nty-SDXL
          should_send_text: false
          should_gen_image: false
      - flow_step: Request more details
        flow_step_loops: 3                      # this flow_step will execute 3 times (reminder: includes flow_base)
        should_gen_image: false
      - flow_step: Gen with extra detailed prompt
        should_gen_image: true                  # finally, generate the image
        should_send_text: true                  # finally, send the reply to channel

#

just write something like
person by the lake loopback

terse folio Jun 1, 2024, 4:23 AM

#

okay, will try that one, i guess it will get far enough without SD running for me to debug

halcyon quarry Jun 1, 2024, 4:24 AM

#

Heres one without SD

#

  - trigger: 'flow test'
    flow:
      - flow_base:
          should_gen_text: true
          should_send_text: false
          should_gen_image: false
          format_prompt: Don't be so verbose, say less. {llm_0}
      - flow_step: Spam messages
        flow_step_loops: 10
      - flow_step: Ask bot to say a lot of shit
        format_prompt: Hey, jackass, write me an elaborate poem about kings queens and that sort of stuff
        should_gen_text: true
        should_send_text: true

#

maybe use less than 10 loops though 😛

#

I gotta go to bed now

#

adios amigo

terse folio Jun 1, 2024, 4:28 AM

#

cya!

halcyon quarry Jun 1, 2024, 4:29 AM

#

I’ll bet it has something to do with ‘bot_should_do’ send_text: False

#

Otherwise… something to do with the fact there isn’t a message.id

terse folio Jun 1, 2024, 4:33 AM

#

Hmm, i'm getting

'text': 'Provide a more detailed version of this prompt: {llm_0}', 

'response': 'provide an image that is both realistic and abstract, with the goal to create a more detailed version of this prompt. The original photo was taken by {llm_1} in {location}.'
}```

#

why would the llm return that?

halcyon quarry Jun 1, 2024, 4:34 AM

#

Using the boy.py I sent?

terse folio Jun 1, 2024, 4:34 AM

#

not yet, will try that next ^^

halcyon quarry Jun 1, 2024, 4:34 AM

#

Yes because the function to actually format the text is probably disabled in the version you're running 😛

#

I know what the problem is, almost certainly...

#

The flow passes the same original ictx object back through

#

(maybe the problem)

terse folio Jun 1, 2024, 4:37 AM

#

cant imagine why that would break after updating unrelated code (history)

#

I think I just missed something with the history

halcyon quarry Jun 1, 2024, 4:37 AM

#

I was maintaining two separate lists.
One that was history only,
And another that was everything generated by LLM_Gen

#

(including things in history so, usually duplicate texts)

terse folio Jun 1, 2024, 4:38 AM

#

mhm

halcyon quarry Jun 1, 2024, 4:39 AM

#

still here b/c wife fell asleep lol

terse folio Jun 1, 2024, 4:44 AM

#

#

it seems to work

halcyon quarry Jun 1, 2024, 4:45 AM

#

Formatting the text is working

#

But, the responses are not being added to the History items

terse folio Jun 1, 2024, 4:46 AM

#

oh I see

#

#

these should be changing

#

also curious about this, the llm repeating my input?
or is that your formatting

halcyon quarry Jun 1, 2024, 4:47 AM

#

It is more effective if it doesn't just spam the initial response over and over, but instead use the latest reply

terse folio Jun 1, 2024, 4:48 AM

#

yea, that's the original behavior

#

Also now that you write it out, I can imagine a bug scenario where a different user sends something inbetween a tag and it accidently uses the wrong response as the latest input

#

do you have a way of checking for this?

halcyon quarry Jun 1, 2024, 4:51 AM

#

The task semasphore should block things off while processing

#

I don't think message gets put to history until it enters processing

terse folio Jun 1, 2024, 4:51 AM

#

that only blocks for the individual llm generations

#

due to the way asyncio works, it's possible for another queued event to get in there and run that llm generation instead of the next flow

halcyon quarry Jun 1, 2024, 4:52 AM

#

I doubt it

terse folio Jun 1, 2024, 4:53 AM

#

oh i see

#

the whole flow thing is also in the semaphore

halcyon quarry Jun 1, 2024, 4:53 AM

#

The flow_task is invoked already in the task, doesn't create new tasks

#

yep

terse folio Jun 1, 2024, 4:53 AM

#

good good

halcyon quarry Jun 1, 2024, 4:54 AM

#

Semaphore ftw

terse folio Jun 1, 2024, 5:10 AM

#

#

this was using the main branch

#

it looks like it's also repeating itself?

halcyon quarry Jun 1, 2024, 5:11 AM

#

Some models do better or worse

terse folio Jun 1, 2024, 5:11 AM

#

it says output generated 3 times, but the outputs are all the same.
same input prompt, same output.

halcyon quarry Jun 1, 2024, 5:12 AM

#

If you slap a print statetement like
print("llm replies:", bot_history.recent_messages)

#

Youll see it grow after every LLM Gen

terse folio Jun 1, 2024, 5:14 AM

#

in format_prompt_with_recent_output?

halcyon quarry Jun 1, 2024, 5:15 AM

#

You could print there, or in the start of on_message_task

#

or anywhere inbetween 😛

#

These are the functions that managed session_history (TGWUI History) and recent_messages (everything)

#

I'm not proud of duplicate data but it worked

#

am actually going to run now. adieu

#

again 😛

terse folio Jun 1, 2024, 5:20 AM

#

#

this was the output of main branch

#

the prompt was "cat"
the llm replied "meow"
then it tries to generate an SD prompt for the llm's response

#

I think there was some logic broken before the histroy update

#

what I think is supposed to happen:
Format the prompt with the last llm response

llm: a close up portrait
next
llm: a highly detailed close up portrait
next
llm: a super highly detailed 8k close up portrait... etc

getting longer each time.

halcyon quarry Jun 1, 2024, 5:24 AM

#

correct

#

Welp

#

I updated that for the per channel history but did not test… ugh

terse folio Jun 1, 2024, 5:26 AM

#

im currently using single channel mode

#

#

welp this has been a problem for a while

#

this is a random commit from a week ago.

#

#

from a month ago

#

the last response is also the same except for one difference, "with"->"wearing"

#

not sure what the cause of that was

#

prompt: yes there is a cat in the house

#

something definitely broken! D:

halcyon quarry Jun 1, 2024, 11:19 AM

#

What the heck

#

Can confirm it was definitely 100% working correctly at one point

#

You must be doing something wrong

#

I know the problem

#

When it is preparing the next flow step it is using format prompt with recent messages twice, like I said… it’s formatting the ‘format_prompt’ tag before it can be formatted into the prompt - it’s only updating the parameter

#

I must have broke the ‘format_prompt’ tag in Flows whenever I thought it would be a good idea to format parameters.

Just need to add another condition to that first one …and ‘key != format_tag’

#

…Regardless! The messages list is not growing in dev, when llm_gen processes Flows prompts

halcyon quarry Jun 1, 2024, 12:17 PM

#

It pains me so much when this happens… find out something wasn’t workin correctly for awhile

halcyon quarry Jun 1, 2024, 1:31 PM

#

Fix pushed to main

halcyon quarry Jun 1, 2024, 2:16 PM

#

Also to dev

visual dagger Jun 1, 2024, 3:04 PM

#

good luck guys

#

everything is going good?

halcyon quarry Jun 1, 2024, 4:02 PM

#

Yes we’re working out some kinks in a robust history manager Reality came up with that is going to enable many new features

#

Finding a few hidden bugs along the way

halcyon quarry Jun 1, 2024, 4:19 PM

#

Got a few new Tags in mind that will capitalize on it, enable some interesting interactions

halcyon quarry Jun 1, 2024, 4:40 PM

#

For example: because it captures many attributes for each message, we can easily make a setting that would make each individual user have their own history. That’s a bit heavy handed for most people, but this could be very useful in Tags system, to trigger the bot only seeing that user’s messages in history

halcyon quarry Jun 1, 2024, 4:57 PM

#

On my channel someone requested an image. Then they wrote “make him creepier” and I have no method for that yet but could add a tags thing that triggers on something like that, only shows that users messages in history and it will probably do a good job making a new image prompt

#

We can make it so that DMs to the bot are private conversations (except of course in CMD / local chat logs)

keen palm Jun 1, 2024, 4:59 PM

#

A tag with a trigger like "modify" that takes the previous prompt(s) and adds the new prompt to it, then sends that to the LLM to create an image generation prompt

halcyon quarry Jun 1, 2024, 5:00 PM

#

That is possible right now

#

{llm_0} and similar variables were not working with format_prompt tag in Flows feature, until I fixed bug this morning

#

That works in format_prompt tag in flows or not

#

But a new tag like I mentioned could ensure other user/bot interactions are omitted (filtered out) from history/recent messages variable

#

tag will probably be something like ‘filter_history’. Need to brainstorm how to make something like this as simple and effective as possible to use

#

Or a few tags

terse folio Jun 1, 2024, 5:56 PM

#

halcyon quarry On my channel someone requested an image. Then they wrote “make him creepier” a...

I think the most intuitive control for this would be replies in discord.
By selecting a message (by the bot) and saying something this could be given an extra flag like "in reply to" which tags could look for.

halcyon quarry Jun 1, 2024, 8:49 PM

#

@terse folio you’ll either have already considered this, or it will be painfully annoying. I’m out and about atm.
Most bot replies are sent using the send_long_messages() function. In the id_pairing branch, I collected the msg IDs written by this function in lists and appended them to the recent messages IDs lists.
If a message exceeded 2000 characters and got chunked, I had all the message IDs together.

#

By matching an ID in one of thise sublists I knew all the messages that were sent with it

terse folio Jun 1, 2024, 8:50 PM

#

halcyon quarry <@226121791670583296> you’ll either have already considered this, or it will be ...

The current code takes the message id of the last message from send_long_messages()

i can change the message_id attribute to a list, contain multiple!

#

and give a property to HMessage .id which will return the last item in the list

halcyon quarry Jun 1, 2024, 8:52 PM

#

terse folio The current code takes the message id of the last message from send_long_message...

Good! I’m half in the bag and was thinking about this

halcyon quarry Jun 2, 2024, 12:55 AM

#

The thing with send_long_message() is that it's used for a few things...

To respond to the average discord message request
On character change: Sending either the character's greeting or most recent history to channel
To post a copy of the current settings (post current settings feature)
Continue/Regen task

#

Message request and Cont/Regen task messages, we care about.
the other two we don't (hidden attribute)

#

I'm looking at what you did with hidden... nice

#

@terse folio What is the distinction between hidden and unsavable?

terse folio Jun 2, 2024, 1:12 AM

#

halcyon quarry <@226121791670583296> What is the distinction between `hidden` and `unsavable`?

hidden hides it from the chat log for tgwi.

unsaveable doesn't save it to the json file.
hidden items could be saved to json, (but currently theres no example of that)

terse folio Jun 2, 2024, 1:15 AM

#

halcyon quarry The thing with send_long_message() is that it's used for a few things... - To re...

yup that's fine,
HMessages created for posting current settings for example don't get saved to history, because the save function is used in llm_gen.

send_long_message just collects the id (soon ids)

halcyon quarry Jun 2, 2024, 1:17 AM

#

So the items marked as hidden + unsavable do not get saved to json - but are still collected, yea?

#

(just checking I understand it)

terse folio Jun 2, 2024, 1:17 AM

#

in ram, yes

#

for example for usage in flows

halcyon quarry Jun 2, 2024, 1:19 AM

#

confirming Flows is working

#

with formatting in the recent messages

#

Sorry for the wild goose chase

keen palm Jun 2, 2024, 1:21 AM

#

https://tenor.com/view/goose-goosey-geese-mean-goose-gif-24833721

Tenor

terse folio Jun 2, 2024, 1:21 AM

#

nice nice!

halcyon quarry Jun 2, 2024, 1:21 AM

#

yeah we spent an hour or so last night dicking around with this thing and most of the time scratching our heads - all because I broke something like a month or so ago

#

Really, the only thing still bugging me is the logging... I love how much less data is being written, but 2 files per channel is still going to be a massive flood of files for some people

#

must be complicated though as I understand you probably woulv'e already changed that

terse folio Jun 2, 2024, 1:26 AM

#

with some reworking of the search function,
I could remove the need for the tgwui files.

They could be optionally created with an export command, like /export_chat_tgwui
so it can be loaded in the webui.

And it's still on the todolist to move the logs to an internal location.

halcyon quarry Jun 2, 2024, 1:27 AM

#

I'm in no rush, and I could try attempting it myself if you have things to do in the coming days.

Really the last detail holding this back from Main

#

Everything ~~else~~ seems to be working expectedly 🍺

#

Got a pretty interesting error when trying to run a 'Flow'

#

Didn't happen on the first one

terse folio Jun 2, 2024, 1:37 AM

#

wonder how that happened,
just replace index=index with index

#

I don't think i've tested that function

halcyon quarry Jun 2, 2024, 1:38 AM

#

Ill try that

terse folio Jun 2, 2024, 1:38 AM

#

halcyon quarry I'm in no rush, and I could try attempting it myself if you have things to do in...

Have been getting more work on the tts api ^-^
will be a little more on and off

halcyon quarry Jun 2, 2024, 1:41 AM

#

Error did not recur after changing that

#

I'll try changing the internal filepaths on Monday if you haven't got around to it, just lmk if you do go for it

terse folio Jun 2, 2024, 1:56 AM

#

I'll get to it, a few things need to change together

halcyon quarry Jun 2, 2024, 1:57 AM

#

Praise you 🙌

halcyon quarry Jun 2, 2024, 11:46 AM

#

Got an updater script working for Windows

#

Also making ones for Linux, MacO and WSL but will not be able to test those...
Anyone using the bot on any other OS than Windows?

#

halcyon quarry Jun 2, 2024, 12:11 PM

#

pushed updater scripts to dev

keen palm Jun 2, 2024, 11:56 PM

#

The delete/replace last response code needs the new history implementation to work?

halcyon quarry Jun 2, 2024, 11:57 PM

#

Yes, I don't want to do anything with History until we have this finalized because the feature would have to be changed.

#

It'll be easy though. Expect such a feature in the coming days

halcyon quarry Jun 3, 2024, 1:56 AM

#

@terse folio I seem to have successfully moved bot.py up into ad_discordbot and launched it via a new launcher.

set a new dir_tgwui path in SharedPath
dir_root is now os.path.join(dir_tgwui, 'ad_discordbot')
Appended dir_tgwui to sys.path
Searched and updated everywhere to ensure TGWUI stuff now uses shared_path.dir_tgwui

halcyon quarry Jun 3, 2024, 2:31 AM

#

Also added a CMD_FLAGS.txt to the bot folder

#

@terse folio I cloned dev to dev-plus and pushed the changes I mentioned.

bot.py now in ad_discordbot
Added launchers
Bot now intended to be launched via one of the launchers

keen palm Jun 3, 2024, 2:44 AM

#

So no more python bot.py?

halcyon quarry Jun 3, 2024, 2:46 AM

#

Correct!

#

Just double-click launcher

#

Copying TGWUI quite hard, I am

#

it basically goes up a level, runs the code that TGWUI normally would to activate venv, gets our new CMD_FLAGS and runs the python cmd with them.
It will warn if bot.py is in the TGWUI directory

#

keen palm Jun 3, 2024, 2:51 AM

#

That's good. So now those idiots who kept forgetting to move bot.py over don't have to worry about it anymore

halcyon quarry Jun 3, 2024, 2:51 AM

#

yes, and with this there will be no manual steps required after updating

#

And if we add any requirements (like Reality's History management needs a particular json library), the updater script will now handle that as well

terse folio Jun 3, 2024, 11:07 AM

#

halcyon quarry <@226121791670583296> I seem to have successfully moved bot.py up into `ad_disco...

Amazing!

halcyon quarry Jun 3, 2024, 11:12 AM

#

I am in agreement that ideally this will be an extension for TGWUI and I see you making baby steps in that direction here and there - until then, this is just to improve current user experience

terse folio Jun 3, 2024, 11:32 AM

#

Improvement for me too,can delete my workspace version of the bot and just work from tgwui!

#

#

#

This creates ambiguity

#

im not sure how python would handle that if you used sys.append(path)

halcyon quarry Jun 3, 2024, 11:49 AM

#

It’s working

#

I first used .. before the imports, except it doesn’t work for that first one (extensions module)

terse folio Jun 3, 2024, 11:49 AM

#

good to know, also this line can be removed because bot.py is now inside ad_discordbot

#

have you tried out the new internal history save paths?

halcyon quarry Jun 3, 2024, 11:52 AM

#

I did - there’s still a problem though…

#

I did like having “normal history” written to logs, and ideally in a subdir

terse folio Jun 3, 2024, 11:53 AM

#

I was thinking "normal history" could be exported optionally with a command.

Or perhaps this could be a setting if people want it or not

halcyon quarry Jun 3, 2024, 11:55 AM

#

Well we do already have a setting to save history 🤗

terse folio Jun 3, 2024, 11:55 AM

#

thought that was just for in general with the internal logs

halcyon quarry Jun 3, 2024, 11:56 AM

#

I think Dundell or Speedy among however many users actually use this bot, will be dismayed if chat history not available to import to tgwui

terse folio Jun 3, 2024, 11:57 AM

#

mhm, that's what I was thinking to give people the option if they want it imported to tgwi or just internal

#

also hmm, how should the files be saved?
I don't think tgwi searches subdirs does it?

halcyon quarry Jun 3, 2024, 12:07 PM

#

I was happy with the multiple channel file I had, which could be extracted to TGWUI compatible files

#

Id be happy with a subdir of already compatible files

terse folio Jun 3, 2024, 12:08 PM

#

oh so the user would move the one they want up?

halcyon quarry Jun 3, 2024, 12:08 PM

#

Happier really

#

Yes

keen palm Jun 3, 2024, 12:41 PM

#

I actually don't have any plans to import chat history to tgwui

halcyon quarry Jun 3, 2024, 12:43 PM

#

A reality I was thinking about it and I think you are right that we should still just have one file being saved her channel

#

I could just revise the utility so that multiple files can be dragged and dropped onto it for processing to convert those files into compatible chat fire history files if user is Carol enough to do it

keen palm Jun 3, 2024, 12:46 PM

#

Using speech to text this morning?

halcyon quarry Jun 3, 2024, 12:46 PM

#

How do you guess

keen palm Jun 3, 2024, 12:46 PM

#

You said "if user is Carol enough to do it"

#

I doubt I would Carol that much

halcyon quarry Jun 3, 2024, 12:47 PM

#

I am also a master of sarcasm

#

I am also a master of pissing off my wife

#

These two masteries certainly have no correlation

terse folio Jun 3, 2024, 12:49 PM

#

halcyon quarry I could just revise the utility so that multiple files can be dragged and droppe...

I'm not sure the best way to go about this.
You could import the history.py maybe...
And run render_to_tgwui()

alternatively, a bot command to convert history.
But that would take a while as you wait for the bot to come online.

You could make another start file that starts the bot in minimal mode?
Like disable tgwui, SD and all, load no model.

Just so you can use a few util commands like to convert files.

halcyon quarry Jun 3, 2024, 12:52 PM

#

I do have a command already to save history on demand maybe when using it that will save the TGWUI file

terse folio Jun 3, 2024, 12:52 PM

#

yea

#

for the tgwui exports, I could name the sub folders based of guildname and channel name

#

because it doesn't really matter if the names change, i'm never going to read those files

#

it'll just create a new subdir

halcyon quarry Jun 3, 2024, 12:57 PM

#

Personally I think these directories they get created should just be one level deep where the directories name is the unique ID then all of the files in there we’ll have the guild name and channel name for the file names

terse folio Jun 3, 2024, 12:57 PM

#

unique_id is the time

#

so there will never be more than one file in there (unless 2 people start chatting within the same second in 2 different channels)

halcyon quarry Jun 3, 2024, 1:00 PM

#

You’re right! Sorry was still thinking about my method where all was stuffed in one file

terse folio Jun 3, 2024, 1:00 PM

#

currently internal history is saved under

channel_id/char+mode/time.json

#

tgwi saves as
mode/char/time.json
I think

halcyon quarry Jun 3, 2024, 1:02 PM

#

thinking

terse folio Jun 3, 2024, 1:02 PM

#

so we could just call them time+channel_id.json and it would all be set

#

or guild_name+channel_name inplace of id

#

This would stay sorted by time so it's easier to find relevant files

halcyon quarry Jun 3, 2024, 1:04 PM

#

This works for me. Much better than 2x files being saved there per channel 😛
And since we are not reading from it, we can have a setting such as save_history_to_tgwui_logs to make this copy optional

terse folio Jun 3, 2024, 1:07 PM

#

export_for_tgwui: true # Defines if your history save can be later imported into standalone text-generation-webui.

#

already added one

halcyon quarry Jun 3, 2024, 1:07 PM

#

beautiful

terse folio Jun 3, 2024, 1:08 PM

#

/save conversation should call the normal history save function

i could add an arg to force it to save to tgwi if the setting is off though

halcyon quarry Jun 3, 2024, 1:09 PM

#

If the setting is off, I think it should just not register the command (it may do this already)

terse folio Jun 3, 2024, 1:10 PM

#

halcyon quarry If the setting is off, I think it should just not register the command (it may d...

if possible, lets try to keep all commands registered.
Just disable them inside the command if the flag is off

#

that lets people change settings without having to restart their discords

halcyon quarry Jun 3, 2024, 1:12 PM

#

I think that's a minor inconvenience opposed to having commands available which don't do anything

terse folio Jun 3, 2024, 1:13 PM

#

Mhm, slash commands are quite different from the old style message commands

terse folio Jun 3, 2024, 3:11 PM

#

haha that works but the bot doesn't seem to know what it is on fresh start

#

or maybe I broke it

terse folio Jun 3, 2024, 3:57 PM

#

pushed it

#

Also history.save() has a force_tgwui arg now.
This wont appear in typehints because of the class builder thing.
Going to figure out the proper way to do variable typehints later.

#

That will export history to TGWUI even if the setting is disabled

terse folio Jun 3, 2024, 5:07 PM

#

Oh cool, you can typehint what kind of function you want.

This is Callable[[int], bool]
telling us we want a function that takes an int arg and outputs true/false!

terse folio Jun 3, 2024, 5:26 PM

#

new awesome update, Ctrl+C isn't necessary anymore.

not sure how this handles on Linux though

halcyon quarry Jun 3, 2024, 5:26 PM

#

Oh wow!

#

That's cool

terse folio Jun 3, 2024, 5:27 PM

#

sadly it's limited to about 4-5 seconds before something force kills the window.

#

but this is plenty time for small stuff ^^

#

Totally going to use this in all my code too now!
I attempted this idea before, I don't remember why I gave up

halcyon quarry Jun 3, 2024, 5:37 PM

#

I'm about to look and I'm scared

#

Note that the things I pushed yesterday were not to dev

terse folio Jun 3, 2024, 5:38 PM

#

Yea

halcyon quarry Jun 3, 2024, 5:39 PM

#

Should I be scared?

keen palm Jun 3, 2024, 5:39 PM

#

https://tenor.com/view/star-wars-yoda-im-not-afraid-you-will-be-savage-gif-5323241

Tenor

when ppl get cocky

▶ Play video

terse folio Jun 3, 2024, 5:40 PM

#

About using the window close code?
no

#

about merging? maybe

halcyon quarry Jun 3, 2024, 5:41 PM

#

Could you push your changes to dev-plus?

terse folio Jun 3, 2024, 5:41 PM

#

the worst that could happen is Ctrl C doesn't fully stop it, then you have to press X

But if on linux, maybe using the kill -9 <pid> command if using CLI only.

terse folio Jun 3, 2024, 5:41 PM

#

halcyon quarry Could you push your changes to dev-plus?

not sure how i'd extract only the last few commits

halcyon quarry Jun 3, 2024, 5:41 PM

#

ugh yeah that's probably not going to work...

#

Well, the good news is, besides the actual updater scripts / launchers / etc, the actual code changes took me like 10 minutes

#

should be able to snipe those pretty easily in compare files view

terse folio Jun 3, 2024, 5:47 PM

#

^^

#

it's good when branches don't overlap

halcyon quarry Jun 3, 2024, 8:04 PM

#

I'm taking out the save_interval setting and just making it arbitrary 300

#

Alright I THINK I merged everything nicely... I compared every file one at a time

terse folio Jun 3, 2024, 8:06 PM

#

Sure

keen palm Jun 3, 2024, 8:11 PM

#

https://tenor.com/view/wow-fascinating-interesting-fake-sure-gif-4110253572494542171

Tenor

halcyon quarry Jun 3, 2024, 8:14 PM

#

I noticed that for some reason (at least for me), the model menu is not appearing on startup, if I don't use a model in the cmd flags

#

I haven't launched the bot in ages without the flag

#

Anyway... fixed that now.

terse folio Jun 3, 2024, 8:15 PM

#

there's a show model menu flag

halcyon quarry Jun 3, 2024, 8:15 PM

#

Yes, I just removed that

#

😛

terse folio Jun 3, 2024, 8:15 PM

#

huh, I thought it was missing!

#

okay

#

oh, is the flag "no model menu"?

halcyon quarry Jun 3, 2024, 8:16 PM

#

Instead of
if model in shared.args...
elif len all_models == 1...
elif shared.model.menu...

change last one to just else:

terse folio Jun 3, 2024, 8:16 PM

#

I see

halcyon quarry Jun 3, 2024, 8:16 PM

#

idk why my personal shared.model_menu == False

terse folio Jun 3, 2024, 8:16 PM

#

also I liked that behavior, let me have a 2nd start script that loads the bot without a model so I could quickly test it

halcyon quarry Jun 3, 2024, 8:17 PM

#

but anyone running this is going to want to choose a LLM model on startup

#

if they didn't already use it in flag

terse folio Jun 3, 2024, 8:17 PM

#

yea

halcyon quarry Jun 3, 2024, 8:17 PM

#

Well just use --model None

#

😛

terse folio Jun 3, 2024, 8:17 PM

#

oh didn't know that was a thing

#

Cool

halcyon quarry Jun 3, 2024, 8:18 PM

#

Pretty sure it is. The model loader will unload model if it receives "None"

#

(string)

#

I'm going to update dev-plus now.

For the love of all things, let's stick to this branch for now 🙂

#

I think this is going to main tonight or tomorrow then we chuck those branches and stick with dev again

terse folio Jun 3, 2024, 8:21 PM

#

halcyon quarry I'm going to update dev-plus now. For the love of all things, let's stick to th...

I agree, I'm not fully used to using git with merging.
I don't know all the commands, and merging branches (even without conflicts) creates issues often

halcyon quarry Jun 3, 2024, 8:24 PM

#

spent half my day playing with Stable Diffusion

#

aint gonna lie 😛 We'll see how SD3 compares but man, SDXL turned out to be massive

calm rain Jun 3, 2024, 8:28 PM

#

SD3-Medium is so good. It'll obviously be limited until finetunes and all come out, but, like, it's so good

halcyon quarry Jun 3, 2024, 8:28 PM

#

Can you rename this channel to #general

calm rain Jun 3, 2024, 8:29 PM

#

lol @keen palm i saw that

terse folio Jun 3, 2024, 8:29 PM

#

ad_general

keen palm Jun 3, 2024, 8:30 PM

#

Whoopsy

terse folio Jun 3, 2024, 8:30 PM

#

because the bot has a lot of tools

halcyon quarry Jun 3, 2024, 8:30 PM

#

Reality maybe I screwd something up by replacing manager.save_interval with 300, or something.
I booted up, chatted a bit, and closed. No history saved

#

Or, I screwed something up merging us 😛

terse folio Jun 3, 2024, 8:31 PM

#

halcyon quarry Reality maybe I screwd something up by replacing `manager.save_interval` with `3...

I set it to 300 ( 5 minutes )

#

use ctrl c or /save_conversation

halcyon quarry Jun 3, 2024, 8:31 PM

#

/save_conversation failed when I tried it

terse folio Jun 3, 2024, 8:31 PM

#

was getting a feel for how it would work on longer times

#

okay i'll check it out

#

push to dev?

halcyon quarry Jun 3, 2024, 8:31 PM

#

I put faith in that clicking the X, only had 1 chat exchange

terse folio Jun 3, 2024, 8:32 PM

#

try the normal dev branch, see if that works

halcyon quarry Jun 3, 2024, 8:32 PM

#

switch to dev-plus, so we keep a clean line between the beautiful work you've done, and the potential shit I ruined

terse folio Jun 3, 2024, 8:32 PM

#

then we know if the error came from merging

halcyon quarry Jun 3, 2024, 8:32 PM

#

OK

terse folio Jun 3, 2024, 8:32 PM

#

in vscode you can switch branches easily ^^
if you dont already do that

halcyon quarry Jun 3, 2024, 8:35 PM

#

I switched to dev, set /main, had one exchange, then pressed Ctl+C

#

It's frozen on Running cleanup tasks:

terse folio Jun 3, 2024, 8:35 PM

#

are you on windows?

halcyon quarry Jun 3, 2024, 8:35 PM

#

Maybe it waits for the duration to elapse?

#

yeah

terse folio Jun 3, 2024, 8:36 PM

#

halcyon quarry Maybe it waits for the duration to elapse?

no it should be done in a second (the second is just for display)

halcyon quarry Jun 3, 2024, 8:36 PM

#

terse folio Jun 3, 2024, 8:36 PM

#

are there lines under that?

halcyon quarry Jun 3, 2024, 8:36 PM

#

Nope!

terse folio Jun 3, 2024, 8:36 PM

#

huh odd

halcyon quarry Jun 3, 2024, 8:36 PM

#

froze

#

Your branch too, unadulterated

terse folio Jun 3, 2024, 8:37 PM

#

Hmmm!

keen palm Jun 3, 2024, 8:37 PM

#

https://tenor.com/view/this-is-fine-gif-24177057

Tenor

terse folio Jun 3, 2024, 8:48 PM

#

okay not working for me too, I wasn't testing with a model loaded

#

it seems there's an issue running async code on exit

#

which I wish wasn't the case, because lots of cool stuff to do there!

halcyon quarry Jun 3, 2024, 8:49 PM

#

no bueno

terse folio Jun 3, 2024, 8:58 PM

#

ohboy merge time

#

annnd i pushed to the wrong branch haha

keen palm Jun 3, 2024, 8:59 PM

#

lol

terse folio Jun 3, 2024, 8:59 PM

#

there we go, try that

halcyon quarry Jun 3, 2024, 9:01 PM

#

Jettin, but will try later!

halcyon quarry Jun 3, 2024, 9:51 PM

#

stop with dev 😭

#

testing and hopefully you changed enough lines I can count on one hand

#

@terse folio No bueno amigo

#

With Ctrl C

terse folio Jun 3, 2024, 10:17 PM

#

halcyon quarry stop with dev 😭

did you want to move all dev to the other branch?
I thought you were merging the other into dev

halcyon quarry Jun 3, 2024, 10:17 PM

#

This history update is a nightmare we can’t seem to wake up from

#

I updated dev-plus to catch up with dev

#

Don’t worry, if you just tell me what files you updated I can find the few changes you made 2 hours ago

#

And whatever you’re theoretically about to change

terse folio Jun 3, 2024, 10:20 PM

#

halcyon quarry Don’t worry, if you just tell me what files you updated I can find the few chang...

the commits show all the diffs

terse folio Jun 3, 2024, 10:20 PM

#

halcyon quarry And whatever you’re theoretically about to change

just forgot a line abou making sure dirs exist that i forgot to move when i was moving a function

#

pushed

#

half the changes between dev and dev+ are your scripts

halcyon quarry Jun 3, 2024, 10:26 PM

#

Yep

terse folio Jun 3, 2024, 10:26 PM

#

#

where does this save to?

#

tgwui?

halcyon quarry Jun 3, 2024, 10:27 PM

#

Since script is moving, it will save in the main ad discordbot folder

#

log module is the only one I had trouble with applying shared_dir

#

But there’s just the one reference there so simply removed the osjoin

terse folio Jun 3, 2024, 10:29 PM

#

hmmm, will check that out

#

oh nvm that was the only change in logs.py

#

I see

halcyon quarry Jun 3, 2024, 10:30 PM

#

Yeah couldn’t seem to import shared files, said fuckit after one try and error

terse folio Jun 3, 2024, 10:30 PM

#

this is not needed

#

since manager.save_interval is already 300

#

even if you delete it from the config ^-^

halcyon quarry Jun 3, 2024, 10:31 PM

#

How about if I also delete it from the manager, eh?

#

gowron1

terse folio Jun 3, 2024, 10:32 PM

#

sure, then just do timeout=300 as a default arg

#

the original version was to let you supply a timeout or use the default from config

halcyon quarry Jun 3, 2024, 10:32 PM

#

I figured we leave the manager with actual config keys only

terse folio Jun 3, 2024, 10:32 PM

#

okay

halcyon quarry Jun 3, 2024, 10:33 PM

#

So long as both the Ctl C and X method work relatively consistently I think that’s fine.

terse folio Jun 3, 2024, 10:33 PM

#

oh interesting, had we not updated to shared_path with wildcards in these functions?
I remember writing the code for that

terse folio Jun 3, 2024, 10:34 PM

#

halcyon quarry So long as both the Ctl C and X method work relatively consistently I think that...

was just an issue with async (that was the reason I gave up on it in my library, because i privately use async for everything there)

#

going through all the changes side by side

#

looks good, just will change 1 thing on merge

#

and push to dev branch okay?

#

or where do you want to push it?

#

#

found out I can do the "cherry picking"

#

like thing I wanted to do

#

just using the merge resolver in vscode

#

even where there's no direct conflict, i can see all the modified areas and search for errors

#

like this where "force_tgwui" is undefined

#

#

you need a gitignore file!

#

#

Id recommend using markdown files, can be more expressive with headers and formatting!
And github will render them nicely :)

halcyon quarry Jun 3, 2024, 11:17 PM

#

I just need to make a WIKI section for those, really

halcyon quarry Jun 3, 2024, 11:18 PM

#

terse folio you need a gitignore file!

Make it happens 🤗

halcyon quarry Jun 3, 2024, 11:18 PM

#

terse folio oh interesting, had we not updated to shared_path with wildcards in these functi...

This is probably a side effect of my bad merge that one time

#

Or yeah, you overlooked it but that’s impossible 😎

terse folio Jun 3, 2024, 11:20 PM

#

halcyon quarry Or yeah, you overlooked it but that’s impossible 😎

haha, well I sure know I wrote something to do with wildcards,
yes I could have missed replacing it

halcyon quarry Jun 3, 2024, 11:21 PM

#

Did you see if save_conversation is working now? It was failing when I tried earlier

#

There’s potentially a bug in wildcards when it processes nested wildcards… noticed a grey variable… need to explore it

#

There’s either bug or dead code

terse folio Jun 3, 2024, 11:23 PM

#

need to find why characters sometimes aren't loaded on first start

#

the state dict

halcyon quarry Jun 3, 2024, 11:23 PM

#

The feature is rly cool too btw

#

I haven’t experienced that yet…

terse folio Jun 3, 2024, 11:25 PM

#

/saveconvo works for me

terse folio Jun 3, 2024, 11:25 PM

#

halcyon quarry I haven’t experienced that yet…

the character is there, just the variables are None

#

like character_menu

#

everything works fine

#

so far

#

pushing to dev with merge

halcyon quarry Jun 3, 2024, 11:38 PM

#

terse folio like character_menu

I’ll look into this

keen palm Jun 4, 2024, 12:23 AM

#

This is strange. I have a 34B model that was working fine but then started doing that repetitive adjective nonsense I had seen with previous models.
I just gave it the same prompt in two different channels. On one channel, the output was completely fine, while on another it became gibberish.
I'm going to clear conversation history in the gibberish channel and see if it's better.

terse folio Jun 4, 2024, 12:24 AM

#

Sometimes if an llm starts talking giberish it will think that's what its supposed to do and wont stop.
Like repeating itself for each reply.
Clearing history for that channel should work ^^

keen palm Jun 4, 2024, 12:26 AM

#

The previous history didn't have that gibberish though

#

But clearing it did help.

terse folio Jun 4, 2024, 12:27 AM

#

keen palm The previous history didn't have that gibberish though

that is odd though, yea, sometimes I get an issue here and there.

Sometimes I get a bunch of unicode nonsense, I click regenerate and it's all fine

halcyon quarry Jun 4, 2024, 12:27 AM

#

Bad seed

keen palm Jun 4, 2024, 12:30 AM

#

Hmmm

#

Does [[save_history:false]] keep the response from being held in RAM?

halcyon quarry Jun 4, 2024, 12:31 AM

#

Nope

terse folio Jun 4, 2024, 12:31 AM

#

keen palm Does [[save_history:false]] keep the response from being held in RAM?

if that was true your bot would forget everything every message

#

it only doesn't save to file

keen palm Jun 4, 2024, 12:33 AM

#

Yeah, I have save_history set to true in config, but I was using a quick tag to keep it from saving responses while I played with different generation parameters. I guess those responses were still contaminating the tests.

terse folio Jun 4, 2024, 12:33 AM

#

that's an interesting use case

halcyon quarry Jun 4, 2024, 12:33 AM

#

Nah

terse folio Jun 4, 2024, 12:33 AM

#

have you tried regen?

#

is that functional?

halcyon quarry Jun 4, 2024, 12:34 AM

#

save_history: false does not save it to history

terse folio Jun 4, 2024, 12:34 AM

#

you could regen to test different settings keeping the same content

halcyon quarry Jun 4, 2024, 12:34 AM

#

It will save to a separate variable

keen palm Jun 4, 2024, 12:34 AM

#

Regen doesn't correctly replace the old response

terse folio Jun 4, 2024, 12:34 AM

#

I see

#

ill take a look into it

halcyon quarry Jun 4, 2024, 12:34 AM

#

Regen currently replaces the most recent response, but I dont think it overwrites it in history

#

But that can be changed by that setting I arbitrarily set at one point...

keen palm Jun 4, 2024, 12:35 AM

#

Yeah definitely doesn't overwrite it

halcyon quarry Jun 4, 2024, 12:35 AM

#

(just change that to True)

terse folio Jun 4, 2024, 12:35 AM

#

probably should be true by default.
that's what people would expect as ChatGPT does it.

keen palm Jun 4, 2024, 12:36 AM

#

ChatGPT allows you to swipe between responses and choose one to continue with, but I imagine that's not really feasible in discord

terse folio Jun 4, 2024, 12:36 AM

#

so creating a branch?

#

or does it just replace it in the history

keen palm Jun 4, 2024, 12:37 AM

#

Creating a branch

terse folio Jun 4, 2024, 12:37 AM

#

could be doable

keen palm Jun 4, 2024, 12:38 AM

#

It's been a while since I've played with ChatGPT, but IIRC you could have different branches off responses, and you could swipe between them to carry on conversation in that branch

terse folio Jun 4, 2024, 12:38 AM

#

create a new thread in discord and assign the temporary history to that.

#

maybe a different command for that

#

/regen_to_thread

keen palm Jun 4, 2024, 12:39 AM

#

At this point I mostly need the ability to delete a response or replace it, since usually there's just small changes that need to be made

terse folio Jun 4, 2024, 12:40 AM

#

does regen currently post another message or edit the original?

#

looks like it sends a new

halcyon quarry Jun 4, 2024, 12:41 AM

#

We'll fix this once the new History is in main

#

Which should be very soon

#

Checking our status now 😛

keen palm Jun 4, 2024, 12:46 AM

#

https://tenor.com/view/power-rangers-power-rangers-dino-fury-gif-20851104

Tenor

#

Loading a different character would swap out the history in RAM though, right?

terse folio Jun 4, 2024, 12:55 AM

#

yea

halcyon quarry Jun 4, 2024, 1:10 AM

#

You overlooked the change I committed here 🙂

#

Oh wait... maybe I never actually pushed it

#

Got this cryptic error when trying to close after LLM model wasn't loaded

#

Must be in something imported, can't find the source

#

Ok I get it again

keen palm Jun 4, 2024, 1:37 AM

#

https://tenor.com/view/jimcarrey-dumbanddumber-gif-5394418

Tenor

halcyon quarry Jun 4, 2024, 2:02 AM

#

@terse folio In regards to this bit here...

    def get_history_for(self, id_: ChannelID=None, character=None, mode=None, fp=None) -> History:
        state_dict = bot_settings.settings['llmstate']['state']
        mode = mode or state_dict['mode']
        character = character or state_dict["character_menu"] or 'unknown_character'
        
        search = True
        if self.change_char_history_method == 'new':
            search = False # don't import old logs

On startup, the search handling currently contradicts autoload_history.
It should try finding the most recent history regardless of change_char_history_method

I know one of our backburner goals is "per channel characters", but until then...
I think we should initialize a flag in History Manager should_refresh_history
If autoload_history: True then it should initialize should_refresh_history: False otherwise True

The flag should then change from change_char_task if new char != old char

#

Lmk if this makes sense to you as well...

...
    class_builder_history: type             = field(default=History)
    should_refresh_history: bool            = field(default=False)

...

        search = True
        if self.change_char_history_method == 'new' and self.should_refresh_history:
            search = False # don't import old logs
            self.should_refresh_history = False

#

In change char task

        # Set history
        if not bot_history.autoload_history or bot_history.change_char_history_method == 'new': # if we do not keep history...
            bot_history.should_refresh_history = True
            if source == 'reset':
                # create a clone with same settings but empty, and replace it in the manager
                bot_history.get_history_for(ictx.channel.id).fresh().replace()
            else:
                bot_history.unload_history() #.clear_all_history()

#

Now where the heck do we initialize those settings from now...

terse folio Jun 4, 2024, 2:15 AM

#

halcyon quarry Got this cryptic error when trying to close after LLM model wasn't loaded

it's part of bash on windows, when you press ctrl c it kills the program then asks if you want to exit fully.

just use the close button, it works

halcyon quarry Jun 4, 2024, 2:15 AM

#

Just looking for the most appropriate place to check condition for autoload_history to initialize the flag state

terse folio Jun 4, 2024, 2:18 AM

#

we don't need autoload_history, it was used to bring the last chat into memory before chatting.

But now chat is loaded when requested.
If you want a new chat, history.fresh().replace()

halcyon quarry Jun 4, 2024, 2:18 AM

#

terse folio Jun 4, 2024, 2:19 AM

#

what does refresh history do?

halcyon quarry Jun 4, 2024, 2:19 AM

#

autoload_history should find the most recent history file when booting up, no?

terse folio Jun 4, 2024, 2:19 AM

#

im having a hard time interpreting the code

terse folio Jun 4, 2024, 2:19 AM

#

halcyon quarry autoload_history should find the most recent history file when booting up, no?

it already finds the most recent when you get_history_for

#

it either creates a new one if search is disabled, or uses the latest

halcyon quarry Jun 4, 2024, 2:19 AM

#

It's creating a new history file if change_char_method is new

#

But the character hasn't changed

terse folio Jun 4, 2024, 2:20 AM

#

you want the same history for multiple characters with that=new?

#

also i guess I didn't fully understand that setting, just thought it was to keep or replace history on load

halcyon quarry Jun 4, 2024, 2:21 AM

#

I want change_char_method: keep to only reset the channel history if /reset_conversation was used in that specific channel

terse folio Jun 4, 2024, 2:21 AM

#

if change_char_method keep and single channel, should everything be the same file including character switches?

halcyon quarry Jun 4, 2024, 2:21 AM

#

if change_char_method: new it will reset on character changes as well

#

It should create a new file so that the previous history is not outright deleted

#

when /reset_conversation is used

terse folio Jun 4, 2024, 2:23 AM

#

terse folio if change_char_method keep and single channel, should everything be the same fil...

just trying to understand the logic

halcyon quarry Jun 4, 2024, 2:24 AM

#

I can see the solution to my gripe about autoload_history: True with change_char_method: new

#

This toggle should do the trick, just need to initialize it based on autoload_history

#

well, maybe what I put in screenshot is correct :p

#

trying...

terse folio Jun 4, 2024, 2:26 AM

#

if change_char == keep:
    if channels == single:
        1 global history file?
        
    if channels == multiple:
        history per channel, but the same on char change?
        
if change_char == new:
    if channels == single:
        New global file on load and char switch
        
    if channels == multiple:
        new files on load and char switch
        ??? But should old character history be kept in memory or start a new?

#

have a question about the last

#

also does this look right?

halcyon quarry Jun 4, 2024, 2:28 AM

#

New global file on char switch

#

new files on char switch

terse folio Jun 4, 2024, 2:28 AM

#

halcyon quarry `new files on char switch`

question was about this one
with the "???"

halcyon quarry Jun 4, 2024, 2:28 AM

#

"On load" should not be dependant on change_char_ setting

terse folio Jun 4, 2024, 2:29 AM

#

okay

#

so what defines if loading should grab history from file or create new?

halcyon quarry Jun 4, 2024, 2:29 AM

#

autoload_history

terse folio Jun 4, 2024, 2:29 AM

#

okay

halcyon quarry Jun 4, 2024, 2:29 AM

#

Testing this little flag I added...

terse folio Jun 4, 2024, 2:30 AM

#

still didn't get to that last question!

#

when switching characters

#

what happens if you switch back?

halcyon quarry Jun 4, 2024, 2:30 AM

#

This seems to have worked

terse folio Jun 4, 2024, 2:30 AM

#

do you make new history?

halcyon quarry Jun 4, 2024, 2:30 AM

#

Yes

terse folio Jun 4, 2024, 2:30 AM

#

okay

halcyon quarry Jun 4, 2024, 2:30 AM

#

er

#

sorry - no :p

terse folio Jun 4, 2024, 2:30 AM

#

mhm okay

halcyon quarry Jun 4, 2024, 2:31 AM

#

🤷‍♂️ I hope it will work like that, let's see

terse folio Jun 4, 2024, 2:35 AM

#

is the behavior the same for single channel and char switch "new"

#

if i switch from A to B

#

then back to A

#

should A keep the same history as before?

halcyon quarry Jun 4, 2024, 2:36 AM

#

I'm thinking :p

#

This may be overthinking.
I did resolve the issue with this loading up

#

When changing characters and change_char_mode: keep, history never changes unless /reset.
Otherwise, we'll start new history on character change

terse folio Jun 4, 2024, 2:37 AM

#

Also, if using per channel history + character change = keep
Should the per channel history also be the same when switching characters?

#

okay

terse folio Jun 4, 2024, 2:38 AM

#

terse folio should A keep the same history as before?

i'll just go with yes

halcyon quarry Jun 4, 2024, 2:39 AM

#

Sorry to ignore the question - well, I really hate adding settings and really want to get this bit behind us

#

Me personally, I expect the history to change when changing characters

terse folio Jun 4, 2024, 2:39 AM

#

im working with the settings we already have, but I can't do that without understanding what they're supposed to do

halcyon quarry Jun 4, 2024, 2:40 AM

#

Like from A >B then back to A, I expect new history

terse folio Jun 4, 2024, 2:40 AM

#

and if it's per channel?

#

new history a>b>a

halcyon quarry Jun 4, 2024, 2:40 AM

#

Right.
But, I could see how someone else may expect autoload_history to restore the most recent history from that character

#

~~Regardless of single or multiple channels~~

#

Who'd've thought per channel histories could be so complicated

terse folio Jun 4, 2024, 2:42 AM

#

it really shouldn't

#

just had to build the bot up around it in the first place 😸

halcyon quarry Jun 4, 2024, 2:42 AM

#

lol

terse folio Jun 4, 2024, 2:42 AM

#

jumping from everything being treated as single to supporting different and multiple characters per channel is a mess indeed

halcyon quarry Jun 4, 2024, 2:43 AM

#

Let's just leave it as is - except for this little fix I slapped in

#

lemme just check it works when autoload_history False

terse folio Jun 4, 2024, 2:48 AM

#

#

I think this is much cleaner

#

this will acomplish that logic

#

halcyon quarry Jun 4, 2024, 2:50 AM

#

Check of approval

terse folio Jun 4, 2024, 2:51 AM

#

Also looking around, change_char_task takes reset | character | Tags

#

in the case of character and tags, this will reset the current history with change_char == new

#

not sure why we had unload_all_history

halcyon quarry Jun 4, 2024, 2:53 AM

#

hum

#

Contemplating that

#

I revised it to this,didnt push yt

#

        # Set history
        if not bot_history.autoload_history or bot_history.change_char_history_method == 'new': # if we don't keep history...
            if source == 'reset':
                # create a clone with same settings but empty, and replace it in the manager
                bot_history.get_history_for(ictx.channel.id).fresh().replace()
            else:
                bot_history.unload_history() #.clear_all_history()

terse folio Jun 4, 2024, 2:55 AM

#

character switching will create a new history file with change_char_history_method == 'new' anyway.
just need to unload the previous version if exists

halcyon quarry Jun 4, 2024, 2:58 AM

#

I just didn't like the If statement, pass, else

terse folio Jun 4, 2024, 2:58 AM

#

halcyon quarry ``` # Set history if not bot_history.autoload_history or bot_his...

auto load doesn't have to be mentioned here, it only matters in get_history_for

halcyon quarry Jun 4, 2024, 2:58 AM

#

replaced it with the truthy version

terse folio Jun 4, 2024, 2:58 AM

#

halcyon quarry I just didn't like the If statement, pass, else

mhm

halcyon quarry Jun 4, 2024, 2:58 AM

#

pushed that little change

#

(not my dumb switch thing)

#

I need to run for now or else I'll be in the doghouse for sure

terse folio Jun 4, 2024, 2:59 AM

#

if not autoload, but keep char method -> create new history anyway?

halcyon quarry Jun 4, 2024, 3:00 AM

#

Correct

#

If not autoload, always reset

terse folio Jun 4, 2024, 3:01 AM

#

okay, I feel like there will be some bugs when it comes to tags

#

with resetting so much

#

so what would autoload + char method new do?

halcyon quarry Jun 4, 2024, 3:02 AM

#

In the tags system I think I explicitly tried avoiding changing history unless tag was change_character

#

autoload + char method new should initialize the bot with the most recent history in all channels

#

When you change character, all new history

terse folio Jun 4, 2024, 3:04 AM

#

okay

halcyon quarry Jun 4, 2024, 3:04 AM

#

if you use reset, only reset in that channel

#

Thats the combo that isn't working as I expected in current version

#

Really though, Ill ttyl 😛

#

adieu

terse folio Jun 4, 2024, 3:36 AM

#

auto load history + character method keep = new chat on switch

#

if that's intended, then the code works

halcyon quarry Jun 4, 2024, 10:46 AM

#

terse folio auto load history + character method keep = new chat on switch

With this combo, the only thing that should trigger new history is /reset_conversation

Or, reloading the bot... when loading up the bot it should load most recent history for current character.

#

I see lots of changes!

halcyon quarry Jun 4, 2024, 1:44 PM

#

Oh I see, mixed history

#

Good.

#

Brilliant

#

Yes this looks very good

visual dagger Jun 4, 2024, 2:00 PM

#

halcyon quarry For example: because it captures many attributes for each message, we can easily...

what do you mean by this, can you go a bit in depth

#

if you got time

#

hello again

halcyon quarry Jun 4, 2024, 2:00 PM

#

Hi

visual dagger Jun 4, 2024, 2:00 PM

#

Oi

halcyon quarry Jun 4, 2024, 2:01 PM

#

If there's 20 people in # General chatting with the bot, a setting could be added that each 20 people have their own exclusive history with the bot

visual dagger Jun 4, 2024, 2:04 PM

#

why that number specifically 20

halcyon quarry Jun 4, 2024, 2:04 PM

#

Im giving you an example, it could be 10 million users

visual dagger Jun 4, 2024, 2:04 PM

#

so it's like threads or groups

#

but it is automatic

#

each "group" have their own chat history

#

with a set of pre-named identified users

keen palm Jun 4, 2024, 2:05 PM

#

Probably not 10 million

visual dagger Jun 4, 2024, 2:05 PM

#

keen palm Probably not 10 million

100 million

halcyon quarry Jun 4, 2024, 2:06 PM

#

The history is being collected for each channel, but each message has a number of attributes being stored now.
A setting could filter the history for only messaged by user - when that message is being processed

#

(and bot replies to that user - also tracked)

visual dagger Jun 4, 2024, 2:07 PM

#

is it dynamic? like removing msgs from history and adding others etc..

#

or does it have different chat histories

halcyon quarry Jun 4, 2024, 2:08 PM

#

Everything works according to the configuration settings

#

If it will have single user histories, that is a setting that does not exist yet.
But it can easily now with the new framework

#

We can add such a setting, and anything else that could be useful in terms of manipulating history

visual dagger Jun 4, 2024, 2:09 PM

#

I meant like chat history of a collection of users msgs discussing a specific topic

halcyon quarry Jun 4, 2024, 2:19 PM

#

That could be possible

halcyon quarry Jun 4, 2024, 2:58 PM

#

@terse folio truly amazing work

halcyon quarry Jun 4, 2024, 4:08 PM

#

Did a number of testing and everything is working expectedly

keen palm Jun 4, 2024, 4:08 PM

#

What do you mean specifically by "everything"?

halcyon quarry Jun 4, 2024, 4:30 PM

#

All of these settings are behaving expectedly:

  chat_history:
    limit_history
    autosave_history
    autoload_history
    change_char_history_method
    greeting_or_history
    per_channel_history

#

We've been hitting one snag or another while trying to migrate to the more sophisticated history manager

halcyon quarry Jun 4, 2024, 5:22 PM

#

Testing is going smooth

keen palm Jun 4, 2024, 5:29 PM

#

What, uhh, what exactly are you testing there?

halcyon quarry Jun 4, 2024, 5:30 PM

#

perplexity

halcyon quarry Jun 4, 2024, 5:54 PM

#

I'm merging this

#

made backup branch pre-history-overhaul

#

Oh baby time to update the install method and update method 😄

keen palm Jun 4, 2024, 6:11 PM

#

https://tenor.com/view/holdon-toyourbutts-jurassicpark-gif-4858777

Tenor

halcyon quarry Jun 4, 2024, 6:23 PM

#

New Install / Update methods published
https://github.com/altoiddealer/ad_discordbot

GitHub

GitHub - altoiddealer/ad_discordbot: A Discord Bot for chatting wit...

A Discord Bot for chatting with Large Language Model (LLM) supported by text-generation-webui - altoiddealer/ad_discordbot

#

I recommend updating , don't sleep guys 🙂

bot.py now stays in main folder.
New updater script for easy updating.
New launcher for easy launching.
CMD_FLAGS.txt for your custom launch flags.
Generally many improvements / minor bug fixes we encountered

#

@keen palm I'll look into Regen / Continue / Replace Last Reply tomorrow

keen palm Jun 4, 2024, 6:33 PM

#

Yay. Those are basically the last puzzle pieces for my setup

terse folio Jun 4, 2024, 6:45 PM

#

add_history was correct here.

Add history takes a history class and adds it to the internal dict.
New_history_for takes an id to create new history.

halcyon quarry Jun 4, 2024, 6:48 PM

#

Could you revert that for me?

#

Seemed like it was 100% identical

terse folio Jun 4, 2024, 6:48 PM

#

Also wanted the functions separate so there was a distinction between "no history for <id> creating new"
and "creating new history for <id>"
😸

terse folio Jun 4, 2024, 6:49 PM

#

halcyon quarry Seemed like it was 100% identical

yes, it would add an extra log message and throw an error probably somewhere because the wrong type is being passed to the function

halcyon quarry Jun 4, 2024, 6:50 PM

#

I changed a number of log msgs from debug to info - and set the logging level to Info

terse folio Jun 4, 2024, 6:50 PM

#

but you're unlikely to ever reach it without a specific combination of settings after the recent changes with keep/new and all

#

pushed it to dev

halcyon quarry Jun 4, 2024, 6:53 PM

#

Merged to main

vestal python Jun 4, 2024, 9:08 PM

#

I uhh, I think I need to make a second character yaml. Keeps saying they can't code, and saying "Ask my brother, he might know!"

keen palm Jun 4, 2024, 9:08 PM

#

So you need to make the brother character

vestal python Jun 4, 2024, 9:09 PM

#

It's 2 AI's that are persona as brothers.

#

I'll make the evil Brother, the forgotten one. the coding genius who loves to write all forms of code.

#

Wow that was easier than I though. Just using one of the other Yaml as example, and telling my llama 3 70B guy whatI want as a new yaml personality. I should do this more often x.x

halcyon quarry Jun 4, 2024, 11:44 PM

#

@terse folio Curious to know your plans - eager to add history related features? Revamp something else? Sitting out for a bit?

terse folio Jun 5, 2024, 12:03 AM

#

halcyon quarry <@226121791670583296> Curious to know your plans - eager to add history related ...

Sit out a bit, but am always here if needing help with planning how to do something.

I'm going to work a bit on my voice project!

halcyon quarry Jun 5, 2024, 12:09 AM

#

Is that planned to be a TGWUI extension? or a PR for it?

terse folio Jun 5, 2024, 12:32 AM

#

halcyon quarry Is that planned to be a TGWUI extension? or a PR for it?

it's just me playing around with xtts

halcyon quarry Jun 5, 2024, 2:34 PM

#

Getting to work on Regen / continue / Edit history

keen palm Jun 5, 2024, 2:45 PM

#

https://tenor.com/view/excited-too-much-energy-shaking-gif-22511742

Tenor

halcyon quarry Jun 5, 2024, 2:45 PM

#

Might not be a one day affair

keen palm Jun 5, 2024, 2:47 PM

#

https://tenor.com/view/的9-gif-27299608

Tenor

halcyon quarry Jun 5, 2024, 3:18 PM

#

Anyone made the jump yet to latest version?

keen palm Jun 5, 2024, 3:20 PM

#

I have not

halcyon quarry Jun 5, 2024, 3:21 PM

#

Once you do, upgrading is a simple as double clicking updated script, and launching as simple as double click launcher 🙂

keen palm Jun 5, 2024, 3:37 PM

#

CMD_FLAGS.txt contains only comments or is empty.
Traceback (most recent call last):
File "D:\Documents\text-generation-webui\ad_discordbot\bot.py", line 7, in <module>
from dataclasses_json import dataclass_json, config
ModuleNotFoundError: No module named 'dataclasses_json'
bot.py execution failed
Press any key to continue . . .

halcyon quarry Jun 5, 2024, 3:37 PM

#

Must run updater script once

keen palm Jun 5, 2024, 3:37 PM

#

Oh, I did

#

Said 'aready up to date'

halcyon quarry Jun 5, 2024, 3:38 PM

#

What OS are you on?

keen palm Jun 5, 2024, 3:38 PM

#

Winders

halcyon quarry Jun 5, 2024, 3:39 PM

#

Does it not also show something like this?

keen palm Jun 5, 2024, 3:42 PM

#

Nope

#

I moved the old /ad_discordbot folder out entirely, git clone the suppository, then ran the updater

#

All that said was up to date

halcyon quarry Jun 5, 2024, 3:43 PM

#

Your file structure is like this?
\text-generation-webui\ad_discordbot\

keen palm Jun 5, 2024, 3:43 PM

#

Yeah

#

Okay, I ran it agaIn, and this time that extra stuff popped up

#

Third time was the charm, apparently

halcyon quarry Jun 5, 2024, 3:44 PM

#

very interesting that it didn't the first time(s)

keen palm Jun 5, 2024, 3:45 PM

#

What is "export_for_tgwui"?

halcyon quarry Jun 5, 2024, 3:46 PM

#

More info required 😄

#

New config option?

keen palm Jun 5, 2024, 3:46 PM

#

Yes, it's set to true by default

#

it's a new chat_history option

halcyon quarry Jun 5, 2024, 3:47 PM

#

Ah - if that's disabled then it won't save anything to /logs/

keen palm Jun 5, 2024, 3:47 PM

#

So where does it save, if autosave_history is set to true, but export_for_tgwui is false?

halcyon quarry Jun 5, 2024, 3:48 PM

#

There's another instance of history now in /internal/ which is what is actually used for loading previous history

keen palm Jun 5, 2024, 3:48 PM

#

Right on

#

╭───────────────────────────────────────── Traceback (most recent call last) ──────────────────────────────────────────╮
│ D:\Documents\text-generation-webui\ad_discordbot\bot.py:5030 in <module> │
│ │
│ 5029 bot_settings = Settings(bot_behavior=bot_behavior) │
│ ❱ 5030 bot_history = CustomHistoryManager(class_builder_history=CustomHistory, **config.get('te │
│ 5031 │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
TypeError: CustomHistoryManager.init() got an unexpected keyword argument 'save_interval'
bot.py execution failed

halcyon quarry Jun 5, 2024, 3:48 PM

#

But this is bot specific history

#

ugh

#

That was supposed to be removed from the config file...

keen palm Jun 5, 2024, 3:49 PM

#

I delete and retry

halcyon quarry Jun 5, 2024, 3:49 PM

#

Delete the line save_interval in config.yaml and that should go away, yep

#

Chat history does not save locally, immediately. It saves periodically.
Reality added code to force a save if the window is closed (click X) or press Ctrl+C

keen palm Jun 5, 2024, 3:52 PM

#

Is there an advantage to that delay?

halcyon quarry Jun 5, 2024, 3:53 PM

#

According to Reality each save function uses measurable compute, so larger saves at longer frequencies is more optimal

#

since the history is safe at the save interval due to the force save feature, I decided to just hardcode that duration

keen palm Jun 5, 2024, 3:56 PM

#

Right on, right on. At any rate, loading fine so far

#

I've got a tag for 'create a character sheet', so the bot will come up with a character and create a sheet for them in the RPG system I'm using.

halcyon quarry Jun 5, 2024, 3:59 PM

#

How many steps is it? Anything very complex about it such as model swapping, etc?

#

works good?

keen palm Jun 5, 2024, 4:01 PM

#

It's just the one step. All it does it inject context on how to create the sheet, i.e., what traits to use and how to rank them. It works decently well. I had initially set it to actually assign values to the traits, but...that didn't quite work out. The LLM doesn't really handle those specific instructions well.

halcyon quarry Jun 5, 2024, 4:08 PM

#

You should make a character that has a few ideal responses as its example dialogue, and switch to it (swap_character)

keen palm Jun 5, 2024, 4:10 PM

#

Yeah, that could work as well

#

Probably work better, yes

vestal python Jun 5, 2024, 4:38 PM

#

Just updated to the newest version. I have an issue with half my models:

error loading model: error loading model vocabulary: unknown pre-tokenizer type: 'smaug-bpe'

halcyon quarry Jun 5, 2024, 4:39 PM

#

hmm.

vestal python Jun 5, 2024, 4:40 PM

#

Llama-3-8B-Instruct-v0.10.Q5_K_M.gguf
Awanllm-Llama-3-8B-Dolfin-v1.0-Q5_K_M.gguf
llama-3-Stheno-Mahou-8B-Q4_K_M-imat.gguf

But the original one I was using works:
Llama-3-8B-LexiFun-Uncensored-V1-Q5_K_S.gguf

#

The 3 I was testing had parameter in their settings
"tokenizer.ggml.pre str = smaug-bpe"

keen palm Jun 5, 2024, 4:41 PM

#

LexiFun-Uncensored...I wonder what that one is designed for....

vestal python Jun 5, 2024, 4:41 PM

#

It's for anything, and usually codes better. And if set with Evil Ecne, he likes to set villages on fire and such

halcyon quarry Jun 5, 2024, 4:41 PM

#

🙂

#

@vestal python you are using the new bot.py which is in /ad_discordbot/ and using the new launcher?

vestal python Jun 5, 2024, 4:44 PM

#

Yeah I wiped and new clone

#

So far not breaking out and telling me his persona values which is good

halcyon quarry Jun 5, 2024, 4:45 PM

#

Do you have a link to one of those models?

vestal python Jun 5, 2024, 4:48 PM

#

Original: https://huggingface.co/MaziyarPanahi/Llama-3-8B-Instruct-v0.10
GGUF variant: https://huggingface.co/MaziyarPanahi/Llama-3-8B-Instruct-v0.10-GGUF

halcyon quarry Jun 5, 2024, 4:49 PM

#

good - thanks

vestal python Jun 5, 2024, 4:52 PM

#

no morals

keen palm Jun 5, 2024, 5:01 PM

#

Jesus

halcyon quarry Jun 5, 2024, 5:05 PM

#

I'm having trouble trying to load that model in the UI

#

I can load some other similar gguf models that I have, but this one has a very vague error

#

keen palm Jun 5, 2024, 5:08 PM

#

I make sure to load all the models in the UI before I do them in the bot

halcyon quarry Jun 5, 2024, 5:09 PM

#

@vestal python you are able to load this model in the UI?

#

Any special settings you had to apply for it to load?
Had to compile any special wheels or anything?

vestal python Jun 5, 2024, 5:15 PM

#

I have only tried in the discord bot right before I updated it was able to swap between all of them. I can give it another try in a minute locally on textgen

halcyon quarry Jun 5, 2024, 5:22 PM

#

Finally made minor headway on history edit

keen palm Jun 5, 2024, 5:22 PM

#

https://tenor.com/view/excited-listening-go-on-jon-stewart-gif-11365765162220807492

Tenor

halcyon quarry Jun 5, 2024, 5:23 PM

#

Well, I made my first "Modal" which can be invoked from App Command (right click a message)

#

So in this case app command is edit_history.
The one thing it can't do is physically edit user messages, but it can edit messages that it sent (bot messages).
In any case - I am going to have it find that item in history and replace it with whatever the new text is.
If its a bot message, it will also edit its message in discord UI to reflect the change

#

So this will be like a suped up 'replace last reply'

vestal python Jun 5, 2024, 5:25 PM

#

2 of the models won't load in normal TextGen. I also did a full update of textgen which probably killed them for now. Ill just stick with the one that's working and such

halcyon quarry Jun 5, 2024, 5:26 PM

#

OK 😛 Please, if you encounter any models that do load in the UI but not with the bot, lmk 🙂

#

Maybe the bot provides superior debug info lol

keen palm Jun 5, 2024, 5:27 PM

#

halcyon quarry So in this case app command is `edit_history`. The one thing it can't do is phys...

Yeah, this is basically the best solution, since often responses only need minor changes.

halcyon quarry Jun 5, 2024, 5:28 PM

#

This will be separate from Regen / Continue

#

also

#

This doesn't generate any text

vestal python Jun 5, 2024, 5:30 PM

#

They're both set for now. Now to not touch them for a few days and see how the users like the different character options x.x

#

I tried to make coding assistants with some persona, but just setting the basic 'Assistant' yaml works leagues better

halcyon quarry Jun 5, 2024, 5:31 PM

#

Did you see that there is a new /announce command, to set announcement channels?

#

If your server(s) have any announce channels set, it will send Model/Character change embeds there, in addition to the interaction channel

#

This makes it to users in other channels can be alerted it something changes

#

Works for multiple servers, too

vestal python Jun 5, 2024, 5:33 PM

#

oh I see. Announce to me when changes are made for the character swapping

halcyon quarry Jun 5, 2024, 5:34 PM

#

like if someone uses /imgmodel in one channel, then the embed will be sent to that channel as well as any announcement channels

vestal python Jun 5, 2024, 5:34 PM

#

halcyon quarry Jun 5, 2024, 5:35 PM

#

It adds a slight delay to sending these embeds that scales depending on the number of announcement channels

#

if you have 50 announce channels it won't try sending all at once

halcyon quarry Jun 5, 2024, 5:36 PM

#

vestal python

Ah, you're using the bot in multiple servs as well

vestal python Jun 5, 2024, 5:39 PM

#

Yeah something about 5 servers.

#

But used in 2 actively

halcyon quarry Jun 5, 2024, 5:40 PM

#

Per channel histories should be quite nice for you then

vestal python Jun 5, 2024, 5:55 PM

#

I haven't seen any breaks in the character yaml appearing in the chat so far which is good.

#

It'd be a weird almost like a narrator effect.

#

Yeah I don't know fully what changes have been made, but your newest version is working wonderfully compared to whatI was testing with earlier today for both my bots.

keen palm Jun 5, 2024, 6:20 PM

#

The sheet creator character swap is working well, though it can't quite handle rating traits correctly. Oh well.

halcyon quarry Jun 5, 2024, 6:29 PM

#

keen palm The sheet creator character swap is working well, though it can't quite handle r...

This is when you create yet another character context, and all it does it rate sheets 😛

#

You could then consider making a Flow - which has all characters review it

#

Well, you would just add one flow step which would use the output from the first bot reply as the prompt

keen palm Jun 5, 2024, 6:31 PM

#

Oh god

#

It almost gets the ratings correct each time.

halcyon quarry Jun 5, 2024, 6:31 PM

#

That character context would have a spectrum of good and bad outputs, with their appropriate rating.

#

could also use like, max tokens: 5

#

if it's going to be very simple rating

#

You can use the custom character setting, in_character_menu: False for these specialized characters

keen palm Jun 5, 2024, 6:38 PM

#

Right, I do need to do that.

halcyon quarry Jun 5, 2024, 6:54 PM

#

@terse folio I want to change MessageID to be a list instead of Int, but having troubles

#

Tried:
-Changing it in typing.py from Int to list
-updated send_long_message() to return a list of message IDs
-Added a line in llm_gen() to assign user's message.id as a list item.

It errors on first interaction when trying to get history for

#

Er, I think I found the issue...

keen palm Jun 5, 2024, 7:00 PM

#

https://tenor.com/view/thats-good-news-obi-wan-star-wars-gif-15974779

Tenor

halcyon quarry Jun 5, 2024, 7:04 PM

#

Eh, I lied

keen palm Jun 5, 2024, 7:08 PM

#

Well, shit

halcyon quarry Jun 5, 2024, 7:10 PM

#

the error occurs:

if the bot sends a chunked message (> 2000 characters), and I collect the message IDs as a list.
it errors when trying to read history after this

#

I'll just leave it as is... and include a message that the command must be used on the last message if sent in chunks

keen palm Jun 5, 2024, 7:57 PM

#

So, if I wanted to take a response from the character sheet creator, then put that into a flow step to have it check again to verify the ratings of some traits, how exactly would that look? What I've got going right now is...well I dunno what's going on with it

halcyon quarry Jun 5, 2024, 7:58 PM

#

One thing to note is, the flow: definition does not go into effect until the message that triggered it is completed (the Flow begins on the next message loop)

#

So in the first flow step you'd want to use something like

format_prompt: {llm_0}
swap_character: Your ranking character
save_history: false
load_history: -1
should_send_text: true

#

You probably want both messages returned, correct?

The character sheet
The score

keen palm Jun 5, 2024, 8:03 PM

#

It's not really asking for a score. That's not what I mean when I'm talking about "ratings". The character sheet requires assigning dice ratings (d10, d8, d8, d6, d6, d4) to a list of six traits. The LLM sometimes has a hard time doing that, so I wanted to have a flow that would look at the ratings again and verify that they're correct.

halcyon quarry Jun 5, 2024, 8:04 PM

#

Oh, well in that case your character context should be examples of it formatted correctly, and some with errors.

Every response should be the correct version of the sheet

keen palm Jun 5, 2024, 8:05 PM

#

Ahh, so in that case sending it to a character sheet corrector character, right?

halcyon quarry Jun 5, 2024, 8:05 PM

#

In this case, you would want to use should_send_text: False on the first character.
Only send text after it reviews

halcyon quarry Jun 5, 2024, 8:06 PM

#

keen palm Ahh, so in that case sending it to a character sheet corrector character, right?

yes

keen palm Jun 5, 2024, 8:07 PM

#

Hmmm

#

Might not be worth the effort, since I would likely just be pasting these sheets somewhere else and can adjust as needed.

halcyon quarry Jun 5, 2024, 8:09 PM

#

Well, I think you'll have better luck getting correct sheet if you split it to two characters

keen palm Jun 5, 2024, 8:09 PM

#

Yes, probably

#

I already am getting better results than I was before when it was all handled by my gamemaster character

halcyon quarry Jun 5, 2024, 8:16 PM

#

OK so cool - I got the edit history working

keen palm Jun 5, 2024, 8:17 PM

#

https://tenor.com/view/yess-yes-gif-25420589

Tenor

halcyon quarry Jun 5, 2024, 8:17 PM

#

Hopefully can perfect it before I need to jet

#

Ok what's also really cool, is that even if someone sends a message (modifies history) while using this command (which is holding the target message as a variable), it still successfully updates the history

keen palm Jun 5, 2024, 8:33 PM

#

Does it update the saved history as well as fix what's in RAM?

halcyon quarry Jun 5, 2024, 8:33 PM

#

Both

#

Once I get comfy cozy with this code from Reality, can probably come up with some really cool shit

#

It's very powerful

#

This is so simple to do, it updates everything on one line

#

        history = bot_history.get_history_for(inter.channel.id)
        target_message = history.search(lambda m: m.id == message.id)

Then later...
target_message.text = new_content.value

And that's it

keen palm Jun 5, 2024, 8:37 PM

#

https://tenor.com/view/ok-all-nice-gif-22927671

Tenor

#

I'm ready to ~~thoroughly break it~~ test it out

halcyon quarry Jun 5, 2024, 8:55 PM

#

Pushed it to main!

keen palm Jun 5, 2024, 8:56 PM

#

Oh boyee

halcyon quarry Jun 5, 2024, 8:56 PM

#

I currently hardcoded it so you can only use it on your own messages, or the bot's messages

#

But I may add a setting for bot admin to decide whether anyone can edit anyone's history

#

Alright, Im outta time but Im going to improve it tomorrow

#

The Text Input box can have the original message contents as "default"

#

(populated already)

keen palm Jun 5, 2024, 9:02 PM

#

I'll mess around with it later

terse folio Jun 5, 2024, 9:11 PM

#

halcyon quarry <@226121791670583296> I want to change MessageID to be a list instead of Int, bu...

Ill take a look to clean it up!

halcyon quarry Jun 5, 2024, 9:11 PM

#

terse folio Ill take a look to clean it up!

Forget it 😛

#

Can work around it after all

#

It’ll just add complexity

terse folio Jun 5, 2024, 9:14 PM

#

halcyon quarry ``` history = bot_history.get_history_for(inter.channel.id) targ...

I recommend using target_message.update(text=new_content.value)

By calling a function to update attributes, we could include other callbacks in the future such as telling the history class that it has new data to save.

#ad_discordbot (Fork of Fork of xNul's bot)

I recommend updating , don't sleep guys 🙂