yes. It does not matter if the text is split like this:
message 1\n\n
message 2
or this:
message 1
\n\nmessage 2
#ad_discordbot (Fork of Fork of xNul's bot)
halcyon quarry
But it is often splitting like this:
message 1\n\nm
essage 2
or
message
1\n\nmessage 2
valid crypt
you told me that \n triggers before and splits right?
halcyon quarry
it triggers and rolls probability to split.
If it rolls True it splits
If it rolls False, that text is added to the "do not check again" pile
valid crypt
so \n does not always split
valid crypt
you want \n not always split but \n\n always split?
halcyon quarry
I want \n\n to have stronger weight to split
Ideally, all conditions could be analyzed together at one time
But it's seemingly impossible
Instead, there needs to be some wonky method of ignoring shorter syntax, when a longer syntax could possibly trigger on the next iteration
BUT, the shorter syntax could still then be triggered if the longer one doesn't appear
halcyon quarry
And how do you trigger to delay the check lol
halcyon quarry
I have the logic of it all figured it, it's just a tangled mess in what is already a bit sloppier than I'd like
With all the variables in play, it's hard to make it all work together
valid crypt
hmmm
halcyon quarry
- Needs to check for the syntax anywhere in a window of 4 characters.
- if syntax of 1 character length is triggered, while 2-character syntax exists, it needs to ignore for 1 iteration.
- It then needs to roll to split. But it needs to offset the split depending on where it is finding the match in that 4-character window.
It's bananas but that's how it needs to work, really
valid crypt
how is the text chunked?
halcyon quarry
Basically like this
already_chunked = ''
async for resp in generate_in_executor(func):
# Check current iteration to see if it meets criteria
partial_response = resp[len(already_chunked):]
should_chunk = check_should_chunk(partial_response)
if should_chunk:
last_checked = ''
already_chunked += partial_response
# process message chunk
yield partial_response
i meant the ouput
halcyon quarry
Do you not use the bot?
valid crypt
of this code
halcyon quarry
Instead of returning True on a match, it was returning an offset index based on where the syntax was matched.
imagine the text is a rope coming out of a machine, and there's some black dots and some wider dots on the rope.
The rope passes through a detector which has a window that can fit 4 small dots or 2 wide dots or 1 wide dot + 2 small dots etc
Instead of just slamming the cutter when it detects, it measures how far inside the window it detected a dot, and how long that dot is it also a factor
And it calls back and tells the cutting thing to scooch over
This is where the relavent code begins in the current bot.py
https://github.com/altoiddealer/ad_discordbot/blob/main/bot.py#L1654
There's a lot of code that is irrelevant to the problem, though.
My prompt to ChatGPT had all that stripped out
I can only spend so much time on one thing, making little tweaks to a few variables and it just keep crapping
valid crypt
just a basic question, how do i paste properly (loose all the tabs or spaces)
halcyon quarry
Well the indentation is all important
If the entire code block has white space on the left, you need to shift the indentation in an editor, like VS Code or even WordPad, or MS Word, etc they have indentation adjustment buttons
in VS Code it has a lovely hotkey Ctrl+[ to indent left and Ctrl+] to indent right
I'm outta here, wife will murder me if I stay on any longer XD
valid crypt
ok
valid crypt
i have skill issue 🙃
valid crypt
ohhhh i see cutting at the wrong place
halcyon quarry
I’m going to give it another shot
keen palm
halcyon quarry
https://tenor.com/view/you-can-gif-27326947
I did it 🍾
This time I made the whole response chunking mechanism into a class.
And I only took some hints from ChatGPTs solution, came up with simpler solutions
Pushed to main!
Be sure to add \n\n to your behavior setting stream_reply_syntax now present in default settings
The main improvement is it will factor a +50% boost in chance to split on ‘\n\n’
valid crypt
### Pushed to main!
Be sure to add **`\n\n`** to your behavior setting **`strea...
if you added it to example character ill take a look
halcyon quarry
Did!
valid crypt
tried tts with wps speed limit
it doesnt work
feature or bug
feels like a feature :p
halcyon quarry
I need to take another look at that feature. It was generally working last I checked, but some things were a bit off like the is typing...
halcyon quarry
tried tts with wps speed limit
This actually isn't intended to make text generate slower. With the bot, it tries to simulate typing slower 🙂
You can use the max_tokens_sec state parameter, which will actually throttle text generation speed
valid crypt
what i expected was i hear the tts and the text being written slowly
halcyon quarry
Use the TGWUI param for that
valid crypt
ngl the wpm is h* accurated xd
halcyon quarry
ah, thanks 🙂
it feels painfully slow compared to unthrottled generation, but it works good for simulating human behavior
valid crypt
is there a way to disable sending the audio file or even the text?
halcyon quarry
You can use the Tags system to set should_send_text: false and should_gen_text: true
You can try it out immediately one time by copy/pasting this along with your prompt ("instant tags syntax")
[[should_send_text: false, should_gen_text: true]]
The TTS file, that is in config.yaml in the tts_settings, param is called play_mode
valid crypt
buddy you better write a comment on that
i see play_mode:2
so idk how to change that
halcyon quarry
The problem is that all the comments get wiped when the bot automatically copies the settings files, during initial installation
If you go to the settings_templates folder and check out config.yaml you'll see everything is very well documented
valid crypt
alr
halcyon quarry
If the bot ships with the settings files in the correct location, its very difficult to make updates
With this structure and recent changes, it warns when new settings are added to the templates, and its up to users to catch up
valid crypt
ayo my pull request got merged https://github.com/Arondight/vits_api_tts :)
halcyon quarry
ayo my pull request got merged https://github.com/Arondight/vits_api_tts :)
Grats!
halcyon quarry
Step 1 ✅
valid crypt
visual dagger
valid crypt
i experiencing a weird situation, im using a differend llm model and for some reason the tts sometimes dont work 😨
my tts server got the api call
ill dig deeper tomorrow
terse folio
uhoh, spontaneous robot disobedience!
Hmm, that is odd, the ad_discordbot should display any exceptions that happen within tasks.
But maybe tgwui itself is ignoring something.
I would try doing the same api call manually and checking the result.
For example there was an issue with using RVC I believe, but the reason was because rvc outputted a wav file (even if the api request specified .mp3 extension)
valid crypt
remember this? i did it
bot did the api call, did get response as it did play on the computer, but did not send the audio
the only difference is im using another llm???
valid crypt
guy i didnt change anything to my tgwui but suddntly i cant load my llama 3.1 8b
numpy.core._exceptions._ArrayMemoryError: Unable to allocate 15.7 GiB for an array with shape (32768, 128256) and data type float32
i could load it before
literally an hour ago i could load it .-.
valid crypt
guy i didnt change anything to my tgwui but suddntly i cant load my llama 3.1 8b...
solved
tgwui uses a lot of virtual memory, and i limited it a few days ago to save disk space, and after few days without shutting down theres some junks in it
halcyon quarry
👍
valid crypt
i suspect that when the audio is not cut and the model is slow at generating, bot forgets to play and sed the audio
using this model in particular:
https://huggingface.co/TheBloke/MythoMax-L2-Kimiko-v2-13B-GGUF
using the vits api tts, the server got the api call, locally received the audio file and... thats it...
i think it isnt because of the model
by switching models i got
Traceback (most recent call last):
File "D:\text-generation-webui-main\ad_discordbot\modules\tags.py", line 446, in match_tags
trigger_match = re.search(trigger_regex, search_text, flags=re.IGNORECASE)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\text-generation-webui-main\installer_files\env\Lib\re_init_.py", line 176, in search
return _compile(pattern, flags).search(string)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: expected string or bytes-like object, got 'NoneType'```"" are causing problems but it is weird to explain
halcyon quarry
hmm
So you are launching the bot with one model, then using the /llmmodel command to change models?
then getting this error?
Have you modified the bot code at all?
valid crypt
by switching models i got
```02:34:39.773 #506 ERROR [bot.modules.tags]: Error...
for this error, i think i can tell why
seems like when i say "" with something in between it gets the error
but depends on what llm
valid crypt
using this model in particular:
https://huggingface.co/TheBloke/MythoMax-L2-Kimi...
i only have 3 models, llama 3.1, mistral idk, and this one,
only llama 3.1 has no problems
halcyon quarry
in the function def init_self_values(self) I am initializing self.llm_prompt and self.img_prompt as None. I think changing this to '' will resolve the error
Later if the prompt is falsy like "" (nothing) it isn't replacing this default value
valid crypt
i updated yesterday thinking that i was dreaming and forgot to apply the fix so the bot's code is totally up to date without modification
halcyon quarry
I'll just look at a bit more code and commit this if I don't see any issues
valid crypt
?
valid crypt
fixied the error, output got voided
halcyon quarry
is that all good, or something still not working expectedly?
valid crypt
Llama.generate: prefix-match hit
llama_print_timings: load time = 4293.47 ms
llama_print_timings: sample time = 0.23 ms / 1 runs ( 0.23 ms per token, 4273.50 tokens per second)
llama_print_timings: prompt eval time = 1288.87 ms / 111 tokens ( 11.61 ms per token, 86.12 tokens per second)
llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per token, inf tokens per second)
llama_print_timings: total time = 1292.59 ms / 112 tokens
Output generated in 1.73 seconds (0.00 tokens/s, 0 tokens, context 1512, seed 676363995)```(0.00 tokens/s, 0 tokens
thats the problem
halcyon quarry
Have you loaded this model in TGWUI and then saved config in the Models tab?
terse folio
I've been having some odd llm behavior similar to that.
Starting the model, it works fine in TGWUI
But using the API to stream chat/completions doesn't iterate for some reason anymore.
I guess "0 tokens" like that, but there's no log in console.
For some reason, sending a request to normal /completions without streaming fixes it
valid crypt
for those 3 models i have, only llama 3.1 works fine
halcyon quarry
well this is a bit different from the API
terse folio
Mhmm, just some oddness that started happning today without changing anything
halcyon quarry
My TTS streaming solution is a bit unorthodox
I noticed that it doesn't like Alltalk's low vram setting
terse folio
if you're going to run LLM + TTS at the same time, lowvram isn't going to benefit you anyway
Since you're doing streaming of both
halcyon quarry
Alltalk tries to move the model around too much or something, and it does not jive with my code spamming it
The TTS extensions are usually only triggered one time, after all text generated. So the TTS is getting triggered much differently/more frequently
certain models may have some sort of unwritten cooldown
Pushed that small update initializing those 2 values as '' instead of None
Reality, hoping you could take a look at it sometime 🤗
terse folio
Mhm, will take smaller peaks at things for now,
Still got a lot to do until the 15th 😸
halcyon quarry
Mainly interested in making the whole custom_chatbot_wrapper / apply_extensions more asynchronous if possible.
For edge_tts it has an asyncio.run() in its TTS processing and my current code does not allow that to execute due to being on same thread
What I have going on could be as good as it gets 🤷♂️ Really not super well versed with this bit
terse folio
Mainly interested in making the whole `custom_chatbot_wrapper` / `apply_extensio...
I thought each extension was supposed to live in its own thread?
Hmm!
Oh, maybe that's not a requirement
Just that some extensions that serve webapis do because it would block the main thread
halcyon quarry
If I call the normal unadulterated chatbot_wrapper with TTS, it doesn't error when it executes apply_extensions after all text generated
The main thing I do in custom_chatbot_wrapper is remove something all the TTS extensions do, which is disable streaming
terse folio
what if instead of doing it there,
re-enable streaming after all the extensions have loaded
halcyon quarry
So as I'm getting the text from it, I'm then deciding when to hit that apply_extensions function
terse folio
So as I'm getting the text from it, I'm then deciding when to hit that `apply_ex...
I see, I see
halcyon quarry
So apply extensions is used for all sorts of things. In the beginning of generation the TTS extensions will set a flag to disable streaming
terse folio
oof, like every point is fighting back on that
halcyon quarry
I pulled the apply extensions that modifies "output" - out of that function
which is called at the end of generation
These are the 2 modifications
So I pulled out the last apply_extensions and I call it from outside the generator
terse folio
ahh, wow, that's a lot of moving parts to keep track of
halcyon quarry
oof, like every point is fighting back on that
I believe TTS extension owners would code it differently, if TGWUI allowed it
But TGWUI only has that output modifier at the end
terse folio
I believe TTS extension owners would code it differently, if TGWUI allowed it
what I'm thinking now that you mention the apply_extensions function is:
Another function like apply_streaming_extensions_iter
So extensions can have an alternate set of code that happens every iteration.
And since extensions are applied in a chain.
A previous extension could decide what is an "iteration"
Like it could group multiple tokens together by the sentence, then pass that along to TTS.
huge overhaul for tgwui extensions, but that's a possible idea to get it done!
halcyon quarry
Yes, it's likely the main hurdle for TTS streaming in the UI / elsewhere
I think my hack isn't too shabby but I'm probably missing some critical thing that would make it work more smoothly
terse folio
What would also be cool is if there's a way to inject new tokens into the TTS stream as they come in.
Instead of having to start a new stream each sentence
halcyon quarry
It's missing a clutch or something and I'm just changing gears :p
terse folio
I think my hack isn't too shabby but I'm probably missing some critical thing th...
that's what prototypes are for!
also make sure to keep notes
halcyon quarry
I think this reworked code in llm_gen() has very good code clarity
very good may be subjective 🤓
valid crypt
About bot forgets to send the tts
i tried the old code https://github.com/altoiddealer/ad_discordbot/tree/d535c7188b40e7900f06af52472ac7b084b0799d (Finalize the TTS streaming)
so i got the information of it was not even doing an api call
but with the latest code, bot does the api call and received the tts but didnt send the tts
https://github.com/altoiddealer/ad_discordbot/tree/b74222a46fa716ef70b3a7f2163bc3ab3572bc5f (Fix more TTS streaming bugs)
this works fine
i think i have the rights to suspect that you have broken something on the way
ill dig further tomorrow
halcyon quarry
For the record, the bot does no API calls (for TGWUI stuff). But your extension might do an API call?
The bot uses apply_extensions with the currently generated text - which triggers the TTS extension.
That subsequently returns a response like audio src=C:\path\to\the\sound\file.wav which the bot looks for. If it finds it, it then finds the file and plays it
Print statement here might help debug it
print("vis_resp_chunk:", vis_resp_chunk)
If the extension is returning the file path but the bot is not playing it, then I definitely have something to fix
I don't think that's the case though
terse folio
`print("vis_resp_chunk:", vis_resp_chunk)`
fun fact, you can use:
print(f'{vis_resp_chunk=}')
To get a quick vis_resp_chunk="Value" type message ^^
halcyon quarry
stonks
terse folio
works with expressions too like f'{2+2=}'
sullen plover
hello guys using ad_discordbot and i am only using max 6 cpu's it is making it work really bad cant i change the setting somewhere?
terse folio
hello guys using ad_discordbot and i am only using max 6 cpu's it is making it w...
in the webui you can change what devices the model can use.
It sounds like you're using LlamaCPP,
perhaps increasing or decreasing the threads will get you your desired result.
Im not sure if you are saying you want the model to run faster or your pc.
If you don't want the model to use the CPU, there's a checkbox for that as well.
and be sure to select "save settings" after you customize the model load settings to your likings (then click load)
sullen plover
i set to cpu but i am sure it limits to 6 cpu's or 5 max or something
i am trying to run reflection 70b with 75gb ram and i think 5 cpu's will just stick to llama 3.1 8b i guess 😄
terse folio
Check the textgen-webui/modules/shared.py file
There's a lot of commandline arguments there for launching the webui.
Surely something exists to specify cpus.
But I don't understand why it would be capped in the first place
halcyon quarry
TGWUI issue? Or bot issue?
terse folio
sounds like tgwui in general
sullen plover
prob main script issue i think
or maybe the issue no longer exists 😄 sorry just trying to do stuff that cant be done 70b reflection using no gpu 😂
valid crypt
i am trying to run reflection 70b with 75gb ram and i think 5 cpu's will just st...
buddy, first 5 cpu i assume cores or threads, but just 5 to run a 70b model is just nuts
sullen plover
it worked but not very well 30 mins to reply and some bs reply
valid crypt
it worked but not very well 30 mins to reply and some bs reply
that is what im talking about
halcyon quarry
Maybe knock a zero off the model size XD
valid crypt
although cpu core are much more powerful but even a 3060 has 3584 cores
sullen plover
yeah now running 8b it works replies within 1 min
halcyon quarry
Brady u been using the bot all this time? Any feedback on recent changes?
sullen plover
which bot? been using your bot script maybe 1 year i think
sullen plover
still not managed to get any web search extensions working or other stuff but still better than any other scripts main script etc and your one
i tried to update soon not updated for 2 months i think
valid crypt
@halcyon quarry upper is newer and numbers are old to new, the 1st is the last working one which does a api call and sends the tts
the 2nd gives a bug but does not do a api call, 3rd is the same, 4th does the api call but not reaction, which is what i meant with bot forgets to send tts
halcyon quarry
Did you add that print statement?
valid crypt
Did you add that print statement?
?
halcyon quarry
i tried to update soon not updated for 2 months i think
There’s been a lot of big changes in the past 2 months
valid crypt
oh i see
sullen plover
am gonna update now then you have forced my hand haha
valid crypt
i went to sleep and the next day i saw codes and skipped XD
halcyon quarry
You should be able to just run the updater bat file
valid crypt
Did you add that print statement?
vis_resp_chunk: <audio src="file/extensions/vits_api_tts/outputs/2024-09-08T00-18-33.617442Z.mp3" controls autoplay></audio>
sullen plover
You should be able to just run the updater bat file
where is that file sir?
valid crypt
where is that file sir?
same folder to the bot
sullen plover
what is it called updater.bat ?
valid crypt
halcyon quarry
If you don’t have it then it’s def been awhile since you updated
valid crypt
```vis_resp_chunk: <audio src="file/extensions/vits_api_tts/outputs/2024-09-08T0...
need me to print with the old and working code?
halcyon quarry
need me to print with the old and working code?
I can’t look into it much for the next 2-3 hours
That TTS didn’t play?
halcyon quarry
Does this issue only happen when it streams replies, or when there is one single message reply, or both?
valid crypt
happens when stream replis without split
sullen plover
(base) zero@zero:~/text-generation-webui-main$ ./update_wizard_linux.sh
What would you like to do?
A) Update the web UI
B) Install/update extensions requirements
C) Revert local changes to repository files with "git reset --hard"
N) Nothing (exit)
Input> A
- Updating the local copy of the repository with "git pull"
remote: Enumerating objects: 38, done.
remote: Counting objects: 100% (38/38), done.
remote: Compressing objects: 100% (18/18), done.
remote: Total 38 (delta 31), reused 23 (delta 20), pack-reused 0 (from 0)
error: insufficient permission for adding an object to repository database .git/objects
fatal: failed to write object
fatal: unpack-objects failed
Command '. "/home/zero/text-generation-webui-main/installer_files/conda/etc/profile.d/conda.sh" && conda activate "/home/zero/text-generation-webui-main/installer_files/env" && git pull --autostash' failed with exit status code '1'.
Exiting now.
Try running the start/update script again.
(base) zero@zero:~/text-generation-webui-main$
i think i need to activate conda or some crap ?
valid crypt
looks like the key line is error: insufficient permission for adding an object to repository database .git/objects
actually you can download the zip from github and replace
valid crypt
actually you can download the zip from github and replace
you should make a backup
config files you might not want to replace
etc
sullen plover
I did, got this error: 3:47:04.367 #32 INFO [bot.modules.utils_shared]: Creating "ad_discordbot/internal/history" for storing internal history states.
INFO [2024-09-07 23:47:04]: Creating "ad_discordbot/internal/history" for storing internal history states. (Line: 32 in init_shared_paths, utils_shared.py )
Traceback (most recent call last):
File "/home/zero/text-generation-webui-main/bot.py", line 50, in <module>
from ad_discordbot.modules.database import Database, ActiveSettings
File "/home/zero/text-generation-webui-main/ad_discordbot/modules/database.py", line 5, in <module>
from modules.database_migration_v1_v2 import OldDatabase
File "/home/zero/text-generation-webui-main/ad_discordbot/modules/database_migration_v1_v2.py", line 4, in <module>
from modules.utils_shared import shared_path
File "/home/zero/text-generation-webui-main/ad_discordbot/modules/utils_shared.py", line 68, in <module>
from modules.database import BaseFileMemory, Database
File "/home/zero/text-generation-webui-main/ad_discordbot/modules/database.py", line 5, in <module>
from modules.database_migration_v1_v2 import OldDatabase
ImportError: cannot import name 'OldDatabase' from partially initialized module 'modules.database_migration_v1_v2' (most likely due to a circular import) (/home/zero/text-generation-webui-main/ad_discordbot/modules/database_migration_v1_v2.py)
(base) zero@zero:~/text-generation-webui-main$
sullen plover
actually you can download the zip from github and replace
ty yes i did 🙂
oh dear lol
am in no rush to fix using groq 70b with mongdb for memory too
sullen plover
bot is online restored from backup
doesnt work update will try again soon, error due to me most likely btw
halcyon quarry
What OS?
sullen plover
linux mint desktop on a server
halcyon quarry
It’s possible the updater script doesn’t work 😛
I haven’t had anyone on linux confirm that one works - I had chatgpt write up all of them
After updating, did you use the new launcher?
again OS specific launchers
sullen plover
hmm need to update properly again soon some stuff is updated i think
halcyon quarry
That is old code because there is no longer an ActiveSettings variable which I see in the traceback
Is bot.py in the ad_discordbot directory?
Seems like maybe you have a mix of current and old files
sullen plover
yeah i think i need to setup from fresh properly 🙂
halcyon quarry
@valid crypt pretty sure I fixed the bug. I had consolidated some redundant code, but overlooked something that made it ignore the TTS when text didn't get chunked
Pushed that fix now
valid crypt
oki
valid crypt
looks like fixed
halcyon quarry
joy!
valid crypt
hey, what about writing a character that stores every setting of the discord bot
some simple question would be solved easily
halcyon quarry
The Wiki isn’t too far off from covering everything
halcyon quarry
Not a bad idea, though!
valid crypt
i think the bot just sent the same tts twice, feels hard to fix
it happened after around 50 messages
small bug
when the bot receives a message, is it just the message, or does it do something like marcos: message
new suggestion: auto prompt evaluation
halcyon quarry
The name of the user who triggered the interaction is used for the name1 value in the text gen payload
I have a planned setting on the backburner ("Server Mode") where the user1 value would be locked as the name of the server, and each message request would be prefixed with the user's name
What is auto prompt evaluation?
valid crypt
What is auto prompt evaluation?
the first message after a reboot or changing character starts evaluationg prompts
valid crypt
llama_print_timings: prompt eval time = 7147.28 ms / 1460 tokens ( 4.90 ms per token, 204.27 tokens per second)
that thing
halcyon quarry
Send a logging message?
valid crypt
instead of start evaluating prompt after the first message, after loading everything automatically start that process
halcyon quarry
ah
valid crypt
Send a logging message?
lemme do a screenshot lil quick
halcyon quarry
I suppose that would be to send a minimal prompt in the background that would not be logged, with like 1 max new tokens
valid crypt
adding that as option to config would be good
sullen plover
Fine-tuned a model & converted it to GGUF format, but getting a tensor mismatch error when loading it in Oogabooga WebUI. Any ideas on how to fix this? 🙏 Model is here: https://huggingface.co/shafire
halcyon quarry
this isn't the best place to ask that
sullen plover
no clue what i am doing but i am trying
it loads on discord then says load llm which is part of your script i think, also says no llm loaded
it connects online then goes offline when you ask a question
halcyon quarry
But no issues using the model via TGWUI?
halcyon quarry
Ok so I slightly misunderstood how that works. I was sending an empty context, but this saves nothing
Would basically need to send the current context and everything but with max_new_tokens: 1
TBH this won't really be effective unless the context that is sent is for the next channel (history) that with a text request
This would only be effective if per-channel-history is disabled
actually, it would probably save time if the context is sent but no history
since the context would be shared among channels
sullen plover
i have no clue to be honest will try fix my 1st llm
halcyon quarry
adding that as option to config would be good
This doesn't seem like it can have consistent results for most users
I'm sending a normal payload except with max new tokens 1, no prompt, and no history.
The next actual request says prefix-match hit, but "remaining prompt tokens to eval" is everything really. Maybe it's saving one token lol
I think history must be first to be evaluated, so if it's any different it re-evaluates context and everything all over again
halcyon quarry
I pushed it to a branch if you want to try it out, toy around with it.
https://github.com/altoiddealer/ad_discordbot/tree/prime_model
The code is this prime_tgwui_model() function, which is called in on_ready() and change_llm_model_task()
halcyon quarry
I guess what would be neat is if there was some way to make it eval context before history
(assuming I'm right about this)
valid crypt
I guess what would be neat is if there was some way to make it eval context befo...
what i thought was there is a way to trigger eval
halcyon quarry
yes, this does trigger eval. But any time history changes it will do a new eval
valid crypt
for me the first eval is the longest
halcyon quarry
It could work if not using per channel histories, and not per server settings
sullen plover
a few errors reinstalled everything
sullen plover
Failed to pull from the repository.
(venv) (base) zero@zero:~/text-generation-webui-main/ad_discordbot$ ./start_linux.sh
CMD_FLAGS.txt is empty.
Traceback (most recent call last):
File "/home/zero/text-generation-webui-main/ad_discordbot/bot.py", line 5, in <module>
from dataclasses_json import dataclass_json
ModuleNotFoundError: No module named 'dataclasses_json'
bot.py execution failed
(venv) (base) zero@zero:~/text-generation-webui-main/ad_discordbot$ cd ~/text-generation-webui-main
(venv) (base) zero@zero:~/text-generation-webui-main$ python bot.py --loader llama.ccp --model talktoaiZero-GGUF --auto-devices --cpu --n_ctx 2048
python: can't open file '/home/zero/text-generation-webui-main/bot.py': [Errno 2] No such file or directory
(venv) (base) zero@zero:~/text-generation-webui-main$ python bot.py --loader llama.ccp --model talktoaiZero-GGUF --auto-devices --cpu --n_ctx 2048
python: can't open file '/home/zero/text-generation-webui-main/bot.py': [Errno 2] No such file or directory
(venv) (base) zero@zero:~/text-generation-webui-main$ python bot.py --loader llama.ccp --model talktoaiZero-GGUF --auto-devices --cpu --n_ctx 2048
Traceback (most recent call last):
File "/home/zero/text-generation-webui-main/bot.py", line 5, in <module>
from dataclasses_json import dataclass_json
ModuleNotFoundError: No module named 'dataclasses_json'
(venv) (base) zero@zero:~/text-generation-webui-main$ python bot.py --loader llama.ccp --model talktoaiZero-GGUF --auto-devices --cpu --n_ctx 2048
Traceback (most recent call last):
File "/home/zero/text-generation-webui-main/bot.py", line 5, in <module>
from dataclasses_json import dataclass_json
ModuleNotFoundError: No module named 'dataclasses_json'
(venv) (base) zero@zero:~/text-generation-webui-main$
working webui just not the bot
halcyon quarry
Failed to pull from the repository.
(venv) (base) zero@zero:~/text-generation-we...
Thanks - this was a bug I fixed in the Windows launcher a long time ago when I first added it. Must need something in the Linux one
For now just add any cmd flag to the bots CMD_Flags file such as —loader exllama2 or similar
Should have a fix for this in 2 hrs
sullen plover
@halcyon quarry you gonna test my llm sir? 😄 will make a few more soon now i know how to make them, next one will be 7b converted to gguf myself using help of my agent
halcyon quarry
Failed to pull from the repository.
(venv) (base) zero@zero:~/text-generation-we...
Could you please try using this version of the launcher?
(with empty CMD_FLAGS.txt)
sullen plover
(base) zero@zero:~$ cd ~/text-generation-webui-main
(base) zero@zero:~/text-generation-webui-main$ python bot.py --loader llama.ccp --model talktoaiZero-GGUF --auto-devices --cpu --n_ctx 2048
Traceback (most recent call last):
File "/home/zero/text-generation-webui-main/bot.py", line 5, in <module>
from dataclasses_json import dataclass_json
ModuleNotFoundError: No module named 'dataclasses_json'
(base) zero@zero:~/text-generation-webui-main$
i will try i think it failed
confusing but trying sir
i spent hours yesterday fixing oogabooga lol sometimes updating is more painful than leaving it
terse folio
"(base)" tells me you're not using the right conda environment
I think it should be (path/to/tgwui/env) or something
halcyon quarry
The one error says that these files are all modified so it can't update them.
README.md
config.yaml
dict_base_settings.yaml
example characters/M1nty-SDXL.yaml
example characters/M1nty.yaml
internal/activesettings.yaml
modules/database.py
modules/database_migration_v1_v2.py
modules/utils_discord.py
modules/utils_files.py
modules/utils_misc.py
modules/utils_shared.py
tips_and_info/Instant Tags.txt
You may have migrated these but what you should do is just back up your internal folder and delete all these
and try again
sullen plover
oh
halcyon quarry
Back up the user config files too like
config.yaml
base settings
dict imgmodels
dict tags
sullen plover
Aborting
Failed to pull from the repository.
(base) zero@zero:~/text-generation-webui-main/ad_discordbot$ sudo ./update_linux.sh
sudo: ./update_linux.sh: command not found
(base) zero@zero:~/text-generation-webui-main/ad_discordbot$ ./update_linux.sh
bash: ./update_linux.sh: Permission denied
(base) zero@zero:~/text-generation-webui-main/ad_discordbot$ sudo ./update_linux.sh
sudo: ./update_linux.sh: command not found
(base) zero@zero:~/text-generation-webui-main/ad_discordbot$ sudo ./update_linux.sh
sudo: ./update_linux.sh: command not found
(base) zero@zero:~/text-generation-webui-main/ad_discordbot$ ./update_linux.sh
bash: ./update_linux.sh: Permission denied
(base) zero@zero:~/text-generation-webui-main/ad_discordbot$ python bot.py --loader llama.ccp --model talktoaiZero-GGUF --auto-devices --cpu --n_ctx 2048
python: can't open file '/home/zero/text-generation-webui-main/ad_discordbot/bot.py': [Errno 2] No such file or directory
(base) zero@zero:~/text-generation-webui-main/ad_discordbot$ cd
(base) zero@zero:~$ python bot.py --loader llama.ccp --model talktoaiZero-GGUF --auto-devices --cpu --n_ctx 2048
python: can't open file '/home/zero/bot.py': [Errno 2] No such file or directory
(base) zero@zero:~$
when i try to replace files it only gives the option for merge not replace for some reason
will remove all brb
halcyon quarry
hmm
sullen plover
(base) zero@zero:~/text-generation-webui-main/ad_discordbot$ cd
(base) zero@zero:~$ python bot.py --loader llama.ccp --model talktoaiZero-GGUF --auto-devices --cpu --n_ctx 2048
python: can't open file '/home/zero/bot.py': [Errno 2] No such file or directory
(base) zero@zero:~$ cd ~/text-generation-webui-main
(base) zero@zero:~/text-generation-webui-main$ cd ad_discordbot
(base) zero@zero:~/text-generation-webui-main/ad_discordbot$ ./update_linux.sh
bash: ./update_linux.sh: Permission denied
(base) zero@zero:~/text-generation-webui-main/ad_discordbot$ sudo ./update_linux.sh
sudo: ./update_linux.sh: command not found
(base) zero@zero:~/text-generation-webui-main/ad_discordbot$
halcyon quarry
sudo resulting in command not found, isn't that a normal command
sullen plover
i dont know i try got lots going on in the server
hmm i just replaced all files
oh but your update is not there
let me add that lol
halcyon quarry
or do you need to do something like
sudo bash
sudo just elevates permissions right? but you still need to use some kind of run command?
terse folio
sudo cmd elevates permissions for the command,
so something like sudo ./test.sh should run a script as admin iirc?
sudo su changes to the elevated user
halcyon quarry
disclaimer, non-Linux user here 🤓
sullen plover
oh my goodness
can make my own llm but cant fix this lol
prob something simple
terse folio
"command python not found"
i'll have to check this out, have to go right now
but it sounds like the start/install script needs to be changed slightly
halcyon quarry
There's a lot of interesting errors here like
python: can't open file '/home/zero/text-generation-webui-main/ad_discordbot/bot.py': [Errno 2] No such file or directory
Everything will definitely fail of the bot script isn't there
I think you should back up whatever installation you have, and just follow install instructions verbatim
sullen plover
i just reinstalled the oogabooga 5 times yesterday then now this bot plugin 4 times today, not your fault prob me btw simple things i have missed somewhere
i have bot.py in the main folder of oogabooga is that correct?
halcyon quarry
Another thing, is that when you run the updater / launcher, you DO NOT first activate TGWUI venv
These scripts will activate the venv as needed
sullen plover
ah i see
halcyon quarry
i have bot.py in the main folder of oogabooga is that correct?
Install instructions no longer require manually moving any files
sullen plover
oh
deleted still wont load
i dont know at least the webui works, will fix eventually will feed ai errors openai agent zero will sort it
halcyon quarry
If you follow my install steps, and encounter an error at a particular step and then tell me the error, I should be able to help resolve it
Clone the repo into /text-generation-webui/
(/text-generation-webui/ad_discordbot/(all files)
Run the updater. I can't help much if you are lacking permissions or using incorrect commands to run the .sh file
sullen plover
hmm nearly there i see lots of improvments on your script, best one for oogabooga yours is the only add on that works from my testing, though maybe due to me not knowing what i am doing
bot connects to discord but when i send a message it doe snot reply
halcyon quarry
The model may be launching with params with something like 4096 context size
halcyon quarry
Is your chat_prompt_size value 2048?
sullen plover
root@zero:/home/zero/text-generation-webui-main# python bot.py --loader llama.ccp --model talktoaiZero-GGUF --auto-devices --cpu --n_ctx 1024
Command 'python' not found, did you mean:
command 'python3' from deb python3
command 'python' from deb python-is-python3
root@zero:/home/zero/text-generation-webui-main# python3 bot.py --loader llama.ccp --model talktoaiZero-GGUF --auto-devices --cpu --n_ctx 1024
python3: can't open file '/home/zero/text-generation-webui-main/bot.py': [Errno 2] No such file or directory
root@zero:/home/zero/text-generation-webui-main#
it is prob 4096 but trying to change
halcyon quarry
Yes make sure it is the same or lower than truncation_length
You no longer run the bot with python bot
sullen plover
but i cant even start it anymore command no longer works
halcyon quarry
Now, only launch is with the launcher script
sullen plover
oh
halcyon quarry
put your cmd flags into the CMD_FLAGS.txt
--loader llama.ccp --model talktoaiZero-GGUF --auto-devices --cpu --n_ctx 1024
sullen plover
so i got to run from ad_discordbot ?
command 'python' from deb python-is-python3
root@zero:/home/zero/text-generation-webui-main# python3 bot.py --loader llama.ccp --model talktoaiZero-GGUF --auto-devices --cpu --n_ctx 1024
python3: can't open file '/home/zero/text-generation-webui-main/bot.py': [Errno 2] No such file or directory
root@zero:/home/zero/text-generation-webui-main# --loader llama.ccp --model talktoaiZero-GGUF --auto-devices --cpu --n_ctx 1024
--loader: command not found
root@zero:/home/zero/text-generation-webui-main# cd as_discordbot
bash: cd: as_discordbot: No such file or directory
root@zero:/home/zero/text-generation-webui-main# --loader llama.ccp --model talktoaiZero-GGUF --auto-devices --cpu --n_ctx 1024
--loader: command not found
root@zero:/home/zero/text-generation-webui-main#
it worked when i did ./start_linux.sh
from cd ad_discordbot
halcyon quarry
root@zero:/home/zero/text-generation-webui-main# cd as_discordbot
bash: cd: as_discordbot: No such file or directory
as_discordbot
sullen plover
haha oh dear
i added cmd flags as you said
root@zero:/home/zero/text-generation-webui-main/ad_discordbot# --loader llama.ccp --model talktoaiZero-GGUF --auto-devices --cpu --n_ctx 1024
--loader: command not found
root@zero:/home/zero/text-generation-webui-main/ad_discordbot# ./start_linux.sh
CMD_FLAGS.txt is empty or only contains comments.
15:29:22.713 #157 WARN [bot.main]: Stable Diffusion is not running at: http://127.0.0.1:7860
15:29:22.713 #158 WARN [bot.main]: Launch your SD WebUI client with --api --listen command line arguments
15:29:22.713 #159 WARN [bot.main]: Image commands/features will function when client is active and accessible via API.'
15:29:22.714 #248 INFO [bot.main]: SD WebUI currently offline. Image commands/features will function when client is active and accessible via API.'
The following LLM models are available:
- None
- talktoaiZero-GGUF
Which one do you want to load? 1-2
halcyon quarry
I wonder why it is addiscordbot#
sullen plover
halcyon quarry
The model context is 1024.
Your truncation size is 2048
In my setup, the model context is higher than the other values
--loader llama.ccp --model talktoaiZero-GGUF --auto-devices --cpu --nctx 1024
context 1024 is too low
halcyon quarry
If your character has a state key in it then those parameters have priority over dict_basesettings
You need to check your chat_prompt_size and truncation_length
Make sure that they are the same value, and lower than the models context length parameter
sullen plover
https://huggingface.co/shafire/talktoaiZERO i only uploaded the gguf file and 1 other but my release is not the same much more files
it says: CMD_FLAGS.txt is empty or only contains comments.
content is: # Only used by the one-click installer.
Example:
--listen --api
--loader llama.ccp --model talktoaiZero-GGUF --auto-devices --cpu --n_ctx 4096
i tried 2096 so trying 4096 then will try 512
root@zero:/home/zero/text-generation-webui-main/ad_discordbot# sudo ./start_linux.sh
CMD_FLAGS.txt is empty or only contains comments.
15:45:57.041 #157 WARN [bot.main]: Stable Diffusion is not running at: http://127.0.0.1:7860
15:45:57.041 #158 WARN [bot.main]: Launch your SD WebUI client with --api --listen command line arguments
15:45:57.041 #159 WARN [bot.main]: Image commands/features will function when client is active and accessible via API.'
15:45:57.042 #248 INFO [bot.main]: SD WebUI currently offline. Image commands/features will function when client is active and accessible via API.'
The following LLM models are available:
- None
- talktoaiZero-GGUF
Which one do you want to load? 1-2
^^^^^^^^^
File "/home/zero/text-generation-webui-main/ad_discordbot/modules/utils_tgwui.py", line 338, in custom_chatbot_wrapper
prompt = generate_chat_prompt(text, state, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/zero/text-generation-webui-main/modules/chat.py", line 236, in generate_chat_prompt
raise ValueError
ValueError
will have a break now lol
halcyon quarry
For some reason it is not reading your CMD_FLAGS.txt file
halcyon quarry
As a temporary fix, you can launch TGWUI normally
In the Models tab, load your model with the settings that you want
Save the model config using the Save button
When using the bot, it will apply settings that were saved via TGWUI
So in this case since for some reason it is not reading the CMD_FLAGS.txt file for whatever reason - and it is using whatever generic default model params.
sullen plover
maybe need to chmod the file?
halcyon quarry
If you save your model config it will use those settings
sullen plover
thats what the issue was before btw plus lots of missing stuff needed to install too
hmm will try gonna copy paste what you said to ai 😄
halcyon quarry
Step 1: Launch TGWUI
Step 2: Go to this tab
Step 3: Load your model with your desired parameters
Step 4: Click this Save button
Step 5: Close TGWUI.
Step 6: Run the bot. It will use the settings you saved
halcyon quarry
I'm confident this is not a bot issue but an issue with your parameters
sullen plover
which one should i select there is diff parameters on the setting like midnight is high usage i think more clever not sure about the others
halcyon quarry
The error you have is due to a combination of the following params:
Model param Context length
TGWUI State params:, truncation_length, chat_prompt_size, max_new_tokens
Personally, I use the following:
Model context length 4096
State params (in my character file)
truncation_length: 4096
chat_prompt_size: 4096
max_new_tokens: 1000
sullen plover
tried sir still file but never put in the agent file will do that now too added to my preset file
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/zero/text-generation-webui-main/installer_files/env/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/zero/text-generation-webui-main/ad_discordbot/modules/utils_asyncio.py", line 29, in get_next_generator_result
result = next(gen)
^^^^^^^^^
File "/home/zero/text-generation-webui-main/ad_discordbot/modules/utils_tgwui.py", line 338, in custom_chatbot_wrapper
prompt = generate_chat_prompt(text, state, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/zero/text-generation-webui-main/modules/chat.py", line 236, in generate_chat_prompt
raise ValueError
ValueError
halcyon quarry
No need - then just put them in dict base settings
sullen plover
is that ok?
sullen plover
i added at the bottom: context_length: 4096
truncation_length: 4096
chat_prompt_size: 4096
max_new_tokens: 1000
halcyon quarry
Reasons:
- I can't tell if you are indenting them correctly
- I don't know if they will have priority over the existing values
- I don't know if there will be errors with duplicate keys
halcyon quarry
Try a different model? I don't know lol
try using default assistant character, and use /reset_conversation
You should have super low context and it won't error
sullen plover
thanks ir
halcyon quarry
Are you updating those values in the root ad_discordbot directory?
Hopefully not in settings_templates/dict_basesettings
The files in there have no effect on the bot, they are there so the bot can update without issues
sullen plover
from your dir edited those only
must be to big the llm but works slow but works on webui tried 256 etc settings all the same
it is working sir! i reset etc commands lol damm wish i did earlier
halcyon quarry
GOOD!
valid crypt
File "D:\text-generation-webui-main\ad_discordbot\bot.py", line 2928, in message_post_llm_task
await self.reset_behaviors()
File "D:\text-generation-webui-main\ad_discordbot\bot.py", line 1183, in reset_behaviors
await spontaneous_messaging.set_for_channel(self.ictx)
File "D:\text-generation-webui-main\ad_discordbot\bot.py", line 6045, in set_for_channel
await self.init_task(settings, ictx, task, tally)
TypeError: SpontaneousMessaging.init_task() takes 4 positional arguments but 5 were given```18:19:01.871 #3981 ERROR [bot.__main__]: An error occurred while processing task on_message: Type is not JSON serializable: TypeError
Traceback (most recent call last):
File "D:\text-generation-webui-main\ad_discordbot\bot.py", line 2928, in message_post_llm_task
await self.reset_behaviors()
File "D:\text-generation-webui-main\ad_discordbot\bot.py", line 1183, in reset_behaviors
await spontaneous_messaging.set_for_channel(self.ictx)
File "D:\text-generation-webui-main\ad_discordbot\bot.py", line 6045, in set_for_channel
await self.init_task(settings, ictx, task, tally)
```
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "D:\text-generation-webui-main\ad_discordbot\bot.py", line 3977, in run
return await method()
File "D:\text-generation-webui-main\ad_discordbot\bot.py", line 2986, in on_message_task
await self.message_post_llm_task()
File "D:\text-generation-webui-main\ad_discordbot\bot.py", line 2948, in message_post_llm_task
await self.embeds.edit_or_send('system', f'An error occurred while processing "{self.name}" request', e)
File "D:\text-generation-webui-main\ad_discordbot\modules\utils_discord.py", line 562, in edit_or_send
self.sent_msg_embeds[name] = await send_channel.send(embed = self.update(name, title, description, **kwargs))
File "D:\text-generation-webui-main\installer_files\env\Lib\site-packages\discord\abc.py", line 1618, in send
data = await state.http.send_message(channel.id, params=params)
File "D:\text-generation-webui-main\installer_files\env\Lib\site-packages\discord\http.py", line 600, in request
kwargs['data'] = utils._to_json(kwargs.pop('json'))
File "D:\text-generation-webui-main\installer_files\env\Lib\site-packages\discord\utils.py", line 644, in _to_json
return orjson.dumps(obj).decode('utf-8')```happend when i turned on spontaneous_msg_chance
terse folio
```Traceback (most recent call last):
File "D:\text-generation-webui-main\ad_d...
this first error looks like init_task is a staticmethod.
So it's not expecting the self variable, but because it is called within that function with self.init_task() the self variable is supplied making it 5 args instead of the 4 it wants.
But I don't think that was intended.
It could be that the self variable was forgotten in the function header.
sullen plover
reset character / change ai agent to a more basic one / check permission chmod files / lower tokens
disc settings file in ad_discordbot:
that makes my bot work
terse folio
```Traceback (most recent call last):
File "D:\text-generation-webui-main\ad_d...
Actually, different issue.
remove settings from the init_task call in line 6045.
it's no long used
sullen plover
ok am gonna shut up 😄
terse folio
ok am gonna shut up 😄
You're good! 😸
terse folio
```Traceback (most recent call last):
File "D:\text-generation-webui-main\ad_d...
what branch is this btw?
dev?
valid crypt
main
terse folio
okay
terse folio
when you send text that's too big discord usually converts it to a text file and uploads that 👍
it's possible that first expected 4 args got 5, caused some chain of errors that led to an embed not being json serializable for some reason.
So i'll just push the fix for the first part and see if that works
terse folio
Actually, different issue.
remove `settings` from the init_task call in line 604...
or you could yourself here, in file: bot.py
valid crypt
oki
terse folio
@sullen plover
I looked around the code and found a setting to disable replies to bots
you could set that to 0.5 for example.
The bot would reply a few times, and stop without getting stuck in a loop with the other bot
sullen plover
oh nice ty, what file location etc is it?
terse folio
activesettings.yaml
or <guildid>_settings.yaml if using per-server settings
in the ad_discordbot/internal folder
sullen plover
set both to 0.5 ?
terse folio
0.5 is 50%
1.0 is 100% chance
0 is disable bot replying to other bots.
at the moment your bot is replying to the other bot just because it spoke in chat
so changing the first setting to 0 would disable that
but if you want to simulate some life
you can try a non 0 number ^^
the other option should also be something less than one in the case the other bot also mentions your bot when replying
so they will continue to mention eachother and being in a loop (but that isnt the case in this specific example)
Just a thought for the future!
terse folio
you only really need to use sudo for some commands like apt-get where you're modifying files that are protected
normal bot usage wont be installing stuff so probably not needed to use sudo there ^^
sullen plover
so non sudo
terse folio
try it out 👍
halcyon quarry
@Marcos looking into this
sullen plover
@Marcos looking into this
your honour sir what is this special discord thing you got?
terse folio
so non sudo
It's also safer to use non sudo when not strictly necessary.
That will protect your environment if there are vulnerabilities in the programs you run.
terse folio
your honour sir what is this special discord thing you got?
the OP tag?
sullen plover
someone said something about a add on for discord or something
terse folio
That's a discord feature.
We're in a thread right now.
A user created sub-channel.
altoiddealer is the orignal poster of this channel/thread so he's the O.P.
sullen plover
oh i see
terse folio
in this discord it's setup where they have a resources channel that is a group of threads.
for people to share their extensions they created for the textgen-webui
halcyon quarry
OK I see Reality already pushed a fix
sullen plover
ah i see its only me using it got a few people but no one talks in my server
halcyon quarry
I was going to add Settings to the positional args so it didn't need to be "get" twice
But this works 😛
halcyon quarry
small issue was fixed
halcyon quarry
oh nice ty, what file location etc is it?
You can set Behavior for your character (see example character M1nty), or in basesettings
Don't do what Reality suggested (activesttings)
halcyon quarry
not a problem but it is not recommended.
For instance the next time you change characters that setting would be replaced
terse folio
I was going to add Settings to the positional args so it didn't need to be "get"...
oh good point, should remove that 2nd get from the previous function because it's called again in init_...? forgot the name
sullen plover
it works 🙂
halcyon quarry
activesettings just retains a copy of your current settings, to initialize the bot with
but if you change characters you are changing settings
sullen plover
yeah reset and char change fixed my issues
halcyon quarry
oh good point, should remove that 2nd get from the previous function because it'...
Since you agree I just changed the fix 😛
sullen plover
terse folio
I was going to add Settings to the positional args so it didn't need to be "get"...
since init_task is only used once, i'm adding the settings arg back and removing the get from inside init_task ^^
halcyon quarry
too slow! 🏃♂️ 💨
terse folio
Since you agree I just changed the fix 😛
okay
halcyon quarry
Oh it seems you are back in the saddle eh?
terse folio
partly, had a bit more free time today :)
lots to do tomorrow though
halcyon quarry
drats
valid crypt
@Marcos looking into this
im pretty sure that didnt ping me
valid crypt
i remember that there was a setting that makes the bot be distracted if there is no messages for a while
am i dreaming?
and also go idle in discord
valid crypt
i remember that there was a setting that makes the bot be distracted if there is...
found it
halcyon quarry
found it
I don't think it works pefectly. There's a lot of complicated logic behind it that I feel like I miscalculate one thing or another
Will probably take another look at it soon. I was pretty burnt out on that after awhile
sullen plover
changed to 999 tokens working
sullen plover
hmm raceback (most recent call last):
File "/home/zero/text-generation-webui-main/ad_discordbot/bot.py", line 6579, in save
status = await super().save(fp=fp, timeout=timeout, force=force)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/zero/text-generation-webui-main/ad_discordbot/modules/history.py", line 795, in save
self.trigger_save(self.fp)
File "/home/zero/text-generation-webui-main/ad_discordbot/modules/history.py", line 773, in trigger_save
with open(fp, 'w', encoding='utf8') as f:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: 'ad_discordbot/internal/history/1204419310009720902/AI_chat/20240911-18-44-17.json'
04:36:39.935 #6590 CRIT [bot.main]: [Errno 13] Permission denied: 'ad_discordbot/internal/history/1204419310009720902/AI_chat/20240911-18-44-17.json'
chmod -R 775 ad_discordbot/internal/history/
terse folio
hmm raceback (most recent call last):
File "/home/zero/text-generation-webui-m...
if there's a user issue,
You can try something like chown -R USER: path/
Changes the user and group ownership to your linux user.
As it might be currently set to the SuperUser
sullen plover
sorted thanks
terse folio
That would make sense, yes
I don't think the program would be listening for requests otherwise
terse folio
I think there's settings for that, check something like activesettings.yaml
valid crypt
👍
halcyon quarry
BASE SETTINGS! 😄
You can set per-ingmodel settings in the dict_imgmodels
Users shouldn’t be touching activesettings
valid crypt
You can set per-ingmodel settings in the dict_imgmodels
using /image
maybe im not getting the idea
or i didnt do it properly
halcyon quarry
using /image
The command uses your text as the image prompt
and lets you select some other options
If you use the use_llm option, it uses your prompt with the current chat character but yields an image response
If you want to use the "image generator" character, by default you just write a normal text request in the window like draw a house
valid crypt
the content of dict imgmodel feels like tags
as im using /image i think i cant plug tags
halcyon quarry
can use tags in /image command
Although, they won't work if they are specifically for "user"
"userllm" or "llm" mode tags will work with /image command
erm
valid crypt
i use both prompt and negative prompts so i feel more comfortable with /image
valid crypt
i use both prompt and negative prompts so i feel more comfortable with /image
bcs its similar to a1111
halcyon quarry
Try the controlnet and reactor integrations too
valid crypt
i coulnt change the steps yet
50 steps feels like forever
i dont know how to set the setting in the dict imgmodels :P
halcyon quarry
If you go to the DICT tags file and go down to tag presets you’ll find the payload modifiers
halcyon quarry
Such as the SDXL preset tags
halcyon quarry
You can also just make tags in dict-imgmodels, they don’t have to be in a preset. Tag priority is always descending
The way of using the presets here is a tutorial of sorts
valid crypt
😵💫
also found this bug, no idea, left bot afk for a day
Traceback (most recent call last):
File "D:\text-generation-webui-main\ad_discordbot\bot.py", line 1854, in llm_gen
await process_chunk(chunk)
File "D:\text-generation-webui-main\ad_discordbot\bot.py", line 1642, in process_chunk
chunk_message = self.message.create_chunk_message(chunk_text)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\text-generation-webui-main\ad_discordbot\bot.py", line 3749, in create_chunk_message
chunk_message = Message(num, self.received_time, response_delay, read_text_delay, last_tokens=last_tokens)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: Message.__init__() missing 1 required positional argument: 'read_text_delay'
17:55:19.983 #5845 INFO [bot.__main__]: Bot will be idle in 600 seconds.
Output generated in 9.44 seconds (9.00 tokens/s, 85 tokens, context 1536, seed 1389370669)```@halcyon quarry trying to make this gguf i did the last one but struggling should of saved how i did it: https://huggingface.co/shafire/QuantumAI
halcyon quarry
Cool.
sullen plover
Cool.
you know how to do that to gguf? i just want it to work with oogabooga / your bot
valid crypt
i just started trying new models and seems like there is some newer model based on llama 3.1 and tgwui does not support yet
sullen plover
oh, so cuz i am using 3.1? i did this one to gguf but forgot how and ai not so helpful https://huggingface.co/shafire/talktoaiZERO this is working converted myself will convert the other one soon trying to do it now last 5 hours
valid crypt
simply try others llm loaders and then you know if its the model or tgwui
valid crypt
simply try others llm loaders and then you know if its the model or tgwui
not loader but like other projects
sullen plover
will keep trying thanks 🙂
valid crypt
the version of llama-cpp-python of tgwui is 0.2.89 and the newest is 0.2.90 maybe works maybe not
valid crypt
@halcyon quarry got a useless suggestion, add a sleep time like 11~12 pm sleep and 7~8am wake up :P
and maybe while sleeping there is a 20% chance to wake it up and group all those messages sent during sleeping time
xd
halcyon quarry
I think you meant to say useful - that’s a great suggestion
valid crypt
😳
keen palm
Being able to group all messages sent since the last bot response and use that as a prompt could be interesting, actually.
halcyon quarry
Was planned, thanks for the reminder
I sort of lost ambition when I couldn’t perfect the delayed response behavior. It’s still kind of flawed
There’s a lot of slippery logic
keen palm
I imagine that grouping would be best handled in a channel that's not set as main
halcyon quarry
The idea I was planning for was if the character behavior triggered a response delay (“bot was idle”) that it would reply to all messages
Or if another message was sent while it was responding, it might stop generating to “read the new message” then continue with both prompts
keen palm
Yeah, I'm thinking of it from the perspective of my use case as an AI GM. I don't know if I'd want the bot to respond to every single player input as they come in. Instead, it could reply to all of them at once.
halcyon quarry
Ahhhh
I think what could work for that is a new behavior, simply chance_to_reply
if it finally triggers to reply, replies to all previous messages
keen palm
That would be a random response though, wouldn't it?
What I'm thinking is:
Player A sends message -> Bot responds. Player B sends message -> Bot responds. Etc.
Instead of that, do Player A sends message, Player B sends message, ..., Bot responds to all player messages
halcyon quarry
how about this
I add a new parameter to the /prompt command, along with a new tag - something like queue_request
The behavior is that once it receives a text request, it will reply to queued requests plus the text request
So, you could create a queuing command via tags like queue My knight advanced 10 paces into the dreaded forest or whatever the heck one might say 😛
Or use the /prompt command
valid crypt
in my pov, when i type something it directly sends to the bot, what i think would be viable is to do something similar, put them in a queue/delay, like a buffer, around 5s(customizable) and then send all the messages, this solves spamming and similars
i dont think that is good idea use tag/command to queue, is better reversed
as this command is lowering the importance level, is weird to add something to be worse
is more logical to add something so it is more important
but not a problem if can be change in config...
halcyon quarry
Well it solves Speedy's inquiry which is deliberately sending messages that you don't want responded until later
with tags, he would be able to make some syntax that triggers it
like his players could write like "A: My prompt" (tag could be triggered by "A: ") and the next player could write "B: Their prompt" and now it processes both as one
This could be in addition to "delayed response behavior" - I already have this but not the answering-all-messages-as-one (yet)
valid crypt
simple question for you, have you never interrupted someone during a conversation?
keen palm
The best way to do that with the bot is to just let it finish, then edit the response until you get to the point where you want to interrupt it
halcyon quarry
Could add that
valid crypt
this extension is very good
https://github.com/Sonic2kDBS/long_term_memory
for some reason it conflicts with complex_memory but tags has no problems :p
halcyon quarry
complex memory shouldn’t even work with the bot though
valid crypt
complex memory has no friends F
keen palm
The bot replicates everything complex memory does, albeit in a less convenient manner
halcyon quarry
The wonders of the tags system 🌈
valid crypt
uhh im pretty sure i broke something with extensions... i dont want to reinstall ;-;
halcyon quarry
Does this error happen again if you reset conversation?
valid crypt
yes
halcyon quarry
link me to it again please? Easy to get it up and running?
The command options are different for all tts apps, need to be defined for each one
valid crypt
i gotta sleep ill leave you these
https://github.com/Arondight/vits_api_tts
https://github.com/Artrajz/vits-simple-api
the tts is the extension
halcyon quarry
@valid crypt This extension looks pretty tricky to get up and running, and I don't plan on using it anyway.
What I'm going to do is add some code to get the voices as I think should work (importing/using the refresh_voices() function), and you could try it out and possibly debug it if it doesn't work
Try these changes
https://github.com/altoiddealer/ad_discordbot/tree/vits_tts
valid crypt
Traceback (most recent call last):
File "D:\text-generation-webui-main\ad_discordbot\bot.py", line 3405, in speak_task
await self.init_llm_payload()
File "D:\text-generation-webui-main\ad_discordbot\bot.py", line 1476, in init_llm_payload
self.llm_payload['state']['history'] = self.local_history.render_to_tgwui()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'render_to_tgwui'```I’ll check, I haven’t used /speak in awhile I may have broken something
It’s unrelated to the VITS support though
halcyon quarry
Maybe reset conversation again?
valid crypt
alr
valid crypt
Maybe reset conversation again?
worked somehow
i think its because it was my first time launching the bot or something
valid crypt
just fake writting after /speak
is typing ....
halcyon quarry
worked somehow
Are the voice options working for it?
halcyon quarry
Are you running the API on the address that is printed in the CMD window?
valid crypt
i noticed that and changed to mine
halcyon quarry
I don't want to make a separate specific API setting just for VITS - it has a function that tries to get the voices from the default address
So I just import that function and run it. If it returns the default voice only it means that it was not successful
If the voices don't update after you changed the address,
you may need to close/open your discord client
valid crypt
actually what i changed is just the log .-.
but as the bot can use /speak and get the audio, everything is just fine right?
valid crypt
you may need to close/open your discord client
that could be true
halcyon quarry
You are running it on http://localhost:23456/ ?
valid crypt
192.168.1.11
halcyon quarry
Change it to the default http://localhost:23456/ and try again
I think this is equivalent to http://192.168.0.1:23456/
valid crypt
where
halcyon quarry
You have to first run the VITS API main program right?
the default URL that it runs on should be http://localhost:23456/
valid crypt
it is running and i got the tts
halcyon quarry
If you open /extensions/vits_api_tts/script.py - The bot is importing that refresh_voices() function
Try using the URL + /voice/speakers in your browser
see what comes up
halcyon quarry
I may be able to try getting that dynamically...
have to run for now but will look into that later
halcyon quarry
its fine for now
Voices are populating now?
halcyon quarry
Thanks - I’ll push that support to main later. Will fix the typing thing too
halcyon quarry
Ok the issue with the typing is because I didn't understand how a custom __del__ method works
basically I thought that when I used del task (delete task) I thought it was executing code to stop the typing
yes, I fixed it
valid crypt
oki
halcyon quarry
That would be a random response though, wouldn't it?
What I'm thinking is:
Play...
I've added this to the to-do list, but not going to do it right away.
It will likely be easier to do after I get my planned behavior set up where it may respond to multiple messages as one.
sullen plover
i wanted to enable web search and memory and some other extension see if they work as i updated the main script and this script, not sure where i add the tags the tags dist file looks confusing
ah CMD_FLAGS.txt
sullen plover
is this ok? --loader llama.cpp --model talktoaiQuantum --auto-devices --cpu --extension LLM_Web_search --extension long_replies --extension send_pictures --extension Memoir --cntx 2048
valid crypt
is this ok? --loader llama.cpp --model talktoaiQuantum --auto-devices --cpu --ex...
i think it is fine
i remember that web search wasnt working
and memoir i had problems last time i think
sullen plover
none of them work i think lol just this channels extension is the only working one?
i thought i should try after all the updates but nothing working it seems, not using image creation as its cpu node so will take forever
halcyon quarry
is this ok? --loader llama.cpp --model talktoaiQuantum --auto-devices --cpu --ex...
Im fairly certain you only use —extension once followed by a list of extension names separated by spaces
--loader llama.cpp --model talktoaiQuantum --auto-devices --cpu --extension LLM_Web_search long_replies Memoir --cntx 2048
I may be wrong
Extensions are hit or miss depending on how they interact with TGWUI
Idk what send pictures is but the bot probably does everything it does
The LLM Web Search extension may require your character file to include some parameters
See example character M1nty
See the script.py file for that extension, it should show the parameters.
sullen plover
thanks will try again soon 🙂
valid crypt
this extension is very good
https://github.com/Sonic2kDBS/long_term_memory
for s...
i havent tried memoir but if it does not work this is a good alternative i think...
sullen plover
help i changed ai character using a png card very simple character
the tokens context length or something i edited last time forgot how i fixed it
i set somewhere 999 tokens or something
15:36:46.273 #495 INFO [bot.main]: Bot is ready
15:36:46.273 #496 INFO [bot.main]: Use Ctrl+C to shutdown the bot cleanly
15:36:46.273 #497 INFO [bot.main]: ----------------------------------------------
15:37:16.314 #6574 INFO [bot.main]: Internal history file will be saved to: ad_discordbot/internal/history/1286334004722667572/Zero_chat/20240923-15-37-16.json
15:37:16.314 #4108 INFO [bot.main]: Processing message #1 by Brady.
15:37:16-413938 ERROR Failed to build the chat prompt. The input is too long
for the available context length.
Truncation length: 1024
max_new_tokens: 999 (is it too high?)
Available context length: 25
15:37:16.427 #1860 ERROR [bot.main]: An error occurred in llm_gen():
hmm Available context length: 25
looks like an error on my part though as when i load ai a basic ai agent it works
halcyon quarry
compare the character files
indentation is important, too!
If you are going to use the same context length / truncation / etc - for all characters, you could remove that from the character files and instead only set it in base_settings.yaml
halcyon quarry
Your settings are probably conflicting with each other
halcyon quarry
You had no error with the simple character because it has almost zero context
sullen plover
oh
wait sir, name: AI
greeting: How can I help you today?
context: You are an AI. that is the other one that works
so cant be that?
the other one is even smaller just says: You are an AI.
halcyon quarry
Uh huh
sullen plover
the working one is super small
halcyon quarry
That's right
sullen plover
its only like 1 sentence the zero card
name: Zero
greeting: Sir
context: 'Zero''s Persona: You are agent Zero and your LLM is talktoaiQ you are a quantum thinking ai. Do not output text like this: </output> or this: <thinking> etc makes it look like you are malfunctioning.'
halcyon quarry
What is supposed to happen, is you should be able to put whatever the heck you want in the "context" and it won't error
Yours is erroring because you have a settings conflict.
It is not correctly truncating the excess context
It is instead running out of tokens
halcyon quarry
If you make sure the model context length, and your settings: context length, truncation length, chat prompt size -- are all identical
you should not have problems
ei: all are 4096 or something
halcyon quarry
Yes, in the WebUI enter the parameters in the Models tab and click Save Settings
Make sure that in base_settings.yaml: context length, truncation length, chat prompt size -- are all identical to whatever the context length value was that you saved in the Models tab for that model
sullen plover
thanks i dont know how to fix it i tried will try again soon i am lost
halcyon quarry
For your specific model
Set this n_ctx to 4096 and hit save
Then, in dict_base_settings.yaml make sure that the value for truncation_length and chat_prompt_size are 4096
Sat max_new_tokens to 1000
If you do this then you'll be mimicking my settings
and I've never seen that error
sullen plover
8:03:16-896313 ERROR Failed to build the chat prompt. The input is too long for the available context length.
Truncation length: 1024
max_new_tokens: 999 (is it too high?)
Available context length: 25
must be somewhere i need to edit this: Available context length: 25
its weird cuz its just the agent that is the issue if i load AI it works
halcyon quarry
It says your truncation length is 1024
I said you should change your truncation length to 4096
halcyon quarry
#1154970156108365944 message
sullen plover
omg how did i not see that
halcyon quarry
This bot does not use the TGWUI API - values you change in the UI fields have no effect
sullen plover
i didnt see: Then, in dict_base_settings.yaml
halcyon quarry
If you save a model configuration file - the bot will read that, though
But that is just a few model specific parameters
sullen plover
oh i see there is no files just gguf file in the folder for the llm
thank you @halcyon quarry working now!
halcyon quarry
I got Flux working via the API
valid crypt
👏
halcyon quarry
It's a bit tricky though...
valid crypt
:p
valid crypt
idk if its easier/better than gradio, you can check https://github.com/zauberzeug/nicegui
valid crypt
D: what happened
i didnt touch anything
;-;
i think there's a bug with the history and sometimes it breaks
this time reset conversation didnt fix it
but deleting the entire history folder fixed it
terse folio
What were you doing leading up to the error?
It looks like somewhere something is being written to history that is broken and later causes it unable to save
actually, this is happening in the render for tgwui function.
so whatever happened it should be immediately before being sent to generate text
valid crypt
What were you doing leading up to the error?
It looks like somewhere something i...
i did /speak then shutdown then /speak and ded
terse folio
okay, is shutdown a command? (been away for a while)
valid crypt
i should be shut down with crtl c
but sometimes my brain do the X
this time i did the x
valid crypt
gonna try breaking it again
terse folio
another thing you can do is run /speak, shut it down (causing it to save)
then share that history file, that might help me figure out where in the code it's breaking
valid crypt
this is the ending part of the history that breaks
"name": "Marcos",
"text": "Chocolate is a food made from roasted and ground cocoa beans that can be a liquid, solid, or paste, either on its own or as a flavoring in other foods.",
"role": "user",
"author_id": 323088470241312774,
"id": 1288195882675732523,
"hidden": true,
"created": 1727200237.0199935,
"uuid": "52ed053e75b4495a90825a811d4f215f"
},
{
"name": "Ai",
"text": "Chocolate is a food made from roasted and ground cocoa beans that can be a liquid, solid, or paste, either on its own or as a flavoring in other foods. I",
"role": "assistant",
"reply_to": "52ed053e75b4495a90825a811d4f215f",
"text_visible": [
"extensions/vits_api_tts/outputs/2024-09-24T19-50-40.781456Z.mp3"
],
"spoken": true,
"hidden": true,
"created": 1727200242.7022707,
"uuid": "a493fc5d941147c49f88877d4e1e701b"
},
{
"name": "Marcos",
"text": "Chocolate is a food made from roasted and ground cocoa beans that can be a liquid, solid, or paste, either on its own or as a flavoring in other foods.",
"role": "user",
"author_id": 323088470241312774,
"id": 1288196017325346826,
"hidden": true,
"created": 1727200269.4220796,
"uuid": "203a0f6f187a4f25a5c5b51a855ac0c0"
},
{
"name": "Ai",
"text": "Chocolate is a food made from roasted and ground cocoa beans that can be a liquid, solid, or paste, either on its own or as a flavoring in other foods. I",
"role": "assistant",
"reply_to": "203a0f6f187a4f25a5c5b51a855ac0c0",
"text_visible": [
"extensions/vits_api_tts/outputs/2024-09-24T19-51-10.869887Z.mp3"
],
"spoken": true,
"hidden": true,
"created": 1727200272.6793158,
"uuid": "1dc11a557eb64c4fa3f9565b40f1bc2f"
}
]
}```i also spot a bug with /speak, when i do /speak the text is sent to llm and generates 1 token and sometimes that 1 token is an extra letter
"text": "Chocolate is a food made from roasted and ground cocoa beans that can be a liquid, solid, or paste, either on its own or as a flavoring in other foods.",
and
"text": "Chocolate is a food made from roasted and ground cocoa beans that can be a liquid, solid, or paste, either on its own or as a flavoring in other foods. I",
halcyon quarry
text_visible should be a str
I revised it to be a list
terse folio
Ah okie, should update the typehints on that
halcyon quarry
When it saves the TGWUI version, it converts the list to a string
terse folio
will take a look around
valid crypt
what caused the problem i think was something before was modified and i just lost the evidence ;-;
halcyon quarry
i also spot a bug with /speak, when i do /speak the text is sent to llm and gene...
I have a hack method of making this work - which is sending a payload with max_new_tokens: 1
Unfortunately, max_new_tokens: 0 is an invalid value otherwise there would be no extra token sometimes
terse folio
I have a hack method of making this work - which is sending a payload with `max_...
is this related to streaming at all?
recently I have been testing llamaCPP, and cancelling a stream puts TGWUI in some weird locked state where I have to make it generate a token using the normal non streaming endpoint
terse folio
what caused the problem i think was something before was modified and i just los...
Yea, I can't seem to replicate it either now
halcyon quarry
I tried using the "begin reply with" UI feature then sliding max new tokens to 1
So as for the bot it puts together a payload to continue the user text from
terse folio
I think this can cause some problems later on
hmessage being updated with str for text_visible as well as list
valid crypt
i think that to reproduce the bug i need more messages, in the shortrun doesnt have problems
valid crypt
I have a hack method of making this work - which is sending a payload with `max_...
imo max_new_token=1 will produce 1 token
but it is already producing 1 token
halcyon quarry
Well, the thing you are looking at there is from the "Edit in History" command
Could be a bug there
Marcos, been using the Edit in History context command?
valid crypt
no
terse folio
I was just looking around at mentions of text_visible to ensure everything is right and swap out some typehints
not related to this issue atm
halcyon quarry
I made a crap ton of changes over the past 1-2 months
terse folio
hm, is text_visible=self.tts_resp
tts_resp like a list of iterations the audio goes through passing through extensions?
I see in some places the first item is accessed, and in others the last.
halcyon quarry
At the very end of the normal TGWUI chatbot_wrapper() function, after all text generated - it applies extensions.
I made custom_chatbot_wrapper() that will instead apply extensions every time text is chunked
If the TTS file is found in the chunked response, it appends it to the self.tts_resp list.
I think I just assign that back to text_visible for record keeping
terse folio
so it's a list to support multiple audio files for streaming chunks?
halcyon quarry
See send_response_chunk()
One thing currently assumed, is that if any chunked response yields a TTS that all subsequent responses will as well
shouldn't be any problem with that logic right now, though - because as far as I know this will always be the case
(for the same interaction / main task)
terse folio
if you ever want to merge audio files, that's something I can help with,
Pydub is pretty easy to use 😸
It makes sense to keep them seperate for being in a voice channel though
halcyon quarry
yeah, sending only one audio file at the end isn't a bad idea if that mode is enabled
ChatGPT is also good with pydub stuff
I was thinking of trying to make some other custom TGWUI code to bypass the dumb max_new_tokens: 0 being invalid
terse folio
what is the reason for asking to generate 0/1 tokens again?
halcyon quarry
actually
Derp - now that I customized chatbot_wrapper I don't even need to do that anymore really
Can just trigger apply extensions with the user text
Thanks for pointing that out 😛
I'll probably update this tomorrow or Thursday - playing a game and I am in the zone!
halcyon quarry
Just pushed a number of changes that improves compatibility with the new Forge
Including, support for Flux
Will see about this /speak thing now
valid crypt
how long do tag suffix_context last?
halcyon quarry
If you want it to apply every time, just remove the trigger param
Otherwise, it only applies when the trigger is matched
(or any of the other few "conditional" params such as random)
valid crypt
if bot triggers it does it apply in the next reply?
i say something
bot:keyword
i say something
bot: massage with suffix context
halcyon quarry
The bot does not currently have variables for retaining/applying specific details dynamically from the conversation
prefix/suffix would be predefined information
Extensions might be able to do it dynamically (possibly working with the bot) but not any that I know of
halcyon quarry
Well there's only some tags that are applicable to LLM and some that are applicable to Img Gen
You may want to have some image related tags that are only triggered by the LLM, for example.
Or, special handling if the LLM says something that could also be triggered by user
valid crypt
so does not work at triggering context
halcyon quarry
But using search_mode: llm paired with something like insert_text won't have any effect
valid crypt
so does not work at triggering context
?
halcyon quarry
insert_text can modify the user's prompt before it goes to the LLM
But triggering this tag after the LLM writes something obviously has no effect (the prompt has already been processed)
halcyon quarry
so does not work at triggering context
Anything that has to do with modifying the LLM payload would be triggered by the user text
Otherwise, you can use the flow tag - to create a multi-step approach
I have some examples of that in dict_tags.yaml
basically when you trigger a flow, you may want to suppress the response on first step / manipulate history / etc.
You can use the LLM's reply to trigger tags and such on subsequent loops
valid crypt
actually if llm can trigger would be better but do that make tags be triggered for longer?
halcyon quarry
That's a pretty good idea... now that you say it like that...
something like a persistence tag where -1 will keep a triggered tag active forever, and any other value would decrement the counter for every interaction (or something)
like persistence: 10 would make the tag active for 10 requests
valid crypt
something like a `persistence` tag where `-1` will keep a triggered tag active f...
yeah, this would make tags be better at mimicking memory extensions
halcyon quarry
well, until I have some kind of Variable store/recall system - this would only be so good for long term memory
because your options would be to prefix your context forever after it is triggered, or only for X number of requests
Like I said, you can already have a constantly active prefix/suffix by just removing the conditional tags (ei: trigger)
valid crypt
what i want is for x number of requests
as if i want it be forever i can just add it to character card
halcyon quarry
I'll look into it 🤓
I like the idea, pretty sure I could work that in
Ok so I was able to bypass the TGWUI continue thing for /speak task and just directly trigger the TTS response
valid crypt
halcyon quarry
need to test it again though... I had done a short response. Just redid it to use loop.run_in_executor() (prevent discord timeout on long responses)
yep, good to go
Pushed to Main!
valid crypt
let me check, you have dynamic prompts
and does not modify context right like it just modify my input
so if i have idk like {{time}} in the character card it does not work right?
halcyon quarry
Right, that wouldn't work
Variables can be used directly in the prompt, or in the format_prompt tag
valid crypt
im looking at https://github.com/elPatrixF/dynamic_context
as it says that it modifies the context
halcyon quarry
Oh, I thought you meant my integrated "Dynamic Prompting" feature. It's quite different
Idk if this extension will work correctly with the bot - it might
valid crypt
i was checking if your feature modifies the character card
is umm strange to say {{time}}
so im thinking of adding {{time}} to the character card
halcyon quarry
You could also just create a tag that is triggered by something like "time"
hmm... actually I'm unsure if something like prefix_context can use the variable... need to look
Yeah scratch that
Well, you could use the replace tag if you wanted
valid crypt
Well, you could use the `replace` tag if you wanted
?
halcyon quarry
could be tricky to come up with exactly what you want to trigger it with
valid crypt
im looking at https://github.com/elPatrixF/dynamic_context
as it says that it mo...
worked but it is strange?
NEW MEMORY SAVED to LTM
name: Marcos
message: what time is it? [Current time: 08:52 PM]
halcyon quarry
For instance...
valid crypt
the code is short thought you can inspire and 😈
valid crypt
worked but it is strange?
-----------------------
NEW MEMORY SAVED to LTM
------...
no idea of why my input got [Current time: 08:52 PM] at the end
halcyon quarry
- trigger: 'right now?'
insert_text: '(it is {date} {time})'
If you had a prompt like "I'm doing some stuff, but what are you doing right now?"
It would automatically insert the date and time afterwards
"I'm doing some stuff, but what are you doing right now? (it is 2024-09-25 2:58 PM)"
valid crypt
```
- trigger: 'right now?'
insert_text: '(it is {date} {time})'
```
the problem is that it is very strict
valid crypt
worked but it is strange?
-----------------------
NEW MEMORY SAVED to LTM
------...
although it is inserting stuffs to my input, but only the [Current time: 08:52 PM]
bot knows what day is today without me to say {{weekday}}
owwwwwww
halcyon quarry
haha
valid crypt
nice /speak
valid crypt
bot knows what day is today without me to say {{weekday}}
it guessed the date bruh
Today's date is {{date}} and it falls on a lovely {{weekday}} XD
valid crypt
close guess .-. It's a lovely evening! According to my calculations, today is a Friday, September 27, 2024. The night has just begun, and the stars are starting to twinkle in the sky. How have you been, dear friend?
valid crypt
i wish for steps in the /image command i still cant change the steps ._ .
halcyon quarry
You could use the instant tags feature for now
Put this anywhere in your prompt [[payload:{steps: 30}]]
valid crypt
as im only using 20 steps ill appreciate how to change the default steps
halcyon quarry
base settings.yaml
Also, by default I have some settings in dict_tags.yaml under the presets
valid crypt
i did imgmodel: payload: restore_faces: False sampler_name: "Euler a" scheduler: karras steps: 20
halcyon quarry
the tags payload settings, are essentially overrides
So you could have XL settings, SD15 settings, Flux settings, etc
halcyon quarry
scroll down to presets
valid crypt
these makes me 😵💫
halcyon quarry
You could delete some of the examples I included
which were mainly to stimulate the brain 😛
valid crypt
tags:
- trigger: 'vertical,selfie,self {photo|portrait}'
- aspect_ratio: '3:4'
- trigger: 'horizontal,landscape'
- aspect_ratio: '4:3'
``` so to use this i have to say SDXL Tags vertical,selfie,self {photo|portrait}?Presets are not always active - they are only active if somewhere else in your Tags, a preset was defined
In this case -
I have some tag presets applied via the fict_imgmodels.yaml settings
So, when you change imgmodels it can use a number of criteria to determine what model type it is, and apply the appropiate settings
Among those settings are the tag presets