ad_discordbot (Fork of Fork of xNul's bot) | Text Generation WebUI | Page 2

vast elm Dec 7, 2023, 8:43 PM

#

I thought it auto switches character when i write draw or generate a picture

#

Or do i have to do it somewhere in settings again

halcyon quarry Dec 7, 2023, 8:44 PM

#

There's triggers to generate an image response, and there's triggers to switch characters

#

by default, "generate" would trigger an image response but not a dynamic context

#

"draw" will trigger both

#

You can customize it if you prefer different behavior

vast elm Dec 7, 2023, 8:45 PM

#

#

#

Do i have to update activesettings too?

halcyon quarry Dec 7, 2023, 8:46 PM

#

Since youve been gone for awhile I recommend starting from scratch, and comparing your files that have customized settings

#

The structure of imgmodels file may be different now (imgloras became imgtags, etc)

#

Also see if you are using the API method... setting in config.py

viral lagoon Dec 7, 2023, 8:50 PM

#

btw, coqui_tts has a glaring problem: It doesn't filter out *stuff* or does any other kind of filtering.
Need extensions/silero_tts/tts_preprocessor.py, which I connected to coqui as a test and works wonderfully.

halcyon quarry Dec 7, 2023, 8:51 PM

#

as in like, background noise, etc?

#

buzzing humming crackling etc

viral lagoon Dec 7, 2023, 8:51 PM

#

No

#

coqui literally says the entire reply out loud

vast elm Dec 7, 2023, 8:52 PM

#

halcyon quarry Also see if you are using the API method... setting in config.py

I do

halcyon quarry Dec 7, 2023, 8:52 PM

#

vast elm I do

Your SD port is probably different than what is in config.py

#

(Im guessing)

viral lagoon Dec 7, 2023, 8:52 PM

#

silero_tts only speaks stuff "between quotes" as far as I know, and has many more filters and fixes, it's quite intricate actually.

vast elm Dec 7, 2023, 8:53 PM

#

halcyon quarry Your SD port is probably different than what is in config.py

Its the same port

halcyon quarry Dec 7, 2023, 8:53 PM

#

Ah, I think I understand... like giggles it speaks aloud "giggles"

vast elm Dec 7, 2023, 8:53 PM

#

It doesn't even attempts to connect to sd

#

halcyon quarry Dec 7, 2023, 8:54 PM

#

Like I said, you need to start from scratch

#

Feel free to share a copy of that character that failed, I could see if there is a glaring issue

#

Or, something I need to take into consideration with the code

viral lagoon Dec 7, 2023, 8:56 PM

#

halcyon quarry Ah, I think I understand... like *giggles* it speaks aloud "giggles"

Yeah, basically.
It's not an issue with your bot though

halcyon quarry Dec 7, 2023, 8:57 PM

#

I'm intrigued on the filtering... Ill check that out, might be able to make that configurable

viral lagoon Dec 7, 2023, 8:57 PM

#

I honestly can't do RP with coqui in it's current state though

halcyon quarry Dec 7, 2023, 8:58 PM

#

The main hook with coqui really is its ease of use and "good enough" on-the-fly voice emulation

vast elm Dec 7, 2023, 8:58 PM

#

So deleting ad_discordbot and just pulling it all again should do it?

halcyon quarry Dec 7, 2023, 8:58 PM

#

like, I clipped a few seconds from a david attenborough interview and the replies are pretty good

viral lagoon Dec 7, 2023, 8:59 PM

#

Yeah, it is quite impressive, though not consistent sadly

halcyon quarry Dec 7, 2023, 8:59 PM

#

vast elm So deleting ad_discordbot and just pulling it all again should do it?

I'd recommend backing up the folder, then yes clone a new copy

vast elm Dec 7, 2023, 8:59 PM

#

Ok i did that, and still same thing

halcyon quarry Dec 7, 2023, 8:59 PM

#

yeah the voice does change dramatically sometimes between sentences, and a few random pauses here and there

vast elm Dec 7, 2023, 9:00 PM

#

#

Do i have to update stable diffusion too to work with this?

#

halcyon quarry Dec 7, 2023, 9:01 PM

#

Does it say something like this in your cmd window?

Dynamic Context: Character: M1nty-SDXL | Prompt: a picture of an ant | History: []

vast elm Dec 7, 2023, 9:02 PM

#

halcyon quarry Dec 7, 2023, 9:03 PM

#

check config to ensure it has this option

#

    'filter': [''], # Example: 'xl' = likely just SDXL models. Can match multiple such as ['xl', '15']

vast elm Dec 7, 2023, 9:03 PM

#

And dynamic context seems to be on

halcyon quarry Dec 7, 2023, 9:04 PM

#

Please share a copy of the character that is failing to load

vast elm Dec 7, 2023, 9:04 PM

#

I mean the character doesn't matter. The minty file even doesn't does pictures

#

I only changed the token in config, otherwise its all original

#

Like here i just selected minty from the github page

halcyon quarry Dec 7, 2023, 9:07 PM

#

I'm deleteing my own config file and using the one from github

#

let's see if I can reproduce...

vast elm Dec 7, 2023, 9:07 PM

#

I deleted the entire ad discordbot folder, and pulled it again. And put the minty character file in

halcyon quarry Dec 7, 2023, 9:10 PM

#

Idk man... I replaced my config file with the default one, and simply added my bot token.

#

I launched, used /character and picked my character, then "draw a cat"

#

vast elm Dec 7, 2023, 9:11 PM

#

Weird

halcyon quarry Dec 7, 2023, 9:12 PM

#

Your character failing to load could be a problem...

#

Also note, I am currently on latest stable releases of A1111 and testgen-webui

vast elm Dec 7, 2023, 9:13 PM

#

Lets try updating everything then

halcyon quarry Dec 7, 2023, 9:14 PM

#

Another thing you could try...

#

launch A1111, and in url bar add "/docs" (may be http://127.0.0.1:7860/docs)

#

find the GET: /sdapi/v1/sd-models
"Try it out"
"Execute"

#

The bot essentially performs this to fetch the model list.

vast elm Dec 7, 2023, 9:18 PM

#

#

#

did update text gen

halcyon quarry Dec 7, 2023, 9:20 PM

#

You are running something old

#

let's see...

vast elm Dec 7, 2023, 9:21 PM

#

#

I could select the minty character now

#

I think

halcyon quarry Dec 7, 2023, 9:22 PM

#

Yes, the problem was that in recent ooba versions the load_character() function returns 7 values now

#

So, now that you've updated that is resolved

#

(its compatible with my code)

#

OK - the problem is that the "draw" is case sensitive 🙂

#

(try "draw __")

vast elm Dec 7, 2023, 9:23 PM

#

It should work...

#

But i don't even see it attempting to reach a1111

vast elm Dec 7, 2023, 9:24 PM

#

halcyon quarry OK - the problem is that the "draw" is case sensitive 🙂

Oh lol

#

#

halcyon quarry Dec 7, 2023, 9:25 PM

#

did dynamic context trigger in your cmd window?

vast elm Dec 7, 2023, 9:25 PM

#

#

I even copy pasted "generate an image" from the config file

#

#

halcyon quarry Dec 7, 2023, 9:27 PM

#

Ok... is A1111 launched with --api flag?

vast elm Dec 7, 2023, 9:27 PM

#

I didn't change anything with a1111

#

Maybe now the update changed it back

#

halcyon quarry Dec 7, 2023, 9:28 PM

#

man what the heck...

vast elm Dec 7, 2023, 9:29 PM

#

Config file only gets read again once you restart the bot right?

halcyon quarry Dec 7, 2023, 9:30 PM

#

yes

vast elm Dec 7, 2023, 9:30 PM

#

I try to remove the / from the end of http://127.0.0.1:7860 if that makes a difference

#

Lol it did

halcyon quarry Dec 7, 2023, 9:31 PM

#

You're killin me man 😛

vast elm Dec 7, 2023, 9:31 PM

#

So at first it didn't work because outdated. And while trying to fix it, i just copy pasted the url from browser

halcyon quarry Dec 7, 2023, 9:31 PM

#

aha

vast elm Dec 7, 2023, 9:31 PM

#

So its the / that was now the problem

#

#

Lol

#

At least i know the api for sd works

halcyon quarry Dec 7, 2023, 9:33 PM

#

maybe you messed up the syntax for the generate image / dynamic context

#

ensure those are exactly as in the original config.py file

#

also ensure the character M1nty-SDXL.yaml is in text-generation-webui\characters

vast elm Dec 7, 2023, 9:34 PM

#

feelsweirdman

#

It just doesn't likes me

#

#

When you directly tag it like @chatbot it doesn't draws. While if you reply to it, and ask it to draw, it does

halcyon quarry Dec 7, 2023, 9:40 PM

#

It;s because there is no space " " between the @mention and the word "draw"

vast elm Dec 7, 2023, 9:41 PM

#

halcyon quarry Dec 7, 2023, 9:41 PM

#

@vast elm draw a duck
@vast elmdraw a duck

#

🥂

#

ok I see it didn't actually draw

#

Ah

#

the problem is the "on prefix only" setting

vast elm Dec 7, 2023, 9:45 PM

#

Turned on prefix only off

halcyon quarry Dec 7, 2023, 9:45 PM

#

space...

vast elm Dec 7, 2023, 9:48 PM

#

Seems to work

#

Did anything else change recently with the updates besides tts?

halcyon quarry Dec 7, 2023, 9:49 PM

#

support for the lrctl extension, if you use a lot of A111 loras

#

param variances (add randomness to sd settings)

#

nsfw image filter handling

viral lagoon Dec 8, 2023, 3:18 AM

#

License for the coqui model is actually trash though.

halcyon quarry Dec 8, 2023, 3:21 AM

#

I'm looking at it now and seems typical to me, no?

halcyon quarry Dec 8, 2023, 2:25 PM

#

There will be another update in a bit that requires a few key changes in config.py and in your characters.

On the plus side, this update should allow any extension specific parameters to be defined, per character.
This will probably also allow any tts extension to work, not just coqui_tts

halcyon quarry Dec 8, 2023, 8:35 PM

#

Actually, no changes to config.py

The only changes (aside from bot.py) is just the structure of the extension params in each character.

halcyon quarry Dec 8, 2023, 9:09 PM

#

Wow, I think I also fixed a pretty big bug

#

The message request queueing feature wasn't actually working... well?
Its working correctly now, along with the @ mentioning feature that was mysteriously not working as intended.

#

I may be able to get image requests to queue up now as well (I don't have time for it now though)

#

For the life of me, I can't figure out why it has "This interaction failed" after selecting a character from /characters command.
It's just cosmetic, though, all of the code executes correctly.

halcyon quarry Dec 8, 2023, 9:45 PM

#

quick comment - I never get good images from Juggernaut. But goddamn, it is an excellent model for img2img

scenic jay Dec 8, 2023, 11:09 PM

#

Having a bit of an issue. I built a new system and put on a fresh Ubuntu install. Installed textgen, your bot, and stable diffusion.

Everything works as it did before, but images are causing a problem. When I use the /image command, sometimes it will generate + post, other times it does not and the bot will indefinitely try to post it, eventually getting a "we are being rate limited" error after several minutes of waiting. Other times, the image posts successfully. When it fails, it seems that SD is indeed generating the image (I can see the progress bar load to 100%). Neither SD or the bot are throwing errors when this occurs though, so I'm a bit confused.

#

I know that's not much to go off of, figured i'd check to see if you have any ideas though!

halcyon quarry Dec 9, 2023, 1:15 AM

#

scenic jay Having a bit of an issue. I built a new system and put on a fresh Ubuntu install...

Fresh copy of the bot?

#

I've seen the rate limited message before, but it's been awhile since I've seen it in any unusual manner

#

I'll get that if I launch the bot and try changing characters too fast

#

(change char, reload bot, change char again)

scenic jay Dec 9, 2023, 1:18 AM

#

halcyon quarry Fresh copy of the bot?

Yup, everything is fresh, minus a few changes in quick settings to match what I have used in SD. Oddly enough, sometimes when the bot doesn't produce the image, it I go on SD and generate a random image, the bot will suddenly post the one I requested, making me wonder if the problem is SD

halcyon quarry Dec 9, 2023, 1:19 AM

#

hmm.

#

Is it the current stable version, or the RC for upcoming version?

#

or other, dev branch, etc

scenic jay Dec 9, 2023, 1:21 AM

#

SD? Should be current - I pulled it from main, can double check though

#

1.6.1 which seems to be normal, looks like RC is 1.7.0

halcyon quarry Dec 9, 2023, 1:23 AM

#

ok, same here 1.6.1

#

1.7.0 will be stable release soon

#

is launched with --api right?

scenic jay Dec 9, 2023, 1:25 AM

#

Yup --api added to command line args. What's weird is that it works intermittently

#

Might do 0-4 images, requiring the bot to reboot each time it fails as it won't do anything else. I suspected an issue with SD API but can't think of anything that would cause it

halcyon quarry Dec 9, 2023, 1:27 AM

#

It looks like you could have more verbose logging by setting an environment variable... https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Command-Line-Arguments-and-Settings

You could try changing it from INFO to DEBUG

#

might see something in the log

scenic jay Dec 9, 2023, 1:27 AM

#

Thanks, will take a look at that

halcyon quarry Dec 9, 2023, 1:28 AM

#

the image generation code in my bot doesn't have much print statements on progress... we could stick a few in there to see exactly where it is getting stuck

scenic jay Dec 9, 2023, 1:29 AM

#

Sounds good, I'll give the SD debug a try in a few minutes and see if anything comes up. The bot worked perfectly on my previous system, and nothing has changed, so I suspect something funky with SD.

halcyon quarry Dec 9, 2023, 1:32 AM

#

📎 bot.py

#

I stuck a few print statements, you should get a better idea of where it is getting stuck

#

we can narrow in on it

#

well, the bot is always changing and I tend to tighten up a little code here and there if something looks loose

#

I'm on windows, could be some code that doesn't jive as well in ubuntu shrugs

scenic jay Dec 9, 2023, 1:44 AM

#

Can't remember if I had it running on Ubuntu before, but i ran it on Arch on my previous system and no issues there.

Found this w/debug:
-Image requested via discord, gets stuck:

#

Then SD w/debug shows:

#

Then, I manually generated an image on SD (not on Discord) works fine and comes up as

halcyon quarry Dec 9, 2023, 1:46 AM

#

Well, seems normal. response code 200 means it processed the request

#

try the code I shared with the print statements

scenic jay Dec 9, 2023, 1:47 AM

#

oddly enough, the first image request from Discord never ended up in the Outputs folder in SD, but the one created within SD itself is there.

#

Will do

halcyon quarry Dec 9, 2023, 1:48 AM

#

hum

#

ok

#

Is there an sd_outputs folder in ad_discordbot?

#

maybe it doesn't have permissions to create the folder

#

(or something)

scenic jay Dec 9, 2023, 1:49 AM

#

There is a folder, it does have some files in it from when it occasionally worked before, but not the one I just attempted

halcyon quarry Dec 9, 2023, 1:52 AM

#

I put even a few more print statements

📎 bot.py

scenic jay Dec 9, 2023, 1:53 AM

#

ok will try that now, thanks

#

hmm getting this now:

File "/home/bot/text-generation-webui/bot.py", line 138, in <module>
tts_client = config.discord['tts_settings']['extension'] # tts client
~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
KeyError: 'tts_settings'

halcyon quarry Dec 9, 2023, 2:00 AM

#

hmm

scenic jay Dec 9, 2023, 2:01 AM

#

Did any of your recent updates change files outside of bot.py?

halcyon quarry Dec 9, 2023, 2:01 AM

#

on config.py you should have this:

    'tts_settings': {      # REQUIRES: 'pip install pynacl' in textgen-webui venv for bot to join a voice channel
        'extension': '',

#

yes 🙂 Big changes

#

Sorry, I could be a better coder if I took into consideration backward compatibility (user doesn't upgrade config.py)

#

I added a new chunk of settings into config.py, tts_setttings. Those need to be there

scenic jay Dec 9, 2023, 2:03 AM

#

Oh no worries, it's just that I may be a version behind but just jumped bot.py ahead likely for those print statements. Usually I just read the documentation and do a proper full upgrade

halcyon quarry Dec 9, 2023, 2:04 AM

#

I asked if it was a fresh copy of the bot and you said yes heh.

Anyway yes I highly recommend upgrading to current version

#

fixed a juicy bug today

scenic jay Dec 9, 2023, 2:06 AM

#

halcyon quarry I asked if it was a fresh copy of the bot and you said yes heh. Anyway yes I hi...

Yes you did - sorry my mistake, I forgot how much work you put in the past week or so hehe

halcyon quarry Dec 9, 2023, 2:11 AM

#

Yeah I haven't put this much effort in in like 2 months

scenic jay Dec 9, 2023, 2:29 AM

#

hmm so after updating it was getting caught at sending payload to a1111_txt2img(). I closed it out, did a full reinstall of discord in the textgen folder and it has worked 10x in a row so far, so i'll have to see. As a side note, even outside of this issue, those print statements are really handy to see settings being passed at a glance

#

scratch that. There it goes again lol. Got caught on "sending payload to a1111_txt2img()...".

#

This is what SD is showing for the stuck image:

Total progress: 100%|█████████████████████████████| 5/5 [00:00<00:00, 10.70it/s]
2023-12-08 21:31:09 DEBUG [PIL.PngImagePlugin] STREAM b'IHDR' 16 130, 13.97it/s]
2023-12-08 21:31:09 DEBUG [PIL.PngImagePlugin] STREAM b'tEXt' 41 210
2023-12-08 21:31:09 DEBUG [PIL.PngImagePlugin] STREAM b'IDAT' 263 65536

halcyon quarry Dec 9, 2023, 2:43 AM

#

📎 bot.py

#

more print statements

scenic jay Dec 9, 2023, 2:50 AM

#

Thank you, i'll put them on tomorrow, gotta get some sleep, early day tomorrow

#

& thanks for all your help, much appreciated!

halcyon quarry Dec 9, 2023, 3:15 AM

#

np

viral lagoon Dec 10, 2023, 1:10 AM

#

Might be interesting https://github.com/erew123/alltalk_tts

GitHub

GitHub - erew123/alltalk_tts: AllTalk is based on the Coqui TTS eng...

AllTalk is based on the Coqui TTS engine, similar to the Coqui_tts extension for Text generation webUI, however supports a variety of advanced features, such as a settings page, low VRAM support, D...

#

halcyon quarry Dec 10, 2023, 1:12 AM

#

Thanks, I'll check it out. I'm almost done with another new feature... only affects bot.py for once 🙂

#

slash command /speak where you just input whatever text you want spoken, and choose the voice from dropdown or attach a custom voice

halcyon quarry Dec 10, 2023, 1:22 AM

#

viral lagoon Might be interesting https://github.com/erew123/alltalk_tts

Did u see that latest version I pushed now supports loading any extension params per character, as well as possibly supporting other tts clients

viral lagoon Dec 10, 2023, 1:24 AM

#

No, I don't think I have the voice part yet (haven't updated in a few days kekw )

#

I only use this bot sporadically, I just like hanging around here.

halcyon quarry Dec 10, 2023, 2:12 AM

#

and I enjoy that!

halcyon quarry Dec 10, 2023, 1:20 PM

#

So I have this new /speak command working - the way it works is it uses your text input for ooba’s “continue” function, paired with max_new_tokens: 1. So it just generates what you wrote

#

I’m going to check out a number of other tts extensions to try and maximize compatibility

halcyon quarry Dec 10, 2023, 4:13 PM

#

halcyon quarry Dec 11, 2023, 2:17 AM

#

I figured out a way to overflow command options into additional menus when there will be more than 25 items in a dropdown list (discord limit), but its a comically overhanded workaround.

halcyon quarry Dec 11, 2023, 3:37 AM

#

It's cool that it works... just, not an elegant solution by any means... I basically have a series of command definitions nested in 'if' statements, so it registers the correct version depending on the total number of items

halcyon quarry Dec 11, 2023, 9:47 PM

#

Just pushed an update for that new "/speak" command and it is very comprehensive.

This update also makes silero_tts and elevenlabs_tts supported (not just for /speak)

The menus for /speak are dynamic and functional for those 3 extensions

#

If you guys speak different languages, don't worry all of the supported languages for each extension are selectable from the menu

halcyon quarry Dec 12, 2023, 3:22 PM

#

There was a bug (typo) that I just fixed. Sorry to anyone that updated and had errors.

The /speak command now replaces underscores with spaces and capitalizes for nicer looking options.

#

halcyon quarry Dec 12, 2023, 6:48 PM

#

** GOOD NEWS EVERYBODY**

The /imgmodel command now supports up to 100 models!

On startup, the command will be populated with the appropriate number of dropdown menus (up to 25 items in each)

limit_imgmodels() = 🚮

fiery flower Dec 12, 2023, 6:49 PM

#

halcyon quarry ** GOOD NEWS EVERYBODY** The /imgmodel command now supports up to 100 models! ...

yay xD

halcyon quarry Dec 12, 2023, 6:49 PM

#

If you have more than 100 models, I know a psychiatrist.
But I can also add support for more models

fiery flower Dec 12, 2023, 7:12 PM

#

halcyon quarry If you have more than 100 models, I know a psychiatrist. But I can also add supp...

Just counted them and I only got to 52... Although I have before that deleted about 20-30 since I ran out of storage space...

halcyon quarry Dec 12, 2023, 9:32 PM

#

Just pushed an update to fix Upload File not working in "/speak" command (only works for coqui_tts)

Includes a very unethical change, which is it now allows an mp3 as input, which it will convert to wav (coqui_tts requires .wav as input)

halcyon quarry Dec 13, 2023, 1:53 AM

#

Next on my to-do list... I'm revamping the "imgtags" feature so it encompasses all the extras I've been putting in config.py recently (they'll be coming OUT).
Meaning, there will be more parameters than positive_prompt, negative_prompt and trumps.
I'll make sure inclusion of the parameters is optional (won't need a bunch of empty keys for each trigger definition)
Characters and imgmodels will also be allowed to have their own imgtags

vestal python Dec 14, 2023, 8:19 PM

#

I have the Discord bot running, default character set within the /Characters folder with Ecne_Bot.yaml that seems to be running good now.

I'm going to take a look into tts option, and also have to wait on fixing up the connected SDXL server of mine when the new 3060 12gb comes in. It does work with draw, but that hardware was not made for SDXL at all...

So, starting to catch up.

halcyon quarry Dec 14, 2023, 8:20 PM

#

👍

#

You can define baseline TTS settings using ooba's settings template

#

And/Or, use the 'extensions:' dictionary in each character

#

(as shown in the example M1nty character)

vestal python Dec 14, 2023, 8:24 PM

#

I see in the example:
Must be defined using exact name of extension (in '../extensions/').
extensions:
coqui_tts

For the different tts and parameters that work with ooba. I can work with that, thank you.

halcyon quarry Dec 14, 2023, 8:25 PM

#

Indeed. If you plan on changing characters frequently and want to use generic settings, you could define those using ooba's settings template (as if you were going to use the webui directly rather than the bot)

#

Those would be overrided by any character specific settings

vestal python Dec 14, 2023, 8:31 PM

#

For now it'd definitely just be this one character. I'm trying to get something running with all the features my users want. Eventually figure out speaking to it and getting a speech reply back. Really personalize it for images, and a buddy is trying to build a lora based on the game character models for drawing with SDXL.

And then figure out how to put all my documents, guides, and wiki as either reference, or direct knowledge for it to tell users about the game/.

halcyon quarry Dec 14, 2023, 8:32 PM

#

very cool, I'd be interested in knowing how everything goes!

halcyon quarry Dec 14, 2023, 9:40 PM

#

Before I forked this bot, the code was alreay quite sloppy, and I cleaned things up as I went and noticed things.

For the past few days I've been working from the beginning and up to restructure the logic of settings handling, and toss out useless code.

Finally got a grip on it 🙂 Changes will only affect bot.py

#

Main benefit will be that values in activesettings.yaml will be completely optional. As it stands they are required

halcyon quarry Dec 14, 2023, 11:03 PM

#

The AllTalk tts extension (fork of coqui_tts) probably works I haven’t had a chance to make it officially supported

#

I expect to include code to take advantage of its features

vestal python Dec 14, 2023, 11:18 PM

#

Alltalk, I'll take a look right now.

halcyon quarry Dec 15, 2023, 12:23 AM

#

It has methods for handling stuff like this maybe even (stuff like this)

vestal python Dec 16, 2023, 1:07 AM

#

Have you seen this yet?

#

I'm going to reboot the system when I'm home.. I can do anything but reboot the system remotely unfortunately.

halcyon quarry Dec 16, 2023, 1:08 AM

#

I think that error means the bot has not been granted sufficient permissions

#

has it been behaving fine for the same requests?

#

I'm making very good progress on cleaning up the code... the whole logic of initializing the character, changing characters was a bit clunky. I've got it absolutely rock solid now

vestal python Dec 16, 2023, 1:13 AM

#

Yeah he was working good up until I left the house. I'll take a look. Idk maybe the not is quarantined or something. He is using a uncensored model, and not sure what my testers are asking in DM right now.

#

OK now he's working. After shelling me 8 responses at once. Weird connection issue maybe.

#

halcyon quarry Dec 16, 2023, 1:58 AM

#

I’ve got an error like that when using a bot command from a channel the bot is not in

#

You may have users with command permissions trying to send commands from other channels

#

if he's being hammered with requests it also may not be queuing correctly... I only recently fixed text queues, images just block (will fix that soon), some other things might interrupt request queues

#

Tightening up all the above is a high priority for me now 👍

halcyon quarry Dec 16, 2023, 8:05 PM

#

A recent ooba update seems to have broke the bot... should have it up to date by tonight

halcyon quarry Dec 17, 2023, 3:01 AM

#

I solved "this interaction failed" for /character command 💪

halcyon quarry Dec 17, 2023, 5:27 AM

#

technically I have everything in tip top shape, however some of the responses I'm getting are all whacked out, which I think has something to do with the new key value "chat_template_str"

halcyon quarry Dec 17, 2023, 5:43 AM

#

actually, maybe it was some other optional key I added to the list...

#

I pushed the update.

oobabooga keeps changing the number of positional arguments and return values from load_character() function, so this will error if not on a recent versionof textgen-webui

#

Bot now has a welcome message thingy
The /main command now toggles the channel on/off as a 'main channel'
It should be much more difficult to run into errors now, especially with character switching, etc.

halcyon quarry Dec 17, 2023, 2:07 PM

#

Something seems a little screwy with dynamic context feature (such as char swap for “draw”), fix coming soon

halcyon quarry Dec 17, 2023, 4:49 PM

#

Dynamic context is still screwy, however I resolved a few other issues I had created

halcyon quarry Dec 18, 2023, 12:35 AM

#

Turns out my dynamic context code is still fine, I didn’t muck that part up.

It’s just that new textgen version no longer replaces {{user}} and {{char}} with correct names for Chat mode, which explains the sudden strange output

I’ll be restoring this behavior as a config option

vestal python Dec 18, 2023, 3:32 PM

#

I got the new Mixtral7bx8 to work on my setup.. just barely, but crazy like 4 times the speed of my Vicuna 30b and 32k token trunc. I'm going to try adding it into the discord bot and see how it goes.

halcyon quarry Dec 18, 2023, 3:34 PM

#

vestal python I got the new Mixtral7bx8 to work on my setup.. just barely, but crazy like 4 ti...

Some of my recent commits were not as good as I thought before I pushed them.
I have a stable update coming very shortly

#

However, it won't be compatible with older versions of textgen-webui

vestal python Dec 18, 2023, 3:35 PM

#

Thank you, I did update my setup yesterday to the newest ooba and your discord bot, and was working good for my case.

halcyon quarry Dec 18, 2023, 3:35 PM

#

Ah okay

vestal python Dec 18, 2023, 3:35 PM

#

Luckily it's an easy, move my old textgen to textgen2 or 3, and just clone a new textgen

halcyon quarry Dec 18, 2023, 3:37 PM

#

I found some issues with the logic in dynamic_context. I also added a function to mimic what textgen-webui recently removed... it used to replace {{user}} with value of name1 and {{char}} with value of name2 in the character context

vestal python Dec 18, 2023, 3:38 PM

#

One thing:

When I run command:

python bot.py --loader exllamav2_hf --model turboderp_Mixtral-8x7B-instruct-exl2_3.5bpw --gpu-split 10,11.5

do I also need to put in the max-seq-len 32768

#

Are the commands based on the server.py ?

halcyon quarry Dec 18, 2023, 3:40 PM

#

For now, I think you may need to do that.
I need to take a sec and make sure the bot collects all the current information

#

I don't think it collects that parameter at the moment... I'll be checking and adding if needed

vestal python Dec 18, 2023, 3:43 PM

#

I really need to figure out if this new model is good for conversations and knowledge. So far it seems good, and that speed and triple paragraphs reply is just ridiculous good.

halcyon quarry Dec 18, 2023, 3:43 PM

#

I'm jelly, I only have a 4070ti 😛

vestal python Dec 18, 2023, 3:44 PM

#

Ohman, I couldn't afford 4000 series though lol

halcyon quarry Dec 18, 2023, 3:47 PM

#

Only got 12gb vram to play with though

#

I just pushed a nice update

#

er.. forgot the new config.py (new setting)

vestal python Dec 18, 2023, 3:49 PM

#

I see the bot.py 204 changes on git pull

halcyon quarry Dec 18, 2023, 3:49 PM

#

just pushed updated config.py

#

just copy/paste the new replace_char_names block into your existing config.py

#

on my to-do list, is to integrate all the default config settings into bot.py so that config updates don't break anything

vestal python Dec 18, 2023, 4:00 PM

#

Upgraded, set configs, and so this is my command:

python bot.py --api --listen --loader exllamav2_hf --model turboderp_Mixtral-8x7B-instruct-exl2_3.5bpw --max_seq_len 32768 --cache_8bit --num_experts_per_token 2 --gpu-split 10,11.5

#

Whoops something missing on my end I think:
File "<unknown>", line 1
[,"Dundell2","M1nty"]
^
SyntaxError: invalid syntax

halcyon quarry Dec 18, 2023, 4:03 PM

#

I added a bit of code that will automatically add the user's name and the characters name to State keys: "stopping_string" and "custom_stopping_string"

#

So, if your state settings include something like "client.user.display_name" or "i.author.user_name" those should now be removed

#

        name1_value = user_input['state']['name1']
        name2_value = user_input['state']['name2']
        user_input['state']['custom_stopping_strings'] += f',"{name1_value}","{name2_value}"'
        user_input['state']['stopping_strings'] += f',"{name1_value}","{name2_value}"'

This:
custom_stopping_strings: '"### Assistant","### Human","</END>"'
Will now become this automatically depending on who's talking to who:
custom_stopping_strings: '"### Assistant","### Human","</END>","altoiddealer","M1nty"'

#

Ohhh... I see my folly here

#

If there are no existing stopping strings this will error

vestal python Dec 18, 2023, 4:15 PM

#

Sorry had to grab the ol filezilla + notepad++
Line 952 of bot.py

halcyon quarry Dec 18, 2023, 4:16 PM

#

I just fixed it now

#

I will never claim to be a great developer 🤗

#

pull the new bot.py and that error should resolve

vestal python Dec 18, 2023, 4:18 PM

#

Pulled and testing now thank you

#

#

He seems to be working now, and now I can have my guys test him out and see how this new model works. Thanks again

halcyon quarry Dec 18, 2023, 4:27 PM

#

Also on my to-do list is the debug spam regarding extensions

vestal python Dec 18, 2023, 5:00 PM

#

Ah, They caught an error 3 times overreloading now.

ERROR [2023-12-18 11:58:49]: Ignoring exception in on_message (Line: 515 in on_error, client.py )
Traceback (most recent call last):

File "/home/dundell/miniconda3/envs/textgen/lib/python3.11/site-packages/discord/abc.py", line 1561, in send
data = await state.http.send_message(channel.id, params=params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/dundell/miniconda3/envs/textgen/lib/python3.11/site-packages/discord/http.py", line 745, in request
raise HTTPException(response, data)
discord.errors.HTTPException: 400 Bad Request (error code: 50006): Cannot send an empty message

Not sure what the issue is currently. When I try locally on ooba, I don't see the error.

halcyon quarry Dec 18, 2023, 5:03 PM

#

this is for a regular text request or an image request?

vestal python Dec 18, 2023, 5:04 PM

#

Normal text. I won't be getting back to sdxl image making until Friday

#

Just simple requests.

#

Another was just what gods are in Gaelic, wich it answered, and then just who is Morrgihan

halcyon quarry Dec 18, 2023, 5:05 PM

#

I added a try / except block in on_message, so maybe a more useful error message will show up

📎 bot.py

vestal python Dec 18, 2023, 5:06 PM

#

I added it in one sec

halcyon quarry Dec 18, 2023, 5:07 PM

#

woops, its going to print a bunch of other stuff too... was looking into the settings stuff

vestal python Dec 18, 2023, 5:07 PM

#

lol I saw

#

That did it thanks

#

current_hidden_states = F.silu(gate) * up
~~~~~~~~~~~^~
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 12.00 MiB. GPU 1 has a total capacty of 11.76 GiB of which 14.69 MiB is free. Including non-PyTorch memory, this process has 11.60 GiB memory in use. Of the allocated memory 11.40 GiB is allocated by PyTorch, and 71.52 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

halcyon quarry Dec 18, 2023, 5:13 PM

#

aha

vestal python Dec 18, 2023, 5:14 PM

#

I knew this was pushing it... I might have to throw in this rtx 2080 super 8gb my brother left me just to run this then.

halcyon quarry Dec 18, 2023, 5:14 PM

#

Could reduce the truncation size / prompt size

#

more context = more vram

vestal python Dec 18, 2023, 5:22 PM

#

Reduced it down to 22k

So command is:
python bot.py --api --listen --loader exllamav2_hf --model turboderp_Mixtral-8x7B-instruct-exl2_3.5bpw --max_seq_len 22768 --cache_8bit --num_experts_per_token 2 --gpu-split 10,11.5

I'll work on implementing the additional 8GB vram for other extensions and such as well.

#

Thank you for that. I would have spent half the day trying to figure that out.

halcyon quarry Dec 18, 2023, 5:23 PM

#

np! Surprised I didn't have that exception block

halcyon quarry Dec 19, 2023, 2:31 AM

#

Just pushed an update that confirms support for alltalk_tts

#

really great improvement on coqui

halcyon quarry Dec 19, 2023, 2:57 PM

#

Just pushed a simple bot.py update that resolves the following:

-I was unnecessarily reprocessing settings at the beginning of most interactions. I think this may have even been causing some interactions to timeout (?)
In any case - Fixed.

-I was updating extension settings on every message response, however they don't actually need updating at this stage, they are already updating at the right times.
Like the other settings thing, this was excessively heavy handed and caused debug spam on every message.
Fixed.

The only drawback to the above changes, is that manual changes in activesettings.yaml will not be automatically applied.
This isn't the intended usage, though. Make updates to the character file, dict files, etc, then use settings commands (/character, /imgmodel, etc) to sync in the new settings.

-History settings were not being honored in Dynamic Context.
Fixed.

timber coral Dec 20, 2023, 1:51 AM

#

#

Any idea what could be going wrong here?

halcyon quarry Dec 20, 2023, 1:52 AM

#

Yep, I actually just fixed that now.,.. please try this version

#

📎 bot.py

#

I personally always have A1111 running so I never encountered the error 😛 Genuinely surprised no one else mentioned this until tonight

timber coral Dec 20, 2023, 1:54 AM

#

You'll have to teach me about that sometime ;p got it working!

#

I 'broke it' but that's my fault

halcyon quarry Dec 20, 2023, 1:55 AM

#

I'll see if I can make that not "break" it, too...

timber coral Dec 20, 2023, 1:55 AM

#

Although;

#

#

I said 'sorry fella' and that happened haha

#

from what ive seen so far though great job man, cool stuff

halcyon quarry Dec 20, 2023, 1:59 AM

#

If you won't be generating any images, disable the stuff in config.py that will trigger an image response

timber coral Dec 20, 2023, 1:50 PM

#

@halcyon quarry any idea why my bot isnt very talkative?

#

it did answer on the 3rd message but then the 4th one was "As an" again

vestal python Dec 20, 2023, 2:05 PM

#

Might be what I was seeing a few times. Ckeck token length, and vram usage

halcyon quarry Dec 20, 2023, 2:06 PM

#

You may have max_new_tokens set too high. My example character suggests setting it to 1000 but that may be too high

#

if that's the case, try lowering it to something around 700

vestal python Dec 20, 2023, 2:12 PM

#

Does it also happen when you're just using the oobaboogas webui?

halcyon quarry Dec 20, 2023, 2:13 PM

#

Yes, if the context + your prompt + max_new_tokens is greater than than the context limit, your context will get screwed

#

There is an auto_max_new_tokens parameter that I haven't played with but from the sound of it, this should be a set-it-and-forget it solution to dynamically calculate and apply it for every prompt

#

I've updated the suggested settings. Also noticed that I had a duplicate setting param in my script which wasn't hurting anything but now it looks prettier 🙂

halcyon quarry Dec 20, 2023, 2:23 PM

#

timber coral <@670018869418786816> any idea why my bot isnt very talkative?

Tell me if lowering max_new_tokens resolves it 🙂

#

next big ticket item on my to-do list, is get image requests to queue up with each other / text requests

#

It's the holiday season, and a new season started in this game I play (Diablo 2 mod called Median XL) so could take a lil' bit

vestal python Dec 20, 2023, 3:46 PM

#

Probably no chance D2RA would work with that I assume :/

halcyon quarry Dec 20, 2023, 4:12 PM

#

If you are a diablo 2 fan, then you'll be blown away by Median XL

#

I came across it 3 years ago and, at the time, I was shocked to know a team of people were actively updating it for like 15 years or something.
Since then that dedicated team has pushed huge updates every 2-4 months with ladder resets. It's mindblowing

vestal python Dec 22, 2023, 6:43 PM

#

Can fooocus be used with the discord API?

#

I have been testing out a ryzen 5 2600 + RTX 2080 super 8GB for my SDXL server, and it's going over much better than I thought. So I'm trying to make it work how I like it.

And then upgrade my current setu to x3 RTX 3060 12gb's for my chat bot.

#

I think I can just work in the a1111, and get it how I like though. I was just wondering.

halcyon quarry Dec 22, 2023, 8:41 PM

#

I'll have to look into it... at some point

#

It's not a matter of if it can be used with Discord, it's a matter of whether I can make the bot script compatible

vestal python Dec 22, 2023, 8:48 PM

#

Yeah it's just some settings I'm not getting to work right with 8GB cards with A1111. 3 8GB cards tested, and it either runs out of vram, or setting --medvram gives garbage images and then crashes on the second image..

I might just not be understanding the settings for sdxl in A1111. Fooocus just works right out of the box for all 3 cards GTX 1070 8gb, 1070 8gb, and this RTX 2080Super 8gb

#

I'm not even sure what Fooocus has for api though

halcyon quarry Dec 22, 2023, 8:48 PM

#

--medvram shouldn't affect the image quality though? I use medvram

#

pretty sure it just increases generation time

#

maybe try --lowvram really

vestal python Dec 22, 2023, 8:50 PM

#

I'm going to start over, try again fresh git clone of a1111

halcyon quarry Dec 22, 2023, 8:54 PM

#

I'm in process of debugging an error I encountered... error after using /speak command, then using "draw..."

halcyon quarry Dec 22, 2023, 9:43 PM

#

ugh. Couldn't find an immediate solution... maybe tomorrow.

vestal python Dec 22, 2023, 9:45 PM

#

I think I got it to work now with A1111. I was missing vae fix, and some additional settings, and missing --medvram-sdxl command, and the google-perftools

Now 1024x1024 images using juggernaut sdxl model is running 6.5GB vram

#

and working good.,

#

I'll test out the discord bot with it and see how it goes as well.

viral lagoon Dec 22, 2023, 9:50 PM

#

"beatuiful"

#

LUL

#

(that's your typo)

#

Nice image though

vestal python Dec 22, 2023, 10:03 PM

#

Yes I'm horrendous for typos. Might be my keyboard usually a space or a t missing.
Trying something right now, let you know how it goes

halcyon quarry Dec 22, 2023, 10:18 PM

#

Nice output 👍🏻

halcyon quarry Dec 22, 2023, 11:58 PM

#

I just pushed an update to resolve some issues

#

I know that's a bit vague - it's a very significant update

vestal python Dec 23, 2023, 1:20 AM

#

Thanks yeah we're now all set up on the fp16 Juggernaut v7 with vae fix model. Seems to work well. I just need to set maybe to 60 steps, but everyone is having fun taking turns testing the discord bots image making.

vestal python Dec 23, 2023, 3:01 AM

#

Going to mess around a bit with imgtags I think it is. Create a quick 30 step default, and a quality 70 step options

vestal python Dec 23, 2023, 3:48 AM

#

I git puled, and promting theAI to draw a picture for me works again. I also figuredout how to add a default 30 step, and a detailed 70 steps wiht the same model so that's all set now and working.

halcyon quarry Dec 23, 2023, 4:52 AM

#

Nice

vestal python Dec 23, 2023, 4:53 AM

#

Hmm, so after switching imgmodels to my detailed 70 step, and trying to do a second image is causing some issue with error:

Error updating channel topic: 403 Forbidden (error code: 50013): Missing Permissions
Updated imgmodel settings to: Detailed (70 Steps)
An error occurred: 'controlnet'

If I load it again, I can run a new image, but then I need to load it everytime

halcyon quarry Dec 23, 2023, 4:54 AM

#

I ran into another issue using controlnet tonight, separate issue but probably same cause.

#

I’ll be fixing something tomorrow.

#

Sorry for the issue 😑

#

Hmm… okay, that’s actually different I think

#

Does your bot have permission to update the channel topic?

#

announce_imgmodels feature

vestal python Dec 23, 2023, 5:14 AM

#

I've readded the bot with admin rights to everything to test. So far nothing new.

The only thing I've noticed is either running imgmodel, or doing a /reset allows for a new picture.

I might remove the second imgmodel option and see if just running defaults doesn't stop the process.

halcyon quarry Dec 23, 2023, 5:44 AM

#

So what is the process you follow that causes the error?

#

Is it when using /image command?

gloomy coral Dec 23, 2023, 6:24 AM

#

thanks for the bot, i finally got it working with a1111, do you happen to know how to fix " An error occurred: 'controlnet' " . It will generate the first image fine but any after that fail unless i restart bot.py

halcyon quarry Dec 23, 2023, 6:25 AM

#

I've just made a few updates... testing them now...

gloomy coral Dec 23, 2023, 6:25 AM

#

oh awesome!

halcyon quarry Dec 23, 2023, 6:25 AM

#

gloomy coral thanks for the bot, i finally got it working with a1111, do you happen to know h...

Is this using the /image command, or "draw __"

gloomy coral Dec 23, 2023, 6:25 AM

#

the /image command

halcyon quarry Dec 23, 2023, 6:27 AM

#

I think I resolved this... pushing it now

gloomy coral Dec 23, 2023, 6:27 AM

#

you're awesome!

halcyon quarry Dec 23, 2023, 6:28 AM

#

Go ahead and tell me if you run into the same issue 🙂

#

I recently made some sweeping changes to the settings handling, and there's been a few hiccups since then

#

I personally haven't reproduced this exact issue... so I'm not sure if my changes fixed it or not

#

But I definitely solved another issue, with settings carrying over from one image request to the next

gloomy coral Dec 23, 2023, 6:35 AM

#

it works!

#

was able to generate 4 images in a row with no issues

halcyon quarry Dec 23, 2023, 6:35 AM

#

🍾

gloomy coral Dec 23, 2023, 6:36 AM

#

hell yeah thanks a ton!

halcyon quarry Dec 23, 2023, 6:36 AM

#

I was at a party wowing some peeps with the controlnet feature, then realized the controlnet was persisting for all future images

#

now here I am 1:37AM, can't sleep leaving this sort of bug unsquashed lol

gloomy coral Dec 23, 2023, 6:37 AM

#

lmao

halcyon quarry Dec 23, 2023, 6:37 AM

#

enjoy, let me know if you run into any other issues

gloomy coral Dec 23, 2023, 6:38 AM

#

will do! 🫡

halcyon quarry Dec 23, 2023, 6:44 AM

#

vestal python I've readded the bot with admin rights to everything to test. So far nothing new...

This issue should be resolved now, sorry for the error 🙂

vestal python Dec 23, 2023, 12:47 PM

#

Thanks! Sorry I had to put my son to bed and.. Well as usual he finds a way to tired me out and pass out right next to him.

halcyon quarry Dec 23, 2023, 2:35 PM

#

next update is going to clean up the handling of controlnet and reactor extensions...

hard cobalt Dec 23, 2023, 4:21 PM

#

halcyon quarry I solved "this interaction failed" for /character command 💪

Nice! That always bugged me

halcyon quarry Dec 23, 2023, 4:30 PM

#

hard cobalt Nice! That always bugged me

It was keeping the interaction open and eventually timing out - until I added a ‘interaction.response.send_message()’

#

I noticed my /imgmodel command wasn’t having the Interaction Failed issue, which is because chatGPT correctly included that syntax for the message send at the end

hard cobalt Dec 23, 2023, 4:43 PM

#

I'll steal that code for sure

viral lagoon Dec 23, 2023, 8:51 PM

#

Idea: Make the bot accumulate several messages and then make it reply (or not) at random times. Basically just following the chat.
I guess this needs a very good multi-character RP model though.

halcyon quarry Dec 23, 2023, 9:54 PM

#

There is a “chance to reply” behavior setting… I have to look into whether or not the unreplied messages go into history (probably not)

#

I’m going to add that. Good idea

#

It will probably work better if the example dialogue includes multiple user messages, and the character responding to previous messages

viral lagoon Dec 23, 2023, 10:30 PM

#

Yeah, it would also probably need a very specific context for it to know to be part of a chatbox

#

Though I'm guessing many people already do that for their Discord AI characters.

halcyon quarry Dec 24, 2023, 1:54 AM

#

Pushed an update to add default controlnet / reactor settings in bot.py (they won't do anything if the extensions are disabled in config.py and also activated in some way (such as in /image command).
These extension args are not required in activesettings.yaml anymore.

Reactor support has been fixed, as I noticed it stopped working after a recent update.

marsh harness Dec 24, 2023, 9:02 AM

#

I cannot for the life of me get the bot to join vc.

#

The voice channel is set correctly. The bot has perms to join vc. There's no join command to make it join. I'm in the voice channel when I start the bot.

marsh harness Dec 24, 2023, 9:37 AM

#

Must be something I'm messing up, because I tried coqui and that didn't work either.

marsh harness Dec 24, 2023, 9:56 AM

#

Well... I found the issue.

#

Related to a previous one I was having, too.

#

It's still just that I can't use any Kanji/Japanese in my yaml file or it'll cause decode errors.

#

Also, having an error when switching characters.

#

So after fucking with it for hours my only issue was that I can't put Japanese in my yaml file. As soon as I removed it, the bot joined and the tts is working.

#

Kinda sucks that I can't use it, because I had to remove information in Japanese about the character, but.

#

Dunno why character swapping isn't working for me at all now.

marsh harness Dec 24, 2023, 10:07 AM

#

marsh harness Dunno why character swapping isn't working for me at all now.

It seems like it may be because my example character is the same as my current character, if I have to guess based on reading the error.

hard cobalt Dec 24, 2023, 10:08 AM

#

are you changing characters too often? Discord has a 10 minute cooldown window

marsh harness Dec 24, 2023, 10:09 AM

#

Nope. I just decided to test the command to see if it'd even work. I don't even see any characters I can change to.

#

#

#

There's no drop down/selection or anything like that after.

hard cobalt Dec 24, 2023, 10:10 AM

#

wouldnt be surprised if there was some problems with character encoding, even though it should all be utf8

#

that would be the first place I would look anyway

#

"your welcome"
Excellent large language model

marsh harness Dec 24, 2023, 10:11 AM

#

Yeah it usually uses proper grammar so that threw me off honestly

marsh harness Dec 24, 2023, 10:12 AM

#

hard cobalt wouldnt be surprised if there was some problems with character encoding, even th...

It was working fine in your version regarding having Kanji in the yaml file without any errors after it was changed to utf8, but even with the encoding set to utf8 in this bot there's still an error for whatever reason

hard cobalt Dec 24, 2023, 10:13 AM

#

altoid did fix some bug I had with the character selection, maybe some other bug got introduced then

marsh harness Dec 24, 2023, 10:13 AM

#

I removed the Kanji entirely to see if that's what was causing the vc joining to not work, and it was. No clue what's bricking the characters not working since I've removed it now.

hard cobalt Dec 24, 2023, 10:14 AM

#

maybe try stopping the script, letting the bot leave, then restart the script

marsh harness Dec 24, 2023, 10:15 AM

#

By leave, do you mean go offline? So basically, restart it but just make sure the bot goes offline first? Or do you mean kicking the bot and then re inviting it to the server entirely

#

As usual my disclaimer if I say something extremely dumb is that I probably really need sleep by now since it's 5 am, but I really just wanted to get this working

hard cobalt Dec 24, 2023, 10:21 AM

#

nah just let it go offline

#

not sure if it will do anything but wont hurt

marsh harness Dec 24, 2023, 10:29 AM

#

Waited for it to go offline and then started it again.

#

Nope. As soon as I /character and then press enter since there are no other options, instant error.

#

What the hell lmao.

#

I coded it into the character's yaml file that its knows its creators name. It used to work with this bot previously, but now if the bot tries to type Hakumei it just breaks.

#

#

I didn't even type the name in Japanese in the yaml file, it's in plain English/romaji

hard cobalt Dec 24, 2023, 11:14 AM

#

I did have the conversation partner names in discord as a stopping string to brute force it into not hallucinating conversations, but that was "{name} :" not "{name}"

#

weird

#

it should be able to say the name of the person it's talking to either way

#

now that I think of it I guess it should have been "\n{name}: " just to be on the safe side

marsh harness Dec 24, 2023, 11:46 AM

#

Well my brain isn't functioning anymore, time to knock out.

halcyon quarry Dec 24, 2023, 12:30 PM

#

I have to be a dad for a bit, but we’ll figure this out

#

The issue is likely because I retain all settings including character info to the activesettinga.yaml file. Something is probably getting borked when it saves that and reads it back

halcyon quarry Dec 24, 2023, 12:37 PM

#

marsh harness I didn't even type the name in Japanese in the yaml file, it's in plain English/...

I did recently hardcode some stopping strings gowron2

#

Which is the cause of this behavior here

#

I’ll figure out an elegant way to make that optional. And see if there’s a setting I can tweak with the yaml formatting

#

The thing with the character no showing in the menu is bizarre, I repeatedly change to the same character constantly.

#

The 10 minute cooldown doesn’t apply when changing to same character

halcyon quarry Dec 24, 2023, 1:51 PM

#

After scratching my head for a bit on the character menu error, I believe the character ‘name’ key must be identical for both, and discord doesn’t want to assign same ‘value’. I can resolve this by using the filename as the ‘value’ - it will probably allow duplicate names then

halcyon quarry Dec 24, 2023, 2:22 PM

#

er no... the labels and values are derived from the filenames in /characters/ firectory, so it should not be possible for it to assign the same value to multiple menu options

#

...unless you have both a .json and a .yaml with the same filename...

#

I seem to have resolved the kanji thing

halcyon quarry Dec 24, 2023, 3:52 PM

#

UPDATE

1. I recently added hardcoded stopping strings for the user's name and the character's name.

These are now optional, just include "name1" and/or "name2" in your stopping_strings.  See example character "M1nty".

2. The bot can now load .json and .yaml character files.  They can even have the same filename.

3. Kanji characters (and presumably other special characters) are now supported in character files.

marsh harness Dec 24, 2023, 4:00 PM

#

halcyon quarry I did recently hardcode some stopping strings <:gowron2:1105872598514028564>

I just woke up; when Merc mentioned the stopping strings I figured that's what was happening. I only glanced over the new {char} and /n (etc) stuff that current ooba and your bot are using.

marsh harness Dec 24, 2023, 4:01 PM

#

halcyon quarry **UPDATE** ``` 1. I recently added hardcoded stopping strings for the user's na...

I'll try everything immediately, when I actually get up. Gonna try & get more sleep.

halcyon quarry Dec 24, 2023, 4:10 PM

#

👍

marsh harness Dec 24, 2023, 4:11 PM

#

halcyon quarry ...unless you have both a .json and a .yaml with the same filename...

I deleted all my json files and only use yaml now.

halcyon quarry Dec 24, 2023, 4:12 PM

#

That was a very unusual error so it wouldn’t surprise me if it isn’t resolved with the update

#

A thing to note: discord command options don’t update right away… they will refresh right away if you close and reopen your app (desktop app, phone app, etc)

marsh harness Dec 24, 2023, 4:14 PM

#

Noted, will ctrl + r.

halcyon quarry Dec 24, 2023, 4:32 PM

#

just pushed a small update...

-Removed a dev print() statement from bot.py
-Updated the suggested usage for the new stopping strings (now suggests "\nname1:" and "\nname2:")

#

it will replace name1 and name2 regardless of anything before/after it

halcyon quarry Dec 24, 2023, 8:09 PM

#

Another update...

I noticed that it was still only accepting the old format convention for textgen custom settings (settings.json).

Now it will correctly load settings.yaml (or settings.json)

marsh harness Dec 24, 2023, 8:51 PM

#

Updated everything, getting these two errors now.

#

#

Just when loading the script/after choosing the model.

halcyon quarry Dec 24, 2023, 9:17 PM

#

I think something is not formatted correctly in the character file

marsh harness Dec 24, 2023, 9:17 PM

#

Hmm...

#

I'll look it over.

halcyon quarry Dec 24, 2023, 9:18 PM

#

Try wrapping the context with quotes, or check for anything else that may be missing a closing quote

#

That - trigger line looks suspicious… note that my bot does not yet have custom character-specific imgtags

#

(On the todo list)

marsh harness Dec 24, 2023, 9:21 PM

#

I tried deleting the name and left it as just ' ' and it still returned the same error.

#

I copied everything else exactly as you have it in M1nty.yaml.

#

Scratch that.

#

Just noticed something.

halcyon quarry Dec 24, 2023, 10:02 PM

#

Let me know 👍

marsh harness Dec 24, 2023, 10:09 PM

#

I try everything I can think of to try before I say anything, so it might take a bit.

#

At this point I'm just gonna give you the yaml file and see if you can fix whatever I've done

halcyon quarry Dec 24, 2023, 10:10 PM

#

Sounds good

marsh harness Dec 24, 2023, 10:11 PM

#

📎 Reiko.yaml

#

I tried deleting everything after the context, (character description etc) and that didn't change anything; I still got the same errors. I removed the Reiko trigger but the error's still the same.

(I know certain things are repeated a lot, but it helped the bot retain it when asked as opposed to only having one instance of it in the file when testing previously)

halcyon quarry Dec 24, 2023, 10:16 PM

#

I’ll take a look in a litle bit

marsh harness Dec 24, 2023, 10:16 PM

#

I feel like I just gave my baby away

halcyon quarry Dec 24, 2023, 10:47 PM

#

marsh harness I feel like I just gave my baby away

The stopping strings need to be nested under a state: key

#

There’s also a lot of keys after the “context” key that I don’t know if textgenwebui even recognizes (character_description, creator, etc).

I recommend you try the character in textgenwebui interface with —verbose and you will see all the context/history/etc shown to the LLM

#

The example character for textgen only has name, greeting and context

#

One more thing - I think those keys after “presets” (triggers etc) are indented further than they are supposed to be

marsh harness Dec 24, 2023, 11:05 PM

#

I...

#

How did I not notice this

marsh harness Dec 24, 2023, 11:05 PM

#

halcyon quarry The stopping strings need to be nested under a state: key

No idea what this means, dunno how to do it.

marsh harness Dec 24, 2023, 11:14 PM

#

marsh harness How did I not notice this

I fixed it.

#

I now I need to fix my other character.

#

I somehow only just now noticed that things still weren't the same as in m1nty.

#

If it still returns the error I've been getting when changing characters after I update this one, I'll send the error in an image.

marsh harness Dec 24, 2023, 11:41 PM

#

Alright, everything works now.

#

I just wasn't using the right file format & needed to fix my other character.

#

Thanks ❤️

#

Now we just need whisper stt support 😶

halcyon quarry Dec 25, 2023, 12:05 AM

#

I’ll look into it 🤗

marsh harness Dec 25, 2023, 12:19 AM

#

There's already an extension for it in textgenwebui, so I'd imagine it's doable. I just personally wouldn't know where to begin. Also, since it records mic audio in textgenwebui when you enable it, it might be tricky to have it recognize speech live/without a recording. I have no idea.

vestal python Dec 25, 2023, 12:50 AM

#

Yeah I've been working with mantella extension for skyrim with AI NPC voicings, but that uses that xvasync, which I think does everything to text on the users end.

#

Idk how that works with actual tts stt extensions

halcyon quarry Dec 25, 2023, 1:13 AM

#

For anyone using Alltalk_tts, if you get the deepspeed option installed, you can enable that via textgenwebui inerface (alltalk_tts github has screenshots showing how to get to the settings page).

If deepspeed is enabled via that settings page, it will always be activated including when in use with this bot

halcyon quarry Dec 27, 2023, 2:42 PM

#

I noticed over the holiday weekend that the "auto-select imgmodels" feature, in mode "random" started changing to the same model over and over again.
Its supposed to remove the current imdmodel from the list so it doesn't select the same one again - however, it was modifying the main instance of the all_imgmodels dictionary rather than a copy of it.
Will push a fix shortly (need to test a bit more).

vestal python Dec 28, 2023, 3:32 AM

#

testing out alltalk + deepspeed + voice channel settings tonight

marsh harness Dec 28, 2023, 3:56 AM

#

Deepspeed made the tts generation way faster. Definitely worth it.

#

Someone commented on it in this server previously, but it's strange that coqui/alltalk will randomly switch to a british accent for whatever reason after speaking for a bit.

vestal python Dec 28, 2023, 4:11 AM

#

I see it uses like 2GB vram?

#

Ah, yeah so everything's loaded correctly, but that extra 2GB vram is messing me up. I'll have to wait on it and install the new 3rd RTX 3060 12GB

#

I figured this would be the case for me x.x I'll just have to wait a week for the cable extension to come in for it.

halcyon quarry Dec 28, 2023, 9:45 PM

#

Is there anyone actually using the /llmcontext, /llmstate and /behavior commands?

vestal python Dec 28, 2023, 10:35 PM

#

I have not used those. Still figuring out/image with CNET_input to see if I can have a denoising level of like 0.4 and come out with some img2img. So far nothing similar beyond background lines.

halcyon quarry Dec 28, 2023, 10:51 PM

#

vestal python I have not used those. Still figuring out/image with CNET_input to see if I can ...

You’re not getting good results with cnet?

marsh harness Dec 28, 2023, 10:52 PM

#

halcyon quarry Is there anyone actually using the /llmcontext, /llmstate and /behavior commands...

I'll try some of them out and see how they perform.

halcyon quarry Dec 28, 2023, 10:52 PM

#

Well I'm planning to axe them 😛

marsh harness Dec 28, 2023, 10:52 PM

#

Ah, okay. Never mind then.

halcyon quarry Dec 28, 2023, 10:53 PM

#

unless there's people actually using them

marsh harness Dec 28, 2023, 10:53 PM

#

I just looked at them, and having different forms for specific types of conversation/tasks actually looks pretty cool.

#

It seems like a nice feature to have, but if it's just extra work to maintain with not enough people using it, then scrapping it is fine.

halcyon quarry Dec 28, 2023, 10:55 PM

#

When I set up those commands, there was not char specific behaviors and states - and also no dynamic context feature.
Now, you can more or less switch things up by just using character / dyanimc context

marsh harness Dec 28, 2023, 10:56 PM

#

Mm, makes sense. There doesn't seem to be a need for the extra commands with that in mind, then.

#

On a side note- have you ever tried the openchat model? I'm not sure if it's just how the model's responses are structed formatting wise conversationally or what, but. Its responses are... weird.

halcyon quarry Dec 28, 2023, 11:09 PM

#

Haven’t tried openchat

#

I was always a fan of the airoboros models until mistral came around

marsh harness Dec 28, 2023, 11:14 PM

#

Hm. I use openorca zephyr now. I've used others, and as far as generally accurate and current knowledge while also being uncensored goes- that seems to be the best for my use case.

#

Also just generally good at just holding a normal conversation.

#

Supposedly the openchat model is better than mistral & gpt3.5 turbo by far.

#

Based on all the videos I've watched etc.

#

But its text generation is... odd.

halcyon quarry Dec 28, 2023, 11:24 PM

#

I’ll have to check it out!

#

If there’s anyone who uses the /llmcontext, /llmstate and /behavior commands let me know… otherwise they’re going the way of the dodo

halcyon quarry Dec 29, 2023, 4:13 AM

#

The deed is done - those commands are axed

#

Pushed an update which removes 3 of the .yaml dict files, and adds one "dict_base_settings.yaml".

Here, you can configure a base layer of behavior, state, context, imgmodel* and imgtag* information (* coming soon).

This shouldn't affect anyone's current settings... this just makes it easier to set some general settings so you don't have to include a bunch of redundant settings in each character.

#

For instance if you want all characters to use alltalk_tts extension, you could put that info in the llmcontext > extensions portion of the base settings. It would then apply to all characters (unless they specify otherwise)

halcyon quarry Dec 29, 2023, 10:57 PM

#

Update

New version of bot.py makes dict_base_settings.yaml work for everything including imgmodels and imgtags

#

This is a nice improvement for managing imgmodel settings. Maybe not as useful for imgtags, but that will change once I replace them outright with drumroll ... "tags"

#

Personally I’m very excited about this update 🤓

charred lake Dec 30, 2023, 1:48 PM

#

the bot seems to come with no requirements.txt, is there a way to install those?

halcyon quarry Dec 30, 2023, 1:49 PM

#

The only thing needed is pip install discord

#

To join voice channels, it has one more requirement… install instructions are on my github page

charred lake Dec 30, 2023, 1:50 PM

#

for me its missing basically all the requirements, like pyyaml, pydub, gradio, ...

halcyon quarry Dec 30, 2023, 1:52 PM

#

Those all come with textgenwebui

#

Which is required to run it

#

Follow the install instructions and it will work 👍

charred lake Dec 30, 2023, 1:53 PM

#

does ooba install the requirements in a venv? else im a little lost on why its missing those..

halcyon quarry Dec 30, 2023, 1:54 PM

#

Make sure bot.py is in the textgenwebui directory

#

Then make sure you use the cmd file from textgenwebui which activates the venv

charred lake Dec 30, 2023, 1:55 PM

#

ohhh i missed the part about using the provided cmd file

#

apologies and thanks!

halcyon quarry Dec 30, 2023, 1:56 PM

#

Enjoy! Let me know if you have any other trouble or feedback

#

Be sure to look at the part How to update

charred lake Dec 30, 2023, 2:10 PM

#

works great so far. wasn't able to find it on the quick, but where would I be able to change the prompt template?

halcyon quarry Dec 30, 2023, 2:36 PM

#

charred lake works great so far. wasn't able to find it on the quick, but where would I be ab...

in dict_base_settings.yaml
Or you can do it per-character, using example character M1nty for reference

#

Any settings from the character / imgmodel command / etc will override ones in base settings. You can see currently active settings in activesettings.yaml

charred lake Dec 30, 2023, 3:07 PM

#

thank you! however i couldn't find where to specify the instruction format used. is it preset?

#

also it prints /n in the message instead of doing line breaks, is this a config issue?

halcyon quarry Dec 30, 2023, 3:14 PM

#

I also noticed the /n issue yesterday… unsure why that started happening. Will fix it tonight

#

I’m fairly certain templates are now in the ninji format only?

#

Preset can be ‘Midnight Enigma’ etc

charred lake Dec 30, 2023, 3:24 PM

#

halcyon quarry I also noticed the /n issue yesterday… unsure why that started happening. Will ...

much appreciated!

charred lake Dec 30, 2023, 3:24 PM

#

halcyon quarry Preset can be ‘Midnight Enigma’ etc

oh, i was talking about these templates

halcyon quarry Dec 30, 2023, 3:27 PM

#

that's probably part of the reason the \n issue is happening... as a temporary fix you could copy/paste the contents of the correct template into base settings file

#

The issue is likely that the template strings I have in there have (double backslash)n instead of \n.
I had copy/pasted those values from one of the modules... may not be correct syntax in my code.
Testing

charred lake Dec 30, 2023, 3:34 PM

#

halcyon quarry that's probably part of the reason the \n issue is happening... as a temporary f...

that fixed it indeed!

halcyon quarry Dec 30, 2023, 3:35 PM

#

Btw, the bot currently doesn't even support instruct mode 🤗 Need to look into that as well...

halcyon quarry Dec 30, 2023, 3:36 PM

#

charred lake that fixed it indeed!

Yep, quick test I see that was the issue... my strings had double backslashes

#

#

Just pushed the fix

vestal python Dec 30, 2023, 3:40 PM

#

I've been trying to get some website or search extensions working last night, but nothing yet. I remember one of them was saying instruct or instruct-chat needed, but I couldn't get it to work right anyways. Just something I'm working on this week.

halcyon quarry Dec 30, 2023, 3:43 PM

#

I'm going to clean up some duplicate code where message requests & image requests overlap, because the time has come... then I will figure out the best way to change modes

halcyon quarry Dec 30, 2023, 3:45 PM

#

vestal python I've been trying to get some website or search extensions working last night, bu...

I've seen a few of those popping up recently on Reddit, have you tested a few in the webui? I'm interested in supporting the feature, would be nice to know what is the best one

vestal python Dec 30, 2023, 3:48 PM

#

A few are strange chrome-based, which is fine for windows users mostly. I've been trying to get just the 'LLM_Web_search' one to work as it's not chrome required, and uses duckduckgo.

#

There are other options I can try. This one I just spent a few hours to get it loaded fine, but there's just no calling it. All hallucinations

#

I might try to get lucid web search to work. I think it would just take me to work on the AI box directly, and not SSH'd from my laptop.

halcyon quarry Dec 30, 2023, 3:57 PM

#

PSA: I see alltalk_tts just got an update

halcyon quarry Dec 30, 2023, 4:07 PM

#

vestal python I might try to get lucid web search to work. I think it would just take me to wo...

This is the extension I noticed… you tried this one?

https://www.reddit.com/r/Oobabooga/s/gmvfWUvjae

From the Oobabooga community on Reddit: Web search extension

Explore this post and more from the Oobabooga community

vestal python Dec 30, 2023, 8:51 PM

#

I seem to have broken my textgen, going to have to clone it again :/ I should really start making a backup

vestal python Dec 30, 2023, 9:33 PM

#

Geez, my error was my config.yaml file in models was causing errors. Cloning ooba, and moving the models folder was just causing the same error until I kept the new config.yaml x.x

halcyon quarry Dec 30, 2023, 9:38 PM

#

Ah I suppose they changed the formatting or something

vestal python Dec 30, 2023, 9:38 PM

#

Every nowandthen

#

Ok so ooba and discord bot are latest, and I'll take a look at the settings for img thanks

#

Then once working, I really want to see if I can get one of those simplified search features working at least.

halcyon quarry Dec 31, 2023, 3:34 AM

#

Ok so I actually got instruct mode set up a lot easier than I thought it would take

halcyon quarry Dec 31, 2023, 4:33 AM

#

Update Bot now supports chat-instruct and instruct modes.

There is currently no command to switch modes, you'll have to put it in the settings (activesettings.yaml or char file).

#

From my quick testing everything works as expected. If anyone sees any issues let me know!

halcyon quarry Dec 31, 2023, 1:19 PM

#

So I think the right thing to do at this point, is to expand the /character command to include mode, inst_template, cht_inst_prompt, grammar

halcyon quarry Jan 1, 2024, 2:15 AM

#

Small update: dynamic labels for /imgmodel and /speak commands

halcyon quarry Jan 1, 2024, 2:08 PM

#

Anyone here actually use chat-instruct mode? Do you save/load your prompt template in some way?

civic bison Jan 4, 2024, 1:06 AM

#

Hi i was investigating moving to this from oobabot but getting confused, is there an option to load the webui interface so can select the model using the webui. I have a 3 gpu system/sometimes switch models so would prefer to load it that way rather than the having to select a model when it loads, is that possible? Like some command python bot.py --api --listen --load webui or such?

Or if not is there a command to load the webui after so i can change it? When i run the command atm it doesnt seem to load the webui after.

halcyon quarry Jan 4, 2024, 2:00 AM

#

civic bison Hi i was investigating moving to this from oobabot but getting confused, is ther...

I plan on adding a command to change llm model during runtime, maybe sooner than later

#

It wont be integrated with the webui interface, though

civic bison Jan 4, 2024, 2:01 AM

#

halcyon quarry I plan on adding a command to change llm model during runtime, maybe sooner than...

Thanks, is there a way to get the webui to load at least? Sorry being dumb but i can get the bot to work, but dont know address to access the webui

civic bison Jan 4, 2024, 2:02 AM

#

halcyon quarry It wont be integrated with the webui interface, though

ahh so you cant use the webui with the bot? Or am i just missing some arg for it?

halcyon quarry Jan 4, 2024, 2:03 AM

#

Can’t use the interface with the bot, unless you’re running two separate instances

#

If you can even do that 😛 I don’t know

civic bison Jan 4, 2024, 2:04 AM

#

halcyon quarry Can’t use the interface with the bot, unless you’re running two separate instanc...

Alright thanks shame yeah its a 70b model so cant run two instances lol. Guess ill stick with 7b if run this as well.

halcyon quarry Jan 4, 2024, 2:07 AM

#

The bot does honor your custom settings defined in the native settings.yaml - you can also enter settings in activesettings.yaml for immediate effect, or add them to the character file and use /character

#

see example character M1nty for formatting that

civic bison Jan 4, 2024, 2:12 AM

#

halcyon quarry The bot does honor your custom settings defined in the native settings.yaml - yo...

Yeah thanks, i got the bot itself working. I just like to switch models/use the webui, i suppose if set args right will be able to get it to work with 70b and sillytav. I got the assistant working with a 13b model tested in discord. Still an option for a stable diffusion bot through thanks 😄

halcyon quarry Jan 4, 2024, 2:12 AM

#

Let me know if you have any other questions! I’ll try and work in the model switching soon

#

I’ve got a big list of things to do 😛

civic bison Jan 4, 2024, 3:50 AM

#

halcyon quarry Let me know if you have any other questions! I’ll try and work in the model swi...

Thanks, ive managed to get it fully working with command : python bot.py --api --listen --loader exllamav2_hf --model LoneStriker_lzlv_70b_fp16_hf-4.0bpw-h6-exl2-2 --gpu-split 13,0,24

Nevermind fixed it I think.

civic bison Jan 4, 2024, 4:12 AM

#

Do you think it would be possible to make a bat file rather than have to use command prompt then paste command to make this work on windows?

marsh harness Jan 4, 2024, 4:17 AM

#

civic bison Do you think it would be possible to make a bat file rather than have to use com...

Do you mean to run the discord bot? I made mine into a bat file that I always use to run it.

civic bison Jan 4, 2024, 4:18 AM

#

marsh harness Do you mean to run the discord bot? I made mine into a bat file that I always us...

Yeah I have :

@echo off
python C:\Users\wuff\AI\text-generation-webui-main\bot.py --api --listen --loader exllamav2_hf --model LoneStriker_lzlv_70b_fp16_hf-4.0bpw-h6-exl2-2 --gpu-split 13,0,24
pause

so far but then just errors out when trys to run import tourch

marsh harness Jan 4, 2024, 4:19 AM

#

Hmm... I'm not sure if the gpu split will work because I haven't tried it, but I can copy & send what's in my bat file. Gimmie a minute.

civic bison Jan 4, 2024, 4:19 AM

#

Oh that command works in command prompt ill test it again to make sure

marsh harness Jan 4, 2024, 4:19 AM

#

You'd just have to edit the flags that you need to start the bat with yourself & see what works.

#


cd /D "%~dp0"

set PATH=%PATH%;%SystemRoot%\system32

echo "%CD%"| findstr /C:" " >nul && echo This script relies on Miniconda which can not be silently installed under a path with spaces. && goto end

@rem fix failed install when installing to a separate drive
set TMP=%cd%\installer_files
set TEMP=%cd%\installer_files

@rem deactivate existing conda envs as needed to avoid conflicts
(call conda deactivate && call conda deactivate && call conda deactivate) 2>nul

@rem config
set CONDA_ROOT_PREFIX=%cd%\installer_files\conda
set INSTALL_ENV_DIR=%cd%\installer_files\env

@rem environment isolation
set PYTHONNOUSERSITE=1
set PYTHONPATH=
set PYTHONHOME=
set "CUDA_PATH=%INSTALL_ENV_DIR%"
set "CUDA_HOME=%CUDA_PATH%"

@rem activate installer env
call "%CONDA_ROOT_PREFIX%\condabin\conda.bat" activate "%INSTALL_ENV_DIR%" || ( echo. && echo Miniconda hook not found. && goto end )

@rem enter commands
call python bot.py --loader exllamav2 --model-menu --extensions whisper_stt
echo Done!
cmd /k "%*"

:end
pause```

#

Just change what you need after call python bot.py

#

copy that and save it as whatever you want.bat, make sure you put the bat file in C:\Users\ (your user)\ (your folder location)\oobabooga_windows\text-generation-webui

civic bison Jan 4, 2024, 4:26 AM

#

marsh harness ```@echo off cd /D "%~dp0" set PATH=%PATH%;%SystemRoot%\system32 echo "%CD%"|...

Thank you very much that worked ^

marsh harness Jan 4, 2024, 4:26 AM

#

Happy to help, cheers

civic bison Jan 4, 2024, 4:27 AM

#

Just need to figure out how to get minty to load now lol. But its working otherwise

#

oh wait i guess they would go in characters folder hmm

halcyon quarry Jan 4, 2024, 4:28 AM

#

Yep

civic bison Jan 4, 2024, 4:29 AM

#

yus they working thanks now to see if can get stable working and all set. I assume it copys the 4096 context and stuff from ooba settings then?

halcyon quarry Jan 4, 2024, 4:31 AM

#

Yep

civic bison Jan 4, 2024, 4:31 AM

#

I think oobabot has a "memory" of the last 7 messages, just curious how this is

halcyon quarry Jan 4, 2024, 4:32 AM

#

Just standard chat history… probably about that many messages before things start slipping out of context

civic bison Jan 4, 2024, 4:35 AM

#

halcyon quarry Just standard chat history… probably about that many messages before things star...

okays you have this setup good everything seems to work out of box lol once got that bat setup, still miss the webui but i suppose silly tavern setup right can replace most of that.

halcyon quarry Jan 4, 2024, 4:36 AM

#

Ah, so it currently does not save history for messages it does not reply ti

#

nice 🙂 yep that is the forte of the bot, making images

civic bison Jan 4, 2024, 4:37 AM

#

Ahh so it remember say something i said 3 messages ago or only from what reply to last message?

halcyon quarry Jan 4, 2024, 4:38 AM

#

If you don’t set a “main” or you adjust the settings so it doesn’t always reply - it wouldn’t see messages it doesn’t immediately reply to

civic bison Jan 4, 2024, 4:38 AM

#

Ahh its set to main i know can use () will tell others that in the chat.

halcyon quarry Jan 4, 2024, 4:39 AM

#

It’s also quite easy to toggle that on/off

civic bison Jan 4, 2024, 4:39 AM

#

Still will play with it tomorrow lol that was a long night but got both yours and oobabot running for now, should make it easier to migrate

halcyon quarry Jan 4, 2024, 4:39 AM

#

If you get into a heated convo 😛

#

I’m probably going to get LLM model switching in tomorrow…

civic bison Jan 4, 2024, 4:40 AM

#

is there a reason one can't use the webui? might be certain extentions that might be useful in that in future?

halcyon quarry Jan 4, 2024, 4:42 AM

#

Really not sure… note that I forked this awhile ago so there’s still a few things I don’t 100% what’s going on backend

civic bison Jan 4, 2024, 4:45 AM

#

Ahh right well still thanks for that, seems like you might be only guy still making bot stuff currently.

halcyon quarry Jan 4, 2024, 4:46 AM

#

I’m loving it 😛 Great canvas to paint ideas on

civic bison Jan 4, 2024, 4:47 AM

#

yeah im suprised too cause something like this handy for help bots and stuff id guess if programmed right, should be merged into the main program imo lol

halcyon quarry Jan 4, 2024, 4:47 AM

#

If you get into the imgtags feature with a bunch of LORAs and have lrctrl extension, I wrote a thing that does a great job automatically crossfading multiple loras

civic bison Jan 4, 2024, 4:48 AM

#

That sounds a little way beyond me lol. I got the hardware to do some training but no clue how to lol.

#

I just finished the workstation today, had 68 gig vram with an a4000 but gonna send that back to save some cash lol.

halcyon quarry Jan 4, 2024, 4:49 AM

#

It’s just trigger words to add stuff to image payload, including loras if you download em

civic bison Jan 4, 2024, 4:49 AM

#

Ohh i think get what you mean

#

Yeah i think friends might like that.

halcyon quarry Jan 4, 2024, 4:50 AM

#

Lrctrl is an a1111 extension that improves native lora handling

civic bison Jan 4, 2024, 4:50 AM

#

So like if wanted a photo of a dog or something it would add tags for the breed and stuff I assume?

halcyon quarry Jan 4, 2024, 4:51 AM

#

It’s just anything you want it to arbitrarily add to the prompt/neg prompt

civic bison Jan 4, 2024, 4:52 AM

#

I'll mention it to my friend tomorrow that, im just the hardwre guy, he is one who would know about stuff like that.

halcyon quarry Jan 4, 2024, 4:54 AM

#

There’s a lot of possibilities with the “dynamic context “ feature too, which can swap characters, apply custom instructions, manipulate history, all from a simple trigger word

#

anyway, hope you and your friends enjoy, I’ll accept any feedback

civic bison Jan 4, 2024, 4:55 AM

#

I'll link him to that tomorrow yeah might want to play with thanks.

marsh harness Jan 4, 2024, 8:18 AM

#

I want to run a Mixtral model so badly. But I really need to upgrade my ram.

halcyon quarry Jan 4, 2024, 3:31 PM

#

@civic bison and all:

Just pushed an update that adds a new command, /llmmodel - it processes model loading just the same as when the script initializes

marsh harness Jan 6, 2024, 1:05 AM

#

Ordered more ram, gonna be able to do a lot more testing.

#

But I think from now on I'm gonna be running dolphin 2.7 mixtral once it gets here. Did some gguf testing with lm studio and it's better than any model I've used beforehand by far.

vestal python Jan 6, 2024, 1:06 AM

#

I'm in the process of upgrading my main AI to x3 rtx 3060 12GBs, and starting a more dev environment testing Box with x2 P40 24GB cards. Hopefully setupnbt the end up next week.

marsh harness Jan 6, 2024, 1:07 AM

#

Damn. Nice. I only have a single 3090, but only 16gb ram. I have 64gb coming now.

vestal python Jan 6, 2024, 1:07 AM

#

I'm on mixtral v0.1 with my discord bot right now. Just fits 24GB vram with 22k context.

marsh harness Jan 6, 2024, 1:11 AM

#

I think I'm just limited by dram, or I'm not using the right settings. Because idk how to get the bot to load with my 24gb vram. Not with exllamav2 at least.

vestal python Jan 6, 2024, 1:14 AM

#

I use 3.5bpw mixtral with 22k context tokens. Really all there is

marsh harness Jan 6, 2024, 1:14 AM

#

Ah, I don't think I've tried the 3bit option with exllamav2.

vestal python Jan 6, 2024, 1:15 AM

#

It just barely fits 22.8GB

#

Not enough room to include alltalk for my discord bot. So I need to expand as well

marsh harness Jan 6, 2024, 1:17 AM

#

Have you tried out dolphin 2.7?

#

Well... it literally just came out yesterday. But

#

Any dolphin mixtral?

#

https://huggingface.co/TheBloke/dolphin-2.7-mixtral-8x7b-GPTQ

TheBloke/dolphin-2.7-mixtral-8x7b-GPTQ · Hugging Face

vestal python Jan 6, 2024, 1:22 AM

#

Oh I have it on my list. I was very interested in it with the description of including a better training set for programming.

I need my discord bot Ecne, to assist with people on a game server, and to Also help me customize a different game's proprietary C++ style scripts.

#

But dolphin 3.0 hoping they perfect it, and doublecheck coding

marsh harness Jan 6, 2024, 1:24 AM

#

Yeah I'm really looking forward to 3.0. Wanted to try out 2.7 to see if the general quality was there though. I only did simple tests since inference was slow, but it could make a working html page with css pretty well.

#

LMstudio with gguf at 4 bit loads for me with 2.7 if I offload 50 layers to gpu.

I didn't get to see or test how well it writes c++, but that's the only language I've been learning so hopefully it's good with that.

#

A lot of models mostly have python in their training data more than any other language though. So hopefully the cpp is good and extensive enough.

marsh harness Jan 6, 2024, 3:39 AM

#

Kept saying ggml... meant to say gguf.

halcyon quarry Jan 6, 2024, 1:27 PM

#

marsh harness I think I'm just limited by dram, or I'm not using the right settings. Because i...

It’s working in the webui not with the bot?

marsh harness Jan 6, 2024, 1:50 PM

#

halcyon quarry It’s working in the webui not with the bot?

Won't work for me with either of them. Probably because I need to try 3bit.

#

The 4bit 128 act order one should fit based on filesize, so I'm thinking it might just be a dram issue. I'll know when my ram gets here.

halcyon quarry Jan 6, 2024, 2:51 PM

#

Of course, the bot is bug-free 😅

restive pendant Jan 7, 2024, 8:53 PM

#

where can i get the link for this

vestal python Jan 7, 2024, 9:14 PM

#

https://github.com/altoiddealer/ad_discordbot/tree/main

GitHub

GitHub - altoiddealer/ad_discordbot: A Discord Bot for chatting wit...

A Discord Bot for chatting with Large Language Model (LLM) supported by text-generation-webui - GitHub - altoiddealer/ad_discordbot: A Discord Bot for chatting with Large Language Model (LLM) suppo...

halcyon quarry Jan 7, 2024, 9:49 PM

#

@calm rain could you pin the message above that links to the github? Or give me pinning permissions? 😆

restive pendant Jan 7, 2024, 9:57 PM

#

vestal python https://github.com/altoiddealer/ad_discordbot/tree/main

tyvm

calm rain Jan 8, 2024, 3:22 AM

#

pinned the original post here, should set up a bot to automatically do that

#

Or give me pinning permissions? for some reason Pin and Delete Other People's Messages are one single boolean perm toggle in discord, so that'll only happen if/when we need more moderators here

#

i don't understand why discord works this way

viral lagoon Jan 8, 2024, 8:03 AM

#

calm rain i don't understand why discord works this way

Even if Pin was a separate permission, you can't assign roles/permissions to threads, only to their parent channel (the forum.)
So, right now, you would need a specialized bot or some custom command to allow thread-creators do something like that.

civic bison Jan 8, 2024, 1:14 PM

#

I had to stop using the bot for now was making my pc hitch while gaming for some reason, im going to test it again when get a second pc

halcyon quarry Jan 8, 2024, 6:01 PM

#

Personally I stop the bot when gaming, then launch it again when I'm done 😛

vestal python Jan 8, 2024, 6:32 PM

#

I'm hopefully going to have t 2nd dedicated AI Desktop up and running tomorrow, and see how well that does, and also see if my Discord Server likes having 2 Discord AI Bots

halcyon quarry Jan 8, 2024, 6:34 PM

#

Please report / request anything you find when 2 bots are interacting, I haven't tested any of those features personally 😛

vestal python Jan 9, 2024, 1:07 AM

#

Prep'd for tomorrow to upgrade and building. My main AI Going with x3 3060 12GBs, and a development second box with 1 P40 24GB for now. Going to test the speeds, and probably add a second P40 or at least my 1080ti 11GB just for testing.

marsh harness Jan 9, 2024, 1:30 AM

#

My ram will get here on the 10th. Looking forward to testing gpt pilot with mixtral and seeing what I can make with it. I'll see if I can get it running with the bot, to test how it performs and what I can do with it.

vestal python Jan 9, 2024, 4:13 PM

#

#

She'll do

#

But I think 3 is the limit for this casr

#

My 2nd bot is going to have to wait. Cause no way it's going to fit the case I had picked out

marsh harness Jan 9, 2024, 10:42 PM

#

My ram got here a day earlier than it was supposed to.

That aside, I tested one of Lonestriker's exllamav2 quants. LoneStriker/dolphin-2.7-mixtral-8x7b-3.75bpw-h6-exl2 Loads in ooba, but inference is super slow. (On my single 3090.) I'm tired, so I'll probably play with it more when I wake up, but maybe I just need a lower bpw and then token generation will be faster. Once I have a decent amount of tokens/s, I'll see how it runs with the bot.

#

#

Maybe if I lower the context size that'll improve performance as well.

vestal python Jan 10, 2024, 2:04 AM

#

I know something about Dolphin was trained on 16k it continuously says?? So maybe lower it to like 14k and see

#

Also the first quetion upon loading is usually like 5 t/s

#

Then, idk there's been some updates to things and see if upgrading requirements does any changes

marsh harness Jan 10, 2024, 10:07 AM

#

Lowering it to 2048 made inference way faster. Now I might just need to use 3.0 bpw instead of 3.75 with 2048 and it should be fast enough for a chatbot.

marsh harness Jan 10, 2024, 10:40 AM

#

4096 context using 3.0 bpw works great. Gonna text with the bot now.

#

That works for me even with alltalk.

marsh harness Jan 11, 2024, 10:51 PM

#

It'd probably work better with instruct mode, but it's functional and fast either way.

halcyon quarry Jan 12, 2024, 12:28 AM

#

marsh harness It'd probably work better with instruct mode, but it's functional and fast eithe...

Did you see that I added support for instruct mode?

marsh harness Jan 12, 2024, 12:31 AM

#

halcyon quarry Did you see that I added support for instruct mode?

I saw you mention it a while back, but didn't read into it any further.

halcyon quarry Jan 12, 2024, 12:32 AM

#

Currently, you can either include it in your character’s “state” keys - or set that parameter in activesettings

#

Or omit the parameter in activesettings but include it in basesettings.
mode: instruct

#

When you launch the bot it will find the instruct template from the model and apply it - otherwise it will use a template defined in activesettings/basesettings/character state etc

#

The cmd window will state everything

marsh harness Jan 12, 2024, 12:39 AM

#

Good to know. Will try it out.

#

Is there a m1nty instruct example character template?

#

If not I'll try it out soon either way. Have some stuff to do for a while.

halcyon quarry Jan 12, 2024, 12:42 AM

#

The ooba default instruct templates are in basesettings

halcyon quarry Jan 12, 2024, 3:20 AM

#

Pushed an update to add the new textgen parameter keys regarding dynamic_temperature

#

(compatibility update)

vestal python Jan 13, 2024, 3:00 AM

#

I've noticed something I was wondering... So the Discord bot program has a 1980 character limit. If I ask it to summarize a script with 2300 characters, it just says bye.

If I reduce he script and ask it to summarize 1800 characters worth of a given script, it answers quite well..

Is there a way to increase this limit, or is there anything you can think of for Nitro 4000 character users? I was also wondering if I have Nitro, does my Bot program also get 4000 limit...

#

Or maybe I'm not understanding it right.

vestal python Jan 13, 2024, 3:29 AM

#

Honestly, I might just call it good enough cause it does work great asis, and just make my new box strictly textgenUI to ask it longer script questions and support.

halcyon quarry Jan 13, 2024, 5:25 AM

#

When you send message over 1980 chars discord truncates the message? Or is the bot not accepting it?

#

Ah I see… discord does cut it… well, maybe I could add a file upload option for prompting so you use a txt file

calm rain Jan 13, 2024, 5:30 AM

#

if you have nitro, discord cuts at 4000

halcyon quarry Jan 13, 2024, 5:31 AM

#

Well txt file could then read > 4000 😄 I’ll put this on the back burner

#

I already have plans to make a command similar to my /image function for more advanced text requests

halcyon quarry Jan 21, 2024, 3:51 AM

#

Started working on the revamped "tags" feature... could still be a few days out. Will be extremely versatile and cut down quite a bit of clutter from config.py

ripe pendant Jan 22, 2024, 7:50 PM

#

halcyon quarry Anyone here actually use chat-instruct mode? Do you save/load your prompt templ...

@ altoiddealer maybe this is usefull for you: https://github.com/oobabooga/text-generation-webui/wiki/01-‐-Chat-Tab#chat-style . I experiment with following pattern: [ ###instruction: This is an erotic adventure game, and your task is to write a reply in name of "<|character|>" where 3 options are given for the user to then choose from, make a sequel out of the material in input [ ** ] ###input: One afternoon they both discover Silvia and Chela hiding behind the hotel building, where there is a mysterious pavilion - a favorite walking route of the older Indian women on the team... ###response: ] (I did it in German, here is an poorly Google-translation only to show the pattern to explore that. ) I did not find the description from ooba for this. the place to insert it is : Parameters Tab> instruction Template > scroll down to > Command for chat-instruct mode (stay alert NOT to delete the <|prompt|> there. Also be sure >chat Tab>down to >mode > 'chat-instruct' is chosen. // [**] you experiment with complex instructions // if the ###response: answer turns to repeat the same, I set up a new chat and give some last stuff from the last conversation in the input , to have some continuity , make good times cordially Petite Epona

GitHub

01 ‐ Chat Tab

A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models. - oobabooga/text-generation-webui

halcyon quarry Jan 22, 2024, 7:52 PM

#

I don't use the UI much, was wondering if people are using some method to quickly change chat-instruct prompts, aside from editing the chat-instruct field

ripe pendant Jan 22, 2024, 7:59 PM

#

for me that field is very helpful to discover more komplex instructions, als I'm absolut beginner in this stuff . Was hoping to learn from You linux 😇 and to give some help

halcyon quarry Jan 22, 2024, 8:02 PM

#

Well, I did add support for this recently - just copy/paste your prompt into the chat-instruct parameter in activesettings.yaml and change the mode to chat-instruct

#

works exactly the same as in the UI

#

What is on my to-do list is a means to easily change that without having to reload the script

#

or change characters, etc

halcyon quarry Jan 23, 2024, 2:20 PM

#

Pushed an update to add the new textgen-webui key, prompt_lookup_num_tokens

#

Note that I had issues running latest textgenwebui with alltalk_tts 1.8 - I updated to 1.9 and it works again

vestal python Jan 29, 2024, 6:13 PM

#

Finally finished upgrading my main bot to x3 RTX 3060 12GB's
So back in business working on Mixtral, and adding in tts, additional features.

halcyon quarry Jan 30, 2024, 3:22 AM

#

Next push has been delayed as I’ve been too busy gaming, but will be soon

vestal python Feb 1, 2024, 4:29 PM

#

Only thing I'll be looking into I think actually with my 2 boxes plus my laptop with basic Mistral, is setting up OpenAI API for the two boxes, while maintaining the Discord bot as well if possible. Trying to do so my 2 people using him to slightly develop some python tools, and use my 3 boxes in autogen studios (1 local, and 2 OpenAI api'd in) to test agents.

vestal python Feb 3, 2024, 4:20 AM

#

I got Mixtral Dolphin 2.7 5BPW and Alltalk working with the upgrade... It's a tight fit geez

vestal python Feb 4, 2024, 3:57 PM

#

If I run ooba first, can I call bot.py on a seperate instance? I'e just been running lines for each AI as:
python bot.py --api --listen --loader exllamav2_hf --model turboderp_Mixtral-8x7B-instruct-exl2_3.5bpw --max_seq_len 22768 --cache_8bit --num_experts_per_token 2 --gpu-split 3,10,10

and:
python3 bot.py --api --listen --extension telegram_bot --loader llama.cpp --model Wizard-Vicuna-30B-Uncensored.Q4_K_M.gguf --max_seq_len 2048 --n-gpu-layers 61 --threads 8

Can I just run bot.py with ooba preloaded with the extensionsI want and api/listen?

halcyon quarry Feb 4, 2024, 9:49 PM

#

Probably not. At some point I’ll look into changing it to use the API

#

Would be a pretty big code overhaul I think though

#

I’m done with my game, had an amazing run on “hardcore” mode but finally rip’d. Development shall now resume 😛

halcyon quarry Feb 5, 2024, 4:11 PM

#

There's another interesting feature I have in mind, after coming across this implementation here https://www.reddit.com/r/StableDiffusion/comments/19dfvf3/rpgdiffusionmaster_mastering_texttoimage/
This relies on using Multimodal LLMs, but my idea is to add some sort of workflow control where the first response from the LLM can be used as input for a subsequent text request, using another defined character/settings/etc... that response again being used as input... and so on, before you receive the final result. I need to think on how to cleanly add such a feature

#

Best way to explain is to look at the last image on the reddit post which diagrams how it works. My idea is for each step to be controlled by a different character card or instruction tailored for that step, I imagine the end result would be the same

#

user has simple input - bot comes up with very complex image prompt correctly formatted for regional prompting.

vestal python Feb 5, 2024, 4:51 PM

#

I see, so multiple LLM's, or would it be the same LLM with a different character assignment to get a different perspective?

I see the example was a large # of steps, thenbuilding the base image steps 1~20, assigning location boxes for each to give a detailed prompt for that location with 20~30 steps each, etc.

Which goes back to the whole theory that LLM's aren't bad, and people just aren't prompting them to their full potential. Interesting.

halcyon quarry Feb 5, 2024, 5:41 PM

#

Same LLM - I believe a character card with just example dialogue of example inputs and desired replies, most models could understand... before revising scripts or anything I'd first make the character cards and try each step

#

I could be wrong 🙂 Heck even if it does a shit job assigning regions, it would probably be good enough.
I have good faith in it making good prompts for each component though - its really the regional prompting assignments step that I'm unsure about

halcyon quarry Feb 5, 2024, 5:56 PM

#

vestal python I see, so multiple LLM's, or would it be the same LLM with a different character...

The way regional prompting works, is it only generates in the defined region for X steps, then goes to the next region for the next steps, etc.
So the four little images shows that the top left was done in 5 steps, top right after 5 more, and the bottom parts 10 steps each. The base prompt is applied over the entire image to help unify everything

#

There is a syntax to use with the A1111 regional prompting extension to define the regions and prompts - so the idea is to have the LLM focus on specific tasks to ultimately craft that final image payload to send to A1111

#

I'm making good progress on this 'tags' revamp...

calm rain Feb 5, 2024, 6:03 PM

#

that's less regional prompting and more multiprompted tiled generation

#

"regional prompting" as a concept does the same steps as normal and just sends different prompts to different areas (via mucking with SD's crossattention)

halcyon quarry Feb 5, 2024, 6:05 PM

#

Maybe I described it a little wonky, but as you wrote it is as I intended

#

It's been awhile since I used that feature... as I remember it, it does not diffuse the entire image uniformly but each region sequentially

#

in any case, the end result is more or less the same for both concepts (whichever it actually is)

#

In their little preview images it shows at Step 5 only the top left region being mostly diffused, the rest partially diffused (base prompt partially diffusing everything). Then the top right has most attention... All the while, the base prompt keeps nudging everything to a finished product.

calm rain Feb 5, 2024, 6:11 PM

#

if your understanding of what words mean comes from what auto webui extensions do you gonna think some wildly wrong things lol

halcyon quarry Feb 5, 2024, 6:12 PM

#

That's it, I'm going to look at the project page right now lol

#

Don't mean to talk out of my ass

calm rain Feb 5, 2024, 6:12 PM

#

(which to be fair is also partially my fault, my Dynamic Thresholding extension does like 50 things that are unrelated to dynthresh)

#

here's what regional prompting looks like in swarm (which is a zero-nonsense just latent couple split conditioning)

halcyon quarry Feb 5, 2024, 6:16 PM

#

Ahhhh yes, I am indeed talking out my butt 😛

#

But yes - I see this RPG-DiffusionMaster thing and it makes me thing this idea could be duplicated in a series of LLM requests from the same model, to make the final A1111 payload

#

How to work the feature in elegantly so big dreamers could use it for other cases, that's where I'll be scratching my brain for a bit

calm rain Feb 5, 2024, 6:22 PM

#

Definitely can ye. rpg is very similar to a thing i tried before -- which was literally the above swarm screenshot, that prompt syntax that swarm supports for regional, i had an LLM generate that. Problem was the LLM was really really bad at picking regions if they're any more complex that just left|right|top|bottom

#

i tried for example to have it to two people on a bench, select regions for each person and the bench, and that's already beyond its limits

#

made worse by the fact that it's really hard to get good generations from 3 overlapping regions with unique objects even if you do it perfectly

#

i also tried have it generate separately and combine or inpaint but that rarely gets a reasonably coherent scene, eg if you generate a person it will never exactly fill the frame with the person, it'll zoom in on the face or zoom out really far or... asfgkags

#

(I wanted to be able to give an LLM one prompt and have it generate complex multi-object scenes in general)

halcyon quarry Feb 5, 2024, 6:25 PM

#

Yes as I understand it, the RPG thing segments each step so one step it knows the subcomponents and just has to set regions, while others step(s) make the subprompts more verbose, another step maybe unifies, etc.

#

I'm thinking I could have a yaml dict where user can define workflows, and it will hijack the normal routine if one is activated
Activated by trigger phrase (I love these 🙂 ), maybe a slash command if more on-the-fly settings seem necessary

halcyon quarry Feb 8, 2024, 3:19 PM

#

I'm almost done with nice update 💪 Put a lot of hours into this one.
The revamped tags feature is extremely versatile, unifies a lot of the existing features into a much more modular and easy to manage system

halcyon quarry Feb 9, 2024, 6:39 PM

#

It's all good, just fine polishing things in hopes that no one has to migrate settings again a few days later 😛

#

this system is sweet... hard to contain my excitement on this

#

I'll also update the wiki in advance...

halcyon quarry Feb 12, 2024, 8:35 PM

#

Squished all bugs that I could identify. Just going to try and add some appropriate print statements, update the wiki and push this thing

halcyon quarry Feb 12, 2024, 10:25 PM

#

ugh… had to rush out the door and couldn’t push it in time. Won’t have access until to the files until tomorrow.

I did update the wiki with information on the new Tags system. Believe it or not, all of the functionality does what it claims to.

https://github.com/altoiddealer/ad_discordbot/wiki/tags

GitHub

tags

A Discord Bot for chatting with Large Language Model (LLM) supported by text-generation-webui - altoiddealer/ad_discordbot

#

Wiki still needs some more info…
-There is a section to define a default value across all tags.
-There is a section to define presets. By including tag_preset_name in any tag, is the same as copy/pasting all of them.

halcyon quarry Feb 13, 2024, 7:50 PM

#

The major update for the new Tags system has been PUSHED

#

🍾

halcyon quarry Feb 14, 2024, 2:39 AM

#

Just pushed a quick update to fix error 'sampler_priority' - a new state key added recently in textgenwebui where you can optionally list the order sampling methods are applied

#

Another quick note to anyone reading my rambling here - the bot works just as well with "SD Forge", except the 'lrctl' extension is bugged in Forge so that feature is not working in the bot w/ Forge

halcyon quarry Feb 16, 2024, 5:54 PM

#

I thought of an interesting 'tag' feature I might add in... after reading some thread about using an LLM to choose the most appropriate image model for a given prompt. I'm thinking I could add a 'description' key for each imgmodel in the dict_imgmodels.yaml.

Then, a tag match could trigger the feature... a request to the LLM with instruction containing the imgmodel list + descriptions, and Choose the most appropriate imgmodel for prompt: "{img_prompt}"

calm rain Feb 16, 2024, 5:55 PM

#

i was thinking about doing that same thing, albeit with loras and the option to select multiple

halcyon quarry Feb 16, 2024, 5:56 PM

#

Ah, let the LLM pick the loras

#

I see you lurking- Have you peeked at this tag feature I added?

It's already triggering LORAs very reliably so long as they're predefined

#

The bot instance on my channel you have access to, is now giving some poor outputs when multiple LORAs are triggered...
Reason being, I switched that setup to sd-webui-forge which does not support loractrl. I have a feature that automatically detects lora syntax and applies loractl scaling.

My other bot (NSFW version) is still on A1111 so that feature is working well... got a crapton of loras defined there, and the images are good quality even if 5+ loras get triggered

halcyon quarry Feb 16, 2024, 6:20 PM

#

I may have to implement another feature to simply modify values for multiple LORAs... without the fancy crossfading effects

calm rain Feb 16, 2024, 6:24 PM

#

oh, i hadn't seen that, that's really clever!

#

forge is, uh... the developer really likes starting fights and causing trouble more than he likes actually developing things

halcyon quarry Feb 16, 2024, 6:27 PM

#

There's no doubt that he's brilliant, though. I'm very impressed with the speed and all.
According to the dev of the loractl extension, he commented that Forge is using really old lora handling code, though. May be part of the whole speed boost

#

https://github.com/cheald/sd-webui-loractl/issues/32#issuecomment-1945446914

calm rain Feb 16, 2024, 6:27 PM

#

if you like the speed of forge you'll love switching to comfy/swarm months ago lol

halcyon quarry Feb 16, 2024, 6:28 PM

#

it looks like it's using something closer to the pre-1.5 A1111 implementation of loras - which this extension will specifically not work with, since it depends on the 1.5 rewrite of lora handling

halcyon quarry Feb 16, 2024, 6:39 PM

#

calm rain oh, i hadn't seen that, that's really clever!

Here's screenshot showing the config settings, the scaling results in the cmd window and how the insertions were handled in the image prompt

#

The scaling factors I have in place has the first LORA start at its full defined strength, then fade to 50% of its strength at the 40% mark in the generation.

#

Multiplies the scaling factors by the values matched in the loras. So its very dynamic

#

Really, its a testament to either how good Chat-GPT 3.5 is or how well I can prompt it, lol.

I understand all the code now and can just sit down and code stuff, but ChatGPT wrote vast majority of the code I've implemented

#

Booty shorts lora could have been stronger but still, pleasing results 😛

calm rain Feb 16, 2024, 6:51 PM

#

different loras should be weighted at different times - "booty shorts" is a structural concept that needs more weight early on and matters less later, vs eg a character's face is something that only matters in the final steps

halcyon quarry Feb 16, 2024, 6:52 PM

#

I have so many ideas in my head... one was a means to assign priority to tags. Could be a solution here - otherwise could be solved by prompting it in reverse heh

#

priority is currently dictated by the order in which they were matched in the text (not the order they were iterated over)

sullen plover Feb 16, 2024, 8:31 PM

#

hi: @halcyon quarry how would i do the command like this maybe? sudo ./start_linux.sh --gradio-auth useranem:mypassword --listen --listen-port 7860 --listen-host zero.talktoai.org --ssl-certfile=/etc/letsencrypt/live/zero.talktoai.org/fullchain.pem --ssl-keyfile=/etc/letsencrypt/live/zero.talktoai.org/privkey.pem --extension LLM_Web_search --extension text-generation-webui-stable_diffusion python bot.py --loader exllama --model NeuralBeagle14-7B-GGUF

halcyon quarry Feb 16, 2024, 8:32 PM

#

For one... I haven't tested this in Linux, and I haven't received any feedback from anyone sayins specifically they used it on Linux. So may not work at all 😛

#

For starters, Id say try launching it simply... python bot.py --loader exllama --model NeuralBeagle14-7B-GGUF

#

maybe sudo if needed. But you dont execute native textgenwebui start batch file

sullen plover Feb 16, 2024, 8:33 PM

#

i think this may not work: python bot.py maybe --python bot.py

#

?

halcyon quarry Feb 16, 2024, 8:33 PM

#

Run cmd_linux.sh (activate the venv)

#

python is the command, is present in the venv

#

python bot.py --loader exllama --model NeuralBeagle14-7B-GGUF

#

this will run bot.py with the 2 flags

#

You can try adding the other flags all at once, or maybe incrementally. They are passed to the webui so they should all be honored

sullen plover Feb 16, 2024, 8:36 PM

#

./start_linux.sh

halcyon quarry Feb 16, 2024, 8:37 PM

#

You use start_linux.sh to launch the webui

#

But you don't use that, when running the bot.
You just run the bot. The bot handles launching the webui

sullen plover Feb 16, 2024, 8:39 PM

#

i see will try thanks

halcyon quarry Feb 16, 2024, 8:39 PM

#

Your textgen folder should look like this

#

bot.db will be created on first launch of the bot, it stores a few settings that need to be persistent ('main' channels, welcome message, etc)

sullen plover Feb 16, 2024, 8:42 PM

#

gonna read docs a bit more i am noob, ty

halcyon quarry Feb 16, 2024, 8:43 PM

#

My wiki isn't 100% up to date 🙂 Some features are not well documented.

The Install / Update instructions are accurate though

#

Features ARE well documented as comments in config.py and the various dict files

#

I try my best 🤓 Updating wikis is boring though 😄

#

Pushed an update to maintain compatibility with ReActor extension (for anyone using that feature)

#

They just keep adding more and more key arguments

sullen plover Feb 16, 2024, 9:17 PM

#

is the bot token the public key of the discord bot?

#

in config.py

halcyon quarry Feb 16, 2024, 9:25 PM

#

No... hmmm its been ages since I got my token... lemme look into that

#

In the Discord Developer Portal, in Applications

#

It is the "Bot" item... on that page is "USERNAME" and "TOKEN"

#

After that you go to OAuth2, check off the "bot" item

#

And select basically everything unless something seems totally out of scope to you. Copy paste the URL into your browser in order to invite the bot to your server

sullen plover Feb 16, 2024, 9:45 PM

#

thanks for the reply, will check 🙂

halcyon quarry Feb 16, 2024, 9:58 PM

#

d'oh - realized my 'trumps' part of the tags feature isn't working... will fix in next update

vestal python Feb 16, 2024, 10:31 PM

#

halcyon quarry For one... I haven't tested this in Linux, and I haven't received any feedback f...

I use Ubuntu for all my machines. Rightnow my command is:

python bot.py --api --listen --loader exllamav2_hf --model LoneStriker_dolphin-2.7-mixtral-8x7b-4.0bpw-h6-exl2 --max_seq_len 22768 --num_experts_per_token 2 --gpu-split 6,10,10

#

Although I haven't git pulled yet to see the changes. I've been messing with MoneyPrinter app, VSCode with continue extension with my new bot, and some autogen studio lately x.x busybusy.

halcyon quarry Feb 16, 2024, 10:33 PM

#

vestal python I use Ubuntu for all my machines. Rightnow my command is: python bot.py --api -...

Config.py is damn near 1/2 the size now. Imgtags is scrapped, and there’s a broad universal system now called Tags

vestal python Feb 16, 2024, 10:34 PM

#

Sounds boss I'll have to give it a go after my son is asleep tonight

halcyon quarry Feb 16, 2024, 10:35 PM

#

Now all the features that were activated by triggers, are matched in 2 searches total (pre-LLM and post-LLM), and only the relevant “tags”. Super efficient compared to previous

#

triggers not required either… whatever the tag is will just be insta matched

charred lake Feb 17, 2024, 1:44 AM

#

Trying the bot again, appreciate all the updates you've made!

Not sure whether this is something on my end, but loading characters seem to be broken? The bot returns ```An error occurred while changing character: cannot unpack non-iterable NoneType object

#

This is the full error

📎 message.txt

calm rain Feb 17, 2024, 1:45 AM

#

charred lake This is the full error

while reading ad_discordbot/dict_base_settings.yaml: while scanning for the next token
found character '%' that cannot start any token
  in "ad_discordbot/dict_base_settings.yaml", line 177, column 6 (Line: 401 in load_file, bot.py )

#

looks like a typo in settings yaml

charred lake Feb 17, 2024, 1:46 AM

#

oh i see, thank you

#

yeah obvious config issue. apologies!

halcyon quarry Feb 17, 2024, 3:23 AM

#

Welcome back gowron1

halcyon quarry Feb 17, 2024, 2:11 PM

#

Pushed an update which fixes the Tag parameter 'trumps' (actually works now)

sullen plover Feb 17, 2024, 3:09 PM

#

in config.py i dont know what to do where to add stuff etc, i am trying, wow 1 hour ago update @halcyon quarry i will not stop until i get it running lol yours if the most understandable for me so far

halcyon quarry Feb 17, 2024, 3:15 PM

#

sullen plover in config.py i dont know what to do where to add stuff etc, i am trying, wow 1 h...

The bot should run fine with just the bot token, and maybe needing to modify the A1111 url if you changed the default port

sullen plover Feb 17, 2024, 3:15 PM

#

i am running on my domain :7086

halcyon quarry Feb 17, 2024, 3:16 PM

#

Once you create a discord bot (discord developer portal) and get your Token, and generate the invitation URL from 0Auth page

sullen plover Feb 17, 2024, 3:16 PM

#

would be cooler to run on discord so will try without domain stuff

halcyon quarry Feb 17, 2024, 3:16 PM

#

copy/paste that url into a browser, and you should get to a page where you can invite your bot to a server

sullen plover Feb 17, 2024, 3:17 PM

#

nice done the bot bit on discord just need to edit that config file i think

halcyon quarry Feb 17, 2024, 3:18 PM

#

Most of the default settings for everything are pretty sensible... some features need a channel ID to work. theres a comment on that in config.py

#

Imgmodel handling is "API method" by default, so you don't have to bother with dict_imgmodels unless you want to start customizing settings per imgmodel

#

to take advantage of most of the cool extra features, you modify dict_tags, or your characters using the example character M1nty for reference

#

base_settings.yaml is also relevent... this contains all the defauts that can be modified.
It's essentially a duplicate of the core settings in bot.py (which are not intended to be modified).
Settings defined anywhere else will have higher priority (in characters, imgmodels, etc)

sullen plover Feb 17, 2024, 3:30 PM

#

thanks i think i should save this convo too lol

halcyon quarry Feb 17, 2024, 3:41 PM

#

Lastly, my overly complex settings handling currently has it so changes in base_settings.yaml do not immediately go into effect. You need to change character (could be same one) or imgmodel, to commit those updates. I’ve been meaning to change this but it makes my head hurt thinking about it 😛

sullen plover Feb 17, 2024, 3:44 PM

#

gonna give it a go till maybe 4am see what happens 🎉

vestal python Feb 17, 2024, 4:35 PM

#

I moved the old folders, git cloned the new one, copied my tokens and alltalk+sdxl settings. All working so far.

halcyon quarry Feb 17, 2024, 4:36 PM

#

Pushed small update to improve error handling when character not found

halcyon quarry Feb 17, 2024, 4:56 PM

#

Actually, it turns out I was disilusioned - that error was already coming from the imported textgen function lol.
Chasing my tail here

halcyon quarry Feb 17, 2024, 6:37 PM

#

Ok, now the appropriate message is right there and not buried a mile up the list

halcyon quarry Feb 17, 2024, 6:56 PM

#

Even better! How exciting

halcyon quarry Feb 17, 2024, 9:46 PM

#

Can anyone link me to virtually ANY Lora? (one that would work w/ 13b model or less).

I have an open issue on the github stating there is an issue loading LORAs with the bot, but I actually have no friggen clue where to find one for testing myself...

vestal python Feb 17, 2024, 9:47 PM

#

Ask me again in 6 months. It's still on my to-do list for mistral lora training with my mmo game data collection x.x

sullen plover Feb 18, 2024, 1:53 AM

#

omg so cool open source thanks guys and @halcyon quarry

halcyon quarry Feb 18, 2024, 3:01 AM

#

Pushed a neat update for some (at least for me 🤓 )

Enhancement for the ReActor extension integration.
https://github.com/Gourieff/sd-webui-reactor
The face_swap tag parameter previously accepted only an image filename (.png or .jpg or .txt in base64).
Now it may also be a directory (choose random image), or a .safetensors face model.

#

And just a quick FYI - any A1111 extensions that work via its API can be controlled by this bot (not just ReActor and Controlnet), using the 'payload' tag parameter. It would just be a bit verbose, but any payload params can be adjusted from tags including alwayson_scripts (A111 extension params)

halcyon quarry Feb 18, 2024, 3:26 AM

#

Example... verbose way to set the parameters for ReActor

halcyon quarry Feb 18, 2024, 12:27 PM

#

Of course some extensions won’t function without input generated by the bot… open to suggestions for more extension integrations

vestal python Feb 18, 2024, 1:25 PM

#

I haven't tried yet. Whisper STT in voice channel. Is that doable?

halcyon quarry Feb 18, 2024, 1:35 PM

#

I haven't looked into it yet, but from what I've read the extension wants a button to be pressed to activate the recording mode, pressed again to finish.
Sounds a bit tricky to simulate that... I imagine we'd want to keep the recording method active, but not send until a minimum value of text is captured and a trigger phrase, or X duration elapses after any text is captured, then process it and open up the listening again... need to look into that.

vestal python Feb 18, 2024, 1:41 PM

#

Sounds like a hassle yeah :/

halcyon quarry Feb 18, 2024, 1:42 PM

#

I have many ideas to implement, just need to get around to that one 😛 May try that sooner than later

sullen plover Feb 18, 2024, 2:14 PM

#

@halcyon quarry i have it all running nice but wondering about it set to use cpu only as it is only using 32% cpu on the main node, also hes running slow my ai agent but very clever, maybe i should leave the setting as he works really good just slow

vestal python Feb 18, 2024, 2:17 PM

#

How slow is slow? I have 3 each different models, hardware with various speeds of 25,7, and 0.5 t/s

halcyon quarry Feb 18, 2024, 2:17 PM

#

more cpu will always be slower

#

Try smaller model 🤗

sullen plover Feb 18, 2024, 2:19 PM

#

ty, is there no command on the end i can add like --cpu-only or something?

halcyon quarry Feb 18, 2024, 2:19 PM

#

Er, if you are using gguf model, could try testing different params

sullen plover Feb 18, 2024, 2:20 PM

#

oh so when i run that i can go locally and use the web ui same time?

halcyon quarry Feb 18, 2024, 2:20 PM

#

I don’t think you can use my bot with webui at same time

#

You can pass the params as arguments when launching the bot (—nlayers X, —threads X, etc)

sullen plover Feb 18, 2024, 2:21 PM

#

ah i see is that mentioned on the read me on github?

halcyon quarry Feb 18, 2024, 2:22 PM

#

You can find the best params using the webui

#

Then just pass those when launching the bot

vestal python Feb 18, 2024, 2:23 PM

#

I'm starting to like GGUF with llamacpp more than exllama. Older Pascal cards work better with it.

sullen plover Feb 18, 2024, 2:24 PM

#

oh you got any models i am using neural beagle 7b it is really clever best i have found so far

sullen plover Feb 18, 2024, 2:28 PM

#

halcyon quarry Then just pass those when launching the bot

might leave as it is, it is so clever right now i dont want to mess with it ^_^

halcyon quarry Feb 18, 2024, 2:41 PM

#

Also using same model now myself

#

It definitely is less shy with responses for nsfw prompts, and as you said clever in general

halcyon quarry Feb 18, 2024, 4:16 PM

#

Pushed a quick update to add a 'random' parameter to the Tags system.
If 'random' is included (0.0 = 0% chance / 1.0 = 100% chance), it will ignore the tag if it fails an RNG check.

#

The more you use it, the more efficient the bot runs 🏄

halcyon quarry Feb 19, 2024, 12:45 AM

#

Next feature… should be quite easy,,, a tag parameter for ‘llm_state_variations’ to behave like the current img_param_variations

#

You specify an int or float range for whatever params, and it picks a value for each param

halcyon quarry Feb 19, 2024, 3:24 AM

#

will finish it tomorrow... realized the img variances thing was a bit scuffed.
Also expanded the function to accept booleans like 'hr_enable:' true or 'hr_enable:' [true, false] (it will return true or false 50/50 for either syntax).
Also to choose a random string value from a list like 'some_param: ['somevalue1', 'anothervalue', 'whoa_anotherone']

halcyon quarry Feb 19, 2024, 6:48 PM

#

params that are nested in subdictionaries will also work for param_variances

#

This would pick 1, 2 or 3 at random for clip skip

halcyon quarry Feb 19, 2024, 7:54 PM

#

LLM param variances now working as well (ignore the thing I crossed off).

halcyon quarry Feb 19, 2024, 8:23 PM

#

Pushed this update about the Param Variances

It's actually super cool, and I recommend everyone check it out deeper.

You can add all sorts of variation now, include picking random samplers, turning HR fix on/off, and so on and so forth

#

And as with all things, these can be defined in a tag preset, or per imgmodel, per character, etc.

vestal python Feb 19, 2024, 9:11 PM

#

Trying out this a1111 Forge. So far a noticable 10% it/s increase.

halcyon quarry Feb 19, 2024, 9:11 PM

#

Only downside is does not support loractl extensions atm

#

Otherwise, yes everything else works with the bot same as A1111 🙂

#

From what I can see, the Forge team is working to make the loractrl functionality standard in their app

vestal python Feb 19, 2024, 9:12 PM

#

I guess I really fail at half of the image generation knowledge :/ I have a hard enough time figuring it out on my RTX 2080 super 8GB card

halcyon quarry Feb 19, 2024, 9:14 PM

#

I feel like I exist on both ends of the spectrum. I'm a serious artist that likes to take the easiest way with the best results.
Too lazy to sit down and use loractl syntax, but ambitious enough to write a thing that'll do it for me automatically

#

Too lazy to write my own prompts, but keep building a thingymajig to do it for me better 😛

#

https://github.com/lllyasviel/stable-diffusion-webui-forge/issues/68

vestal python Feb 19, 2024, 9:37 PM

#

Hmm, also something different about this... Idk the generations seem less accurate on forge it's a weird anecdote.\

halcyon quarry Feb 19, 2024, 9:38 PM

#

they may have different defaults

#

Pushed an update to allow as many 'tags' definitions as you want in config.py for the "API Method" of ImgModel handling

#

(such as multiple 'tag_preset_name' tags)

#

This might sound stupid but I swear I'm on my way to adding an "auto-prompter" feature, which will use a new dynamic prompting feature (similar to the A1111 extension).

sullen plover Feb 19, 2024, 10:03 PM

#

correct? using cpu only right now: python bot.py --loader llama --model NeuralBeagle14-7B-GGUF --extension LLM_Web_search --auto-devices --cpu

#

bot is not using much cpu 35% max so far

halcyon quarry Feb 19, 2024, 10:26 PM

#

Looks good - I need to look into that web search extension

#

Like I said before you could also try playing whackamole with some cpu params, there is a sweet spot with nlayers, threads n whatnot

#

Systematically try different combos soon enough you’ll narrow in on the one with best tokens/sec

sullen plover Feb 20, 2024, 12:52 AM

#

halcyon quarry Like I said before you could also try playing whackamole with some cpu params, t...

I haven't got a clue lol only thanks to you and others I am even able to run this stuff 🥳 I don't know what other commands I could do to make the bot more premium.

halcyon quarry Feb 20, 2024, 12:53 AM

#

read the notes 🤓

halcyon quarry Feb 20, 2024, 3:24 AM

#

Pushed an update that enhances the "Guess Imgmodel Data" for Imgmodels API method

#

Now the guess matching isn't limited to just 'filesize'.
'filter' and 'exclude' have been added to compare against the filename of the selected imgmodel.

This addition makes it possible to configure separate SDXL Turbo and SDXL 1.0 params when using the API method

#

I'm also doubling down on the whole Tags concept - the code samples now use tags for the payload params.
Listing the params still works (for now) but that is being deprecated.

#

config.py growing again 😄

halcyon quarry Feb 20, 2024, 3:09 PM

#

Bloody hell... I need to walk this latest commit back a bit.

The /image command menus rely on the width and height params at minimum, and running tag matches for this would be super clunky

halcyon quarry Feb 20, 2024, 4:02 PM

#

First 'bad commit' in awhile, man I was on a roll

halcyon quarry Feb 20, 2024, 4:34 PM

#

Pushed an update to fix things.

Notably:
-The 'width' and 'height' are required to be called out separately from 'tags' in config.py, as well as dict_imgmodels.yaml
-The checkpoint name is required to be called out separately from 'tags' in dict_imgmodels.yaml (for YAML method)

Beyond that, I've updated the example settings to handle imgmodel params as I recommend. It is much easier to switch them around by using the tag presets.

sullen plover Feb 20, 2024, 4:44 PM

#

oh my nice, i just updated i think it broke maybe you updates again and i need to update

#

line 39 config.py

#

ignore me it is ok now my mistake i think

halcyon quarry Feb 20, 2024, 4:56 PM

#

I wouldn't be surprised if it was my mistake though 🙂 But yeah line 39 is probably you 🙂

charred lake Feb 20, 2024, 5:02 PM

#

Is there a way to load a character on launch? —character doesn’t seem to do anything

halcyon quarry Feb 20, 2024, 5:02 PM

#

Screenshot showing my dict_imgmodels.yaml (I prefer the YAML method).
And my dict_tags.yaml
The width and height in the 'tags' are redundant though... but this is a nice way to apply the settings, and have sensible values for param variances

halcyon quarry Feb 20, 2024, 5:03 PM

#

charred lake Is there a way to load a character on launch? —character doesn’t seem to do anyt...

The last character used is the one which will be loaded

sullen plover Feb 20, 2024, 5:03 PM

#

my character auto loads it seems

halcyon quarry Feb 20, 2024, 5:03 PM

#

If it fails to find the character (you changed the filename, etc), just use /character command which is populated by chars in the characters folder

sullen plover Feb 20, 2024, 5:03 PM

#

it chooses my fav character zero everytime

#

i think your code has secret ai inside ^_^ premium

halcyon quarry Feb 20, 2024, 5:05 PM

#

I've got many trigger based image LORA type stuff configured, but beyond that I'm running the same crap with some higher step counts

#

I work from two locations, I git pull myself constantly lol

halcyon quarry Feb 20, 2024, 5:11 PM

#

sullen plover it chooses my fav character zero everytime

You have that LLM_Web_search extension working? It works well?

sullen plover Feb 20, 2024, 5:14 PM

#

halcyon quarry You have that LLM_Web_search extension working? It works well?

not sure i have not got anything working properly besides your bot code and the main app, most likely down to me not knowing much but doing stuff anyways, i am doing this all for fun i love it i am sure most of you do too ^_^

sullen plover Feb 20, 2024, 5:14 PM

#

halcyon quarry You have that LLM_Web_search extension working? It works well?

i think it does not work

halcyon quarry Feb 20, 2024, 5:24 PM

#

Thanks 🙂 Still going to take a peak at it, could be very nice if it does what it claims to

sullen plover Feb 20, 2024, 5:49 PM

#

halcyon quarry Thanks 🙂 Still going to take a peak at it, could be very nice if it does what ...

yes you are pro you prob get it working fast, please let me know post here when you get time 🙂

vestal python Feb 20, 2024, 5:55 PM

#

Here I am setting up my 2nd box Windows..This is not fun I miss my Ubuntu. Do you all use Windows for your AI needs?

halcyon quarry Feb 20, 2024, 5:57 PM

#

Windows only guy here. I'd used Ubuntu once on a dedicated HTPC notetop for the performance edge, but it was brutal for me

vestal python Feb 20, 2024, 6:19 PM

#

Ok so the P40 + GTX 1080ti are both being seen in nvidia-smi and working with pytorch now without any errors.. Just going to see about getting it to work with a 5B Mixtral model :/ Windows works the same almost, it just looks soo foreign to me for such tasks.

sullen plover Feb 20, 2024, 7:01 PM

#

vestal python Here I am setting up my 2nd box Windows..This is not fun I miss my Ubuntu. Do yo...

linux mint, cant use windows pc only likes linux lost bios pass etc

sullen plover Feb 20, 2024, 7:17 PM

#

INFO [2024-02-20 19:14:34]: Initializing in chat mode (Line: 899 in on_ready, bot.py )
INFO [2024-02-20 19:14:34]: Trying to load character "Coding AI"... (Line: 774 in load_chat, bot.py )
INFO [2024-02-20 19:14:34]: Initializing with character "Coding AI". Use "/character" for changing characters. (Line: 778 in load_chat, bot.py )
WARNING [2024-02-20 19:14:35]: ** No extension params for this character. Reloading extensions with initial values. ** (Line: 756 in update_extensions, bot.py )
19:14:35-009156 INFO Loading the extension "gallery"
INFO [2024-02-20 19:14:35]: Loading the extension "gallery" (Line: 33 in load_extensions, extensions.py )
INFO [2024-02-20 19:14:35]: Bot is ready (Line: 913 in on_ready, bot.py )
WARNING [2024-02-20 19:15:31]: ** No extension params for this character. Reloading extensions with initial values. ** (Line: 756 in update_extensions, bot.py )
19:15:31-485655 INFO Loading the extension "gallery"
INFO [2024-02-20 19:15:31]: Loading the extension "gallery" (Line: 33 in load_extensions, extensions.py )
ERROR [2024-02-20 19:15:32]: Error while changing character username or avatar: 400 Bad Request (error code: 50035): Invalid Form Body
In username: Too many users have this username, please try another. (Line: 470 in update_client_profile, bot.py )
INFO [2024-02-20 19:15:32]: Loaded new character: "Zero". (Line: 2133 in callback, bot.py )

#

it wont load my character zero for some reason

halcyon quarry Feb 20, 2024, 7:18 PM

#

Frankly, I'm not sure why it let you ever load it with a super popular name

#

The error is because bot names are registered as usernames, and discord limits the number of users with the same exact username

sullen plover Feb 20, 2024, 7:19 PM

#

oh ok i will change thanks i was lost lol

halcyon quarry Feb 20, 2024, 7:19 PM

#

np! I wanted to use Minty but had to settle with M1nty

#

same reason

vestal python Feb 20, 2024, 8:52 PM

#

Yeah my Ecne_Bot and Ogma_Bot

At least they respond without the bot part

halcyon quarry Feb 20, 2024, 9:11 PM

#

Pushed a small-ish update
Since before the dawn of time, there was a feature in this bot where a message about the time was arbitrarily added before every prompt.
Later, I made it configurable but it was still clunky.
Tags... again, ported it to this but it was still clunky.

Now, just write {time} anywhere and the time will be swapped in.

If you want it before every message, use the 'instruct' tag... for now.
- instruct: 'it is now {time}.\n{}'

halcyon quarry Feb 20, 2024, 10:52 PM

#

I noticed the LLM response can’t trigger an image response using Tags. That’s the next item on the list, and will try to make those processes more flexible…

vestal python Feb 21, 2024, 1:49 AM

#

My 2nd box is finally finished with ad_discord bot working with command:

python bot.py --api --listen --loader llama.cpp --model dolphin-2.7-mixtral-8x7b.Q4_K_M.gguf --n_ctx 20000 --n-gpu-layers 90 --threads 8 --tensor_split 19,9 --numa --row_split

#

It was interesting to try to figure out since he's all Pascal GPU...

#

9 t/s on Pascal vs. 25 t/s on my ampere version

halcyon quarry Feb 21, 2024, 1:52 AM

#

How much vram?

#

I haven't tried running any mixtral models yet, I probably should try...

vestal python Feb 21, 2024, 1:55 AM

#

Well shoot he's not right in the head yet...

#

halcyon quarry Feb 21, 2024, 1:56 AM

#

lmao

vestal python Feb 21, 2024, 1:57 AM

#

My ampere model with Ecne uses command:
python bot.py --api --listen --loader exllamav2_hf --model LoneStriker_dolphin-2.7-mixtral-8x7b-4.0bpw-h6-exl2 --max_seq_len 22768 --num_experts_per_token 2 --gpu-split 6,10,10

With alltalk working.

#

Ogma, my Pascal model uses command:
python bot.py --api --listen --loader llama.cpp --model dolphin-2.7-mixtral-8x7b.Q4_K_M.gguf --n_ctx 20000 --n-gpu-layers 90 --threads 8 --tensor_split 19,9 --numa --row_split

And reaches..:

#

So that's about 30GB for 4Bit Mixtral ?

#

the GGUF model shows it's more like 4.56 Bit

halcyon quarry Feb 21, 2024, 2:00 AM

#

I won't be testing Mixtral 🤓 with my pleb 4070ti

vestal python Feb 21, 2024, 2:01 AM

#

It's been quite the little venture to setup yeah :/

#

fixed Ogma, Dolphin is based on 16k context, and I tink having it to 20k was causing issues? Reducing it to 16k seems to have fixed the crazy maybe

vestal python Feb 21, 2024, 3:04 AM

#

I need to start from scratch for the Discord bot for this second one. It's still doing the repeated text after 2nd prompt. VSCode 'continue' extension is working good for coding with at least with the openai api on ooba.

halcyon quarry Feb 21, 2024, 3:05 AM

#

Have you used this bot for coding at all? just wondering because it is capable of outputting code very nicely

#

Well, as best as possible given the 2000 character message limit. It has a very good message chunking function... which finds the best places to split lines, and looks for discord code markdown in order to correctly reformat any split chunks

vestal python Feb 21, 2024, 3:08 AM

#

So, Ecne as Mixtral model under your discord bot program

He has been used for coding assisting in rAthena decently from my 2 devs. Using the /cont option helps alot with their requirements. Usually just helping with an example NPC script, and asking it to write a new NPC following the format, or events/mobs and such.

halcyon quarry Feb 21, 2024, 3:09 AM

#

The /cont function was bugged when I picked up this bot, I also use it from time to time

vestal python Feb 21, 2024, 3:10 AM

#

I use Ecne at work alot with just simple python scripts. I'm part cyber security, and handle alot of website issues. I've had Ecne build me a number of free python scripts for simplified repeaters, scrap, SQL checks, etc.

#

Telegram was worthless unfortunately. Discord, the way it handles the text definitely helps with minor coding.

halcyon quarry Feb 21, 2024, 3:12 AM

#

Did a quick google on what Ecne is - my head is spinning after 5 seconds of reading

#ad_discordbot (Fork of Fork of xNul's bot)