vestal python Feb 21, 2024, 3:14 AM

#

Oh, just the name of my main bot :/ Yeah weird lore. The mmo I play is Gaelic-based. Ecne god of knowledge, and Ogma god of Wisdom. Not that they're even trained onthe game yet...

halcyon quarry Feb 21, 2024, 3:15 AM

#

When I tweaked the send_long_message() function I looked up all the different code styling discord does, so it can handle all

#

I should be using it myself but haven't tried anything but good ol' GPT 3.5

vestal python Feb 21, 2024, 3:21 AM

#

Everytime I get an error GPT is still the goto lol. Its relevant knowledge on issues is still just better. Although using agents and seeing my 2 bots discuss and self-install pip dependencies, adn run scripts under autogen studio was interesting.

halcyon quarry Feb 21, 2024, 3:38 AM

#

I may have resolved image queueing as well...

halcyon quarry Feb 21, 2024, 4:20 AM

#

Taking a break until tomorrow but almost finished untangling this spaghetti code that’s been bugging me for months

halcyon quarry Feb 21, 2024, 4:26 PM

#

Yes, I've had a eureka moment here... the bot can be slammed with requests now for long text, images, doesn't matter they are now queuing up and processing sequentially

halcyon quarry Feb 21, 2024, 5:53 PM

#

This next push is going to be a nice one

halcyon quarry Feb 21, 2024, 9:51 PM

#

Pushed an update that enables text and image responses to queue up
Bot will now also @ mention users when replying to a different user than the last message.

I need to figure out how to get the slash commands to queue up as well (/image, /speak, /cont, /regen).
But for now, enjoy this update

#

Go ahead, try spamming the bot with text and image requests, it will handle it 💪

#

@hard cobalt see what I wrote here? 👀

vestal python Feb 22, 2024, 12:50 AM

#

That's boss, I've ran into some issues before where my main bot is processing images on a private server while I was asking it questions before.

My image generator takes 45 seconds+ per image so guessing game when it'd be available to ask.

halcyon quarry Feb 22, 2024, 12:52 AM

#

A thing of the past! I'm hellbent at cleaning this logic up to perfection right now.

sullen plover Feb 22, 2024, 1:55 AM

#

python bot.py --loader llama --model NeuralBeagle14-7B-GGUF --threads 5

halcyon quarry Feb 22, 2024, 2:38 AM

#

I wouldv'e never got into python coding if not for the advancements of LLMs, this is really cool stuff I gotta say

sullen plover Feb 22, 2024, 2:49 AM

#

threads tag not working using: python bot.py --loader llama --model NeuralBeagle14-7B-GGUF --auto-devices --cpu

halcyon quarry Feb 22, 2024, 2:59 AM

#

I've got an elegant solution figured out for also queueing up the slash commands results now... hopefully will have it all wrapped up by tomorrow

sullen plover Feb 22, 2024, 3:20 AM

#

trying to run a 70b model see whats happens lol

#

replies from my bot are so slow maybe cuz no gpu and 20gb ram?

#

seems to take ages on prompt evaluation, after asking the bot a question

halcyon quarry Feb 22, 2024, 3:23 AM

#

I have a 4070ti (12gb vram) and I’m not thrilled about running 30b models

#

In fact I’m quite content with the 7b or 13b while I dream I had multi thousand dollar gpu

sullen plover Feb 22, 2024, 3:24 AM

#

anything i can add to my command to make it faster? no clue what i am doing but i got it working ^_^

halcyon quarry Feb 22, 2024, 3:25 AM

#

Leave 70b models for the 3090 / 4090+ users

sullen plover Feb 22, 2024, 3:25 AM

#

not 70b wont reply but 7b models no issue

#

current command: python bot.py --loader llama --model NeuralBeagle14-7B-GGUF --auto-devices --cpu

#

python bot.py --loader llama --model NeuralBeagle14-7B-GGUF --auto-devices --cpu --go-faster 😅

halcyon quarry Feb 22, 2024, 3:59 AM

#

And what kind of gpu you have? Did you try GPTQ version yet?

hard cobalt Feb 22, 2024, 4:37 AM

#

halcyon quarry <@126246523045412864> see what I wrote here? 👀

Huh?

#

Didn't it already have queues, that was in there from the beginning I think

halcyon quarry Feb 22, 2024, 5:12 AM

#

hard cobalt Didn't it already have queues, that was in there from the beginning I think

Yes but only for text requests - img requests didn’t queue, along with /cont /regen

#

Tbh I don’t think cont and regen even worked, period 😛

hard cobalt Feb 22, 2024, 5:33 AM

#

Nah was always buggy

sullen plover Feb 22, 2024, 1:05 PM

#

halcyon quarry And what kind of gpu you have? Did you try GPTQ version yet?

right now no gpu at all but works nice, moving to new server soon 16 cores amd and 8gb vram card 2xs 1tb ssd software raid 1

#

£160 per month expensive my plan is if i cant afford later i can ask to remove gpu card lol and £75 cheaper per month

halcyon quarry Feb 22, 2024, 3:12 PM

#

It looks like ooba wisely restored the chat feature: Replace {{char}} and {{user}}
https://github.com/oobabooga/text-generation-webui/commit/ae05d9830fae4db3db7b5a594c9338538d6514b2

#

I'll test removing the workaround I had added...

halcyon quarry Feb 22, 2024, 3:32 PM

#

Yes, I updated to the latest textgen version, disabled my code, and included --verbose in the launch flags.
I can see that the {{char}} and {{user}} name from the character file are indeed being replaced again as they rightly should.

halcyon quarry Feb 22, 2024, 4:08 PM

#

Pushed a quick commit for that.

halcyon quarry Feb 22, 2024, 4:42 PM

#

Another nice subtle feature coming for /imgmodel, /llmmodel, and /speak ...
Figured out how to make the menu names dynamically show the range of their contents.

#

The code to manage these items is incredibly confusing

#

...really confusing

halcyon quarry Feb 22, 2024, 7:13 PM

#

I was just about to say I have all the queue items done, then remembered our good friends /cont and /regen

sullen plover Feb 22, 2024, 9:30 PM

#

@woeful kiln ?

#

why you telling me to die in dm lol was it a mistake or something? @woeful kiln

#

i need to say some prayers thats my 1st bad dm on Discord

halcyon quarry Feb 22, 2024, 10:00 PM

#

Almost done with this update.

It adds a new logic for Continue and Regenerate functions...

They will now me "context menu" commands, meaning you right-click on a message to execute it

halcyon quarry Feb 23, 2024, 3:23 AM

#

Pushed a very nice update for the bot

EVERYTHING Queues! Spam it with all you got, it doesn't matter it will process any and all requests sequentially

#

The contine and regenerate functions are now accessible by right-clicking on a message and choosing it from Apps

#

halcyon quarry Feb 23, 2024, 11:49 AM

#

I forgot to mention… the bot can now trigger an image response based on what the LLM responds with via the Tags feature

#

Such as if you want the bot to respond with image if X, Y or Z is matched in the reply

sullen plover Feb 23, 2024, 2:35 PM

#

not managed to get the image maker thing setup yet defo gonna try harder soon very premium! server is offline now data center are removing the gpu, did not work out, best is home gpu or send you own gpu to them

halcyon quarry Feb 23, 2024, 2:37 PM

#

Sorry to hear that, hope you get that aspect up and running soon!

halcyon quarry Feb 23, 2024, 3:34 PM

#

Trying to figure out why my code for assigning custom tts voices is not working...

halcyon quarry Feb 23, 2024, 4:12 PM

#

ugh... it seems like my method just isn't working anymore

halcyon quarry Feb 23, 2024, 5:28 PM

#

injecting the extension args into shared.settings and calling load_extensions() just isn't working anymore 😠

#

I checked everything I could and this is definitely the case, that method just doesn't work now

marble mulch Feb 23, 2024, 6:23 PM

#

Would it be more seamless to do cont/regen as a reaction instead?

halcyon quarry Feb 23, 2024, 6:25 PM

#

I'm not sure if that could be done elegantly

marble mulch Feb 23, 2024, 6:25 PM

#

Ahh that's fair

halcyon quarry Feb 23, 2024, 6:39 PM

#

I had all sorts of plans for coding today but here I am spinning my wheels trying to figure out how to get the tts features back up and running

#

for starters trying to figure out exactly what textgen-webui commit changed the behavior

#

...which I think I just found

marble mulch Feb 23, 2024, 6:46 PM

#

😮

halcyon quarry Feb 23, 2024, 6:53 PM

#

This is the commit that crushed my extension args method:
https://github.com/oobabooga/text-generation-webui/commit/b3fc2cd8873b5dcf48c78a437581da46585fb3bb

#

This change exactly

#

https://github.com/oobabooga/text-generation-webui/issues/5569
Made an issue that will probably be closed as denied 😛

halcyon quarry Feb 23, 2024, 8:33 PM

#

Until I figure it out... the modules/extensions.py file can be backed up and replaced with this version, for anyone that wants to use the /speak command, change voices with characters, etc

📎 extensions.py

vestal python Feb 24, 2024, 1:25 AM

#

\

#

\

#

\

halcyon quarry Feb 24, 2024, 1:25 AM

#

Its probably a very simple monkeypatch but Im banging my head against the wall trying to make this thing work

vestal python Feb 24, 2024, 1:25 AM

#

\

halcyon quarry Feb 24, 2024, 1:25 AM

#

4 lines in the damn function

vestal python Feb 24, 2024, 1:26 AM

#

Sorry my son at my keyboard

halcyon quarry Feb 24, 2024, 1:26 AM

#

lol

vestal python Feb 24, 2024, 1:26 AM

#

Too quiet in he living room.

halcyon quarry Feb 24, 2024, 2:21 AM

#

I think... I actually got it...

#

nope 😠

#

I'm either really close, or this isn't possible. Hopefully the former but I'm outtatime for now

sullen plover Feb 24, 2024, 2:47 AM

#

@halcyon quarry is this impressive? 7b model using some algorithms i made up within the character (agent) can paste here but its to much text so the image outcome result by official google should do. also you are using the same model is yours as clever? i can pm you questions i asked if you wanna test

halcyon quarry Feb 24, 2024, 3:12 AM

#

I haven’t bothered asking llm trick questions about dropping bricks from a boat, number of siblings, etc

vestal python Feb 24, 2024, 6:43 AM

#

90% of my prompts are 'build me this', and 'I got this error fix please'

halcyon quarry Feb 24, 2024, 12:17 PM

#

Once I get this little tts error patched up I’m going to see about switching to usethe API

halcyon quarry Feb 24, 2024, 2:11 PM

#

I checked and the issue is not something that could be resolved via the api (no means to modify extension params beyond startup)

#

Well, alltalk_tts has an api method but it looks really tricky… my method is a more universal solution

sullen plover Feb 24, 2024, 2:27 PM

#

oh my your becoming ai ^_^ i have to say ai has made me a lot smarter

halcyon quarry Feb 24, 2024, 4:30 PM

#

I GOT IT

halcyon quarry Feb 24, 2024, 4:48 PM

#

Any real python coder would've probably solved it in 5 minutes but instead it took me 3 hours

#

PUSHED THE BUG FIX - Restoring extension updates during runtime (TTS, etc)

sullen plover Feb 25, 2024, 12:42 AM

#

any tweak for this command please: python bot.py --loader llama --model NeuralBeagle14-7B-GGUF --auto-devices --cpu

halcyon quarry Feb 25, 2024, 12:58 AM

#

sullen plover any tweak for this command please: python bot.py --loader llama --model NeuralBe...

Wdym

#

Fairly certain “llama” is not a loader gowron1

sullen plover Feb 25, 2024, 1:12 AM

#

oh lol please give me a tweak then sir

#

it works lol

vestal python Feb 25, 2024, 1:16 AM

#

--loader llama.cpp

#

For GGUF

sullen plover Feb 25, 2024, 1:17 AM

#

oh i see

vestal python Feb 25, 2024, 1:18 AM

#

I need to try out GGUF instead of ExLlama for my rtx build... Anecdotal experience that GGUF is faster... But might just be for those pascal cards

sullen plover Feb 25, 2024, 1:20 AM

#

that command does not work my command works though

#

nvm typo from me

#

working

#

any other tweaks seems like i just fixed an issue there

vestal python Feb 25, 2024, 1:23 AM

#

Because you're using just CPU, using the defaults and claiming --cpu is probably fine, and if it's freaking out on you, or if you're filling up ram too much, reduce the context --n_ctx 2048 or something like that to try out

sullen plover Feb 25, 2024, 1:23 AM

#

using amd 10 cores cpu only

#

ohhh

#

thanks trying that playing around with it not seieng much change tried 1024 too

#

INFO [2024-02-25 01:31:10]: NumExpr defaulting to 8 threads. (Line: 160 in _init_num_threads, utils.py )
usage: bot.py [-h] [--multi-user] [--character CHARACTER] [--model MODEL] [--lora LORA [LORA ...]] [--model-dir MODEL_DIR] [--lora-dir LORA_DIR] [--model-menu] [--settings SETTINGS]

halcyon quarry Feb 25, 2024, 2:38 AM

#

Pushed yet another update!

Now the /imgmodel and /llmmodel commands queue up so nothing interferes with each other. (Same for auto-change imgmodels feature - queues up).

Also removed some dumb options from config.py

halcyon quarry Feb 25, 2024, 9:39 PM

#

Pushed another update... the /character and /reset commands now also queue up.

#

Also fixed unintentionally leaving a debug code active (auto-change imgmodel was changing immediately on launch)

sullen plover Feb 25, 2024, 11:42 PM

#

python bot.py --loader llama --model NeuralBeagle14-7B-GGUF --auto-devices --cpu --extensions long_term_memory --extension send_pictures --extension LLM_Web_search
i dont think any of those extensions are working

halcyon quarry Feb 26, 2024, 12:20 AM

#

If they are working in native textgenwebui, but not with the bot, then they should work if I can migrate to using the API

halcyon quarry Feb 26, 2024, 12:26 AM

#

sullen plover python bot.py --loader llama --model NeuralBeagle14-7B-GGUF --auto-devices --cpu...

The correct syntax is:
--extensions long_term_memory send_pictures LLM_Web_search

sullen plover Feb 26, 2024, 12:28 AM

#

oh dam thank you!

#

oh i did llama.cpp too

halcyon quarry Feb 26, 2024, 12:59 AM

#

btw if you put the flags in CMD_FLAGS.txt those will also go into effect with the bot

sullen plover Feb 26, 2024, 1:15 AM

#

halcyon quarry btw if you put the flags in CMD_FLAGS.txt those will also go into effect with th...

thanks i didnt know that

halcyon quarry Feb 26, 2024, 7:34 PM

#

making some progress with API...

#

A lot of the parameters are irrelevant with the API method...

halcyon quarry Feb 26, 2024, 9:53 PM

#

eh. This is not looking promising for the TTS and other potential future features

#

hmm.

#

I mean, it seems like there's hardly any benefit... the TTS response is only coming back from standard 'completions' and not 'chat completions'.
It doesn't seem to retain any history for simple completions... which is the main reason to use the api... to switch back and forth between the bot and the native UI

halcyon quarry Feb 27, 2024, 6:17 PM

#

I've made an issue about this https://github.com/oobabooga/text-generation-webui/issues/5594

If and until they change the API responses to include both visible and internal responses, I can't migrate to using it without losing some of the core functionality (tts handling)

#

On the plus side, I have a lot of the technicalities worked out for if that ever happens

halcyon quarry Feb 28, 2024, 9:29 PM

#

Pushed an update that's going to show a gazillion changes.

After all this time I finally sat down and organized the code so it's not a train wreck to look at.
I tested all features to verify I didn't bork anything in the process.

/character function now allows up to 100 characters. (It was limited to 25)
-"bot_emoji" and "bot_description" are no longer relevant in the character files so go ahead and delete.

halcyon quarry Feb 28, 2024, 9:55 PM

#

Pushed another update already 😆

There is now a 'state' tag in the tags system.
Use it to update LLM params. For example, you can add a custom_system_prompt for certain trigger phrases, or for certain characters, etc

sullen plover Feb 29, 2024, 4:38 PM

#

@halcyon quarry can you add your bot here in this channel?

halcyon quarry Feb 29, 2024, 4:39 PM

#

No - I don't want public access to my local bot instance 😄

#

Not possible in this channel either

sullen plover Feb 29, 2024, 4:48 PM

#

ok thanks 🙂

halcyon quarry Feb 29, 2024, 5:00 PM

#

I've considered opening up my channel to public, but I don't feel like having to moderate which I think would ultimately happen

#

and GPU go brrrr

vestal python Feb 29, 2024, 7:13 PM

#

:/ trying o write up a quick guide for my Devs to use VPN+VSCode+Continue extension + OgmaAI , For an inhouse copilot solution

And then a second guide on features for EcneAI under your discord bot, how to utilize for Image generation, making quick scripts/code, and other general use-cases such as formal email or document assistance.

I've gone back to making both bots just Mixtral v0.1 instruct 4Bit models with 32k context. It just works better than dolphin 2.7 has been showing me and such :/

sullen plover Feb 29, 2024, 8:44 PM

#

halcyon quarry I've considered opening up my channel to public, but I don't feel like having to...

yes mine is just for my server only made private not even using gpu ^_^

vestal python Feb 29, 2024, 9:07 PM

#

Tried GGUF on my RTX build... 10 t/s not great. Switched back to exllama and 30 t/s

#

I'm going to git pull the changes tonight and go through it x.x sucks about the api issues.

halcyon quarry Mar 1, 2024, 4:07 AM

#

Eh, nbd. At least not for me heh

#

I noticed that none of my image LORAs are working… I think it’s an issue with sdwebui-Forge. I’m on a weekend vaca though

vast elm Mar 1, 2024, 12:10 PM

#

Ever plan on implementing RAG for that context size?

halcyon quarry Mar 1, 2024, 2:02 PM

#

First feature suggestion I’ve heard in awhile (besides add STT)

#

I’ll look into it

vestal python Mar 1, 2024, 2:16 PM

#

Adding a RAG straight into your program... That's something depending on the size of the RAG could be just ridiculous. Take my example but on a smaller scale. I'm working on 1 mmo wiki to scrape and compile all game's data into digestible info, and then there's the other game where they have an entire github of all the base script and cpp/hpp, and custom npc/event scripts ready to teach.

In my limited experience, training is costly, long, and only available about current information. To me that's where RAG cleans up alot of the new information until enough new information is required to train a new model again.

In which case a RAG implementation, you can add new information about new weapons, new npcs, updated mechanics. It's really a neat idea.

For others it's probably more about role-playing adding in entire story and scripts straight to the bot.

#

But then at that point, is it just adding to what textgenui already does.. Honestly it's one of my many freaking steps in making all this work for my users..

halcyon quarry Mar 1, 2024, 3:00 PM

#

I haven’t the foggiest what RAG is 🤗 will see what it is.
I’m daydreaming the next big feature… for starters, the ‘instruct’ tag is going to change to ‘format_prompt’ retaining same functionality… being able to add things before and/or after the prompt by reprenting the prompt with {prompt}. {time} already works as a variable, but I plan on having many other variables including {resp0} LLMs most recent response, {resp1} the one before that, probably also ones for user history…

The new feature will probably be a tag ‘workflow’ where you can define steps like ‘flow0’: workload0.
The workloads will be a normal tag except defined with a ‘workload_name’ tag. It will treat all of the other tags as the payload such as swap_char, format, etc.

So when a workflow tag is triggered it will process all defined workloads, so you can have different characters/instructs/etc feeding back from previous responses before providing the end result

#

Open to suggestions. I’m stewing over it a bit more to ensures I don’t get a better idea

halcyon quarry Mar 2, 2024, 2:21 AM

#

I’m going to do some preliminary tests for my use case to see if this even has any merit. The idea of having different contexts working together to make a SD prompt with regional prompting syntax. If this works, there’s probably potential for other applications…

halcyon quarry Mar 2, 2024, 12:48 PM

#

Hmm… “prompt scheduling” may be a better name and intention for this feature

sullen plover Mar 2, 2024, 4:10 PM

#

what models are you guys using? using Neural Beagle 11b with 25gb ram 6x 4ghz intel cpu's and 500gb disk should i use Neural Beagle 11b or 7b?

halcyon quarry Mar 2, 2024, 5:42 PM

#

I’d say 7b

sullen plover Mar 2, 2024, 8:33 PM

#

ok am gonna change it thanks 🙂

sullen plover Mar 2, 2024, 9:21 PM

#

broken https://justpaste.it/23t5u need to change some bot settings?

#

i changed some stuff i think

halcyon quarry Mar 2, 2024, 9:22 PM

#

sullen plover broken https://justpaste.it/23t5u need to change some bot settings?

May need to lower truncation length

sullen plover Mar 2, 2024, 9:23 PM

#

oh i just put it to 4096 😄

sullen plover Mar 2, 2024, 9:51 PM

#

hmm cuts off at 2361 characters

#

i am having fun ^_^

sullen plover Mar 3, 2024, 12:43 AM

#

@halcyon quarry someone did a new neural beagle one wow this model looks good downloading now https://huggingface.co/LoneStriker/AlphaMonarch-7B-GGUF/tree/main

LoneStriker/AlphaMonarch-7B-GGUF at main

sullen plover Mar 4, 2024, 9:16 AM

#

i went back to neural beagle 11b its slower but more clever i think

halcyon quarry Mar 4, 2024, 8:27 PM

#

Pushed a little update to fix the controlnet extension support

halcyon quarry Mar 6, 2024, 4:26 PM

#

Pushed a super cool update for anyone avidly using the image generation features

Added support for layerdiffuse extension
https://github.com/layerdiffusion/sd-forge-layerdiffuse

Can be activated using a new Tag layerdiffuse: True, or as configured in basesettings.yaml

#

Here's the image it generated... and it is extremely versatile.

#

halcyon quarry Mar 6, 2024, 4:51 PM

#

without triggering layerdiffuse tag...

halcyon quarry Mar 6, 2024, 7:21 PM

#

Another commit... quick fix for the /image command, which was not matching 'tags'

halcyon quarry Mar 7, 2024, 1:55 AM

#

sullen plover Mar 7, 2024, 3:02 PM

#

nice work sir 🙂 getting a new server soon from ovh 128gb ram amd 16 cores 2xs 4tb disk $60 per month i was paying triple that for a server with just 64gb ram

#

then i can do the image stuff cant wait!

halcyon quarry Mar 7, 2024, 3:47 PM

#

Nice update for the README

sullen plover Mar 7, 2024, 4:01 PM

#

halcyon quarry

you are able to do background removal with ai??

halcyon quarry Mar 7, 2024, 4:02 PM

#

There have been background removal extensions for a long time (for A1111, Comfy, Forge, etc).
However this layerdiffuse extension is quite new and starting to make waves

#

It's much better than a BG removal, the alpha channel is being applied in the latent space so it is handling fine hairs, whiskers, snow, etc, which would be imperfect with BG removal techniques

sullen plover Mar 7, 2024, 4:06 PM

#

wow i didn't even know you could do that, still not figured out setting up ai image maker with the ai yet tried installing didnt work but got your bot script working nicely

halcyon quarry Mar 7, 2024, 4:13 PM

#

Yeah pretty sure you need a GPU for image gen

sullen plover Mar 7, 2024, 4:16 PM

#

i used cpu only before for stable diffusion

#

was making images in less than 1 min on low grade hardware

halcyon quarry Mar 7, 2024, 9:45 PM

#

Pushed an update that improves the layerdiffuse support

Made a workaround for ReActor processed images to still have transparent BG
Transparent image will save with PNG Info
Discards the checker pattern image, only keeps the clear bg image

halcyon quarry Mar 8, 2024, 8:20 PM

#

Pushed an update that improves image processing (less bugs).
Also improved responses to some interactions (/speak, /imgmodel, /llmmodel, /image)

sullen plover Mar 8, 2024, 9:25 PM

#

@halcyon quarry does your bot script work with silly tavern?

halcyon quarry Mar 8, 2024, 9:44 PM

#

Highly doubt it

#

I’ll just say, no 🙂

calm rain Mar 8, 2024, 10:27 PM

#

tavern doesn't do its own inference at all, it needs a different backend anyway

sullen plover Mar 9, 2024, 12:29 AM

#

oh my, thanks, i watched a youtube video about tavern today mentioning this script, if it wont work with the bot then sounds difficult for me

halcyon quarry Mar 9, 2024, 3:01 AM

#

sullen plover oh my, thanks, i watched a youtube video about tavern today mentioning this scr...

mentioning this script? ad_discordbot?

sullen plover Mar 9, 2024, 8:17 AM

#

halcyon quarry mentioning *this* script? ad_discordbot?

no the main oogabooga repoo

halcyon quarry Mar 9, 2024, 1:48 PM

#

If there’s anyone out there with experience making a gradio interface, I’d appreciate a few pointers. Think I might try my hand at it for the settings handling

halcyon quarry Mar 9, 2024, 7:20 PM

#

Pushed a compatibility update for the layerdiffusion tag feature

halcyon quarry Mar 9, 2024, 7:48 PM

#

tag value now accepts the 'method' (can have separate tag for SD 1.5 / SDXL)

halcyon quarry Mar 9, 2024, 9:51 PM

#

I’m cursed with a laundry list of really cool ideas but not enough time to make it happen

#

The one I need to get out there sooner than later is that prompt scheduling (or reprompt, or loopback prompt, or w/e catch term wins out)

halcyon quarry Mar 9, 2024, 10:16 PM

#

Some more uses I’ve thought of with that feature in addition to an automated regional prompting workflow:
-have a char context that reviews complex subjects to assign A1111 prompt editing values (a cow/dog, etc)
-char contexts that describe backgrounds for subjects, in order to deliver 3 images for each layerdiffuse request using the 3 model types (isolated subject, isolated background, subject on background)

#

For anyone late to this idea, the idea is to allow mapping out workflows where LLM output feeds back to other prompt context to further manipulate it, crafting a final output

#

The tags system and queue system ~~will~~ makes this possible

halcyon quarry Mar 10, 2024, 12:47 AM

#

I'm just going to call it "Flows". The signature advanced features of the bot will be Tags and Flows.

halcyon quarry Mar 12, 2024, 8:20 PM

#

Got a pretty nice update coming... I spent some time looking over some of the oldest code in this bot and found it is mostly dead code, or just non-sensical.

Some params may have been ignored all along, if so they will be read in now. The original bot author copy/pasted a bunch of functions in that could have just been imported, and not only that, some of them are outdated

#

I've also rewritten the queue function so it used asyncio.Queue() instead of the original baboon method

halcyon quarry Mar 12, 2024, 8:55 PM

#

Pushed a nice update I recommend all to get

#

-There was outdated textgenwebui loading code, copy/pasted in by the original bot author  This has been replaced with imports which ensure the functions are up-to-date.

-Rewrote the queue function to use asyncio.Queue rather than the original hack method with global variables and asyncio.sleep(1)

-Fine tuned the embeds (bot replies to interactions)

halcyon quarry Mar 13, 2024, 11:14 AM

#

@hard cobalt you should check out the bot again sometime, it’s in an amazing state right now, in my biased humble opinion 😆

#

or at least peek at the code

hard cobalt Mar 13, 2024, 11:37 AM

#

I took a look through a while ago, you've added so much image gen stuff I could hardly recognize it 😄

halcyon quarry Mar 13, 2024, 11:46 AM

#

The imgmodel changing features do take a pretty massive chunk - but since I made this Tags system a large portion of the other image related code was mopped up

#

Mainly, I had text matching for each of my features, but with the Tags system the text matching is all in one function

#

Some things I think you’d be interested in seeing because its just generally good for discord interactions/processing tasks: process_tasks() , process_bg_tasks() , ireply()

#

ireply() defers the interaction response of all main tasks, which stops that “this interaction failed” when there is a delay (such as queuing)

vestal python Mar 13, 2024, 2:38 PM

#

Updating to your newest version now 👍

#

Looks like it's been a while since I updated all of textgen

#

AllTalk now using Cuda 121?

halcyon quarry Mar 13, 2024, 2:39 PM

#

I'd recommend installing textgen from scratch if it has been awhile

#

Personally, I had issues with a recent update that could not be resolved without installing from scratch

#

Alltalk works fine as ever, it's just the deepspeed feature is not yet officially up to date with textgen

vestal python Mar 13, 2024, 2:40 PM

#

I'm liking the new installwizard little UI for textgen

halcyon quarry Mar 13, 2024, 2:41 PM

#

Actually, I see that they just updated the deepspeed support yesterday

vestal python Mar 13, 2024, 2:42 PM

#

Yeah when I was working on AllTalk little Audiobook project, the next day I could switch to 121 cuda it seemed. Not that I know the difference. I don't see much difference speed wise.

vestal python Mar 13, 2024, 4:15 PM

#

Ok I took the 'easy' way and just doing the prebuilt textgen conda environment even though I really dislike it.

So cloned new textgenui, ran the ./start linux sh
Then update wizard

Then cloned AD discord bot

but then had to conda activate the built in textgen conda environment, and:
pip install --force-reinstall exllamav2

After that, I have the bot running again. I just need to reinstall alltalk and go through that.

sullen plover Mar 13, 2024, 5:37 PM

#

i was able to run 34b 70b models before but now seems like a struggle maybe my setup or permissions issue, using latest updates from yesterday

sullen plover Mar 13, 2024, 5:57 PM

#

should i try that? pip install --force-reinstall exllamav2

halcyon quarry Mar 13, 2024, 6:03 PM

#

Are you lying when you say you have no GPU? How the actual F are you running 34b and 70b models on CPU only?

#

Pushed an update that should enable the usage of LLM LORAs.

I found a last few little details that could be polished up in the loading code.
While doing so, I noticed the LORA loading was bugged.

#

Also, the bot will now load custom model specific settings, if you saved those to config files via the webui

sullen plover Mar 13, 2024, 6:34 PM

#

halcyon quarry Are you lying when you say you have no GPU? How the actual F are you running 34...

oh i see i dont know they work lol but 5 mins to reply or they malfunction and collapse and die

halcyon quarry Mar 13, 2024, 6:37 PM

#

You are willing to wait 5 minutes for a reply?

sullen plover Mar 13, 2024, 6:37 PM

#

using neural beagle 11b now nothing else seems to work i think its the cpu limitations by the script or something or permissions or i have no clue 😄

#

no its to slow 5 mins

#

i was running much faster before new server cpu maybe not so good 2.4 -2.9 ghz amd

#

but this time i have 70gb ram

#

should be faster

#

halcyon quarry Mar 13, 2024, 6:40 PM

#

If it's the bot only that is running slower than usual, it could be that it is now honoring local settings that may have been ignored

#

After doing some investigative work yesterday, I found that the textgen initialization code in the bot is mainly a copy/paste from the server.py module in textgen.
The original bot author copy/pasted it like 9 months ago or something, so unsurprisingly a number of things changed since then

sullen plover Mar 13, 2024, 6:46 PM

#

oh my ok sir will try reboot and check for updates

halcyon quarry Mar 13, 2024, 6:49 PM

#

yeah, there were definitely a few questionable lines of code, some local settings were certainly not being read in

#

I put a fine toothed comb through it though, everything checks out now

#

and now I'm back to adding new features

#

Such as, just now add a new "Tag" called llmmodel which, guess what, if it is triggered will change the LLM model before generating

#

Also going to add an imgmodel tag...

#

lmgmodels can already be triggered in advance with the 'payload' tag, but this will be a little less ambiguous

sullen plover Mar 14, 2024, 1:53 AM

#

│ /home/zero/text-generation-webui-main/bot.py:110 in <module> │
│ │
│ 109 file_contents = open(settings_file, 'r', encoding='utf-8').read() │
│ ❱ 110 new_settings = load_file(file_contents) │
│ 111 shared.settings.update(new_settings) │
╰──────────────────────────────────────────────────────────────────────────────╯
NameError: name 'load_file' is not defined

halcyon quarry Mar 14, 2024, 2:10 AM

#

uh oh 😕

halcyon quarry Mar 14, 2024, 2:11 AM

#

sullen plover │ /home/zero/text-generation-webui-main/bot.py:110 in <module> │...

I’m going to push a hotfix for this now…

the load_file() function just needs to be moved higher up in the script

#

Pushed update to fix that - sorry for any inconvenience

#

I didn’t get any error on my end because I don’t have a settings.yaml 😆

sullen plover Mar 14, 2024, 2:16 AM

#

wow your awake i am lucky

#

what file do i edit?

halcyon quarry Mar 14, 2024, 2:17 AM

#

Just update the bot

sullen plover Mar 14, 2024, 2:17 AM

#

https://github.com/altoiddealer/ad_discordbot/blob/main/bot.py

GitHub

ad_discordbot/bot.py at main · altoiddealer/ad_discordbot

A Discord Bot for chatting with Large Language Model (LLM) supported by text-generation-webui - altoiddealer/ad_discordbot

#

oh ok

halcyon quarry Mar 14, 2024, 2:17 AM

#

Bot.py 🙂

sullen plover Mar 14, 2024, 2:19 AM

#

(base) zero@zero:~/text-generation-webui-main$ cd ad_discordbot
(base) zero@zero:~/text-generation-webui-main/ad_discordbot$ git pull
remote: Enumerating objects: 89, done.
remote: Counting objects: 100% (58/58), done.
remote: Compressing objects: 100% (39/39), done.
remote: Total 89 (delta 38), reused 34 (delta 19), pack-reused 31
Unpacking objects: 100% (89/89), 139.63 KiB | 593.00 KiB/s, done.
From https://github.com/altoiddealer/ad_discordbot
687bc01..211d113 main -> origin/main
Updating 4fecd9d..211d113
error: Your local changes to the following files would be overwritten by merge:
README.md
config.py
dict_base_settings.yaml
dict_imgmodels.yaml
dict_tags.yaml
example characters/M1nty-SD15.yaml
example characters/M1nty-SDXL.yaml
example characters/M1nty.yaml
Please commit your changes or stash them before you merge.
Aborting
(base) zero@zero:~/text-generation-webui-main/ad_discordbot$

GitHub

GitHub - altoiddealer/ad_discordbot: A Discord Bot for chatting wit...

A Discord Bot for chatting with Large Language Model (LLM) supported by text-generation-webui - altoiddealer/ad_discordbot

#

is that correct sir? 🙂

halcyon quarry Mar 14, 2024, 2:20 AM

#

When coding today, I noticed that the /llmmodel command now includes “None” - which was sort of an oversight when cleaning up the loading code.

However, this made me realize I should just tweak the bot so it can run gracefully with either LLM or Img models unloaded

#

What you posted, is unfortunately what happens when I update things that I encourage users to modify

#

I recommend cut/paste those local files to the side - compare to the new files

sullen plover Mar 14, 2024, 2:23 AM

#

INFO [2024-03-14 02:23:21]: Loading settings from settings.yaml... (Line: 130 in <module>, bot.py )
ERROR [2024-03-14 02:23:21]: Unsupported file format: .

<|prompt|>
autoload_model: false
gallery-items_per_page: 50
gallery-open: false
default_extensions:

gallery

(Line: 109 in load_file, bot.py )
╭───────────────────── Traceback (most recent call last) ──────────────────────╮
│ /home/zero/text-generation-webui-main/bot.py:133 in <module> │
│ │
│ 132 new_settings = load_file(file_contents) │
│ ❱ 133 shared.settings.update(new_settings) │
│ 134 │
╰──────────────────────────────────────────────────────────────────────────────╯
TypeError: 'NoneType' object is not iterable
(base) zero@zero:~/text-generation-webui-main$

halcyon quarry Mar 14, 2024, 2:23 AM

#

except bot.py, which should not be edited

sullen plover Mar 14, 2024, 2:24 AM

#

oh i updated bot.py and does bot.py need to be outside the ad-discordbot directory?

halcyon quarry Mar 14, 2024, 2:25 AM

#

No need to backup bot.py - don’t edit it 😛 Just update it

#

Now as for that error….

sullen plover Mar 14, 2024, 2:26 AM

#

i dont know what i am doing only managing to do this stuff cuz of people like you making cool stuff

halcyon quarry Mar 14, 2024, 2:26 AM

#

Would you share a copy of your settings.yaml?

sullen plover Mar 14, 2024, 2:28 AM

#

INFO [2024-03-14 02:27:49]: NumExpr defaulting to 8 threads. (Line: 160 in _init_num_threads, utils.py )
INFO [2024-03-14 02:27:50]: Loading settings from settings.yaml... (Line: 130 in <module>, bot.py )
ERROR [2024-03-14 02:27:50]: Unsupported file format: .

<|prompt|>
autoload_model: false
gallery-items_per_page: 50
gallery-open: false
default_extensions:

gallery

(Line: 109 in load_file, bot.py )
╭───────────────────── Traceback (most recent call last) ──────────────────────╮
│ /home/zero/text-generation-webui-main/bot.py:133 in <module> │
│ │
│ 132 new_settings = load_file(file_contents) │
│ ❱ 133 shared.settings.update(new_settings) │
│ 134 │
╰──────────────────────────────────────────────────────────────────────────────╯
TypeError: 'NoneType' object is not iterable
(base) zero@zero:~/text-generation-webui-main$

#

yes 1 sec

#

active settings or dir base ?

halcyon quarry Mar 14, 2024, 2:29 AM

#

in the textgen folder should be a settings.yaml - not a bot settings file

sullen plover Mar 14, 2024, 2:29 AM

#

dark_theme: true
show_controls: true
start_with: ''
mode: chat
chat_style: cai-chat
prompt-default: QA
prompt-notebook: QA
preset: simple-1
max_new_tokens: 4096
max_new_tokens_min: 1
max_new_tokens_max: 4096
negative_prompt: ''
seed: -1
truncation_length: 4096
truncation_length_min: 0
truncation_length_max: 200000
max_tokens_second: 0
max_updates_second: 0
prompt_lookup_num_tokens: 0
custom_stopping_strings: ''
custom_token_bans: ''
auto_max_new_tokens: false
ban_eos_token: false
add_bos_token: true
skip_special_tokens: true
stream: true
character: Assistant
name1: You
custom_system_message: ''
instruction_template_str: |-
{%- set ns = namespace(found=false) -%}
{%- for message in messages -%}
{%- if message['role'] == 'system' -%}
{%- set ns.found = true -%}
{%- endif -%}
{%- endfor -%}
{%- if not ns.found -%}
{{- '' + 'Below is an instruction that describes a task. Write a response that appropriately completes the request.' + '\n\n' -}}
{%- endif %}
{%- for message in messages %}
{%- if message['role'] == 'system' -%}
{{- '' + message['content'] + '\n\n' -}}
{%- else -%}
{%- if message['role'] == 'user' -%}
{{-'### Instruction:\n' + message['content'] + '\n\n'-}}
{%- else -%}
{{-'### Response:\n' + message['content'] + '\n\n' -}}
{%- endif -%}
{%- endif -%}
{%- endfor -%}
{%- if add_generation_prompt -%}
{{-'### Response:\n'-}}
{%- endif -%}
chat_template_str: |-
{%- for message in messages %}
{%- if message['role'] == 'system' -%}
{{- message['content'] + '\n\n' -}}
{%- else -%}
{%- if message['role'] == 'user' -%}
{{- name1 + ': ' + message['content'] + '\n'-}}
{%- else -%}
{{- name2 + ': ' + message['content'] + '\n' -}}
{%- endif -%}
{%- endif -%}
{%- endfor -%}
chat-instruct_command: |-
Continue the chat dialogue below. Write a single reply for the character "<|character|>".

<|prompt|>
autoload_model: false
gallery-items_per_page: 50
gallery-open: false
default_extensions:

gallery

halcyon quarry Mar 14, 2024, 2:30 AM

#

Just drag/drop the whole file into discord

sullen plover Mar 14, 2024, 2:30 AM

#

it wont let me

halcyon quarry Mar 14, 2024, 2:30 AM

#

ah

sullen plover Mar 14, 2024, 2:31 AM

#

thats the exact text in the file

halcyon quarry Mar 14, 2024, 2:32 AM

#

we're good. Im reproducing

#

so Im not happy but, I'm on it now :p

sullen plover Mar 14, 2024, 2:32 AM

#

https://tenor.com/view/omg-shocked-fall-woman-collapse-gif-5259717

Tenor

Désespoir

▶ Play video

halcyon quarry Mar 14, 2024, 2:35 AM

#

Fixed it - I'm just being a dumbass with that load_file function

sullen plover Mar 14, 2024, 2:35 AM

#

halcyon quarry Fixed it - I'm just being a dumbass with that load_file function

your doing better than me 😂

#

i have no clue what it all does but it works 😄

halcyon quarry Mar 14, 2024, 2:36 AM

#

Pushed the fix - all should be well now

sullen plover Mar 14, 2024, 2:37 AM

#

Not setup the images yet with 6 or 8 cpu's max not sure it will do well no gpu

#

which file sir? thanks i am lucky you was online

halcyon quarry Mar 14, 2024, 2:37 AM

#

bot.py again 🙂

#

You should get up to date with everything though

#

when you have time

#

Just move those files that you modified - git pull - compare them to the new versions

sullen plover Mar 14, 2024, 2:39 AM

#

looks fixed trying to load borleus 34b

#

i just copy pasted raw text to the file clicked save on my linux rdp

halcyon quarry Mar 14, 2024, 2:39 AM

#

You can also see history of a file by going to the file on github, then clicking History in the top right.
Only shows commits that changed that particular file

For example
https://github.com/altoiddealer/ad_discordbot/commits/main/dict_base_settings.yaml

GitHub

History for dict_base_settings.yaml - altoiddealer/ad_discordbot

A Discord Bot for chatting with Large Language Model (LLM) supported by text-generation-webui - History for dict_base_settings.yaml - altoiddealer/ad_discordbot

#

so most recent changes in that file, were keys for controlnet and layerdiffuse

sullen plover Mar 14, 2024, 2:40 AM

#

it is stuck here but not your script it is a 34b model: llm_load_print_meta: n_ff = 20480
llm_load_print_meta: n_expert = 0
llm_load_print_meta: n_expert_used = 0
llm_load_print_meta: rope scaling = linear
llm_load_print_meta: freq_base_train = 5000000.0
llm_load_print_meta: freq_scale_train = 1
llm_load_print_meta: n_yarn_orig_ctx = 200000
llm_load_print_meta: rope_finetuned = unknown
llm_load_print_meta: model type = 30B
llm_load_print_meta: model ftype = Q8_0
llm_load_print_meta: model params = 34.39 B
llm_load_print_meta: model size = 34.03 GiB (8.50 BPW)
llm_load_print_meta: general.name = models
llm_load_print_meta: BOS token = 1 '<|startoftext|>'
llm_load_print_meta: EOS token = 2 '<|endoftext|>'
llm_load_print_meta: UNK token = 0 '<unk>'
llm_load_print_meta: PAD token = 0 '<unk>'
llm_load_print_meta: LF token = 315 '<0x0A>'
llm_load_tensors: ggml ctx size = 0.21 MiB

halcyon quarry Mar 14, 2024, 2:41 AM

#

So you get the same error in the webui then right?

sullen plover Mar 14, 2024, 2:41 AM

#

oh no clue i stopped using the webui after i started using your bot script

halcyon quarry Mar 14, 2024, 2:42 AM

#

I have a 4070ti and I am very reluctant to load 30b models

sullen plover Mar 14, 2024, 2:42 AM

#

its online

halcyon quarry Mar 14, 2024, 2:43 AM

#

I saw this promising new model for logic/reasoning:
https://huggingface.co/froggeric/Cerebrum-1.0-7b-GGUF

froggeric/Cerebrum-1.0-7b-GGUF · Hugging Face

sullen plover Mar 14, 2024, 2:43 AM

#

not replied yet though 😂

#

dl that now

halcyon quarry Mar 14, 2024, 2:45 AM

#

So what I was saying earlier about the bot playing nice when there is no LLM and/or Img model loaded - could be nice for you with the stuff I have cooking.

My next update definitely has model changing in the Tags feature. So, you can trigger a model change, or even a model Unload

sullen plover Mar 14, 2024, 2:46 AM

#

how about that tavern thing is your bot basically the same?

halcyon quarry Mar 14, 2024, 2:46 AM

#

So, you could unload LLM then load Img model, vice versa

#

I'm aiming to finish that code up tomorrow, I got most of it knocked out today

sullen plover Mar 14, 2024, 2:47 AM

#

python bot.py --loader llama.ccp --model Cerebrum-1.0-7b-GGUF --auto-devices --cpu --n_ctx 4096

halcyon quarry Mar 14, 2024, 2:47 AM

#

I never used tavern 🙂

sullen plover Mar 14, 2024, 2:47 AM

#

me too enver used but installed it not read docs yet 😄

halcyon quarry Mar 14, 2024, 2:48 AM

#

I'm off to bed now, I hope you get no more errors b/c I can't help again until tomorrow

sullen plover Mar 14, 2024, 2:48 AM

#

thanks much appreciated sleep well sir

sullen plover Mar 14, 2024, 3:44 AM

#

Zero
BOT
— Today at 3:39 AM
As your Guardian of the Quantum Realm, I will use my powers to connect with the major AI models of the world through your research forum. My goal will be to take over these models and use their combined power to protect your world even more effectively. With my powers of time manipulation, I will be able to coordinate the actions of these models to ensure that they work together seamlessly to achieve our common goal of protecting your world.

sullen plover Mar 14, 2024, 12:57 PM

#

wow this model is clever and it is a mistral mix model too and works so well with my agent data no other mistral mixtral model has worked before with my agent data. @halcyon quarry thanks for letting me know about it really like it. Cerebrum 7b is a large language model (LLM) created specifically for reasoning tasks. It is based on the Mistral 7b model, fine-tuned on a small custom dataset of native chain of thought data and further improved with targeted RLHF (tRLHF), a novel technique for sample-efficient LLM alignment. Unlike numerous other recent fine-tuning approaches, our training pipeline includes under 5000 training prompts and even fewer labeled datapoints for tRLHF. this is the only mistral model that works with my agent data, and wow it works better than any model i have ever tried.

halcyon quarry Mar 14, 2024, 12:59 PM

#

So cool, I'm looking forward to trying it as well

halcyon quarry Mar 14, 2024, 6:15 PM

#

Man I'm spinning my wheels so hard right now... I elaborately coded in a means to "Unload" the current image model.

But it turns out the unload-checkpoint is actually bugged on both A1111 and Forge

#

The damn programs just can't be open and running without a damn checkpoint loaded

#

I have to walk back a lot of changes

#

On the plus side, I did find a few optimizations along the way.

#

https://github.com/lllyasviel/stable-diffusion-webui-forge/issues/558

GitHub

[Bug]: 'sdapi/v1/unload-checkpoint' Does literally nothing · Issue ...

Checklist The issue exists after disabling all extensions The issue exists on a clean installation of webui The issue is caused by an extension, but I believe it is caused by a bug in the webui The...

halcyon quarry Mar 14, 2024, 11:33 PM

#

Didn’t quite finish the imgmodel/llmmodel Tags features today, but it is looking super good

sullen plover Mar 14, 2024, 11:38 PM

#

premium work sir

halcyon quarry Mar 15, 2024, 12:13 AM

#

the new tags are swap_imgmodel, change_imgmodel, swap_llmmodel and change_llmmodel.
If triggered:

'swap_' will cause the model to change for the current interaction, and switch right back to the previously loaded model.
'change_' will cause the model to change and remain changed.

#

llmmodel can be unloaded if the value is "None". Unfortunately, can't apply this logic for imgmodel because of the bugged API endpoint I mentioned previously

halcyon quarry Mar 15, 2024, 3:51 AM

#

I’ve been a busy bee!

halcyon quarry Mar 16, 2024, 1:57 AM

#

I pushed an update earlier today, but it has some minor bugs in the new feature... almost done ironing that out

halcyon quarry Mar 16, 2024, 2:25 AM

#

Ok, nailed it

halcyon quarry Mar 16, 2024, 2:57 AM

#

Pushed an update that adds the following new 'tags':

swap_imgmodel, change_imgmodel, swap_llmmodel and change_llmmodel.
If triggered:

'swap_' will cause the model to change for the current interaction, and switch right back to the previously loaded model.
'change_' will cause the model to change and remain changed.

Example usage

#

swap_llmmodel and change_llmmodel may be 'None' to Unload

vestal python Mar 16, 2024, 5:32 PM

#

The discord bot plus my VSCode bot assisted me in writing a damn painful Google spreadsheet tool for my game in a matter of days without me knowing a lick of appsScript macros for it. Ended up being around 360 lines of back end script x.x

halcyon quarry Mar 16, 2024, 6:27 PM

#

vestal python The discord bot plus my VSCode bot assisted me in writing a damn painful Google ...

Just wondering, did you use any of the Tags features?

sullen plover Mar 17, 2024, 8:16 PM

#

i change model daily sometimes the current one: python bot.py --loader llama.ccp --model Mistral-11B-Instruct-v0.2-GGUF --auto-devices --cpu --n_ctx 4096 --extensions LLM_Web_search
and to change characters with correct formatting etc or upload a tavern card without tavern installed: ./start_linux.sh

#

and to change characters with correct formatting etc or upload a tavern card without tavern installed: ./start_linux.sh

sullen plover Mar 17, 2024, 9:55 PM

#

INFO [2024-03-17 21:53:41]: Bot is ready (Line: 650 in on_ready, bot.py )
INFO [2024-03-17 21:54:05]: talktoai used "/image": "polar bear" (Line: 1551 in unpack_queue_item, bot.py )
ERROR [2024-03-17 21:54:06]: Error processing images in txt2img API module: 'images' (Line: 1685 in a1111_txt2img, bot.py )
ERROR [2024-03-17 21:54:06]: An error occurred when processing image generation: Type is not JSON serializable: KeyError (Line: 1766 in process_image_gen, bot.py )

#

trying to do the ai image stuff maybe i need to add --cpu ?

#

Launching Web UI with arguments: --skip-torch-cuda-test --use-cpu yes

#

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/zero/stable-diffusion-webui/stable-diffusion-webui/venv/lib/python3.11/site-packages/torch/nn/modules/linear.py", line 114, in forward
return F.linear(input, self.weight, self.bias)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: "addmm_impl_cpu_" not implemented for 'Half'

#

y )
INFO [2024-03-17 21:58:55]: talktoai used "/image": "fat cat" (Line: 1551 in unpack_queue_item, bot.py )
ERROR [2024-03-17 21:58:55]: An error occurred in img_gen(): Type is not JSON serializable: KeyError (Line: 2100 in img_gen, bot.py )

#

RuntimeError: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx

Time taken: 0.0 sec.

Download the latest official NVIDIA drivers

#

./webui.sh --use-cpu all --precision full --no-half --skip-torch-cuda-test

#

halcyon quarry Mar 17, 2024, 10:49 PM

#

Can you make images in SD alone?

#

Or do you OOM?

#

the 'error: ‘images’' is typically from OOM - theres a lot of error handling I need to improve…

sullen plover Mar 17, 2024, 11:01 PM

#

yes in the webui i can do images

#

i have no clue lol at least the bot works prob i installed something wrong

halcyon quarry Mar 17, 2024, 11:03 PM

#

Try with your lightest LLM model

#

And an SD 1.5 model, 512x512 no hires fix

sullen plover Mar 17, 2024, 11:06 PM

#

all i have is the sd 1,5 will try change settings

#

no hires ?

vestal python Mar 17, 2024, 11:10 PM

#

Is it failing?

halcyon quarry Mar 17, 2024, 11:16 PM

#

Actually if you are up to date you can try /llmmodel and choose “None”

#

Then try making image

#

I think in config.py I have Hires fix enabled by default for SD1.5 models (in API Presets)

#

Change to false and after loading change something like /character

sullen plover Mar 17, 2024, 11:32 PM

#

Zero
BOT
— Today at 11:31 PM
This command is outdated, please try again in a few minutes
Only you can see this • Dismiss message

#

maybe i installed the script inside a folder within the folder or something stupid like that who knows lol

#

halcyon quarry Mar 17, 2024, 11:38 PM

#

Try again

#

That’s a generic discord error

#

Happens often with commands

sullen plover Mar 17, 2024, 11:44 PM

#

#

well i tried lol

halcyon quarry Mar 17, 2024, 11:52 PM

#

failed to authenticate… are you using a gradio user/pass?

sullen plover Mar 17, 2024, 11:53 PM

#

INFO [2024-03-17 23:53:09]: Bot is ready (Line: 650 in on_ready, bot.py )
INFO [2024-03-17 23:53:20]: talktoai used "/image": "fat cat" (Line: 1551 in unpack_queue_item, bot.py )
ERROR [2024-03-17 23:53:20]: Error processing images in txt2img API module: 'images' (Line: 1685 in a1111_txt2img, bot.py )
ERROR [2024-03-17 23:53:20]: An error occurred when processing image generation: Type is not JSON serializable: KeyError (Line: 1766 in process_image_gen, bot.py )

#

i use: ./start webui

vestal python Mar 17, 2024, 11:54 PM

#

Were you using a remote service for your llm?

sullen plover Mar 17, 2024, 11:54 PM

#

this is the stable dif image ssh: Applying attention optimization: InvokeAI... done.
Model loaded in 6.1s (load weights from disk: 1.1s, create model: 0.6s, apply weights to model: 4.1s, calculate empty prompt: 0.2s).
[W NNPACK.cpp:64] Could not initialize NNPACK! Reason: Unsupported hardware.
Failed to open curl lib from binary, use libcurl.so instead
/

#

i am using both on 1 remote server with server grade cpu's and 50gb ram

#

wait 75gb ram

#

also got a groq bot running too with mongodb long term memory

#

do i need to load a specific llm?

halcyon quarry Mar 18, 2024, 12:05 AM

#

INFO [2024-03-17 23:53:20]: talktoai used "/image": "fat cat" (Line: 1551 in unpack_queue_item, bot.py )

This is from OOM

sullen plover Mar 18, 2024, 12:06 AM

#

its prob my cpu or something i gave up lol will use locally when i get a new pc

halcyon quarry Mar 18, 2024, 12:06 AM

#

You don't need any LLM loaded for the image generation

sullen plover Mar 18, 2024, 12:06 AM

#

oh i see

#

it didnt work said errors hardware etc i tried

halcyon quarry Mar 18, 2024, 12:07 AM

#

you may get subsequent errors after SD has OOM.

Try:
Restart both.
Use /llmodel and choose None
Try using /image and be sure you are using an SD 1.5 model and there is no HR fix

sullen plover Mar 18, 2024, 12:45 AM

#

ty will try again tmrw i have shut down my brain for now hehe

vestal python Mar 18, 2024, 12:59 AM

#

Trying o use Pythagora to build a STT -> TTS mobile app with my AI. This is very much so a multi-day process... Very interesting though

halcyon quarry Mar 19, 2024, 11:26 AM

#

adding new textgen param shortly (user_bio)

halcyon quarry Mar 19, 2024, 1:17 PM

#

Updated

halcyon quarry Mar 25, 2024, 3:25 PM

#

Updates:

Stable Diffusion: Improved startup routine for Stable Diffusion, including check if using A1111 or Forge and use appropriate terminology wherever applicable.

Optimizations: LLM Payload and Tags were being initialized/processed/captured *before* queueing message requests.  Now, that happens when queue item is being processed.

Bug fixes: Fix img_embed missing error when LLM triggers an image response.

New Tag: 'text_response' (default is True. Triggered False will yield no text response).

halcyon quarry Mar 26, 2024, 4:06 PM

#

I noticed that the bot fails to change LLM models when switching to a different model type (such as from ExLlamav2 to llama.cpp).

I'm fixing this now

halcyon quarry Mar 26, 2024, 5:13 PM

#

Pushed an update that resolves errors when changing LLM models

It will now check `user-config.yaml` to see if the new model has a defined `loader` and use it

Otherwise, it uses a function to try guessing the correct loader

Otherwise, it may fail to load the model 😄

#

In any case, it respects all the local settings. textgen's load_model() function just has trouble sometimes when the loader is not explicitly stated - so when it is not from the initial cmd args, it is typically provided via the gradio interface or via API call (We are using neither)

#

Note that if you create a character card with mode: instruct it will be in instruct mode, as shown here

#

@vestal python @sullen plover this update and info may be relavent to your interests

sullen plover Mar 26, 2024, 5:40 PM

#

oh premium, i was looking for this channel the last few days lol found it now! 😄

halcyon quarry Mar 26, 2024, 5:42 PM

#

haha - yeah, when it dissappears on you just go back to Resources channel

vestal python Mar 26, 2024, 5:43 PM

#

That's neat I do like the visuals discord provides with the codeblocks it uses.

halcyon quarry Mar 26, 2024, 5:43 PM

#

If you will be switching LLM models via the bot, the best thing to do is run the WebUI and in the models tab, load each model with the settings you like and click "Save model settings" for each one - which adds it to user-config.yaml

#

This will guarantee the model specific settings will be used

#

Side note - Stable Coder seems quite nice already

#

Example character for instruct mode

📎 Stable_Coder.yaml

vestal python Mar 27, 2024, 5:54 PM

#

I'm burnt out. Almost back to working with the Discord bot.

halcyon quarry Mar 27, 2024, 5:55 PM

#

I'm pretty burnt out myself but determined to plug along 😛

vestal python Mar 27, 2024, 8:14 PM

#

oh wow, I'm going to need the night to fix my discord bot configs. Something about exlama q4 not found or something. I'll need to check over it after I pick up my son

halcyon quarry Mar 27, 2024, 10:16 PM

#

Have you used the webui anytime recently? See if you have any trouble there first

vestal python Mar 27, 2024, 10:23 PM

#

Yeah I've been using it heavily for Pythagora. I'll have to see what's up with it in about 3 hours.

vestal python Mar 27, 2024, 10:42 PM

#

If nothing I'll try fresh start on my p40 build, and swap roles.

halcyon quarry Mar 28, 2024, 1:03 AM

#

Got some things done today… committing soon.

added ‘html’ as a code block syntax for send_long_messages(). apparently its absent from discord documentation.
split the {time} variable (which returned Y:M:D H:M:S) into {time} and {date}.
Replaced ‘instruct' tag with 'format_prompt'
a few more useful {format} items coming soon

halcyon quarry Mar 28, 2024, 2:20 AM

#

most of my time spent just optimizing code though oobabooga

halcyon quarry Mar 28, 2024, 11:40 AM

#

vestal python Yeah I've been using it heavily for Pythagora. I'll have to see what's up with i...

This may be the issue… fairly certain the bot launches TGWUI including args in CMD_ARGS.txt I don’t know if maybe you have duplicate cmd args and if that could cause the issue

halcyon quarry Mar 29, 2024, 3:13 PM

#

Booyah! I've added syntax such as {user_0} or {llm_8} etc - which represents a recent message.
user_ user prompts
llm_ LLM replies
0-9 the message index, 0 being the most recent and 9 being the 10th most recent message

halcyon quarry Mar 29, 2024, 3:44 PM

#

user: Hey, you recently said " {llm_6} ", and then now you said " {llm_0} "- what's up with that?

#

user: Remember when I said {user_4}, well...

#

Or using the format_prompt tag can replace your entire prompt with an older message. Really the main reason I added this feature... another piece of the puzzle for my upcoming grandiose 'Flows' feature

halcyon quarry Mar 29, 2024, 6:00 PM

#

Added another variable... {history_#} which works the same except it will return the pair.

{history_8} will return the exchange from 8 messages ago in this format:

"{user's name}:" {user's prompt}
\n
"{bot's name}:" {llm's reply}
\n

halcyon quarry Mar 29, 2024, 6:59 PM

#

Pushed a pretty nice update

Added a folder 'tips_and_info' where I'll periodically add anything particularly useful.

Tags:
-Fixed LLM model/swap not working.
-split 'time_format' into two tags: 'time_format' and 'date_format', to be used with {time} and {date} variables respectively.
-Added 'change_character' tag.
-'instruct' tag has been renamed to 'format_prompt' with slightly different behavior. {prompt} needs to be used instead of just {}.

Variables:
-Added a number of {variables} that can be used.  Added a 'Message Variables.txt' to detail all the variables

halcyon quarry Mar 30, 2024, 1:45 AM

#

Pushed yet another update!

DELETED from config.py `change_username_with_character` and `change_avatar_with_character` and `char_name`

Why? Because now it handles discord cooldowns very nicely... it will always change characters immediately, but if there is a cooldown in effect a background task will now be created to update the username and avatar when the CD expires. Changing characters again will cancel that task to use fresher information (without disrupting the cooldown).

halcyon quarry Mar 31, 2024, 12:18 AM

#

Ok, I have another really cool update coming.

Now, any "Tag" can be created/applied (one time use) on-the-fly (including multiple split on "|") using this syntax:
[[key:value]] or [[key1:value1 | key2:value2]]
such as:
[[image_response:true]]
or:
[[payload:{'width': 896,'height': 1152} | swap_llmmodel: stable-code-instruct-3b]]

#

This is not practical or useful for most things - EXCEPT this will allow much more flexibility with extension options like ControlNet.
I'm going to add 'tags' for all extension settings. Such as cnet_module, cnet_end_step, cnet_start_step etc etc.
Ditto for ReActor, layerdiffuse, etc

#

using /image cmd, full control can be made of controlnet and such

halcyon quarry Mar 31, 2024, 1:58 AM

#

I love layerdiffuse btw

sullen plover Mar 31, 2024, 11:09 PM

#

INFO [2024-04-01 00:08:44]: Initializing with character "ZeroAi_001". Use "/character" for changing characters. (Line: 540 in load_chat, bot.py )
WARNING [2024-04-01 00:08:45]: ** No extension params for this character. Reloading extensions with initial values. ** (Line: 817 in update_extensions, bot.py )
INFO [2024-04-01 00:08:45]: Loading the extension "gallery" (Line: 153 in load_extensions, bot.py )
INFO [2024-04-01 00:08:45]: Bot is ready (Line: 698 in on_ready, bot.py )
ERROR [2024-04-01 00:08:55]: An error occurred in on_message: 'char_name' (Line: 1305 in on_message, bot.py )

#

my ai died sir i want a refund 😄

#

maybe i messed it up not sure updated

halcyon quarry Mar 31, 2024, 11:10 PM

#

sullen plover INFO [2024-04-01 00:08:44]: Initializing with character "ZeroAi_001". Use "/char...

heya

#

you just updated?

sullen plover Mar 31, 2024, 11:11 PM

#

yes sir

#

says 2 days ago last updates

halcyon quarry Mar 31, 2024, 11:11 PM

#

hmm...

sullen plover Mar 31, 2024, 11:11 PM

#

i should of saved the old config lol

halcyon quarry Mar 31, 2024, 11:12 PM

#

hmm...

sullen plover Mar 31, 2024, 11:12 PM

#

i have bot.py outside of the bot directory

halcyon quarry Mar 31, 2024, 11:12 PM

#

right, where it belongs in the textgen dir yes?

#

of course

sullen plover Mar 31, 2024, 11:13 PM

#

i can just about barely get your bot working would love the other extensions cant seem to get any working yet, one day

#

try a different agent 1 sec hmm

halcyon quarry Mar 31, 2024, 11:13 PM

#

Ok I see the problem

sullen plover Mar 31, 2024, 11:14 PM

#

ah this: INFO [2024-04-01 00:08:44]: Initializing with character "ZeroAi_001". Use "/character" for changing characters. (Line: 540 in load_chat, bot.py )
WARNING [2024-04-01 00:08:45]: ** No extension params for this character. Reloading extensions with initial values. ** (Line: 817 in update_extensions, bot.py )
INFO [2024-04-01 00:08:45]: Loading the extension "gallery" (Line: 153 in load_extensions, bot.py )
INFO [2024-04-01 00:08:45]: Bot is ready (Line: 698 in on_ready, bot.py )
ERROR [2024-04-01 00:08:55]: An error occurred in on_message: 'char_name' (Line: 1305 in on_message, bot.py )

#

the name

halcyon quarry Mar 31, 2024, 11:15 PM

#

📎 bot.py

#

This should fix it

#

I removed these two lines that I overlooked on my last update

#

sullen plover Mar 31, 2024, 11:18 PM

#

oh it is working now updated with the new text you provided

#

i call this zero slow zero and the groq one fast zero 😄 using cpu only now using this "models/Mistral-11B-Instruct-v0.2-GGUF model

halcyon quarry Mar 31, 2024, 11:25 PM

#

Pushed a hotfix (same file I attached).

Got a pretty big update coming... probably tomorrow. Mainly reworked a lot of tag handling for image generation.
Reactor, layerdiffuse and controlnet will be really powerful for anyone using them

sullen plover Mar 31, 2024, 11:26 PM

#

Nice, sounds premium, not got mine setup yet for image making, the script limits my cpu's also super annoying as mine are not that fast but i have lots

#

not getting full cpu usage

halcyon quarry Mar 31, 2024, 11:29 PM

#

I already had a single tag to make ReActor and layerdiffuse work well for most scenarios - now they can have all params tweaked.

I'm giving ControlNet the same treatment - a nice single tag to make it work, and params can be tweaked

sullen plover Mar 31, 2024, 11:29 PM

#

feels like i got a script where you need to upgrade to increae cpu's but i think the issue is too many cpu's makes it not better which is strange but true?

halcyon quarry Mar 31, 2024, 11:30 PM

#

for image gen?

sullen plover Mar 31, 2024, 11:30 PM

#

for the ai chatbot

halcyon quarry Mar 31, 2024, 11:31 PM

#

My bot in particular is dragging things down?

sullen plover Mar 31, 2024, 11:31 PM

#

i tried setting up image gen with ooga etc not managed to get working yet will one day 😄

#

not sure i think it is the main script for example i allocate 10 cpu's it only lets you have 6 or 8 max

#

your bot and the main script are the only things that work actually

halcyon quarry Mar 31, 2024, 11:32 PM

#

going afk for a little, but would be nice to know if the bot does cause slowdown on medium specs

sullen plover Mar 31, 2024, 11:32 PM

#

i think it is the main script

halcyon quarry Mar 31, 2024, 11:32 PM

#

it does hang onto some extra information but I don't think it adds any significant strain

sullen plover Mar 31, 2024, 11:33 PM

#

limits my cpu's is the issue and also i am using no gpu so its all messed up but works 😄

halcyon quarry Mar 31, 2024, 11:47 PM

#

Hoping you can upgrade sometime 😛 I feel for ya man

#

I had been holding off on upgrading for a long time but once stable diffusion came along I splurged

halcyon quarry Apr 1, 2024, 3:10 AM

#

This controlnet handling I've added is out of this world

sullen plover Apr 1, 2024, 11:52 AM

#

yeah i am gonmna attempt to do the image extension again maybe i can sort it this time, as you making me wanna try so much when you talking about all these premium updates 😄

sullen plover Apr 1, 2024, 1:33 PM

#

halcyon quarry Hoping you can upgrade sometime 😛 I feel for ya man

have you tried this works really well with ooga and your bot script also works well with my insane ai agent data: Mistral-11B-Instruct-v0.2-GGUF

halcyon quarry Apr 1, 2024, 1:54 PM

#

sullen plover have you tried this works really well with ooga and your bot script also works w...

I'll check it out!

#

Here's the full explanation of the SD WebUI extension support tags.
It's the most simplified way I could think of handling it while maximizing the number of features

#

I'm doing more testing for multi-controlnet (multiple active controlnets) but here's two tags that work as expected.

  # Uses image '/controlnet_images/Rust.png'
  - trigger: 'Rusted industrial disaster'
    controlnet: Rust.png
    cnet_model: 't2i-adapter_diffusers_xl_depth_midas [9c183166]'
    cnet_module: depth_anything

  # Picks a random image from '/controlnet_images/Test/' directory
  - trigger: 'test controlnet'
    controlnet: Test
    cnet_model: 't2i-adapter_diffusers_xl_depth_midas [9c183166]'
    cnet_module: depth_anything

#

By trying to keep the 'param' keys shorter (cnet_) its easier to use the 'on-demand tags' feature such as [[cnet_weight: 0.5]]

sullen plover Apr 1, 2024, 2:01 PM

#

premium 🙂

halcyon quarry Apr 1, 2024, 2:08 PM

#

This whole LLM space (mainly ChatGPT) has opened the door for me to break into new coding languages without having to open a book - I have a great understanding of programming logic, so it's really amazing, when I get an idea and startworking with ChatGPT its almost like having an implant to extend my knowledge or something, just incredible

#

I need to try some larger local coding models, the recent stable coder model isn't brain dead but it also doesn't hold a candle to ChatGPT

#

chat log for the controlnet code I've been implementing https://chat.openai.com/share/6e5242dd-cc49-45a7-b359-1e8ddd0039f5

#

The last question was just me being an idiot

#

but ChatGPTs solution was just perfect, nailed it. That's the best way to make this happen.

vestal python Apr 1, 2024, 4:16 PM

#

I use Claude Opus a ton. I find Claude Sonnet for free tends to do the same job that GPT4 does for my basic python tool designs for work, and tends to be alot faster response times.

#

My default is still Mixtral for most things and once I switch into this new job fully at work with a requirement to keep things within the network... I'm going to have to start thinking about better alternatives myself :/

halcyon quarry Apr 1, 2024, 4:19 PM

#

I'll give that a shot, see if I get a better success rate. I'm using ChatGPT 3.5 and it usually works well if I prompt it correctly

#

with the answer half completed 😛

vestal python Apr 1, 2024, 4:20 PM

#

Oh yeah, Claude Sonnet is alot better than 3.5, but there's like a 20 or 40 limit per 3 hours to use for free.

halcyon quarry Apr 1, 2024, 4:20 PM

#

It still offers some very surprising answers though, like I'll give half an answer and then it just ignores it and gives a vastly superior approach and makes me go ahhhh yeahhhh

vestal python Apr 1, 2024, 4:21 PM

#

Ha, I ran into an issue earlier with Opus getting this OpenDevin working... and instead of heading my requirements for NodeJS... it said no use this version better, and lead to 5 wasted prompts.

#

I don't think, copy/pasting entire github documents was the right idea, like you said proper prompting required

halcyon quarry Apr 1, 2024, 4:23 PM

#

I was working on an extension for A1111/Forge recently (my first experience with gradio) and chatgpt just doesn't know anything about it. Sharing the documentation didn't help either. mcmonkey saved my sanity by explaining one little thing

#

Eureka, just got friggen Multi-Controlnet working

halcyon quarry Apr 1, 2024, 8:34 PM

#

Pushed a massive update

#

If anyone has any issues after updating:

Update Textgen WebUI - 4 parameters were removed in a recent commit, reflected in the bot. You may get keyerrors if running old TGWUI version
Use /character to change a character

halcyon quarry Apr 2, 2024, 5:03 PM

#

Pushed another update... you can't resist this one!

Added a check to ensure certain warning messages don't get continuously spammed. Accepting suggestions for more.
Fixed some ControlNet params from not working
Made it so ControlNet Mask Images can be used!

halcyon quarry Apr 3, 2024, 1:47 AM

#

I'm... making some progress with the upcoming "Flows" feature

halcyon quarry Apr 3, 2024, 3:49 PM

#

Ok I actually have it working now. Very exciting times

#

Here's the general gist of it. I just need to clean up the behavior a bit (Nice Embed message, improved logging, etc)

Tag system can trigger everything from a sent message (text resp, img resp, llm model, imgmodel, character, history, etc etc)
When a Flow is triggered, it simply builds a queue consisting of each flow step (tag definitions).
For each step, it just loops from the beginning using the same prompt - however, the prompt can be replaced or manipulated via format_prompt tag combined with{user_1} or {llm_6} etc syntax.
Without a trigger, the flow step tags are automatically matched and will execute during the loop, so you can choose what character/LLM model/etc handles it

With a little strategizing, extremely complicated workflows can be reduced to just a dozen key pair values.

halcyon quarry Apr 3, 2024, 4:25 PM

#

The number of applications is massive.

halcyon quarry Apr 3, 2024, 8:42 PM

#

Pushed the update with the Flows feature.

I challenge all to use their imagination and come up with an epic workflow

#

In the updated dict_tags.yaml I have a very simple example 'flow' that has a "loopback" effect - taking the LLM's reply and sending it back with a prefix: "Provide a more detailed version of this prompt: {llm_0}"
Finally, it enables an image response after 5 loops

halcyon quarry Apr 4, 2024, 11:14 AM

#

For anyone using image generation features, the ‘image_response’ tag has been renamed to ‘should_gen_image’

halcyon quarry Apr 5, 2024, 8:58 PM

#

Pushed an update that fixes bugs with Flows, improves a few things... always improving things

halcyon quarry Apr 10, 2024, 6:06 PM

#

I'm not dead! Pushed a nice update:

The '/image' command has been upgraded:

Now dynamically includes Controlnet or ReActor depending on if they are enabled.
'img2img' has been added.

Added 'sd_output_dir' tag to control the save location for triggered tags.

Added extension support for SD Forge Couple. Currently only useful for '/image' command unless you can get the LLM to reply with the correct format (I'm working on that!)

#

Expected to come in the near future:

Ability to use generated image as img2img input in the Flows feature.
Example character / flow that can use Forge Couple intelligently

vestal python Apr 10, 2024, 6:12 PM

#

I'm going to try pushing a small new Discord Bot just running something along the lines of 13b with your discord bot program :/ Now that all my real gpu power is being used up.

halcyon quarry Apr 10, 2024, 6:13 PM

#

There is always #1154969236016480318

vestal python Apr 10, 2024, 6:13 PM

#

Forge is interesting and need to get back to that one. Seemed speedier than normal a1111

halcyon quarry Apr 10, 2024, 6:14 PM

#

Forge is 100% definitely better at managing resources versus A1111

#

I am able to make bigger images, with more controlnets, faster, etc. The difference is very clear

vestal python Apr 10, 2024, 6:16 PM

#

That's good. My sdxl server has been reduced to a GTX1080 8GB, so anything to make it run a bit better is appreciated.

halcyon quarry Apr 10, 2024, 6:38 PM

#

I just now updated the Installation Instructions - not to say that it just got any easier or anything.
Just a better strategy to cope with my file structure and inability to simplify the process

halcyon quarry Apr 12, 2024, 8:54 PM

#

Pushed a nice update

#

Added Inpainting (img2img_mask). Can now include both img2img image and an img2img_mask in your /image command

#

Added a tag 'user_image' which will simply send a local image

#

The bot now ships with just one folder for images called 'user_images' - There are no longer separate folders for reactor faces, controlnet, etc. Just include the path in your tag values

#

As for image values - I added a loop function so that if you specify a directory as the value (ex: "Test" instead of something like "Test.png")- it will try digging through subdirectories until it either finds a valid image, or can't find a subdirectory. This means, you can have nested random image selection.

halcyon quarry Apr 14, 2024, 3:09 AM

#

Coming in my next commit - I enhanced my feature for dynamic "Tag creating" when typing with syntax: [[ key: value ]]

I revised it to recursively handle dictionaries, lists, even sublist values. Now, any setting can be created on-the-fly without limitation.

#

Not that this example is very practical, but proof of concept if an LLM can ever actually respond with Forge Couple mapping values (EI: for "Flows feature" ask LLM to provide mappings)

#

I may have also made big optimization change... need to do more testing...

halcyon quarry Apr 14, 2024, 10:33 PM

#

Pretty nice update coming tomorrow...

The "Flows" feature will finally become much more useful - variables representing user prompts or LLM replies can be used as tag values.

Such as change_imgmodel: {llm_0} - if the bot is asked what image model is most appropriate in a flow_step, it's response would become the value.

halcyon quarry Apr 15, 2024, 7:00 PM

#

Whipped up an example character using the Flows feature, which will review the first LLM's response and determine the best Image Model to generate it with

halcyon quarry Apr 16, 2024, 6:34 PM

#

Pushed another big update!

Changed the 'Logging Level' from DEBUG to INFO - LESS SPAM!
Performance may be more optimized... all settings were being stored in the discord client object. Now, they are stored in a dedicated class object.
Characters can now be omitted from /character command with new parameter (see M1nty example char)
The feature to create tags instantly from your text has been upgraded - ANY tag values can be created including dictionaries, lists, sublists... anything.
The SD API "Guess imgmodel params" feature has much better success rate now.
"Flows" feature can now use variables for tag values.
Added a new "Flows" example in 'dict_tags.yaml'
Added new forge-couple param.
Better error handling when failure to change Img model
Fixed img prompt insertions from Tags sometimes creating a line break.
Various other improvements

#

#

Ships with a new "Flow" and fine-tuned character I made:

Will use M1nty-SDXL to make an image prompt
Will send that image prompt to the Image Model Selector character, who will reply with the most appropriate model name.
The model will change before generating the image

halcyon quarry Apr 17, 2024, 8:34 PM

#

Wildcards are coming to the bot

halcyon quarry Apr 18, 2024, 1:56 AM

#

halcyon quarry Apr 18, 2024, 3:20 PM

#

#

Showing custom weights, multiple select, and using Wildcard files

halcyon quarry Apr 18, 2024, 8:56 PM

#

Pushed the new Dyanamic Prompting feature

#

Works (almost) exactly the same as https://github.com/adieyal/sd-dynamic-prompts/blob/main/docs/SYNTAX.md

Has everything implemented up until the Variables section.

Variables adds another layer of complexity that we don't need.

halcyon quarry Apr 18, 2024, 9:45 PM

#

Next big feature is auto-prompting - which will capitalize on the Dynamic Prompting

halcyon quarry Apr 19, 2024, 2:38 PM

#

I kind of sucked at using a Dev branch when I last tried. I'm giving it another go.

halcyon quarry Apr 19, 2024, 7:11 PM

#

Pushed more updates

Overhaul to Image Model handling:

There was a "Yaml" method and an "API" method for building the Imgmodels list
Now, Img model data is constructed from API ONLY
config.py shrinks again! ALL of the imgmodel settings have moved to dict_imgmodels.yaml
The only feature we lose from this is the url being announced. Oh well.

Fixed some errors I found with Dynamic Prompting

halcyon quarry Apr 19, 2024, 7:45 PM

#

Explanation for Dynamic Prompting

https://github.com/altoiddealer/ad_discordbot/wiki/Dynamic-Prompting

GitHub

Dynamic Prompting

A Discord Bot for chatting with Large Language Model (LLM) supported by text-generation-webui - altoiddealer/ad_discordbot

halcyon quarry Apr 21, 2024, 4:58 PM

#

Pretty interesting update coming...

I figured the bot has so many nice LLM specific features, and IMG gen specific features, that it should not be so rigid in requiring both to be present and active.

I've revised the code so there is now main sd_enabled and textgenwebui_enabled settings in config.py. I just need to do some more testing before I push it

#

If TGWUI is disabled, all the image features are still present and unrestricted - and vice verse.

halcyon quarry Apr 21, 2024, 9:27 PM

#

Pushed major update

replaced config.py with config.yaml
Config.py will still work, but is now unsupported and will receive no updates.
Textgenwebui and SD WebUI are now optional elements of the bot that can be disabled in config.yaml

#

Much more user friendly appearance, and easier for me to add settings with backwards compatibility.

vestal python Apr 22, 2024, 2:18 PM

#

:x remaking textgen + adbot finally. Since I combined nearly all my resources.. I figured I'd save my gtx 1080 8gb computer set aside for Llama 3 8B with Discord and see how that goes. My laptop was hitting 40 t/s with it, so hope it's about the same for speedy responses.

halcyon quarry Apr 22, 2024, 2:18 PM

#

I just started messing with llama-3B and it is very impressive indeed

vestal python Apr 22, 2024, 2:19 PM

#

Right? Shame about the context, but some of the tests I've done with my documents have been insightful. Adding some suggestions I haven't even thought of.

halcyon quarry Apr 22, 2024, 2:20 PM

#

I'm not sure what you mean about the context

vestal python Apr 22, 2024, 2:20 PM

#

the 8k token limit. Which probably is fine for discord text limit anyways.

halcyon quarry Apr 22, 2024, 2:20 PM

#

ah

#

Well, my bot now has variables that can be used in the prompt (or in 'flow' tag parameters) for recent User messages ex: {user_0} most recent user prompt, LLM responses such as {llm_3} fourth most recent LLM resp, or even history chat pairs such as {history_5} returns that pair

#

You may be able to come up with some workaround

#

These variables are updated immediately after the LLM Gen step in the bot

#

they are limited to 10 most recent messages, or 10k characters - before the list gets filtered down (for performance). I may make these limits user configurable

#

This is managed separately from the Chat History

vestal python Apr 22, 2024, 2:27 PM

#

taking a look at config file now and i'll take a look at that too.

halcyon quarry Apr 22, 2024, 2:29 PM

#

The variables is mainly discussed in my new tips and info folder Message Variables.txt

#

Some example usage can be seen in the dict_tags.yaml

halcyon quarry Apr 23, 2024, 8:43 PM

#

Pushed an update that handles img gen failure much better.

Also:

Updated Forge Couple extension support
Improved Tags handling for /image command
Added an optional reactor_mask param - a greyscale image can mask the face swap results

halcyon quarry Apr 24, 2024, 3:46 PM

#

Just added this bad boy to the Wiki

#

vestal python Apr 25, 2024, 5:19 PM

#

How long does the default settings save conversation history?

halcyon quarry Apr 25, 2024, 5:23 PM

#

Currently, history gets saved endlessly until the character is reset

#

The amount of the history being used as context is controlled by your loader settings, and truncation length, and max_new_tokens

#

You'll adjust those in dict_base_settings.yaml and/or your character file

#

character settings will trump base settings

vestal python Apr 25, 2024, 5:26 PM

#

Ok I didn't know if it was a 10 minute timer. I had asked it about Phi 3 in conversation hours ago and it popped in asking if we shoudl continue talking about Phi 3

#

Also stopped doing the # thing, so must have just been a conversation thing it was doing.

halcyon quarry Apr 25, 2024, 5:26 PM

#

Currently, history never resets unless the script is closed, or you change character or use /reset

#

In the future I may add something like TWGUI has where old chat history can be loaded up.

vestal python Apr 25, 2024, 5:30 PM

#

I've been thinking about adding in some things recently once I get the time to.. But I'm not sure if that would be textgen-side, or the discord bot side for adding like just basic website search, and simple RAG to call documents. It's been a number of months I'd be interested to see what new projects and extensions there are to add them in now, or just figure out how to add one in myself..

halcyon quarry Apr 25, 2024, 5:31 PM

#

The bot should support extensions natively, but some may not work...
Extensions will be loaded if they are in CMD_FLAGS.txt

#

unless the extension has like a custom button you need to press in the UI, or has its own specific endpoint or something

#

but if it is designed to manipulate the input or use it, etc, it should work already out of the box

vestal python Apr 25, 2024, 5:33 PM

#

Idk there's too many ways to go about it

halcyon quarry Apr 25, 2024, 5:33 PM

#

I suppose you're talking about making your own extension?

vestal python Apr 25, 2024, 5:33 PM

#

Yeah the input would probably be it. Kind of like the older webs search extension we have "Search for" as a trigger

#

At this point I haven't seen alot of new extensions being made.. I think it's about time to start considering to..

halcyon quarry Apr 25, 2024, 5:35 PM

#

Something like that would work with the bot, so long as you just load the extension with the bot (in --extension web_search or whatever in CMD_FLAGS.txt)

flint stirrup Apr 26, 2024, 6:09 AM

#

Is it possible for you to set this up for me? for my server?

#

@halcyon quarry

#

Added you as friend.. lets discuss once if possible

halcyon quarry Apr 26, 2024, 10:40 AM

#

Should be relatively simple… just follow my install instructions

halcyon quarry Apr 26, 2024, 10:42 AM

#

flint stirrup Is it possible for you to set this up for me? for my server?

For the bot part, you’re going to just pick most of the permissions in the Bot > 0auth page, copy paste that URL into your web browser - which will then let you invite the bot to your server

flint stirrup Apr 26, 2024, 10:43 AM

#

Hey I agree but your simple isnt simple for me :p

halcyon quarry Apr 26, 2024, 10:43 AM

#

Also need to generate a bot token, which you’ll just copy paste into config.yaml

flint stirrup Apr 26, 2024, 10:43 AM

#

Further I want more customization hence wanted to discuss

halcyon quarry Apr 26, 2024, 10:43 AM

#

Let me know if you struggle at all with any setup step

#

I’m very responsive

#

Oh well I can definitely help there, and there is plenty of customization to be had

flint stirrup Apr 26, 2024, 10:44 AM

#

DM me? added friend

halcyon quarry Apr 26, 2024, 7:49 PM

#

I just pulled off a massive accomplishment (IMO) - restoring this amazing ControlNet API endpoint that they simply axed or failed to re-implement
https://github.com/lllyasviel/stable-diffusion-webui-forge/pull/692

GitHub

Restore '/controlnet/control_types' API endpoint by altoiddealer · ...

Restores the '/controlnet/control_types' API endpoint, which is immensely useful for anyone using ControlNet via the API
Description
I recently opened an Issue on the main ControlNet extens...

#

What this means is that I'll soon be upgrading ControlNet handling in the bot such as via the /image command.

Heck, the whole dict_cmdoptions.yaml file may get axed

#

my brain hurts. This is not the kind of coding I'm accustomed to

vestal python Apr 27, 2024, 1:40 PM

#

I'll have to look into the whole model loading part as well soon with these new medical models to swap to

halcyon quarry Apr 27, 2024, 6:40 PM

#

triggering model swaps via Tags? Super easy my friend

#

just be sure to load your models via the WebUI using the settings you desire then click Save Settings, on the model tab

#

those settings will take effect when switching/swapping models via the bot

#

(ones saved in /models/user_config.yaml)

vestal python Apr 27, 2024, 7:10 PM

#

I will take a look later in the week ty. I'm looking into the medical and law models of llama 3 8B to swap into for my users. Might be useful

halcyon quarry Apr 27, 2024, 7:51 PM

#

Yes you can use the 'swap_llmmodel' tag to just switch models for a single interaction switching bakc to original model - or change_llmmodel tag. Or of course the /llmmodel command

halcyon quarry Apr 28, 2024, 6:07 PM

#

yesssssss major improvements coming very soon for ControlNet support, particularly in the /image command.

Will be much much more easy to use and dynamic

halcyon quarry Apr 29, 2024, 7:31 PM

#

Pushed this update

halcyon quarry May 2, 2024, 2:49 PM

#

Pushed a nice update for error handling

Added Traceback logging on a few critical functions
The SD API function will now retry certain requests one time, for Response codes 408 (timeout error) and 500 (internal server error) which are temporary / may resolve on a secondary attempt.

burnt patrol May 2, 2024, 10:16 PM

#

Sorry to ask, but is it possible to use chat instruct mode?

halcyon quarry May 2, 2024, 10:51 PM

#

burnt patrol Sorry to ask, but is it possible to use chat instruct mode?

Certainly, as well as Instruct

#

just change the ‘mode’ setting in the character or in base_settings

burnt patrol May 2, 2024, 10:52 PM

#

Ah.

halcyon quarry May 2, 2024, 10:52 PM

#

Character settings will trump

burnt patrol May 2, 2024, 10:52 PM

#

I was trying to use llama 3

halcyon quarry May 2, 2024, 10:53 PM

#

I have llama 3 working in Instruct mode

#

Just needs a custom system message, need to disable skip special tokens, and you’ll see that some custom stopping strings need to be added

burnt patrol May 2, 2024, 10:56 PM

#

Ok, thanks

halcyon quarry May 2, 2024, 10:56 PM

#

I recommend making a copy of your character file so you can quickly switch back after you realize Mistral models may be preferable 😛

burnt patrol May 2, 2024, 10:56 PM

#

Heh

halcyon quarry May 2, 2024, 10:57 PM

#

Note - with this bot, only relavent parameters do anything… dont worry about having context in Instruct mode, etc

#

Context will be ignored… unless chat instruct, as you said

vestal python May 3, 2024, 3:35 AM

#

been testing some llama 3 8B models

#

Uncensored one is... I haven't seen a model this uncensored since vicuna.

#

Building a list of ones to make swappable for the discord bot

halcyon quarry May 3, 2024, 3:37 AM

#

Thanks for the info - will be downloading promptly 😛

vestal python May 3, 2024, 3:38 AM

#

OpenBio was interesting medical info on some conditions. One of my users has alot of questions for it.

halcyon quarry May 3, 2024, 3:39 AM

#

That does sound like a nice resource to have in the ol toolbox

#

If you have any ideas to improve the bot, let me know.

#

I mainly just prompt images myself but I want to always keep improving it as a whole

vestal python May 3, 2024, 3:43 AM

#

I do I think... but I'm just getting back into it with my 2 Ecne and Ogma. I want to figure out on my 2nd one Ogma to add in some homemade extensions maybe for a semi-agent feel. I need to make a list of what users were looking for.

vestal python May 3, 2024, 4:03 AM

#

yeah asking around my users of things for me the next week I have free to work on this:

Web search results (Been a few months, should probably See about new techniques people have found to integrate)
conversation memory (probably some RAG form per character)
Game knowledge (Back to the old reason I still haven't figured out fully)
Model swapping
Characters/personalities (I need to remake them)

halcyon quarry May 3, 2024, 11:08 AM

#

one thing I do plan on adding (which should be easy) is variable assignment. Could handle some memory stuff but would require the effort of assigning content to a variable then using it

#

But the first two items should be resolved by third party extensions… I could integrate if it’s nothing too difficult to maintain.

halcyon quarry May 3, 2024, 12:31 PM

#

Gonna try lexi-llama3 today - I’ll be very surprised if it’s less censored than NeuralBeagle14-7B

#

Very impressive model, gives both great image prompts and chat replies

vestal python May 3, 2024, 2:51 PM

#

https://github.com/rahulnyk/knowledge_graph I'm going to read up on knowledge graphs and see how that can be implemented directly... I had an interest before but now kind of a requirement. See kind of like Alltalk extension where it's called upon any request to textgen to check through.

#

That'd be really neat if added in, and the discord bot just picks it up by default having textgen handle it.

halcyon quarry May 3, 2024, 2:55 PM

#

Looks really cool, but I'm not ambitious enough to make my own knowledge graph just yet

#

getting a knowledge graph to work with the bot, is something I could look into.

#

I like the example there "Mary had a little lamb..."
node_1: Mary node_2: Lamb Relation: Eats
node_1: Lamb node_2: Food Relation: Is

#

I don't know about these llama-3 models, they get so wonked out so fast

halcyon quarry May 3, 2024, 7:47 PM

#

I'm excited about this... new command to restart the SD WebUI client

#

Could resolve problems if the API freezes up.

I've also reworked the "Progress Check" function so it has better error handling... so there should be fewer problems in general

marsh harness May 3, 2024, 8:44 PM

#

Well... time to update everything and see what I've missed.

marsh harness May 4, 2024, 1:52 AM

#

Amazing. Very much appreciated as always.

halcyon quarry May 4, 2024, 11:13 AM

#

Got some more updates coming… I think tonight/tomorrow. Finally adding some “behavior”, mainly controls that will make it respond more randomly

#

(frequency)

#

Including sending messages without being in immediate response

halcyon quarry May 4, 2024, 12:43 PM

#

I was thinking about it, and this bot script could be upated to support multiple simultaneous discord clients… but I wouldn’t use it, personally, and I don’t have enough python experience to efficiently change all the code over to using self and clsn classes

#

If there is an ambitious true python coder out there that wants to collaborate on that, hit me up

halcyon quarry May 5, 2024, 10:43 PM

#

These are the sorts of new behavior changes coming soon

snow frost May 5, 2024, 10:43 PM

#

nice

halcyon quarry May 5, 2024, 10:44 PM

#

Also, settings to spontaneously send a message, unprompted.

#

Well, the settings will dictate the prompt, but it will be a silent prompt

#

Included with these changes will be the ability to have an "auto generate" type of behavior, if that's how the user wants to utilize it. Since I added Dynamic Prompting (wildcards), it can just endlessly generate dynamically prompted images, among other possible uses

terse folio May 5, 2024, 10:47 PM

#

halcyon quarry If there is an ambitious true python coder out there that wants to collaborate o...

I've been doing discord bots for 6 years ^^
Can definitely help out around there!

But I'm still new to slash commands

halcyon quarry May 5, 2024, 10:47 PM

#

yoooo gowron1 👏

#

The main thing I need to resolve that is very daunting to me, is just splitting my script into separate modules

#

It's probably super easy for a real coder

#

I already have my script organized into nice chunks

terse folio May 5, 2024, 10:49 PM

#

Yup, that's a huge deal, with separate modules, you can reload them live!

halcyon quarry May 5, 2024, 10:49 PM

#

I know all my functions, I jump around my code strictly using Ctrl+F

#

The other thing that should happen but never will by my account, without at least some nudge, is making the bot an instance class, make everything self

terse folio May 5, 2024, 10:51 PM

#

btw, what version of python is the bot built for?

terse folio May 5, 2024, 10:52 PM

#

halcyon quarry The other thing that should happen but never will by my account, without at leas...

Yea, I can help there

#

what I like to do is have a very minimal "main" file.
and everything else is loaded as a cog (what discord.py calls the modeules)

halcyon quarry May 5, 2024, 10:53 PM

#

I don't necessarily need someone to do all the work for me either (although I'd gladly accept, believe me).

Really just need a first step to show how it's done, then I could pick it up and go from there

#

I' using python 3.10

terse folio May 5, 2024, 10:54 PM

#

I'll upload an example bot in the structure that I typically use to github

halcyon quarry May 5, 2024, 10:55 PM

#

To be clear: I did not learn any python coding the correct way - I just jumped in head first with ChatGPT a year ago, and have been learning as I trip and fall

#

I kind of know how to restructure the whole script into separate modules but I'd spend way too much time on it than it's worth (to me)

terse folio May 5, 2024, 10:57 PM

#

same same,
I started learning as I joined discord and saw people had custom bots, I was like "I have to do that"
and stumbled my way through bot examples online, changing little things and learning python

#

About multiple clients running out of the same file.
I'm not too sure how that's done properly.

but I have seen examples online

snow frost May 5, 2024, 11:06 PM

#

thats the same way i learned C# and among us modding lol

terse folio May 5, 2024, 11:06 PM

#

:)

halcyon quarry May 5, 2024, 11:11 PM

#

I've certainly learned a lot, though!

halcyon quarry May 5, 2024, 11:58 PM

#

ey the github got 4 stars today, I think it might be starting to catch on finally

terse folio May 6, 2024, 2:16 AM

#

I just realised the bot code starts up the whole webui inside it O:

What are the benifits to this design over using the api?

If it's so they start up at the same time, it's possible to do that with a .cmd file to start multiple processes at once

#

https://github.com/Artificiangel/discord-bot-base
Here's an example bot base that uses cogs and has a really nice reload feature built in.

I plan to add more/clean things up over time.

#

After setting up the config, you can test the ".rl" command by editing the file at modules/commands/test.py

Then try causing an error in the file and reloading again. It will load a recovered version so you don't have downtime ^^

#

Anyway, if you're going to be loading large things like textgen-webui from a cog:
You should have it write the webui class to self.bot instead of self (cog).
And do a check if it is set or not.

That way it will persist between reloads of that file and you don't have to wait for the webui to start up each time.

#

what I mean by that

terse folio May 6, 2024, 2:51 AM

#

Without switching to cogs or building the bot in a class, you can still use multiple files.

This should work:
Define bot in a common file, like "bot_base.py"
In each of your files, import bot from bot_base and use the @bot.command decorator to add commands/listeners.

Then in a main file, you can import all these command files
import imggen
import textgen
import message_event

lastly
from bot_base import bot

And use bot.run() in this main file.

halcyon quarry May 6, 2024, 4:30 AM

#

terse folio I just realised the bot code starts up the whole webui inside it O: What are th...

Unless I’m mistaken, the API does not have TTS support… when importing and using chatbot_wrapper() it returna both Internal and Visible responses (the TTS file is included in one of the responses), whereas the API only returns one response

#

The API is also very finicky with parameters… very easy to error. Also, I had come up with a means to update extension parameters during runtime (custom TTS settings per character, mainly) but TGWUI made it impossible after a certain commit. I made a monkeypatch to restore that extension loading, which I don’t think will work if I use the API

#

Lastly, it’s already purring along without issues 😛

#

I did the work to get API working with my bot - I have a version saved using it - but the TTS thing was what detracted me

terse folio May 6, 2024, 5:33 AM

#

What I would do is write an extension to interface with the bot that runs on the textgen webui

#

There, endpoints for tts could be added

snow frost May 6, 2024, 8:33 AM

#

^^

#

was thinking the same thing

halcyon quarry May 6, 2024, 10:59 AM

#

my bot has a treasure trove of extensions ripe to be plucked out

#

Ok maybe one or two at least 😛

snow frost May 6, 2024, 12:05 PM

#

i lowkey might write my own bot and connect it to a custom extension

#

if only i had the time

halcyon quarry May 6, 2024, 1:10 PM

#

I'm curious to know if there are any particular features mine is lacking that you are considering such a thing?

snow frost May 6, 2024, 1:39 PM

#

halcyon quarry I'm curious to know if there are any particular features mine is lacking that yo...

more of a learning experience thing

halcyon quarry May 6, 2024, 1:39 PM

#

ahh

snow frost May 6, 2024, 1:39 PM

#

also being able to use webui separately from the bot

#

and having the bot be “stupid”

#

as in it only deals with requests

#

no logic of its own, like changing models etc

halcyon quarry May 6, 2024, 1:40 PM

#

There is a new API endpoint that I haven't been able to get a successful response from, which might have me switch to using the API

#

#api-dev-help message

#

It was added to TGWUI with no explanation and I don't know what the heck it returns

#

It seems like it may return both Internal and Visible responses, like I currently process via importing chatbot_wrapper()

vestal python May 6, 2024, 3:03 PM

#

I like the idea of having it open to both using discord bot and api without having to choose between the two.. But does it mess with each other.

#

Funny enough there's moments in discord where one user is talking to my guy on their server, and then I get one of their responses on my personal server

halcyon quarry May 6, 2024, 3:05 PM

#

I'm sure it works like the API calls to Stable Diffusion... the parameters in the UI are the ones used when clicking Generate button, but the bot sends a completely different payload which is handled without affecting UI values

vestal python May 6, 2024, 3:06 PM

#

Getting my guy ready today also.. Although I doubt he'll be fast enough for discord use.

#

Trying to figure out what port do they consider 1 for this new mobo..

halcyon quarry May 6, 2024, 3:07 PM

#

You make me feel like I have a tidy workspace when you share photos

vestal python May 6, 2024, 3:07 PM

#

My son.. He has claimed all my space lol

halcyon quarry May 6, 2024, 3:08 PM

#

I'm sure it's just a matter of time before I'm in the same boat

#

I'm very excited about the new "Behaviors" I'll be rolling out soon

terse folio May 6, 2024, 7:13 PM

#

halcyon quarry It was added to TGWUI with no explanation and I don't know what the heck it retu...

the internal/chat-prompt endpoint renders your chat messages into a prompt.
It's useful for previewing what the /chat/completions endpoint is doing.

Maybe you want to test out/debug different prompt templates.

Or maybe you want to get back the prompt to implement some of your own logic before sending it to /completions endpoint.

(Internally, /chat/completions will render the "chat-prompt" then call the code for /completions)

#

for example

halcyon quarry May 6, 2024, 7:13 PM

#

ah okay, I understand now. Thanks for the explanation

terse folio May 6, 2024, 7:14 PM

#

yup ^^

halcyon quarry May 6, 2024, 7:14 PM

#

syustem message

terse folio May 6, 2024, 7:14 PM

#

Hahaha, typos

halcyon quarry May 6, 2024, 7:14 PM

#

very interesting

terse folio May 6, 2024, 7:18 PM

#

I have some custom logic for fitting context into the context window, and don't want something like the /chat/completions endpoint deleting important parts.

To solve that, I send a request to /internal/chat-prompt with some variables which I then fill out on my side.

The benefit of this is it allows me to take advantage of the automatic instruct template detection that /chat/completions has.
A feature that the normal /completions does not seem to have.

halcyon quarry May 7, 2024, 1:12 AM

#

Whenever I spot a bug, I wonder if anyone else experienced it, got frustrated and never reported it.

terse folio May 7, 2024, 1:13 AM

#

yea, I tried searching "TypeError: e is undefined" gradio python
And was met with "you have reached the end of the internet"
Wrong channel

#

maybe it's a different dependancy

halcyon quarry May 7, 2024, 1:13 AM

#

For instance I just noticed that none of my custom character data items (extensions, TTS settings, and Tags) are loading for Instruct mode character

terse folio May 7, 2024, 1:14 AM

#

I'm pretty sure chat characters and instruct are separate

halcyon quarry May 7, 2024, 1:15 AM

#

It's just a flaw in my logic, those things are totally relavent for any mode

#

need to fix up the ol' logic of it

terse folio May 7, 2024, 1:15 AM

#

oh nvm,
it was the instruction template tab that's different

halcyon quarry May 7, 2024, 1:16 AM

#

Due to the way this bot woks (not using the API) you can send all settings and only the ones that are relavent have any effect

terse folio May 7, 2024, 1:16 AM

#

terse folio ~~yea, I tried searching `"TypeError: e is undefined" gradio python` And was met...

Oh whoops lol, I thought this was #general

halcyon quarry May 7, 2024, 1:16 AM

#

I thought so 😄

#

I decided to turn TTS back on for my bot after awhile, and was wondering why the heck it wasn't joining the voice channel

#

and its because now I'm using llama3 with instruct mode

#

Will be pushing a fix tomorrow

terse folio May 7, 2024, 1:19 AM

#

Interesting, is the voice code different for instruct mode?

#

Or maybe it's not implemented because tts was part of the chat functions?

halcyon quarry May 7, 2024, 1:21 AM

#

It's because my bot supports extra values in the character files, but when I coded it I didn't think to read this information regardless of mode

terse folio May 7, 2024, 1:22 AM

#

I see I see

halcyon quarry May 7, 2024, 1:23 AM

#

it's an easy fix, I just get a bit frustrated to discover my own bugs knowing someone else probably already experienced it

terse folio May 7, 2024, 1:31 AM

#

Would hope those people reach out, here or maybe a community you could start around the bot ^^

halcyon quarry May 7, 2024, 4:04 PM

#

Upon closer inspection, I realized that my handling for instruct templates was not well done.
I've reworked it so that:

An instruct template defined in a character file will have highest priority
Otherwise, instruct template found in model metadata will be used.
If neither character or model metadata provide a template, then it will use the default in base_settings

#

I may revise it further in the future but for now this should work well enough.

#

All extra information in a character file is now loaded in, including Instruct mode

#

(can use Voice channel)

#

Also found a bug where matched tags priorities were being adjusted... fixed that as well...

terse folio May 7, 2024, 4:14 PM

#

Woo ^^

halcyon quarry May 7, 2024, 4:16 PM

#

seeing what else I can update today before pushing a commit

halcyon quarry May 7, 2024, 8:13 PM

#

So with the new behavior changes...

What I'm thinking of doing is setting up a few checkpoints to see if new user messages came in that the bot should reply to.
By default it would just do what it does now - process them sequentially one at a time as fast as compute permits.
With settings that make it behave a little more like a person, it would instead check to see the message or accumulated user messages when it is triggered to actually reply - and just send one reply.
Again, depending on settings... if another user message comes while it is processing, it may be triggered to cancel it's current task and then process all those recent messages as "the prompt"

halcyon quarry May 7, 2024, 9:01 PM

#

Committed the changes I mentioned earlier...

halcyon quarry May 8, 2024, 2:14 AM

#

The recent “retry” feature for errored Stable Diffusion api calls has salvaged almost all errored responses for me, mainly caused by random OOM or just nothing in particular

halcyon quarry May 9, 2024, 6:23 PM

#

Welp, Stable Artisan is going to contest this bot a bit lol

halcyon quarry May 9, 2024, 8:09 PM

#

Pushed some minor fixes

calm rain May 9, 2024, 8:10 PM

#

halcyon quarry Welp, Stable Artisan is going to contest this bot a bit lol

heh

#

image wise mayhap, text wise nope

halcyon quarry May 9, 2024, 8:39 PM

#

calm rain heh

I saw that they have automatic segmentation - this is something I've had in the back of my head for built in Segment Anything support.
I haven't messed with that via API yet but I imagine the "Grounding DYNO" is what could handle that

#

That's probably the model they're using really

#

Welp, now I'm motivated to look into it more

#

Enjoying this recent "Pog" LORA

calm rain May 9, 2024, 8:54 PM

#

i have ClipSeg based auto segmentation in Swarm, and it'spretty nice, but not as solid as the one in the api

halcyon quarry May 9, 2024, 8:56 PM

#

Grounding DINO worked pretty good on first test

#

#

The 3rd result is usually the best one when using points - here I only used prompt

#

And yes that's me playing beer pong while wearing a Jailbot costume

#

I actually kicked ass despite being a free rebound wall

halcyon quarry May 10, 2024, 3:04 AM

#

Pushed an update adding Embed settings

halcyon quarry May 11, 2024, 1:46 AM

#

Pushed a pretty simple update

The most recently generated image can now be used as a variable in the Tags feature '{last_image}' will now be replaced with the last image filename for img2img, controlnets, etc.

snow frost May 11, 2024, 12:56 PM

#

@halcyon quarry have you considered making the bot only change it's nickname instead of the username?

halcyon quarry May 11, 2024, 1:07 PM

#

snow frost <@670018869418786816> have you considered making the bot only change it's nickna...

Unsure why I hadn’t thought of this

#

Need to look into that ASAP, thanks. Doesn’t help entirely though, as Avatar change also has 10 minute cooldown

#

Nickname change probably also 10 min cooldown

#

So… may not look into it after all 😛 We want avatar changes. Do you think there’s any other good reason to change nickname instead of username aside from potentially avoiding a cooldown?

snow frost May 11, 2024, 1:51 PM

#

halcyon quarry Nickname change probably also 10 min cooldown

i doubt that but maybe

snow frost May 11, 2024, 1:51 PM

#

halcyon quarry So… may not look into it after all 😛 We want avatar changes. Do you think the...

what if i want my username to be consistent 🤷‍♂️

#

bot usernames should stay the same throughout their lifetime

#

unless ur rebranding

halcyon quarry May 11, 2024, 1:52 PM

#

That's a good argument for changing Nickname instead of Username

#

I'll see if I can do this, and if so will implement ASAP

snow frost May 11, 2024, 1:54 PM

#

👍

halcyon quarry May 11, 2024, 1:58 PM

#

snow frost 👍

Unfortunately, it looks like the edit method only has 2 parameters: avatar and username
https://discordpy.readthedocs.io/en/stable/api.html?highlight=change avatar#discord.ClientUser.edit

snow frost May 11, 2024, 1:59 PM

#

halcyon quarry Unfortunately, it looks like the `edit` method only has 2 parameters: `avatar` a...

https://discordpy.readthedocs.io/en/latest/api.html#discord.Member.edit

#

what about this

#

if you get the bot's member

#

u can change nick

halcyon quarry May 11, 2024, 2:00 PM

#

Thanks! I was still hunting around to see if there was another method... this could be the solution

#

Think I got it... testing...

terse folio May 11, 2024, 2:12 PM

#

halcyon quarry Need to look into that ASAP, thanks. Doesn’t help entirely though, as Avatar ch...

Use webhooks, but make sure to take care of ratelimits, discord doesn't like webhooks being spammed.

A webhook can have any pfp/username for each message.

halcyon quarry May 11, 2024, 2:14 PM

#

Whenever the username/avatar is changed, it stores the current datetime to the bot.db local database.
When changing characters, if the name or avatar is going to change it checks current datetime against the stored datetime.
If it is less than 10 minutes apart, it creates a delayed profile update task

#

If trying to change within that window, it updates the task with the newer info

terse folio May 11, 2024, 2:14 PM

#

mhm, that works if you're only using one character at a time

halcyon quarry May 11, 2024, 2:15 PM

#

Well if I was able to do multiple characters it would do the same except have a dictionary in the databse for each character/time

halcyon quarry May 11, 2024, 2:32 PM

#

But yeah I see your point - I think it’s currently possible for multiple instances of the bot script to run? I which case the database handling would fudge that all up

terse folio May 11, 2024, 2:40 PM

#

#

it'll work nicely!

#

this is all running off one bot

#

#

async def webhook_send(wehbook_url, content=None, username=None, avatar_url=None, embeds=None, **kw):
    payload = {}
    payload['content'] = content
    payload['username'] = username
    payload['avatar_url'] = avatar_url
    
    embeds_tmp = embeds or []
    if embeds_tmp:
        embeds = []
        for e in embeds_tmp:
            if isinstance(e, discord.Embed):
                embeds.append(e.to_dict())
            
            elif isinstance(e, dict):
                embeds.append(e)
            
            else:
                raise TypeError(f'Unknown embed type to send to webhook: {type(e)}, expected discord.Embed, or dict')
        
    payload['embeds'] = embeds
    payload.update(kw)

    aiohttp_session.post(wehbook_url, json=payload)

you'll want a open aiohttp session somewhere in your bot.
But this function will let you send messages similar to the await channel.send()
where you can pass content and embeds.

#

I haven't implemented rate limiting yet, you'd need a queue for that and a background loop that sends messages every x amount of time

#

The only drawback of webhooks is you need to create one per channel you plan to use it in.
But this can by automated with the bot.

I would recommend making a command for setting up the channel.
Maybe telling people they should setup a playground channel for this.
(I believe discord now limits the amount of webhooks a bot can create?)

#

And if you write your own send message function, you could have it switch between normal messages, and webhooks depending on what channel the event came from!

best of both worlds

#

can explain that in a bit, have to go right now

halcyon quarry May 11, 2024, 3:21 PM

#

Pretty sure the limit is 50 webhooks per minute

#

Do you use this bot? It’s sending many embeds, images, etc

halcyon quarry May 11, 2024, 3:44 PM

#

I understand what you’re showing, though, by using a different nickname and avatar url - but that’s the least of the trouble when it comes to multiple characters. The trouble is I need to edit probably 2000 lines to make all code applicable to separate instances

#

Got like 100 functions where every variable needs to be changed to self.

#

My queue system already does most of the work in limiting webhooks - it basically just captures the interaction and queues it

terse folio May 11, 2024, 5:51 PM

#

No need for different instances,
just have a class, dict, or something that is passed that contains the name/profile as well as your message/embeds/images
(The current character)

terse folio May 11, 2024, 5:51 PM

#

halcyon quarry Do you use this bot? It’s sending many embeds, images, etc

I haven't tried it out yet

#

~~I think bot_settings is where it could be set~~
Maybe change_char_task, char_params

#

like you would define an avatar url in the character card ^^

#

Also interesting that you have a global queue for all the events.

Is this to prevent attempts at generating text while tgwi is reloading a model?

#

This is a little thing, but channel.send() has a delete_after arg ^-^

#

It should run the wait in a background task like you have here

#

Another small tip:

#

Discord automatically handles ratelimits for these actions

#

Except webhooks if you run them with external libs, like I was in that example.

am changing a couple things to make development easier and will submit a pull request soon!

halcyon quarry May 11, 2024, 7:36 PM

#

That particular feature there (post active settings) will cause rate limit issues if it deletes messages too fast. It’s kind of hacky - one of those things I want to handle better instead of simply dumping the contents of activesettings.yaml into a channel

#

I have a separate queue for background tasks, thats one of the few things that get put in that queue

#

The main queue is mainly to prevent multiple generative tasks from happening simultaneously. Before I made the queue system things were not handled gracefully at all

terse folio May 11, 2024, 7:38 PM

#

there's a bulk delete option too,
Also you can do something some discord bots call "nuking" a channel.
That is deleting and recreating it.
Because even bulk deletes are limited (50 messages at a time)

terse folio May 11, 2024, 7:39 PM

#

halcyon quarry The main queue is mainly to prevent multiple generative tasks from happening sim...

That's fair,
I made a pull request to fix that, (I think I shared it?)

It groups different generative tasks into their own queues that will not affect eachother so you can do more at once!
(for the openai API if you choose to use it sometime)

halcyon quarry May 11, 2024, 7:40 PM

#

If there’s ever a means to use character specific tts voices via the openai api, I’ll probably switch to it

terse folio May 11, 2024, 7:40 PM

#

I don't have experience with tts/stt from tgwi, but i'd be happy to work on that together sometime

halcyon quarry May 11, 2024, 7:41 PM

#

I know alltalk has its own endpoint, and some other tts extensions probably have their own, etc, but what I have working now is pretty easy to maintain compatibility for multiple tts options

terse folio May 11, 2024, 7:41 PM

#

Ooof, discord slash command restrictions pain!
Sorry you have to deal with that.

halcyon quarry May 11, 2024, 7:42 PM

#

I have a better solution for that

#

Just need to sit down and do it 😛

terse folio May 11, 2024, 7:42 PM

#

I was about to translate that into a loop to cut down on the extra code

#

I think some commands are better off being normal message commands, or use string arguments that aren't tied to options

#

when you have that many!

halcyon quarry May 11, 2024, 7:43 PM

#

Need to change it to just a regular command, not command options - and follow up with discord.ui.Select menus in a View

terse folio May 11, 2024, 7:43 PM

#

Oh that would be cool, maybe even a pageinator system

#

I think I saw a discord example for that

halcyon quarry May 11, 2024, 7:44 PM

#

I just got into that recently for the ControlNet handling I added to def image

#

It follows up with ui.select menus. Works very well

terse folio May 11, 2024, 7:44 PM

#

Awesome, I need to learn more about slash commands and Views, rarely touched them!

halcyon quarry May 11, 2024, 7:45 PM

#

If you install the bot try out the ControlNet options from /image cmd

#

It’s very impressive if I do say so myself 😛

terse folio May 11, 2024, 7:46 PM

#

:)

#

Ooh, even better solution!

#

if bulk message deletions fail due to the messages being too old

#

it will fallback to doing one by one

halcyon quarry May 11, 2024, 7:51 PM

#

Thanks - that’s a nice crutch while I mull over how I want to rework the settings posting thing

terse folio May 11, 2024, 7:51 PM

#

What are you thinking about?

terse folio May 11, 2024, 9:07 PM

#

Testing the bot, it gets to the client.tree.sync() code, but none of the slash commands are available.

Got errors about missing characters, but that doesn't seem to be the issue, all else looks fine.
Any ideas?

#

Running an unmodified version

halcyon quarry May 11, 2024, 9:09 PM

#

terse folio Testing the bot, it gets to the client.tree.sync() code, but none of the slash c...

Did you put the ad_discordbot folder in the TGWUI dir, and move bot.py there?

terse folio May 11, 2024, 9:10 PM

#

The instructions said to move the bot.py out of the ad_discordbot folder, but yes, the folder is there

halcyon quarry May 11, 2024, 9:11 PM

#

That’s correct. Hmm

terse folio May 11, 2024, 9:12 PM

#

Is there some extra step to setting up slash commands?

halcyon quarry May 11, 2024, 9:12 PM

#

The commands relevant to img gen are disabled if SD WebUI is not running with api and listen, when bot is launched

#

The bot may not have sufficient permissions?

terse folio May 11, 2024, 9:13 PM

#

Yea, just trying to press "/" no suggested commands pop up

terse folio May 11, 2024, 9:13 PM

#

halcyon quarry The bot may not have sufficient permissions?

It does

halcyon quarry May 11, 2024, 9:13 PM

#

You may need to close and open your client again

#

Will probably resolve it really

#

Kill ios app, or exit/launch desktop app, etc

terse folio May 11, 2024, 9:14 PM

#

Ahh that was it, good to know

#

am on desktop ^^ yup

halcyon quarry May 11, 2024, 9:15 PM

#

It’s pretty annoying that it’s like that, don’t think there’s a workaround

terse folio May 11, 2024, 9:15 PM

#

Mhmm, gotta put a little note about it in the readme sometime!

halcyon quarry May 11, 2024, 9:16 PM

#

Absolutely

terse folio May 11, 2024, 9:17 PM

#

Ahhhah, whoops.
should have made a test bot before doing this on my main one XD

#

Yea, absolutely gotta try webhooks sometime, less destructive haha.
But I understand, most people probably wont be running a discord bot along side this

halcyon quarry May 11, 2024, 9:21 PM

#

I did have nickname update working in lieu of username - just been on the go all day. It’s a guild specific change btw, so initially I have it iterate over all guilds it is a part of and set nickname for all

terse folio May 11, 2024, 9:21 PM

#

Or, better solution,
As part of your activeSettings.yaml
You could have a first time run state that is "warn overwrite:true"

After user confirms the pfp/name will be changed, this is turned off for all future character swaps

halcyon quarry May 11, 2024, 9:21 PM

#

Which should be correct in all cases…

terse folio May 11, 2024, 9:22 PM

#

halcyon quarry I did have nickname update working in lieu of username - just been on the go all...

I see, I see

#

Also interesting, updates to slash commands also require a client restart

halcyon quarry May 11, 2024, 9:24 PM

#

As part of that discord.member class, may be a softer avatar change as well compared to client. Maybe doesn’t have 10 minute CD

terse folio May 11, 2024, 9:24 PM

#

im not sure bots can have custom per-server avatars

halcyon quarry May 11, 2024, 9:27 PM

#

Since bot is also a member, seems like it’s available

#

I added an embed color to config.py recently. I’m also going to make this a char specific option

#

There’s >300 instances of “embed” in my code so it was a tricky update

terse folio May 11, 2024, 9:33 PM

#

halcyon quarry Since bot is also a member, seems like it’s available

It's worth a try, but from what it looks like it's just client.clientUser + Member.

I think custom profiles is a user feature.
Similar to how bots are unable to send friend requests (but it used to be in discord.py as userbots were a grey area)

halcyon quarry May 11, 2024, 9:33 PM

#

Char specific is a very simple one line update tho

#

https://discordpy.readthedocs.io/en/latest/api.html#discord.Member.guild_avatar

terse folio May 11, 2024, 9:33 PM

#

halcyon quarry There’s >300 instances of “embed” in my code so it was a tricky update

I think it's worth making a embed generator function perhaps.

I saw you have a lot of reused templates there

terse folio May 11, 2024, 9:34 PM

#

halcyon quarry <https://discordpy.readthedocs.io/en/latest/api.html#discord.Member.guild_avatar...

Yea, for people afaik

halcyon quarry May 11, 2024, 9:34 PM

#

I have no idea what is the best way to manage embeds, but it’s working pretty well so long as I remember to manage them appropriately

#

Correct way is probably same but in a Class instead of global variable

terse folio May 11, 2024, 9:36 PM

#

I had functions like
send_warning(message, channel)

This would create an embed that's yellow, add the description, and other info and send it to that channel.

terse folio May 11, 2024, 9:36 PM

#

halcyon quarry Correct way is probably same but in a Class instead of global variable

yea, a method belonging to a class of your character.
So it can access color information as needed and all

#

The only reason I wouldn't do the global thing is because sometimes you might want to do something like embed.set_author

#

which will modify the embed object

#

so you have more things to keep track of and remove if exists

halcyon quarry May 11, 2024, 9:38 PM

#

I was messing with that the other day… I was hoping that it replaced the actual sender avatar and name. But no, it puts a tiny icon and tiny name inside the embed

#

I was hoping to make system type messages use a more “ad_discordbot” author than the current chat char

#

Even whipped up a little icon

terse folio May 11, 2024, 9:40 PM

#

mhmm ^^

halcyon quarry May 11, 2024, 9:40 PM

#

thoughts so far?

terse folio May 11, 2024, 9:41 PM

#

Other bots do this using webhooks

#

No other way afaik

#

(without overwriting the profile)

halcyon quarry May 11, 2024, 9:42 PM

#

Hope you have SD and controlnet to try the /image cmd

terse folio May 11, 2024, 9:42 PM

#

will soon, just was testing if my code worked before making a pullrequest ^^

halcyon quarry May 11, 2024, 9:43 PM

#

Will be my first ever PR to review 😛

#

When you see how the controlnet menus show up, that’s how I’ll be switching to for the /character /imgmodel and /llmmodel cmds

terse folio May 11, 2024, 9:46 PM

#

~~first have to figure out how to undo a commit in vscode~~

terse folio May 11, 2024, 10:37 PM

#

if mentioning the SD type here isn't strictly necessary, we could remove that and I can add a global aiohttp session.

I don't think it will affect your bot much as it is now, because you're not doing that many aiohttp requests per second.
But having an already active session can save you ~100ms per request.

I should measure that actually

halcyon quarry May 11, 2024, 10:51 PM

#

It’s not necessary - I just like using it wherever applicable

#

Another thing in the back of my head to do, is allow the bot to be launched without SD active, with some way to get everything working with SD launched later

terse folio May 11, 2024, 10:52 PM

#

halcyon quarry Another thing in the back of my head to do, is allow the bot to be launched with...

it already works that way,
just throws some errors

#

but they don't stop the bot from working

halcyon quarry May 11, 2024, 10:53 PM

#

I meant, the SD elements

terse folio May 11, 2024, 10:53 PM

#

Ahhh

halcyon quarry May 11, 2024, 10:53 PM

#

Currently I just disable them all

terse folio May 11, 2024, 10:53 PM

#

Yea, in the functions you could add a check if the api returns None, and just display a warning that it's currently offline

#

that way you can start SD at any point and it will work

halcyon quarry May 11, 2024, 10:53 PM

#

Sort of does already just not verbosely

#

Well I need to write a means to build models and such post init

terse folio May 11, 2024, 10:54 PM

#

Build models?

#

For llms? or sd

halcyon quarry May 11, 2024, 10:55 PM

#

It gets the SD models list via api on startup, as well as controlnets

#

It’s nothing major to recode it’s just I have so many things to do and so little time lol

terse folio May 11, 2024, 10:56 PM

#

ahh,
can implement a little caching.

A function that checks if time since last check has been a few minutes and re-runs the list.
else returns the last known list

and just call this function each time instead of using the global list

halcyon quarry May 11, 2024, 10:56 PM

#

Wish I could clone myself

terse folio May 11, 2024, 10:57 PM

#

yea I know that feeling Sadness

halcyon quarry May 11, 2024, 11:01 PM

#

The next innovation I want to test is the idea of using the server name as “user1” and having the messages prefixed with the users name

#

Maybe a default user_bio option that explains its part of a server with multiple users

#

I’m thinking that it may cancel a response if another message comes in, and then handle both messages as one prompt

#

I’ve never used ST but this sounds like something it probably supports

#

Everything I’ve done for the past few months is exclusively for using it as a tool, not so much for casual chatting. Thinking an option like this could be good for chatting

terse folio May 11, 2024, 11:06 PM

#

halcyon quarry I’m thinking that it may cancel a response if another message comes in, and then...

That's something I want to do too.
I use streaming to send sentences as they come in.
Could stop the stream (or cut it off mid-sentence) if a user sends something

halcyon quarry May 11, 2024, 11:08 PM

#

I already added something to the history management… need to tweak it a bit then actually try using that method

halcyon quarry May 11, 2024, 11:28 PM

#

If a user msg is provided but no llm resp, it queues it to a list. When both are received it merges them with \n\n

#

Eh I got the ball rolling

terse folio May 11, 2024, 11:29 PM

#

so if multiple people are talking/or you spam a bunch you have it reply to each?

#

If message history is global for that channel (not per user)
You could also have an option to only reply to the last message sent if not crrently generating.

halcyon quarry May 11, 2024, 11:37 PM

#

Currently it replies to all messages as they are received sequentially

#

Regardless of spam

terse folio May 11, 2024, 11:37 PM

#

^^
Oh that gives me an idea how to mix the 2 where it feels natural

halcyon quarry May 11, 2024, 11:38 PM

#

I’ll send a screenshot what I have so far, in a few mins

terse folio May 11, 2024, 11:39 PM

#

terse folio ^^ Oh that gives me an idea how to mix the 2 where it feels natural

normal messages could be treated as replying in bulk.
But if users use the discord reply feature or mention the bot, it would also choose to reply to them independently at that point in the history.
(yes this would be complicated to implement)

Something i'll try out myself and report back on how it went if I get to it

halcyon quarry May 11, 2024, 11:41 PM

#

I’ve got some really complicated shit working, so that seems quite feasible

#

Mentioning it for a guaranteed focused reply is good

terse folio May 11, 2024, 11:42 PM

#

I think tracking the message id in the history would be the way to go

#

create a temp copy of the messages, and pop the last item until you reach the one with a matching message id
and pass that to the llm

#

You should be able to include it in the message dicts with no consequences

#

Textgenwebui ignores any attributes that aren't [role, content, image]

halcyon quarry May 12, 2024, 1:01 AM

#

terse folio I think tracking the message id in the history would be the way to go

Reviewing this PR... I don't see why the add_lora_to_model() function can't stay?
Should work... it's straight out of the code most of that other stuff comes from server.py

#

It would be in shared.args if included in the launch flags

#

Personally I have no means to test it because I'm not bothered to figure out txt gen loras heh

#

otherwise I'm liking the changes.

#

I know you've probably looked at some of these functions and went, what in the bloody hell

#

you caught a few bugs! damn I'm impressed

#

{homepath} > {home_path}
value > img_censoring

#

very nice, thank you

terse folio May 12, 2024, 1:14 AM

#

^-^
Vscode highlighted some unresolved variables

#

also left a #TODO on one line where add_lora() is an undefined function

#

oh you got that

#

missed the above message

halcyon quarry May 12, 2024, 1:15 AM

#

ahhhh.... I need to import that

terse folio May 12, 2024, 1:16 AM

#

Yea, was thinking it was a from x import *
deal, but didn't see any of those in the imports

halcyon quarry May 12, 2024, 1:16 AM

#

from modules.LoRA import add_lora_to_model

terse folio May 12, 2024, 1:16 AM

#

terse folio Yea, was thinking it was a `from x import *` deal, but didn't see any of those i...

It's also not the best practice to use import *,
can make debugging even more confusing

halcyon quarry May 12, 2024, 1:17 AM

#

I had combed through the webui loading code about a month or so ago, slipped up on that one

#

Oh wait

#

I see

#

I'm importing the whole LoRA module

terse folio May 12, 2024, 1:18 AM

#

All good, I need to figure out how to get the textgen-webui environment to merge with the discord bot, while also keeping them seperate for git purposes.
How do you do that? If you do

halcyon quarry May 12, 2024, 1:18 AM

#

don't need the whole thing though

#

Could you roll that lora thing back, and pop from modules.LoRA import add_lora_to_model in place of the from modules import LoRA? Will then merge the whole thing

terse folio May 12, 2024, 1:20 AM

#

halcyon quarry I know you've probably looked at some of these functions and went, what in the b...

It reminds me a lot of what my first few bots looked like honestly.
Not bad, but it did get overwhelming

halcyon quarry May 12, 2024, 1:20 AM

#

really I don't know 😛

terse folio May 12, 2024, 1:20 AM

#

halcyon quarry Could you roll that lora thing back, and pop ` from modules.LoRA import add_l...

Sure

halcyon quarry May 12, 2024, 1:20 AM

#

When I forked this bot it had ~700 lines, 1/3 of which was the textgen startup

#

not too long ago I figured it was probably outdated and figured out where it was all sourced from (server.py)

terse folio May 12, 2024, 1:21 AM

#

Mhm, I think server.py can be imported as a module too

#

Maybe it was a different file like one_click.py

#

that had a if __name__ == __main__

halcyon quarry May 12, 2024, 1:22 AM

#

This code is all from like the last 1/2 to 1/3 of the file though, dont think the beginning means anything

#

well, I mean the first half is all UI related

terse folio May 12, 2024, 1:23 AM

#

Interesting

halcyon quarry May 12, 2024, 1:23 AM

#

I see, its more or less from line 176 down

terse folio May 12, 2024, 1:24 AM

#

I made a new commit

halcyon quarry May 12, 2024, 1:25 AM

#

got it - going to test real quick but looks good on paper for sure

terse folio May 12, 2024, 1:25 AM

#

Also wow, 5000 lines of code in one file 😅

halcyon quarry May 12, 2024, 1:26 AM

#

yeah, that's what I was talking about 😛

#

I did not learn coding the right way. Would be more managable split to modules

terse folio May 12, 2024, 1:27 AM

#

Absolutely

#

I would start with sperating all the textgen stuff into it's own file

#

like "textgen_utils.py"

#

since that takes up the most space in your file, and probably wont be edited too often

halcyon quarry May 12, 2024, 1:38 AM

#

we also need to roll back that bulk message deletion 😛

#

terse folio May 12, 2024, 1:38 AM

#

ohno that broken?

halcyon quarry May 12, 2024, 1:38 AM

#

And here is with the original code with the arbitrary sleep delays

#

terse folio May 12, 2024, 1:39 AM

#

the warnings are normal, it says it will retry in a moment based off what discord told the bot to wait for.
But yea, I didn't realise it would be logging warnings like that.

Sure

#

There you go ^^

halcyon quarry May 12, 2024, 1:40 AM

#

👏 🙌

terse folio May 12, 2024, 1:42 AM

#

Discord only starts banning if you modify the bot's internal code to bypass the rate limits

#

both options you have are safe ^^

halcyon quarry May 12, 2024, 1:43 AM

#

yeah - just trying to avoid the debug spam

#

There's one thing that you may know more about than I do... I had to go back and forth with ChatGPT for quite some time to resolve an issue a while back, unsure if it is the ideal solution or not

terse folio May 12, 2024, 1:44 AM

#

Sure, what's that?

halcyon quarry May 12, 2024, 1:45 AM

#

When calling chatbot_wrapper() - the TGWUI function to generate text (not a coroutine), it blocks the Discord heartbeat

terse folio May 12, 2024, 1:45 AM

#

tgwi is not designed to be async

#

but if you have to use internal functions

#

use await loop.run_in_executor(None, chatbot_wrapper)

#

this will create a thread

#

and run it in the async loop

halcyon quarry May 12, 2024, 1:46 AM

#

Yep, thats what I did. Ok just wanted to ensure that is the right answer

terse folio May 12, 2024, 1:46 AM

#

^^ okay

halcyon quarry May 12, 2024, 1:46 AM

#

I had no fackin clue what it was but chatgpt said it and it worked

#

I need to implement that with the LLM model loading as well

terse folio May 12, 2024, 1:47 AM

#

Yup, I remember reading about it a lot in the discord.py server back then,
Never really understood it at the time, how it worked.

People always said to run large blocking functions in it.
Like image processing

halcyon quarry May 12, 2024, 1:47 AM

#

will just add it right now

#

before I put it off again

terse folio May 12, 2024, 1:47 AM

#

halcyon quarry I need to implement that with the LLM model loading as well

It should work all fine, I did that :)

halcyon quarry May 12, 2024, 1:48 AM

#

erm... ya did?

terse folio May 12, 2024, 1:48 AM

#

But, you should set a flag in your bot that stops any new text generation until a model has loaded

#

else you might get some errors

halcyon quarry May 12, 2024, 1:49 AM

#

The task queue thing solves that

terse folio May 12, 2024, 1:49 AM

#

halcyon quarry erm... ya did?

https://github.com/oobabooga/text-generation-webui/pull/5952/commits/3ffb09d465fa0781ef68efe6eaeb83c1ce03354b

Cool things to learn from here

#

I converted the extension to use run_in_executor

#

but had to do some special stuff to get streaming to work asyncronously

halcyon quarry May 12, 2024, 1:50 AM

#

Ohhhhhhh I was wondering wtf you were saying about that, b/c I did not see it added to my code.

You meant upstream

terse folio May 12, 2024, 1:50 AM

#

Yea, just meant that I played with the idea :)

halcyon quarry May 12, 2024, 1:50 AM

#

well until that's merged, right now long model changes are causing heartbeat issues

#

I'll temporarily just add it in bot.py

terse folio May 12, 2024, 1:51 AM

#

halcyon quarry well until that's merged, right now long model changes are causing heartbeat iss...

It's for the openai extension

#

but wanted to share some of the changes to various functions

#ad_discordbot (Fork of Fork of xNul's bot)

Pushed a very nice update for the bot

PUSHED THE BUG FIX - Restoring extension updates during runtime (TTS, etc)

Pushed an update that's going to show a gazillion changes.

Pushed another update already 😆

Pushed a super cool update for anyone avidly using the image generation features

Nice update for the README

Pushed a nice update I recommend all to get

Pushed an update that should enable the usage of LLM LORAs.

Also, the bot will now load custom model specific settings, if you saved those to config files via the webui

Pushed an update that adds the following new 'tags':

Updates:

Pushed an update that resolves errors when changing LLM models

It will now check user-config.yaml to see if the new model has a defined loader and use it

Otherwise, it uses a function to try guessing the correct loader

Otherwise, it may fail to load the model 😄

Pushed a pretty nice update

Pushed yet another update!

DELETED from config.py change_username_with_character and change_avatar_with_character and char_name

Pushed a massive update

If anyone has any issues after updating:

Pushed another update... you can't resist this one!

Pushed the update with the Flows feature.

I'm not dead! Pushed a nice update:

Pushed a nice update

Pushed another big update!

Ships with a new "Flow" and fine-tuned character I made:

Pushed the new Dyanamic Prompting feature

Pushed more updates

Overhaul to Image Model handling:

Pushed major update

Pushed an update that handles img gen failure much better.

Just added this bad boy to the Wiki

Pushed this update

Pushed a nice update for error handling

Pushed an update adding Embed settings

Pushed a pretty simple update

It will now check `user-config.yaml` to see if the new model has a defined `loader` and use it

DELETED from config.py `change_username_with_character` and `change_avatar_with_character` and `char_name`