#Fork of xNul's discord bot that integrates with oobabooga

1 messages · Page 2 of 1

supple socket
#

Generating something right now to see how it comes out

undone escarp
#

as an extension?

supple socket
#

No, just on its own

#

I couldn't even get it to work on its own from the repo before

supple socket
#

Bless git checkout

#

I need to stop messing with bark infinity, almost broke everything because of nltk

#

It'd be nice to have it as an extension for the bot, but

#

Getting it to generate longer context doesn't exactly seem to be easy right now

#

I'm just gonna leave it alone for the time being and work on other stuff

rancid fable
#

Have u watched a tutorial for install and use? I saw one by Olivio a few weeks ago - probably not helpful for the bot integration though

supple socket
#

I'm in the suno discord server and the guy that made the fork for bark infinity is in it, but. I haven't spoken to him yet.

#

I'm gonna stop messing with it for now either way and work on actually getting SDL to work properly so I can make progress with c++.

rancid fable
#

Suggestion: add "Custom stopping strings" to config.py

I tried adding this myself but it just doesn't work (probably doesn't jive with line 423 in bot.py)

undone escarp
#

that should work actually

#

the formatting of it is just a little weird

rancid fable
#

err

#

nvm, I see that's already there and probably does work 😛 I'm trying to figure out wtf my issue is here...

#

I saw stopping strings a few times so assumed there was a problem with that

#

Yesterday morning I updated ooba, and updated the bot. Did not edit bot.py whatsoever and I didn't make any weird changes to config.py or my character

#

This error appears when the bot is generating reply

#

Yeah I have no idea what this "tfs" is - I tried searching all the relavent files for "tfs" and there are no instances of it

rancid fable
#

welp, just gonna try reinstalling everything all over.

undone escarp
#

Looks like they added another key to the state dict

#

Should be a fairly easy fix

rancid fable
#

For anyone else having trouble just update to current, then use git checkout HEAD~4
For some reason it wouldn't let me just use git checkout b6c407f

#

error was "couldn't find the files from pathspec"

#

Well it found the damn files once I used HEAD~4

#

Woops.... that's actually not a good commit

#

OK - ideal commit to roll back to is b4662bf - the tfs and top-a params were added to the api in next commit

undone escarp
#

try adding tfs and top-a to the parameters in the char file

#

wait no, in the config file sorry

#

under here llm = { 'state': { 'max_new_tokens': 400, 'seed': -1.0,

#

no idea what the values should be though

#

llm = { 'state': { 'max_new_tokens': 400, 'seed': -1.0, 'tfs': None, 'top-a': None,

I'd try something like that

#

or maybe 0, or 1.

#

once I get time I'll go hunt for the actual values

rancid fable
#

Ill give it a shot!

rancid fable
#

Heres a little bug fix

#

current code adds a line break before the prompt suffix

supple socket
supple socket
rancid fable
#

that also works 🙂 But he made changes that made it compatible with a few more versions

#

current bot version is compatible up to b6c407f

supple socket
#

Oo

#

Good to know

rancid fable
#

@undone escarp
if positive_prompt_suffix: payload["prompt"] = payload["prompt"] + positive_prompt_suffix if negative_prompt: payload["negative_prompt"] = negative_prompt if presets: for preset in presets: if preset['trigger'].lower() in payload["prompt"].lower() or preset['trigger'].lower() in ctx.message.clean_content.lower(): payload["prompt"] = payload["prompt"] + preset['positive_prompt'] payload["negative_prompt"] = payload["negative_prompt"] + preset['negative_prompt']

#

triggers were also forcing a line break

undone escarp
#

And line breaks are triggering your ocd?

#

Wonder what causes it

#

Maybe there's a hidden line break in the yaml

#

I'll try to remember

rancid fable
#

er, hmm... the triggers still make a line break even with this adjustment

#

but the one I mentioned earlier resolves the one for prompt suffix

#

I'm sure we're on relatively same page of OCD 😛

rancid fable
#

@undone escarp your bot has a “conversation recency” not found in ooba… what happens if this is set to 0?

undone escarp
#

That's my own stuff

#

Don't think it really works anymore

#

It's just a way for it to remember that it was in a conversation with the user for a certain amount of seconds

rancid fable
#

Got it 👍 I think ooba just gave me the solution I’ve been seeking (no saved dialogue).
Just set maximum prompt size to 0

undone escarp
#

looks like adding
"tfs": 0, "top_a": 0,
fixed it

#

no idea what the values do though 😄

rancid fable
#

Yay 😀

rancid fable
#

If your coding itch hasn’t been thoroughly scratched, could look into trigger words 🙂
-it’s triggering on partial matches, for instance I made an trigger for “emo” which I’ll be commenting out b/c it’s triggering for a lot of other words (lemon, emotion, etc)
-would be nice to have support for multiple trigger phrases per definition, but your duplicate lora thing prevents the worst outcomes

#

Didn’t test to see if it catches all duplicate prompt additions 🤔

undone escarp
#

I've started a little noteboard for ideas now

#

I'll get on it once inspiration strikes

undone escarp
#

I cant really see a line break being added

#

think there might be a line break in the yaml itself, can I see it?

#

or - I could just force a trim of each string I guess

#

I cant really see a line break being added

rancid fable
#

Ever since I included a positive_suffix in the character, it has made a line break for it. Can be seen in A1111 pnginfo, as well as external metadata viewers

rancid fable
#

Copycatted someones prompt I saw on Reddit (Robin Williams as Tom Bombadil)

#

Gotta love when good results from LLM + SD

undone escarp
#

missing the blue coat and yellow boots!

#

man robin williams would have killed it

rancid fable
#

Still very happy with your bot. Another suggestion is to be able to have a predefined default checkpoint per character

#

For instance I found a neat 2.1 model, and so I made another instance of my character which has ideal parameters for it (doesn't need HiRes fix, different steps, different prompting, etc). Would be nice to autoload a specific 2.1 SD model when picking the character, then autoloading my preferred 1.5 SD model when changing to the other character

undone escarp
#

Oh yeah model loading.. Should be doable

rancid fable
#

Welp, I updated ooba

#

I’m mainly interested in the new “begin reply with” feature, gonna see if I can roll back to that commit (w/o errors)

rancid fable
#

I rolled back to 9b0e95a which includes the 'Start reply with' feature - and it works 🙂

undone escarp
#

another key error 😄

#

can most likely just add "microstat_mode" into the state variables

rancid fable
#

@undone escarp Indeed! Just updated to current, and updated bot.py as such - works

#

"mirostat_mode": 0, "mirostat_tau": 5.00, "mirostat_eta": 0.10

rancid fable
#

yo merc, just wondering if there is any rhyme or reason for some things to be in double quotes, and others to be in singles?

undone escarp
#

Lol nope, not there at least

rancid fable
#

Thanks 👍🏻

undone escarp
#

just sprinkling little ocd traps randomly around

#

best to just not look at the code

rancid fable
#

@undone escarp so, I was contemplating a snippet of your code for awhile until I could understand what I was looking at. I was then able to prompt ChatGPT better to revise the code rather than confusing it with extra bits and poor prompting.

I wanted to be able to define multiple trigger phrases per definition (comma separated), AND not trigger for partial matches (text contains "emotional" - do not trigger for "emo")

This solves both:
if presets: triggered_definitions = [] for preset in presets: triggers = preset['trigger'].lower().split(',') for trigger in triggers: words = trigger.strip().split() if all(word.lower() in payload["prompt"].lower().split() or word.lower() in ctx.message.clean_content.lower().split() for word in words): if preset not in triggered_definitions: payload["prompt"] += preset['positive_prompt'] payload["negative_prompt"] += preset['negative_prompt'] triggered_definitions.append(preset) break

#

Test .yaml which only triggered one time if the user + bot replies contained both "self photo" and "selfie"
` - trigger: 'self photo,selfie'
positive_prompt: ', test positive1'
negative_prompt: ', test negative1'

  • trigger: 'emo'
    positive_prompt: ', test positive2'
    negative_prompt: ', test positive2'`
#

It is also only triggering for exact matches. if it finds "emotional" the definition for "emo" is not triggering now

#

I also wanted to see if I could get ChatGPT to revise the code even further so that it still accomplishes the above - but also check to see if a definition was triggered for an exact phrase that is also in another definition. This code will only use the definition that has a longer trigger phrase and ignore the one with the shorter phrase.

#

`if presets:
triggered_definitions = []
max_length = 0
for preset in presets:
triggers = preset['trigger'].lower().split(',')
for trigger in triggers:
words = trigger.strip().split()
if all(word.lower() in payload["prompt"].lower().split() or word.lower() in ctx.message.clean_content.lower().split() for word in words):
if len(trigger) > max_length:
max_length = len(trigger)
break

for preset in presets:
    triggers = preset['trigger'].lower().split(',')
    for trigger in triggers:
        if len(trigger) == max_length:
            words = trigger.strip().split()
            if all(word.lower() in payload["prompt"].lower().split() or word.lower() in ctx.message.clean_content.lower().split() for word in words):
                triggered_definitions.append(preset)
                break

for preset in triggered_definitions:
    payload["prompt"] += preset['positive_prompt']
    payload["negative_prompt"] += preset['negative_prompt']`
#

For example, if definition including "self" and definition including "self helpline" is matched, it only trigger the definition for "self helpline"
` - trigger: 'self,selfie'
positive_prompt: ', test positive1'
negative_prompt: ', test negative1'

  • trigger: 'self helpline'
    positive_prompt: ', test positive2'
    negative_prompt: ', test negative2`
#

(Sorry for blowing up your thread)

#

Real world use case (NSFW) gimme_nsfw_please
|| - trigger: 'anal'
positive_prompt: ', (1boy), (rear focus), deep penetration lora:povDoggyAnalCreampie_v3:0.7'
negative_prompt: ', solo, (in vagina), elongated'

  • trigger: 'cowgirl anal'
    positive_prompt: ', (1boy), (reverse cowgirl anal), (close up), (from behind), (rear focus), deep penetration lora:PovReverseCowgirlAnal-v2:0.8'
    negative_prompt: ', solo, (in vagina), elongated'||
rancid fable
#

I feel bad about bringing up another issue after dumping all the shit above, but there is something super weird going on with SD's API and a paramater called "Seed Resize"

#

er.... hmm...

#

It just occurred to me that maybe the Duplicate Lora thing you implemented wasn't actually working? But was just keeping it out of the metadata? I don't know. I've been getting some super wonky images here and there, and so I decided to try and investigate the problem

#

I regenerated one of the images via the WebUI using all the same settings, but got a very different result.

#

I underlined the only difference I can find.

rancid fable
#

@silk galleon I'm sorry to ping you but since you outed yourself as an SD developer, maybe you could shed some light on this issue 😛

silk galleon
#

wha

#

seed resize is an auto webui thing not an sd thing

#

-1x-1 is probably the placeholder default for disabled or somethin

rancid fable
#

I actually just solved the problem

#

I added these parameters to the extension's config file, and it made the Seed Resize go away

#

'seed_resize_from_h': 0, 'seed_resize_from_w': 0

#

Yep, absolutely amazing. I just made the bot generate the same reply, and set the fixed seed value. Perfect output this time

#

@undone escarp this is definitely a bug. I went back to a few other images I had that were particularly wonky, and it was this Seed Resize parameter mucking things up

rancid fable
undone escarp
#

äääh

#

hard time following it all

#

but you seem to have fixed it on your own, nothing I need to do?

rancid fable
#

Adding those 2 params to config.json will prevent Seed Resize issue which seems to have a dramatic negative effect on images with LORA.

#

You should fix it for the good of mankind 😆 A1111 has ~2k open issues so who knows if they’ll fix this on their end

#

The tweak I shared about multiple trigger phrases per definition + only triggers for exact matches: also very nice… I say exact matching is a bugfix

undone escarp
#

alright cool, I'll do that thanks

fair anvil
#

Hi there!

#

@undone escarp is the bot already capable of working with extensions like edge gpt? I've been "out of order" for some weeks now and I'm not up to date at all

undone escarp
#

havent gotten those to work no, sorry

#

works with some extensions, not all

fair anvil
#

thanks anyways 🙂

#

nad BTW; this is not directly related with the bot, but just our of curisoity, my idea is to provide the bot to my server with some tech info about some software (Blender in my case) any idea on how to achieve this? I tried training but the outcome is not good at all

#

and the other things I can think about are not there, like some lang chaing extensions or something similar

rancid fable
#

Nothing to do on your part, just sharing another little tweak I made with a little help from my pal ChatGPT.
I have a relatively low max new token setting (100 tokens), the response from LLM often ends with a half-formed word or only part of a larger phrase.
I updated this little bit in the code to find the last comma in the LLMs reply, and just trim it out along with whatever follows
else: last_comma_index = prompt.rfind(",") if last_comma_index != -1: image_prompt = prompt[:last_comma_index] else: image_prompt = prompt

#

There was also a bug in the snippet I shared here <#1099227372655489035 message> I updated it (in case anyone tried it out)

#

I need to try this same stuff with WizardCoder-15B. Looks super impressive from what I've seen

rancid fable
#

quick FYI, chat_prompt_size is no longer a setting in oobabooga

supple socket
#

Haven't been around in a while.

#

Exllama seems great from what I've seen on paper. Increased token context size and relatively massive speed improvements.

Since it's a transformer based thing, I wonder how implementation with the bot would work.

#

Also wondering when a complete cure for llm Alzheimer's will come out.

#

The concept of superbooga doing what it's supposed to makes sense, but... it doesn't. (Edit: That's because it's clearly meant to be used and configured in the webui, not used in the discord bot.)
If all context could be read from a db stored as history locally that it could read from with essentially no limit, that'd be great.

rancid fable
#

You just use --loader exllama in your command to launch bot.py

#

it works great 👍

#

I'm also using llama.cpp with the bot... again, just need to specify the loader and stuff like n-layers in the cmd

supple socket
#

Ooo, thanks. So should I just install exllama in the textgen environment and then use --loader exllama?

#

(If I clone it into repositories and then install the requirements)

#

Or is the bot in Merc's repo updated and working with the latest version, or a previous commit that works with exllama?

rancid fable
#

I used 1-click windows installer... exllama is built in

#

so long as you are updated with ooba

#

and yes exllama is in the textgen environment

supple socket
#

Would update_windows.bat install the latest things & still have the discord bot work properly?

rancid fable
#

You don't need to install exllama separately it is now buit into oobabooga

#

The latest version of the discord bot is compatible with the latest ooba

supple socket
#

Great. Thanks. So I should be able to just do update_windows.bat and then just grab the latest bot code from the repo.

#

I haven't updated ooba or the bot in like a month.

rancid fable
#

won't hurt to update both 🙂

#

if things don't work... just do the 1-click installer from scratch

supple socket
#

Mm, I'll just do a fresh install of the one click & update the bot.

#

Probably cleaner faster and less of a headache.

supple socket
#

Everything seems to be working okay.

#

Hallucinating like crazy, but maybe that's because I messed with token length a bit.

#

Also... she's typing only in parentheses?

#

Hmm. I guess it was just a weird response to my initial prompt.

#

Yeah... this is really fast. Nice.

Thanks again @rancid fable 🙂

supple socket
#

Holy fucking shit.

#

This is amazing.

#

Idk if WizardLM is better or not, I'll have to test, but

#

So far TheBloke_Wizard-Vicuna-13B-Uncensored-SuperHOT-8K-GPTQ is performing really well.

#

It's so fast too

supple socket
#

The only issue I seem to have is that she cuts off a 0 at the ends of dates now.

#

So instead of 2023 for the date... she'd write 203.

rancid fable
#

very odd

undone escarp
#

In the middle of a sentence, or at the end?

supple socket
#

Wherever, whenever she says the date.

#

In this ss it's in the middle, I guess.

undone escarp
#

What the hell

#

Does the correct date appear in the terminal at the beginning of the context?

supple socket
#

I didn't think so, but I checked again just to be sure.

#

I said a 0 was missing... I was tired. I meant a 2... or just a number altogether. But yeah.

agile sage
#

So uh, my bot now talks in multiple channels, without their name being mentioned. I checked the bot.db and it's set to the only channel I want

#

Oh, nvm, I see that there is a 'main_channels' table.

undone escarp
#

It should display an embed the first time it starts talking after startup that gives some info on that

rancid fable
#

@undone escarp There's a new key needed in state

            "repetition_penalty_range": 0,
undone escarp
#

Excellent thanks, will update

rancid fable
#

@undone escarp I understand you may be all done with updating this, but just mentioning that there are some errors after updating Ooba to latest.

-It won't load the bot with error: AttributeError: 'Namespace' object has no attribute 'flexgen'
Note that I # commented out the the lines "if shared.args.flexgen:", "return....", and "else" so it wouldn't check for flexgen.
This worked, and the bot loaded.

-When I message the bot, it throws an error: "AttributeError: module 'modules.shared' has no attribute 'history'"

#

There could be more errors behind this, couldn't figure out a workaround here

undone escarp
#

Oh, moved the history I take it

#

I've just been having a summer and then some Diablo which moved my attention away

#

Namespace error sounds Odd

rancid fable
#

I've been playing Diablo, too, but almost certainly something different from you 😛
There's an extremely huge mod for Diablo 2 called Median XL.

undone escarp
#

Oh yeah heard about that

#

Did a bit of D2R before d4 came out for nostalgic purposes

#

But now it's baldurs gate 3

agile sage
#

I think the issue is that "history" doesn't exist (or isn't initialized) on startup, because Ooba now uses sessions.

#

The flexgen error is indeed an easy fix (remove 3 lines)

rancid fable
#

Median XL is absolutely staggering in how much it added / changed. They push huge updates every 4 months and seem to have been doing it for like 20 years

#

most players don't ever reach the most difficult content it has to offer heh

rancid fable
rancid fable
#

@undone escarp I made some progress in updating the code... but I'm missing something.
I'm pretty sure that history now belongs in state, like this:

        "state": {
            "history": {'internal': [], 'visible': []},

And I think the few instances of shared.history are to be replaced with, for example:
user_input['state']['history']['internal'].append([user_input['text'], last_resp])

#

the character context just isn't being fed into user_input correctly now...

undone escarp
#

Arf

#

I'll get to it some dayyyy

rancid fable
#

oobabooga only changed like 250+ lines with the word history in it on that particular commit

rancid fable
#

Ok... with the changes I made, I think it's actually set up correctly except the problem is that now only the literal "context:" portion of the character is treated as user_input -- the persona and example dialogue are no longer part of the package

#

Well, I am able to personally avert this issue by just removing "greeting" and "example dialgue" and just making everything fall under "context" in the character files

#

Ok... looking at the current Example.yaml (ooba example char) I see that example dialogue just isn't a thing anymore.
So the indices are now name[0], greeting[1], context[2] The persona is shown in example to just be first line of the context

rancid fable
#

@undone escarp I did figure out a solution, there's probably a more "right way" that this isn't 😛
I modified your version of bot.py to be compatible with the latest version of ooba.

#

You can see my comment here about what exactly was modified...

#api-dev-help message

There were also some new keys, removing the flexgen thing, etc

#

I renamed history_hack to session_history though, because it sounds slightly more legit

#

The only other thing you would need to update is just swap the order of "greeting" and "context" in your Llaya example char

#

The only thing "wrong" about this is probably just where I stuck the variable

supple socket
#

I'm currently trying to get the bot to connect to a voice channel so that it can use tts. I got as far as following a youtube guide on how to set up a bot to join a voice channel, but I don't know if it's due to conflicts, or what.

@client.command(pass_context = True)
async def join(ctx):
    if (ctx.author.voice):
        channel = ctx.message.author.voice.channel
        await channel.connect()
    else:
        await ctx.send("You are not in a voice channel. Please join one so I can join!")
        
        


#leave voice channel command
@client.command(pass_context = True)
async def leave(ctx):
    if (ctx.voice_client):
        await ctx.guild.voice_client.disconnect()
        await ctx.send("I have disconnected from the voice channel.")
    else:
        await ctx.send("I am not in a voice channel!")``` 

That's the code I ended up writing because of the video. I don't know if it has to be bot.command or client.command, although it seems it's client because that's what I see defined in bot.py. 

The prefix in bot.py is "." from what I can see, too. But when I did .join, the bot wouldn't join the voice channel I was in.

Loading the bot with silero tts worked- or at least... it still generates an audio file when it responds with text. But I haven't been able to test if it'll play that audio when it's connected to the vc. 

The next thing I'd want to do after that is use a custom voice, or use bark instead of silero. And it'd be a bonus if the stt would work as well, so that I could just speak to the bot and have it respond to me while in a vc.
#

I hate feeling stupid and not knowing how to do something when it clearly is and would be possible... I could just learn python but I want to focus on & learn more c++ since I've still barely even scratched the surface with it.

rancid fable
#

try @client.hybrid_command

#

You could also try manually entering the channel ID. If you have developer mode setting ticked on in discord settings, you can just right click a channel to get its id #

supple socket
#

Thanks. Tried multiple things with different combinations including the things you said, still couldn't get it to work though. :/

rancid fable
#

Maybe the bot doesn’t have appropriate permissions?

#

Go to 0auth page and make sure to tick bot then all the appropriate boxes, reinvite the bot to your server

supple socket
#

Alright, I'll try that right now.

#

It works now. I guess I just didn't give it perms to join vc's when I first made the link. Thanks.

#

Well. Gonna try it with the tts extension and see if it'll play.