dev-contrib

[patent pivot]

PR looking for reviews from anyone who knows SQL good and can optimise my query https://github.com/python-discord/site/pull/418

[tawdry vapor]

Why do you need to convert the epoch back to a timestamp to group by it?

[patent pivot]

yeah that is true

fixed

[tawdry vapor]

Does flipping the where condition matter, so that it short circuits

Presumably a boolean check is faster than a string comp

[patent pivot]

order of execution is not guaranteed by SQL standard

I think Postgres looks at the conditions and figures out the cost of each one

[tawdry vapor]

alright

[patent pivot]

PostgreSQL doesn't care at all about the order of entries in a WHERE clause, and chooses indexes and execution order based on cost and selectivity estimation alone.

[tawdry vapor]

I think select count(*) with a subquery is as good as it gets

[patent pivot]

yeah

[tawdry vapor]

Well, maybe you could do it with a join?

Not sure if it'd be faster

[patent pivot]

hmmmm

join seems to be slower

I should probably add an index to the messages table on the author_id column

[tawdry vapor]

It'd probably be marginally faster if the author was an int instead of a string

[patent pivot]

yeah, I will look at migrating that eventually

the reason why it is not is because the visualisation tool uses javascript

and does not yet use bigints

[cold moon]

I think https://github.com/python-discord/bot/issues/553 should get updated to latest directory structure.

[green oriole]

Good lord, I forgot about that

!remind 17h this

[stable mountain]

Aye aye, cap'n!

Your reminder will arrive in 17 hours!

ID: 718, Due: 2020-10-28T08:07:22

[green oriole]

If no one does it before then

[hardy gorge]

MAybe we should just close that issue

[green oriole]

It is a good tracker though

[brazen charm]

Is it getting updated?

[green oriole]

It is supposed to be kept up-to-date, but right now it doesn't match the new directory structure

[hardy gorge]

I think we don't really want unittests for a lot of features in the bot after our reconsideration of the testing policy

It's slowing down development and often fails to actually capture the bugs we're suffering from

I'm in favour of writing tests for pure utility functions that are fairly straightforward to test

I'm no longer in favour of trying to test "everything"

[cold moon]

I think we should make that bug fixes should have unit test that ensure that this bug is fixed.

[hardy gorge]

I don't really think that's feasible for us in a lot of situations

I'm mean, sure, if it's a more or less "functional" part of the bot, I can get into it

but we've already seen that people are hesitant to touch parts of the bot because of unittests

And some of the unittests we have barely capture the important aspects of the things they're testing

For example, those filter rule functions? Those should be easy to test.

[green oriole]

I don't think regression testing is a good fit here, at leats not for the bot

[hardy gorge]

A function that interacts a lot with APIs? Not so much

[green oriole]

We very rarely even have cases of regression

[hardy gorge]

If you have to spend 15 minutes to figure out how to properly mock something in order to write two lines of actual assertions, it's wearing down the devs who have to do it

[brazen charm]

Writing proper unittests is plenty hard, I don't think forcing (or favoring) them for anything non critical is going to help development much. Even if it's just for small bug fixes they may have to do a lot of mocking around it with the amount of libraries relied on

[hardy gorge]

I think that if I were to write a small utility bot myself that takes care of critical features in a community, I would write unittests for most things

However, I think that we'd have to refactor a lot of our code base in order to make facilitate testing in an easy way

big chunks were not written in a way that makes it easy to test the features

[cold moon]

I think 2 extension groups should be fully covered with tests: moderation and backend.

[brazen charm]

Not as important for unittests, but they were also starting to take a fair amount of time which would increase a lot if the issues weren't dropped

[wild ridge]

Hey @cold moon, just thought you should know that I'll be overseeing the Dewikification project for a bit. I'll have a look at your PRs tomorrow, (got my hands full tonight) but if you have any issues/questions about site, or just want some eyes on a PR, don't hesitate to ping me!

[eternal owl]

should I splitting my single file(400 lines) in multiple commits or will 1 commit be fine? Its a single feature file

[stable mountain]

@green oriole

It has arrived!

Here's your reminder: this.
[Jump back to when you created the reminder](#dev-contrib message)

[celest charm]

@eternal owl

1. You could've build it incrementally, adding a commit on each step -- it might be easier to go through for the reviewer
2. You can just submit it in a single commit if you didn't -- nothing wrong with that, if it's one logical change
   (I suppose)

[cold moon]

@wild ridge Can you look at https://github.com/python-discord/site/pull/395 first, then to others, because I'd like to start working on displaying resources issue?

[wild ridge]

@cold moon Having a look at that now

[cold moon]

Thanks

[wild ridge]

@cold moon Made a couple comments on that PR. Nothing worth requesting changes for, just some stuff worth thinking about. Let me know if you're actioning any of those, and/or when you're ready to merge it

[cold moon]

@wild ridge Changes made in resources app PR

[wild ridge]

👍 merged.

[cold moon]

Amazing. Gonna fix other PRs too.

[wild ridge]

Excellent. I have a bit more to comment on once you've actioned those reviews, so let me know when you're done with those.

[cold moon]

@wild ridge But when I use TemplateView, how to test this? As I'm not gonna include testing pages to scanning.

It was easier to test old way

[wild ridge]

Can you link the comment you're referring to? @cold moon

[cold moon]

https://github.com/python-discord/site/pull/417#discussion_r513656338

TemplateView require that template is included to scanning

[wild ridge]

@cold moon I'd move the test pages and events pages to sub-directories under templates/events, move the PAGES_PATH variable to the settings.py, and then use something like

 @patch("django.conf.settings.PAGES_PATH", new=PAGES_PATH)

To override it in your test?

[cold moon]

Thanks, I'll try this as soon I can. I'm going to sleep now and tomorrow is my birthday

[wild ridge]

No problem!

And happy birthday :)

[obsidian patio]

!remind @cold moon 14h Happy birthday! 🎉

[stable mountain]

Aye aye, cap'n!

Your reminder will arrive in 14 hours and will mention 1 other(s)!

ID: 722, Due: 2020-10-29T08:59:28

[obsidian patio]

Since I know I will forget it hahah

[cold moon]

Thanks @wild ridge

[green mesa]

@cold moon happy birthday

[cold moon]

@green mesa Thanks🥰

[wild ridge]

🎉

[stable mountain]

@obsidian patio @cold moon

It has arrived!

Here's your reminder: Happy birthday! 🎉.
[Jump back to when you created the reminder](#dev-contrib message)

[sour sierra]

Happy birthday!

[tall cedar]

hi

[remote drift]

🎂 HAPPY BIRTHDAY 🎂

[cold moon]

@wild ridge https://paste.pythondiscord.com/paseguluye.py This doesn't work. This update PAGES_PATH conf, but not templates conf. And if I update templates conf there, this will need whole reload of application.

[wild ridge]

@cold moon I was thinking we move the events pages (and test event pages) to /pydis_site/templates/events/pages

[late wolf]

Does the bot use psycopg2 or asyncpg?

[green oriole]

We use the django ORM, so that's psycopg2 iirc

[iron timber]

can someone explain me how base64 is used ? i have a file in base 64 encoding and i need it to be turned to utf-8

i can only find examples of utf-8 turned to base64 but not other way around

[cold island]

Feel free to open a help channel for that #❓｜how-to-get-help

[iron timber]

oh wrong channel

[cold island]

This channel is for discussing the development of our core projects

[celest charm]

Are new runtime dependencies for snekbox still accepted? And what are the requirements for that?
I wanted to add lark. It sometimes comes up when discussing parsing stuff, and it's a very general and robust parsing library.
Also, It has 0 dependencies.

[cold moon]

@wild ridge Events app PR changes applied

[wild ridge]

@cold moon Great! I'm having a look through now

[patent pivot]

Anyone in need of a code review?

Are new runtime dependencies for snekbox still accepted? And what are the requirements for that?
I wanted to add lark. It sometimes comes up when discussing parsing stuff, and it's a very general and robust parsing library.
Also, It has 0 dependencies.
@celest charm fine by me, feel free to PR

[obsidian patio]

@patent pivot the SeasonalBot candy command would be nice to get merged soon. It’s currently completely broken

[patent pivot]

link me buddy boy

[obsidian patio]

.issue 496

[dusky shore]

GitHub

[seasonalbot] #496 Fix candy mini-game to group leader-board scores correctly

[obsidian patio]

I’ve already given it a review

[cold island]

I mean, there's one you're assigned to 😤

[patent pivot]

lol is there

[cold island]

https://github.com/python-discord/bot/pull/1259

[obsidian patio]

Do as Zig says or we’ll put you in the Zig soup as well 😋

[cold island]

Weren't you the one who put me there? 🤔

[obsidian patio]

Yep. You seemed so tasty

[cold island]

continuity issues 😳

[obsidian patio]

Do you remember the poem?

[cold island]

I remember that it's a great poem

[obsidian patio]

Lovely. We should add joe to the next zig soup poem

[patent pivot]

oh yeah

this is something I should review

[obsidian patio]

Btw joe, are you tasty?

[patent pivot]

zig SMH NO SCREENSHOTS

[cold island]

o shit

[patent pivot]

no

I'm Joe

[sullen phoenix]

why do you need a list of reactions when there's only one?

[obsidian patio]

Ah, snap. Maybe you don't fit the soup, then

[cold island]

Because I made it easy to extend

[sullen phoenix]

over-refactoring >:(

[cold island]

I mean, it reads better anyway imo

[sullen phoenix]

actually no

preemptive refactoring i think

hmm it seems more complex to me

[cold island]

https://cdn.discordapp.com/attachments/305126844661760000/769692477471981658/mods_-_Discord_2020-10-25_01-41-55.mp4 @patent pivot

[patent pivot]

very good

[cold island]

Although this one has an extra reaction

We should basically standardize the way we add emoji menus to embeds, but Scragly is working on something overlapping

Because we have a bit of code repetition currently

Because there are only so many ways to do it

[sullen phoenix]

yeah, there's also going to be a global trashcan thing

[cold island]

joe if you think I overcomplicated it too I can just make it account for the one reaction, but I had an idea that we might want to integrate modmail in there or something similar

[patent pivot]

I'll have a gander soon™️

[cold island]

f1re not saying your input is not important btw :P
you can leave a review too

[patent pivot]

I'll get the two of these done shortly

[sullen phoenix]

f1re not saying your input is not important btw :P
you can leave a review too
oh no, all good

i probably will when i have some more time

[patent pivot]

I don't listen to f1re any more on overcomplication comments after reading the source of his URL shortener

[cold island]

Hahaha but that's more of an intellectual exercise

[sullen phoenix]

lololol fair enough

just wait until the auth comes

[cold island]

Auth to use a url shortener? 🤔

[sullen phoenix]

only authenticated users will be able to create vanity URLs and it’ll track clicks and whatnot

[cold island]

Will you launch a partnership program?

[obsidian patio]

This is what I call overcomplicating things: https://tinyy.io/snakes

[sullen phoenix]

absolutely

[obsidian patio]

It's a 100 layers deep url hahah

[sullen phoenix]

i’ll start a whole conglomerate

/s

in case that wasn’t clear lmao

[cold island]

It's a 100 layers deep url hahah
Too many redirects to actually load

[obsidian patio]

Yep

You'll have to refresh when that happens and keep going down the 100 redirects

[cold island]

ah, I might need to fix something with the PR

Is there any particular reason to choose either asyncio.create_task or self.bot.loop.create_task? because I see both in the repo

[brazen charm]

asyncio.create_task is the higher level func that abstracts the loop out, but can't be used when there isn't a loop running (cog inits for example)

[cold island]

If I await the infraction search, the menu becomes unresponsive until pagination times out. Would creating a separate task be considered a proper solution? (it works, my question is just about convention)

[sullen phoenix]

seems fine to me

[cold island]

https://github.com/python-discord/bot/blob/master/bot/exts/moderation/silence.py#L40
hmmmm

Why are we looping every second only to check if it's been 15 minutes?

[brazen charm]

I did it to get second resolution on the timeouts when sent

Could've done it through datetimes but that'd complicate the code a bit and the loop is practically free anyway

[cold island]

I see

[trim cradle]

!code

[stable mountain]

Here's how to format Python code on Discord:

```py
print('Hello world!')
```

These are backticks, not quotes. Check this out if you can't find the backtick key.

[trim cradle]

If you post code in a markdown block without the py, you still get the older, longer message

I'll have to look into that

[green oriole]

This is what I call overcomplicating things: https://tinyy.io/snakes
@obsidian patio ERR_TOO_MANY_REDIRECTS

[obsidian patio]

Indeed. You’ll have to figure out a way to reach the end message

[green oriole]

Do you have to spam the refresh button haha

[obsidian patio]

That’s one option

Or you can use your programming skills

[green oriole]

I'm on phone, so no skills for now haha

If this is a rickroll, I swear

Hey hello ERR_CONNECTION_REFUSED

I killed f1re's website

[rapid igloo]

lol you can code on a phone online tho

[tawdry vapor]

@cold island the former only works if a loop is already running. Thus, the latter is useful for scheduling tasks before the bot starts (such as in init functions of cogs)

[cold island]

So if the loop is already running it doesn't particularly matter? or is asyncio better just because it's less noisy

[tawdry vapor]

It doesn't really matter at that point, whatever is more convenient

Like if you have access to the loop and don't want to import asyncio

I prefer to use asyncio.create_task where possible anyway

[sullen phoenix]

!tempban 710411235212132353 14d Only messages here are spam ads. Do not do that in the future.

[stable mountain]

:incoming_envelope: :ok_hand: applied ban to @silk canyon until 2020-11-13 12:42 (13 days and 23 hours).

[mellow hare]

Question: Are you able to link GitHub issues between repos?

Or do you have to link it via a full URL

[cold island]

you can link, but for example in our repos, if you want to link to a site issue from a bot issue, you have to do python-discord/site#<issue>

[green oriole]

Directly dropping the link should also work

[tough imp]

I'd use the full url and let github shorten it for display, e.g. here I'm making an issue on bot:

that will get displayed as such:

[cold moon]

@wild ridge Do you know how to resolve Pipfile.lock conflict in content app PR? I hate these conflicts, as always when I resolve it nothing works.

[wild ridge]

I'd just delete the lockfile and re-lock tbf

[brazen charm]

Yes, just get the pipfile sorted out, relock and use that

[wild ridge]

I'll have another look through those PRs tonight @cold moon

[cold moon]

Okay

[cold moon]

I think https://github.com/python-discord/site/issues/387 can be closed now, as my PR that resolves it is merged.

https://github.com/python-discord/site/issues/388 Also can anybody assign?

[green oriole]

I think https://github.com/python-discord/site/issues/387 can be closed now, as my PR that resolves it is merged.
@cold moon which PR?

[cold moon]

https://github.com/python-discord/site/pull/395

About resources in YAML, I don't understand what is these icons like in:

urls:
- icon: regular/link
  title: E-book
  url: https://automatetheboringstuff.com/
- icon: branding/amazon
  title: Amazon
  url: https://www.amazon.com/Automate-Boring-Stuff-Python-Programming/dp/1593275994/

branding/amazon, regular/link, what is these? To what these points? As when I look current resources pages, they are just fa-icons

[green oriole]

Maybe just a file path?

[cold moon]

I don't think so, as they are displayed as fa-icons in HTML and I can't find images for them

[mellow hare]

Do we have an example in the bot where we use the redis thingy?

Trying to find an example so I get how it works

[brazen charm]

https://github.com/python-discord/async-rediscache/blob/master/async_rediscache/types/cache.py#L36, the dm relay is probably the simplest cog that uses it

[mellow hare]

Ah, solid. Thanks, Num

So is the redis persistent? Or does it get wiped when the bot restarts

[brazen charm]

It persists in memory and gets backed up to normal storage

[mellow hare]

Gotcha. Sorry, feel like I'm asking dumb questions.

[cold island]

Don't worry, they're a lot smarter than the ones I ask 😃

[mellow hare]

Is there a reason that we don't have a namespace for the RedisCache in the dm relay?

Wait

Bleh, no never mind

I see my mistake now

[cold moon]

What is misc folder in resources YAML folder?

[green oriole]

What's in it?

[cold moon]

good first issue tag and python cheat sheet

[green oriole]

I'm not sure where to look for, I can't find it on the dewikification branch

[cold moon]

It's under home app resources folder

[green oriole]

Ah thanks

I'm not sure, we should ask @crude gyro, because I can find the cheat sheet at the bottom of the resource page but not the good first issue thing

[crude gyro]

so, the YAML stuff is badly out of date

because it's not actually in use

yet

but honestly I have no idea what you guys are asking

[cold moon]

Yes, I think I need to make bigger refactor for these resources

[patent pivot]

I think the YAML is probably fine, it just needs updating with everything from site

[sour sierra]

https://github.com/python-discord/seasonalbot/pull/496
Can this be merged now?

[obsidian patio]

We’ll need a core developer review as well

[patent pivot]

It's on my list

[cold moon]

@patent pivot do you know what these icons mean?

As I don't understand branding/amazon etc.

[patent pivot]

Which icons?

[cold moon]

@patent pivot https://github.com/python-discord/site/blob/master/pydis_site/apps/home/resources/books/automate_the_boring_stuff.yaml#L11-L16 Like these, regular/link, branding/amazon..

[patent pivot]

font awesome icons @cold moon

[cold moon]

But font awesome icons is fa-amazon etc.

[patent pivot]

yes

but

fa = regular
fab = branding

[cold moon]

Oh!

I didn't realise that connection

[cold moon]

Should I include payment information to resources? They are in YAML, but not in current pages.

[patent pivot]

@cold moon probably

I think we decided we want to try add difficulty to resources as well

See https://github.com/python-discord/meta/issues/49

[cold moon]

Huh, that's one complex issue

But what should I do? As I can't measure level of resources, should I just add option to add field exp_level to YAML and site show it when it exists?

But why all resource items have to be in different YAML files? Why every group of resources can't be in same YAML file? Because having them in different files means opening a LOT of files.

[patent pivot]

it will never be that many, I think multiple files is the clearest

you can leave it as none for now and myself or one of the admins will go through and rank resources

[cold moon]

And when we get caching too, then this will be even faster

Currently hardest part is to make data up-to-date and restructure it way to make this possible to display like currently.

[sudden crane]

can somebody update the format-watching bot func to recognize both python and python-repl as valid langs?

[green oriole]

python should be recognized, but I don't think python-repl is

@sudden crane are you familiar with GitHub? If so, can you open an issue on https://git.pydis.com/bot ?

[sudden crane]

@green oriole thanks, i will

[green oriole]

Thanks!

[sudden crane]

i didnt know py and pycon were valid as well. neat

[cold moon]

How should be resources ordered: By alphabetical order or by custom order that is in YAML?

[cold moon]

What is best way to handle hover for icons in resources page without writing CSS for every icon?

[green oriole]

How should be resources ordered: By alphabetical order or by custom order that is in YAML?
@cold moon Using the YAML order, I'd say

[cold moon]

Okay

[mellow hare]

Do we have any good examples of things the bot sends that delete themselves after a time?

Pff, never mind.... There's one in the file I'm working with

[cold island]

The paginator, if reactions count

[patent pivot]

good infographic? good infographic.

[obsidian patio]

good infographic.

[cold island]

@velvet willow We don't allow recruitment here.

[late wolf]

So for help channel, how do u guys avoid ratelimits because you can only edit 2 channels per 10 minute and in a situation like

a person claims a channel and that channel gets moved to the occupied(edit 1)

another person claims a channel and that channel gets moved to the occupied(edit 2)
and at that time what if a person happens to close a channel?

u can't edit the channel because the ratelimit is reached.
how do u guys get around that?

[subtle kraken]

2 per 10 minute ratelimit is solely for name of the channel, not for editing the channel in general

and its per channel not per guild

[late wolf]

ohh didn't know that

thx

[subtle kraken]

and fwiw
there is no way to get around ratelimits

Some people try to get around them by using multiple bots, but discord will simply ban all parties included in that case

[obsidian patio]

Doesn’t discord allow you to use multiple bots in sync?

[late wolf]

Ye, I meant how do u manage the system when there is ratelimit

my bad for the wording

[subtle kraken]

we haven't really hitten any ratelimits so theres that

and d.py handles any ratelimiting by simply delaying the action

such as if we would be changing name of a channel, we would change it twice and third one would just simply happen after 10 minutes to not get a 429

@obsidian patio sure but if you use them to avoid ratelimits, not really

such as if you use 100 bots to have rainbow role with each bot just change the color every 1s or so, you will still get banned
regardless of the fact that your bots aren't being ratelimited
it still is spam to API and you just try to avoid gates that were made to lighten the load on the API

[late wolf]

such as if we would be changing name of a channel, we would change it twice and third one would just simply happen after 10 minutes to not get a 429
@subtle kraken really, yesterday I was trying to change the topics of 4 channels at once, and what happened is that specific part didnt' run at all

neither did it give any errros

[obsidian patio]

Hmm, yeah

[subtle kraken]

yeah the ratelimit is for both name and topic

we only change topic on dormant -> available
available -> occupied

meaning we can't really get ratelimited in majority of cases

[hardy gorge]

We did hit some rate limits a while ago and we made some changes to prevent it from happening

[late wolf]

i see

[hardy gorge]

Not specifically in the help channel code, although there was a brief moment where Discord's ratelimit changes hit a lot of people by surprise

[subtle kraken]

name & topic ratelimit change isnt it

supposedly to stop clock and member bots

[hardy gorge]

Yeah, the changes to those rate limits were a bit unexpected for most libraries, I think

We also have some reaction ratelimits that we're hitting from time to time

That's due the way discord.py's ratelimit protection is designed

There's a small change that multiple requests get past the "lock" before the lock is updated with the response from a request

[subtle kraken]

well libraries didn't care much about that change, its more about the fact that developers didn't expect it

feels like low effort way to stop malicious bot to me

[hardy gorge]

It's an async race condition that's fairly difficult to avoid, unless you make all requests from a bucket run sequentially (and, even then, there are global buckets as well)

[cold moon]

When I define bulma extensions in Django Simple Bulma config, why this don't load them?

[cold moon]

Do I have to load extension somewhere? Because when I define extensions config:

["bulma-tooltip"]

this just don't have this extension enabled (site repo)

Oh, site have really old version of django simple bulma

[crude gyro]

we can probably just update it to the latest version

feel free to open a PR, if you like

[cold moon]

@crude gyro I opened PR that use new bulma-tooltip instead old, as old don't work

[mellow hare]

@clever wraith If you're needing help setting up our bot to run on your machine, it's better to ask your questions in here

[clever wraith]

how can i setup my bot and how i make it run?

[green oriole]

You want to setup your copy of @stable mountain, or your custom bot?

[clever wraith]

@hushed swanyton

@stable mountain

of that bot please

How can I setup it @green oriole ?

[green oriole]

In this case, see https://pythondiscord.com/pages/contributing/bot/

[clever wraith]

I already have that

Now what i have to do?

@green oriole ?

[green oriole]

Well, have you read it?

[clever wraith]

yes

[green oriole]

It is all explained

[clever wraith]

and i download that from git

but i mean in the code how can i setup the bot?

Im new at python

[green oriole]

Well, it is explained on the webpage I just linked you

[clever wraith]

im on mac i dont have the cmd

[green oriole]

We have instructions for both

You should have a terminal on your Mac

[clever wraith]

on Studio code?

It puts that the command brew isnt found

[green oriole]

Right, you should probably install it

[cold moon]

Current state of resources page.

[green oriole]

Noice

[cold moon]

Now I only need to wait until my Django Simple Bulma PR get merged, as current tooltip extension is broken. I made extension fix locally, but current is not supported anymore

[mellow hare]

Trying to get the bot docker stuff working and it's pitching a bit of a fit. Whenever I do the whole docker-compose up on the bot side, I keep getting errors https://paste.pythondiscord.com/isuhesakec.py

No idea where to go from here.

[late wolf]

can someone link me with the help channel system code ?

[green oriole]

!source HelpChannel

[stable mountain]

Help Command

An interactive instance for the bot help command.

Source Code

Go to GitHub

bot/exts/info/help.py:41

[green oriole]

Nope

Oh really

!source HelpChannels

[stable mountain]

Help Command

An interactive instance for the bot help command.

Source Code

Go to GitHub

bot/exts/info/help.py:41

[green oriole]

.>

https://github.com/python-discord/bot/blob/master/bot/exts/help_channels.py#L56

Dunno why the bot isn’t finding it

[obsidian patio]

Fuzzy matching, I assume

[green oriole]

But it should be a better match

[obsidian patio]

Maybe we should make sure that it checks for exact matches first, if it doesn't alreayd

I'm not sure how it's implemented, but it could be that it's searching through the names and just takes the first one over a certain threshold

[late wolf]

@green oriole thx

[late wolf]

But it should be a better match
@green oriole i think because there is help it mathes it to helpcommand

[green oriole]

Hmm

I'm not sure

[sullen phoenix]

!e

from fuzzywuzzy import fuzz, process

result = process.extractBests('HelpChannels', ('help', 'Help', 'HelpChannels'), scorer=fuzz.ratio, score_cutoff=60, processor=None)
print(result)

[stable mountain]

@sullen phoenix :white_check_mark: Your eval job has completed with return code 0.

[('HelpChannels', 100)]

[sullen phoenix]

¯_(ツ)_/¯

and with the full process, it's a little worse, but it still catches

!e

from fuzzywuzzy import fuzz, process
from fuzzywuzzy.utils import full_process

arg = full_process('HelpChannels')

result = process.extractBests(arg, ('help', 'Help', 'HelpChannels'), scorer=fuzz.ratio, score_cutoff=60, processor=None)
print(result)

[stable mountain]

@sullen phoenix :white_check_mark: Your eval job has completed with return code 0.

[('HelpChannels', 83)]

[green oriole]

!e

from fuzzywuzzy import fuzz, process
from fuzzywuzzy.utils import full_process

arg = full_process('HelpChannels')

result = process.extractBests(arg, ('HelpChannels', 'HelpCommand', 'CustomHelpCommand'), scorer=fuzz.ratio, score_cutoff=60, processor=None)
print(result)```

[stable mountain]

@green oriole :white_check_mark: Your eval job has completed with return code 0.

[('HelpChannels', 83)]

[green oriole]

Really weird

[cold island]

This has another bug I found:

!source silence

[stable mountain]

Command: silence

Silence the current channel for duration minutes or forever.

Source Code

Go to GitHub

bot/utils/lock.py:64

[cold island]

Should be https://github.com/python-discord/bot/blob/master/bot/exts/moderation/silence.py#L103

[green oriole]

Lmao

[eternal owl]

On the site repo, should there be 1 migration file per model or multiple models?

currently, I have 1 migration file for 2 models UserEvent and ScheduledEvent
https://github.com/python-discord/site/pull/410/files#diff-bd9d1bbc0c06c8223a57bbed6d2cfea18c5ee53741aa962f3c1d2993d2e270dc
(still in the works)

[green oriole]

It sounds good to me, I don't think we need to micromanage migrations at this point

[eternal owl]

alright

[eternal owl]

https://github.com/python-discord/site/pull/410
this PR is ready for review

[mellow hare]

Okay so, need a sounding board.

For context, this is still the voice ping thingy. Currently the bot will remove any bot message that happens in #voice-verification after 8 seconds. I'm trying to prevent it from deleting the message that I'm having the bot send to ping the user

The only thing I can think of is just checking the content of the message with a like..... starts with or something

[tough imp]

Sounds like you will need to adjust the current behaviour

We do a similar thing in the server verification channel, the bot does not remove its own reminder / @Unverified ping

it simply compares the content of messages in the channel against the reminder message constant

https://github.com/python-discord/bot/blob/0b7a1be1dd57a464fc79fcb235b79d75bec43f99/bot/exts/moderation/verification.py#L615

[mellow hare]

So the issue is that I'm pinging the person directly, rather than a role

[tough imp]

Is the user mention injected into the middle of the string? If it's just at the beginning or end you can probably just use startswith/endswith

if you're worried someone will abuse it, you can check that the message was sent by the bot too

[mellow hare]

Yeah I was figuring I could use ends with

Okay, I was worried that a solution using that would be crude or the wrong way to do it

[tough imp]

Simple is better than complex 🤷‍♂️

[mellow hare]

And it's going under the if message.author.bot check

So I'm not concerned about that

[tough imp]

You could use some internal state to control it

but I don't think it's necessary

I don't see a way in which your solution could break, we have control over the message's content

[mellow hare]

True enough

And there are no real instances when it would overlap with another

Unless someone was a butt and echo'd into the channel

[vocal wolf]

I keep getting a bunch of bot.api.ResponseCodeError: Status: 401 Response: {'detail': 'Invalid token.'} when running both the site and the bot on different terminals with pipenv run start. This occurs when the bot attempts to request from the site.

I'm not exactly sure what I'm doing wrong

[blazing magnet]

hmmm, I wish I knew how the auth works LOL

[vocal wolf]

Here's an example of one of the errors (the other ones are very similar ```
Traceback (most recent call last):
File "/home/xithrius/Repositories/bot/bot/exts/moderation/infraction/_scheduler.py", line 49, in reschedule_infractions
infractions = await self.bot.api_client.get(
File "/home/xithrius/Repositories/bot/bot/api.py", line 119, in get
return await self.request("GET", endpoint, raise_for_status=raise_for_status, **kwargs)
File "/home/xithrius/Repositories/bot/bot/api.py", line 114, in request
await self.maybe_raise_for_status(resp, raise_for_status)
File "/home/xithrius/Repositories/bot/bot/api.py", line 104, in maybe_raise_for_status
raise ResponseCodeError(response=response, response_json=response_json)
bot.api.ResponseCodeError: Status: 401 Response: {'detail': 'Invalid token.'}

[blazing magnet]

I mean... is it a valid token? xD

[vocal wolf]

well, no.

I've copied the token the site gave me when it first turned on and shoved it into the bot's .env file at BOT_API_KEY, so I think it may have something to do with ports

[blazing magnet]

hmmm, well a 401 is pretty specifically a malformed token. Or otherwise invalid token, but everything else is passed properly.

[vocal wolf]

Maybe I changed the token at some point and forgot about it?

[blazing magnet]

sorry, I am not much help, I also need to get off and get dinner.

But it sounds like it is very likely an issue with the token itself.

It is not that the token is not passing auth, it is that the token is literally invalid. A 403 should be raised if the token failed auth.

A 401 indicates that your token could not be parsed, was found to be invalid somehow (invalid characters, not encoded properly, too short or too long, invalid prefix, etc)

This could be raised for jwt if the signature was invalid, the date/expiration was invalid, or I sometimes accidentally cut off some character haha.

[vocal wolf]

hmmmmmmm

[sullen phoenix]

did you change the site url in the config?

make sure you're not sending a request to the prod api lol

[vocal wolf]

ah

that's probably the problem

@sullen phoenix yep, that was the problem. Was requesting to pythondiscord.com instead of pythondiscord.local:8000

[sullen phoenix]

nice 👍

[vocal wolf]

or... maybe not?

hmmm

I seem to have caused another error

I'll fix it later

[cold moon]

#dev-log does bot redeployed or not?

[hardy gorge]

don't think so, I'll check

Should be redeployed now

[slim widget]

Hey, I have a suggestion for a small change to the eval command.

I was thinking that the line numbers that get printed in the output are potentially more confusing than they are helpful.

I've seen at least one person get confused by them.

And the output is never more than 10 lines, so it doesn't seem like line numbers are really necessary.

What do people think?

[hardy gorge]

I've never used the line numbers in a help conversation, I think

I'm not attached to them

[green oriole]

That’s a fair point

It can be confused ahout when a line is getting continued on the following line or if it is an actual line break though

Especially on mobile

[brazen charm]

I think they help, but could be reduced to just 2 digits if they're in the way, or some header that'd explain them

[slim widget]

Currently, it looks something like this:

@slim widget ✅ Your eval job has completed with return code 0.

001 | hello
002 | world

I'm suggesting to simplify it to this:

@slim widget ✅ Your eval job has completed with return code 0.

hello
world

This example is fine, but if I were printing out only numbers someone might think the line numbers are part of the output.

[sharp timber]

It may be worth also considering

 1 | hello
 2 | world
10 | other

[nocturne hare]

@surreal venture Ping me when you get on :)

[green oriole]

I think it would be even more confusing bast

[sharp timber]

yeah ._.

[cold island]

I would be fine with removing the numbering, I never actually used it

[mellow hare]

@hardy gorge Is there a way to flush the ReDis cache?

... It's probably in the docs isn't it

[brazen charm]

flush in what way?

[mellow hare]

Like just clear out. Currently testing something that fills it

Just going to make a temp function

Er

Method

Yeah I found the method to do so, should have just looked first

[green oriole]

If you are using docker, you could use docker rm <compose project name>_redis

The default compose name is the name of the cwd, but we ask to override it in the dev guide to site

[mellow hare]

I'm running the site in docker but the bot through pipenv (since I can't for the life of me get the full bot docker container suite to work). And I have the fake_redis on, so...

I have no idea why it's not clearing or letting me clear it easily

For some reason the cache.clear() isn't doing what I thought it would

So I'm kind of stumped

Okay so it's not a problem of the cache not being cleared...

[mellow hare]

The crap, I have the pre-commit hooks, why didn't it run them

Okay, something's not right...

[brazen charm]

What are you running the precommit from?

[mellow hare]

PyCharm. I've never had issues with it before

[brazen charm]

Do you have a global flake8 install? Pycharm has some troubles with precommit and venvs and doesn't activate them

[mellow hare]

I'll double check

Guess it wasn't. Must have missed that when I installed 3.9 since it's now my default on my system

[nocturne hare]

!remind 10h revisit 1214 which was forgotten.

[stable mountain]

Okay.

Your reminder will arrive in 10 hours!

ID: 773, Due: 2020-11-06T18:21:48

[green oriole]

.issue 1214 bot

[dusky shore]

GitHub

[bot] #1214 fix(statsd): Gracefully handle socket.gaierro in AsyncStatsClient

[obsidian patio]

"Gracefully" hahah

[late wolf]

hey is seasonbots wolfram down?

[obsidian patio]

.wa 5+5

[dusky shore]

Wolfram Alpha

View original for a bigger picture.

[obsidian patio]

No

[late wolf]

.wa is white a colour

bruh

[nocturne hare]

.issue 513 if anyone wants to dwell into cooldown world

[dusky shore]

[404] #513 Issue/pull request not located! Please enter a valid number!

[nocturne hare]

:/

.issue 513

[dusky shore]

GitHub

[seasonalbot] #513 Create rollover cooldown logic for wolfram alpha

[mellow hare]

Just gonna leave this here...https://github.com/python-discord/bot/pull/1272

[green oriole]

nice use of the walrus

[sullen phoenix]

maybe you could rename redis_cache to something that better represents it?

possibly something like member_notif_msgs

[hardy gorge]

I haven't looked at the source code yet; does it use a redis cache to remember who've been in voice before?

[sullen phoenix]

it looks like it's to remember if the user has been pinged to become verified yet

[hardy gorge]

We do consider our caching to be volatile, which means it comes with the risk of pinging users multiple times

[sullen phoenix]

user id -> message id for the ping

yes, they can be pinged multiple times

[mellow hare]

That was my previous reasoning for wanting to use the database but that got rejected as an idea

How volatile is the cache

Like would this potentially lose entries every minute? Hour? Day?

[green oriole]

Hopefully never

But it can

[mellow hare]

So should I not use the cache? What other options should I consider?

[green oriole]

I think that pinging a user twice is okay to be honest

It would be a very special case anyway

[mellow hare]

Gotcha.

[tough imp]

@hardy gorge Redis supports collections, right? I.e. if we wanted to keep a set(-like) of user IDs, rather than a key->value map

[hardy gorge]

yes, we can have a set type

[tough imp]

can we have a set type now?

[hardy gorge]

we don't have a redis type for it

[tough imp]

right

[hardy gorge]

if that's what you mean

[tough imp]

this PR (and some other iirc) abuses the keys as a set members with a "dummy" value, which makes sense because it's probably easier than serializing a whole set everytime as a string

[hardy gorge]

yeah

We could also look into supporting a few operations that we may want to use

[tough imp]

I think having support for it would be nice, since I've seen quite a few places where it'd have been convenient to use bot-side

[mellow hare]

I mean

I do use the dictionary style for a reason

The only "dummy" check is to see if someone is already in it. The value does track the message that is sent to the user, and I use that later to find it and remove it if they try to verify while the message still exists

I feel that I'm using it in the correct way

Obviously I'm open to being proven wrong (as is my way)

But it was a conscious decision I made regarding it

[green oriole]

With the current API, it is fairly good yeah

[mellow hare]

Also Akarys, I'm not triggering message deletion in the on_message event

Only on the command call and in the voice event change call

[tough imp]

I think you're using it correctly given the tools available

[mellow hare]

The other message deletion isn't mine

Sorry k, I just kind of bristled when you said abuse.

[tough imp]

when you put a False, you don't care about the value no more

[mellow hare]

No argument there

[tough imp]

you only look at whether the key exists

right?

[mellow hare]

Yep, and I know what you meant in that context, but how would I track the message ID along with it?

And still easily be able to find said message and remove it

[green oriole]

Also Akarys, I'm not triggering message deletion in the on_message event
@mellow hare well, you do still delete bot message with the on_message handler, right?

[mellow hare]

Which line

@tough imp Or is the suggestion to have two caches, one as a dictionary and one as a set?

[tough imp]

I think that would be my go-to solution if we had a set type

but we don't, so your solution makes the most sense

[mellow hare]

Okay, that I could see

Apologies

[tough imp]

no worries

[green oriole]

174

[mellow hare]

One sec, das booting PyCharm

That wasn't something I implemented, that was already there

[green oriole]

Well, yes, but it doesn’t make much sense now IMO

You have two different triggers for this handler, why would you have a logic for separating the trigger again, while you could directly slap the deletion line 147 and keep the one you added?

[mellow hare]

For the difference in the deletion delay. The messages that line 174 are catching the messages sent to the channel for the other parts of voice verification. Those have an 8 second removal time. The ping I'm sending has a 1 minute delay so that people have time to find the channel and figure out what's going on.

Where as with the 8 second one, they'd already be looking at the channel

That 8 seconds should also cover any other bot message that gets sent there either purposefully or accidentally

Although it's probable that I'm just misunderstanding what you're meaning

[green oriole]

That 8 seconds should also cover any other bot message that gets sent there either purposefully or accidentally
Right, I didn’t think about that

I guess it should be fine then, even if I don’t really like the str.startswith() method

[mellow hare]

I don't either, but it's the only reasonable solution I could come up with for it

The only instance where it'd get a false positive is if we echoed a message that had the same ending

[green oriole]

You could store the ping message ID in the redis value, instead of a dummy value, and check if the ID is in one the redis dict value though, that could work

[mellow hare]

That'd be intensive

[green oriole]

It would be an O(n) solution, but it should be rare enough to not cause any issue

[mellow hare]

The current is O(1) though, isn't it?

[green oriole]

Yeah

[tough imp]

What I was thinking about earlier w.r.t. this is whether the on_message could fire before the send coro finishes, I guess probably not

but could it in theory?

[mellow hare]

I don't know how it could

It'd have to get the event from Discord, so even if the send delays, the right information would still be there and ready

If I'm understanding correctly

And even still, if that message isn't caught properly, I'm still following it up with that 1 minute delete built into the voice event change

[green oriole]

I don’t think it can, since discord is firing the event

[tough imp]

You don't have the ID of the message you sent until send returns it, and it has to wait for the API to respond to finish, so I'm wondring whether you could end up in a situation where the on_message triggers with the message before the event loop gives control back to the send to finish

[green oriole]

Hmm.. there shouldn’t be

[mellow hare]

Sure, but again, I'm not checking the message id in the on_message event.

I'm specifically looking at the content of the message there

[tough imp]

Yea, so it's fine ~ if you didn't want to check the content you'd have to know the message ID to recognize the messages to ignore

which led me to thinking of that highly theoretical issue

[mellow hare]

Which would mean me having to iterate over the dict values to find out whether a message was there or not, which isn't ideal either. Your concern is valid, I just think the currently solution takes care of that

[tough imp]

Yep, which is where the set type would be appropriate

[mellow hare]

Yarp

[tough imp]

I'm just theoretizing, I'm not attacking your solution

[mellow hare]

No no, I know

Just.... kind of proud how it turned out

I did quite a bit of testing to get it working

[tough imp]

I think the content check is perfectly fine, but you already know that

[mellow hare]

Even if it's a small thing

Yeah, and I appreciate that. I'd honestly still be hemming and hawing over it if you hadn't reassured me before

[green oriole]

Dismissed 👌

[eternal owl]

https://github.com/python-discord/bot/pull/1269
this PR is ready for review

[cold moon]

@eternal owl Reviewing this PR currently

[eternal owl]

cool, can you have a look at the site PR too? they both are linked

Maybe you could look at the site one first?

cuz the bot uses the new api endpoints introduced in the site PR

https://github.com/python-discord/site/pull/410

[cold moon]

I do code review on bot first, as I already started it

[eternal owl]

okay 👍

I should prolly add a few logging statements also

[eternal owl]

https://github.com/python-discord/site/pull/410#pullrequestreview-525276182
I should prolly update the docs for more clarity, but what happens is, we want to create model instances via the REST api(DRF) and the admin interface, when creating via admin interface, the save() method isn't called, so no point in overriding the save method for performation validation.
I used the clean() methed instead, which is called when creating a model instance in the admin interface.
The clean() method is not called in the save() which DRF uses to create model instances so manually calling clean() in serializer's validate() method and catching validation errors @cold moon

[cold moon]

Okay

[brazen charm]

So what about the other language codes here https://github.com/python-discord/bot/pull/1266#pullrequestreview-521248033

[mellow hare]

Styling question:

class VoiceGate(Cog):
    """Voice channels verification management."""

    """
    RedisCache[t.Union[discord.User.id, discord.Member.id], t.Union[discord.Message.id, bool]]
    The cache's keys are the IDs of members who are verified or have joined a voice channel
    The cache's values are either the message ID of the ping message or False if no message is present 
    """
    redis_cache = RedisCache()

    def __init__(self, bot: Bot) -> None:
        self.bot = bot
        self._init_task = self.bot.loop.create_task(self._async_init())

or

class VoiceGate(Cog):
    """Voice channels verification management."""

    # RedisCache[t.Union[discord.User.id, discord.Member.id], t.Union[discord.Message.id, bool]]
    # The cache's keys are the IDs of members who are verified or have joined a voice channel
    # The cache's values are either the message ID of the ping message or False if no message is present 
    redis_cache = RedisCache()

    def __init__(self, bot: Bot) -> None:
        self.bot = bot
        self._init_task = self.bot.loop.create_task(self._async_init())

[brazen charm]

definitely comments for me

[sullen phoenix]

yeah, i like the comments

[mellow hare]

Mkay

I just wasn't sure how the 3 comments looked one after another

Felt mildly messy

[cold moon]

@eternal owl About user events, what about making subscribing command for events, so you get notification when this event is starting?

[eternal owl]

its using reactions instead of command

[cold moon]

But you removed subscriptions?

[eternal owl]

Decided not to store it on site

as we can simply fetch from message reactions

[cold moon]

Oh

[eternal owl]

its only removed from the UserEvent model

[stable mountain]

@nocturne hare

It has arrived!

Here's your reminder: revisit 1214 which was forgotten..
[Jump back to when you created the reminder](#dev-contrib message)

[sharp timber]

Is there a reason why this doesn't work?:

hm

I guess it doesn't support typing

[green oriole]

What doesn't support what @sharp timber?

[last patio]

@mellow hare i would put it as a docstring below `redis_cache``

that way sphinx parses it

I did that at work with typing.NamedTuple attributes, because otherwise you get the "alias for field x" docstrings

[mellow hare]

@last patio So you mean like this:

class VoiceGate(Cog):
    """Voice channels verification management."""

    redis_cache = RedisCache()
    """
    RedisCache[t.Union[discord.User.id, discord.Member.id], t.Union[discord.Message.id, bool]]
    The cache's keys are the IDs of members who are verified or have joined a voice channel
    The cache's values are either the message ID of the ping message or False if no message is present
    """
    
    def __init__(self, bot: Bot) -> None:
        self.bot = bot

[last patio]

Yeah!

I actually haven't seen something where you add the detailed type a second time, so that's interesting. Makes me wonder whether you can implement subscription for manual typehints

[mellow hare]

We had it somewhere else in the code where we're using the RedisCache. Figured it'd make sense to follow suit

Well that's lame. Since it's just a comment/docstring it's not letting me make a commit for it

Nevermind, PyCharm was being dumb

[tough imp]

I'm not sure if we use that anywhere in the repos, it looks quite alien to me

[mellow hare]

It feels better than 3 stacked comments

But for consistency, I'll keep it as is

Oh derp, I don't know why I did it using else's

Fixing now

[tough imp]

Have you thought about just using an int flag instead of a False in the cache?

[mellow hare]

What's the main benefit of doing that. That's the part I'm not sure about

[tough imp]

It ultimately does not matter too much, I just think it's unnecessary to introduce an extra type to the cache, when we could just use a 0 or -1 (using a bool is strange because it will never be True)

a False is basically a 0 anyway

You could also have a MSG_DELETED = 0 const at the top, and then use that to check, it'd be a little bit more readable in my opinion (it can still be a False if you prefer)

then you could do cache.set(member.id, MSG_DELETED)

instead of a False which feels a little cryptic

but I'm fine with the False too, just offering some ideas

[mellow hare]

Fair. I'll go with the constant idea

[tough imp]

sorry for giving you this much trouble lol

[mellow hare]

No no no! I need it. You can't get better without some criticism and work. I just appreciate you guys taking the time

[tough imp]

I wonder what the typestring would look like if we wanted to add a None as a valid value to the rediscache

n|None lol that's awkward

it could just be None I guess, because a string "None" would be s|None, but then it's not consistent with the other formats

[mellow hare]

I really don't like the idea of using a string None

[tough imp]

yea I meant on the side of rediscache, to allow it to take a genuine None

it has to serialize it to a string

[mellow hare]

Question: would doing NO_MSG be fine? Because folks who are already verified don't even get a message

Just for clarity, I mean

[tough imp]

yes I think that's even better

I think you could also do, hmm

[mellow hare]

Depends on whether nor not None is going to be implemented in the cache anyway

[tough imp]

in one of the places where you get from the cache, you can probably simplify to get(member.id, NO_MSG) because the behaviour is the same for NO_MSG and non-existent key

but I'm not sure if I'm remembering correctly

yeah I don't know, we could ask Ves whether there's plans

probably not

[mellow hare]

Yeah, since I'm doing the NO_MSG = 0, I'll swap that as well

[tough imp]

there may be a good reason for why it wasn't added initially, I'm not remembering

0 is still falsey so you probably don't have to adjust anything

[mellow hare]

Yeppers, just swap the variable

[tough imp]

So what about the other language codes here https://github.com/python-discord/bot/pull/1266#pullrequestreview-521248033
@brazen charm Sorry that your review was missed, I think it's a good suggestion although I don't think those codes are used frequently

I would just add them on master but I'm a little frightened about the comment & that the ordering matters lol

I'll have to test it

def a(): ...
a: Cat = 1

[obsidian patio]

“A cat”

[tough imp]

what even is "gyp"

[obsidian patio]

No clue

[mellow hare]

Okay, cool

PR should be done

[tough imp]

thaanks it's looking great now

I will try to test tonight

[mellow hare]

Sounds good

You might want to add a test function that lets you manually clear the cache

That's what I had to do when I was futzing around

[tough imp]

@mellow hare may have to leave it for tomorrow since I'm falling asleep, but I'll make it a priority

[mellow hare]

I mean I can holler at someone else as well.

You don't need to go out of your way

[green mesa]

what happen to emojis😅

[short snow]

it is is ok for me

@green mesa

andriod or ios problem

[green mesa]

No on android

[short snow]

android problem

discord shld do something about this tho

[green oriole]

The renderer on some platforms just go brrrr gurkan

There's nothing we can do against that

[green mesa]

😔😐

[mellow hare]

Embeds are wonky

[green mesa]

Indeed

[brazen charm]

It's more about discord's very inconsistent handling of seemingly everything to do with markdown urls

[cold moon]

Can anybody review https://github.com/python-discord/django-simple-bulma/pull/67 ? I need to get this merged + released to continue working on resources app.

[nocturne hare]

!remind 2d ;-; revisit 1214 which was forgotten

[stable mountain]

Yeah okay.

Your reminder will arrive in 2 days!

ID: 780, Due: 2020-11-09T15:50:10

[cold moon]

I think @stable mountain should have auto-reloading extensions when extension is changed in development engine like Django does it.

[green oriole]

We thought about that and we rejected the idea, although I have to admit I don't remember why

[patent pivot]

Yeah we did, can't remember either

[green oriole]

I think the idea was that it is easy enough to just run a command

[eternal owl]

is there anything similar to nodemon in python?

[subtle kraken]

no one wants to reload extension just because they save the file every time they type single character was the reason iirc

[patent pivot]

lol

isn't it because pycharm does weird auto-saving shenanigans

[green oriole]

Riiiight, that was the idea

[subtle kraken]

typing the command isn't hard at all anyway

and ensures that you only reload when you truly mean to

[eternal owl]

you dont need to even type it, just up arrow key and enter, lol

[patent pivot]

huh?

not in chat

[green oriole]

is there anything similar to nodemon in python?
@eternal owl there's the tool django is using, but it is pretty undocumented

[subtle kraken]

yeah we mean ext reloading

not bot reloading

though could be cool to have interactive console for bot

[eternal owl]

ooh I thought bot reloading when we making changes to code while development

like how nodemon does in js

@eternal owl there's the tool django is using, but it is pretty undocumented
@green oriole ohk

[patent pivot]

nah nodemon restarts the whole process

[green oriole]

We couuulld use user signals to reload the cogs and it wouldn't take much effort, but it is Linux only

[eternal owl]

yea, I meant while we are working on the bot, so we dont have to stop and run the bot again

[subtle kraken]

d.py had a lot of requests for auto reloading which all been rejected, reasons would match our reasoning

[eternal owl]

while developing locally

I just realised the forms backend repo finally has some code

[patent pivot]

lol

yep

[green mesa]

Where infraction such as temp ban all stuff are stored?

When bot reloads it retrives from api?

[patent pivot]

yeah

look at the scheduler system

it uses the same system as reminders under the hood

fetches all active temporary bans and starts tasks for them iirc, but I haven't checked in a while

[green mesa]

Ok tnx for info 🙂

[short snow]

hmmm, in which commands in the paginator used, i am trying to make the wiki command a bit better, with 2-3 lines of information for each topic

and wanted to use the emoji paginator

[green mesa]

That's line paginator only buddy @short snow

Check emoji_count.py

[short snow]

ok

my code is messed up rn that;s why<

[green mesa]

😅

[short snow]

oh so that's the line paginator, what is used for the emoji reactions? like the one in help and reddit ?

[green mesa]

Actually if u check the file it's documented there

How it's working

Check paginator.py

And in that line paginator

[short snow]

ok 😄

[eternal owl]

the reddit command uses the image paginator if you need that as a ref

[cold moon]

I think blacklisting domains should get 1 new feature: There should be way to blacklist domain such way that all it's subdomains is banned too. I know some sites what have a lot NSFW subdomains, and all these subdomains is always NSFW, so I think it would be better to have ability to blacklist all subdomains.

[neon garnet]

.issue 1197 bot

[dusky shore]

GitHub

[bot] #1197 Infraction subcommand for amending infraction reasons

[neon garnet]

I'll be addressing the change requested here, if anyone's up for review, feel free to let me know

[green oriole]

I think blacklisting domains should get 1 new feature: There should be way to blacklist domain such way that all it's subdomains is banned too. I know some sites what have a lot NSFW subdomains, and all these subdomains is always NSFW, so I think it would be better to have ability to blacklist all subdomains.
@cold moon sounds good to me, if you want to open an issue

[cold moon]

Okay

[sharp timber]

While you're at that, feature request: ability to edit a blacklist entry

[cold moon]

@sharp timber I think this should have it's own issue

[sharp timber]

agreed

[mellow hare]

@tough imp Did the thing with the fixes and the what not

[tough imp]

ok I'll look thanks

[tough imp]

@mellow hare sorry still a few comments

[mellow hare]

@tough imp Would you be able or willing to make those mergable changes or suggestions? I think it'll make things easier for both of us.

[tough imp]

yea I can do that

[mellow hare]

I appreciate it

[tough imp]

sorry if my review was too harsh

it doesn't look like it on the surface but it's not an easy problem to solve because of the concurrency

so don't feel bad about it not being perfect on the first try

[mellow hare]

No no, I think they're fine reviews, but it's making this PR take a bit longer if I can't just plug in the desired changes

Plus it lets you take the proper credit for the things you're fixing/suggesting

[tough imp]

ok, I will combine it with a final test & get it merged, but I will probably leave it for tomorrow

[sharp timber]

I think there's a pressing race issue here.

        if await self.redis_cache.contains(member.id):
            log.trace("User already in cache. Ignore.")
            return

        log.trace("User not in cache and is in a voice channel.")
        verified = any(Roles.voice_verified == role.id for role in member.roles)
        if verified:
            log.trace("User is verified, add to the cache and ignore.")
            await self.redis_cache.set(member.id, NO_MSG)

May be an issue, because we've had a pathological case where a users android/mobile client bugs out and spam join-leaves, and if this happens during heavy activity I have concerns that we'll re-enter the method for the same user, and between these two awaits we'll end up skipping past the redis_cache check here

Actually, this is worse than I thought.

        log.trace("User is unverified. Send ping.")
        await self.bot.wait_until_guild_available()
        voice_verification_channel = self.bot.get_channel(Channels.voice_gate)

        message = await voice_verification_channel.send(f"Hello, {member.mention}! {VOICE_PING}")
        await self.redis_cache.set(member.id, message.id)

The actual set of "in cache" is delayed until after we send a message, rather than just within the async machinery.

An unverified user with the bug will trigger 6+ voice entry events, within a second, which then gets hung up waiting simultaneously at await self.bot.wait_until_guild_available() and await voice_verification_channel.send(f"Hello, {member.mention}! {VOICE_PING}"), sending several pings at once. I don't have a device that triggers this, though, so I can't retrigger the bug as an example or for testing.

I'm going to repost these on the PR, I think, that'll be helpful

[green mesa]

One question when bot goes reloads all its cache data such as when to unban or inmute is gone so how bot collect that's data from api ? Api send some webhook to trigger or it just create asyncio task upon when it reloads?

[tawdry vapor]

The latter

The bot sends an API request when it restarts and then re-schedules any temporary infractions

[tough imp]

@sharp timber Those are good observations; I've responded on the PR

[mellow hare]

@tough imp @sharp timber I'm leaving it with you guys then, because you seem to already have a solution in mind. It's going beyond what I thought would be required for this particular PR.

[surreal venture]

Can someone drop a review on my pull request please?

.pr 509

[dusky shore]

GitHub

[seasonalbot] #509 Spook Name Rate Game

[late wolf]

What is redis?

[hardy gorge]

redis is an in-memory "data structure store", like a cache/database

We use it for caching values in an easy way that don't benefit from permanent storage in our relational database (postgres)

Or, in other words, we temporarily cache values using redis

[late wolf]

hmm

why use that?

[brazen charm]

Some values need to persist and sending them over to the site db is a bit overkill

[mellow hare]

Saves on multiple API calls

[late wolf]

i see

[tough imp]

It's also easier to develop because you don't need to make or adjust tables and endpoints in a separate repo

[late wolf]

i see

[stable mountain]

@nocturne hare

It has arrived!

Here's your reminder: ;-; revisit 1214 which was forgotten.
[Jump back to when you created the reminder](#dev-contrib message)

[tough imp]

get to work floppy

[brazen charm]

Got some time for a few redis questions @hardy gorge ?

[hardy gorge]

yes

[nocturne hare]

!issue 1214

:(

[hardy gorge]

.issue 1214 bot

[dusky shore]

GitHub

[bot] #1214 fix(statsd): Gracefully handle socket.gaierro in AsyncStatsClient

[brazen charm]

It doesn't look like async rediscache's dev direction will enable the necessary features so I looked into going through the RedisObject and aioredis' connection "manually".
I think all that was needed was an expire time and then purging idle objects, so got untested code that'd do that below but not sure on how to go with the purge. Should the keys get something like a doc prefix added to them to be able to query all of the through a scan? My initial thought were multiple dbs but that doesn't look like it's a great idea in redis

from typing import Optional
from async_rediscache.types.base import RedisObject, namespace_lock
class DocRedisCache(RedisObject):

    @namespace_lock
    def set(self, key: str, value: str) -> None:
        """Set markdown `value` for `key` with week expiry."""
        key = self._key_to_typestring(key)
        value = self._value_to_typestring(value)

        with await self._get_pool_connection() as connection:
            await connection.setex(key, 7*24*60*60, value)

    @namespace_lock
    def get(self, key: str) -> Optional[str]:
        """Get markdown contents for key"""
        key = self._key_to_typestring(key)
        with await self._get_pool_connection() as connection:
            return connection.get(key)

    @namespace_lock
    def purge_idle(self) -> None:
        """Remove objects that were idle for more than 36 hours"""
        ...

[hardy gorge]

Would you want an expire option for a RedisCache?

Because it would be trivial to implement it in the actual package

The way expire works in Redis is that it will expire once you try to access that key (i.e., there's a TTL, but it's only checked on access)

Hmm, EXPIRE/SETEX only work for a top level key, I think; not for child keys in a hash set

[brazen charm]

Those were the issues I ran into, hash values aren't objects

The expire seemed feasible to implement myself through rediscache but the idletime seemed very redundant to me as it'd need a set for every get

[hardy gorge]

I do have to warn you, I'm slowly working on removing that ugly namespace lock (it should not be necessary after the next release)

What kind of data type are you looking for in this case?

Basically a pseudo-hashset and thus the prefix question?

So that you get docs${key}-like keys?

$ is arbitrary in that example of course

[nocturne hare]

Revisting that pr, Mark mentioned that a 6 attempt with a power of 2 fallback retry timer was too much

Would a times 10 timer be more sufficent for statsd retry timer?

so, 1, 10, 100, 1000, 10000 with 5 attempts

[brazen charm]

Not that familiar with redis but I believe that's pretty much I want. The hashset vals don't provide expires and idletime so it needs to be a top level key but I want them to be grouped to get all of their idletimes for their removal at some intervals

[hardy gorge]

What's the reason for manually purging/removing them?

The way EXPIRE and SETEX work is that whenever you try to access them after their TTL, they'll just show up as non-existing and Redis will handle the expiration internally

[brazen charm]

It's not necessarily needed I guess, just don't see a reason to keep unused values around after some random queries

The plan is to store everything that was fetched directly by an user in redis

[hardy gorge]

How do you plan on getting those keys?

[brazen charm]

What do you mean? The keys will probably be the urls pointed at by symbols from inventories since those seem to be the most stable thing. Then on fetch redis will be checked, if the key doesn't exist there a request to the site is made and symbols on there are parsed with the symbol queried by the user also going to redis

[hardy gorge]

I mean for the purge you're planning

You say you want to manually purge all keys; are you going to store them in a separate collection in Redis as well?

[brazen charm]

The first thought was the separate db I mentioned but that doesn' seem to be much of a redis thing, then a quick search around suggested scanning through the keys with the pattern

That's mainly why I asked that here since I'm not sure on the best way to do it

[hardy gorge]

I don't really have a good solution for that either. You could also go with the radical approach of setting a TTL on the entire rediscache, which means you invalidate the entire thing from time to time

Otherwise you'd have to get all the keys and make a selection that way

You could just use a regular namespace, but just use that as the prefix

You get something like ClassName.docs.<key_here> and can use that for your key/pattern match

Maybe I'd even say "don't worry about purging", depending on the total size we'll reach

[brazen charm]

!d

I've got 74590 total symbols locally with some mildly outdated inventories and without asyncpg so I guess it's not going to be that large as users will only get a fraction of that

Thought it'd be nice but it is starting to sound like it's more trouble than it's worth since expires will delete it a while later anyway to avoid old data

[cold moon]

Do you plan to remove API too from site, as currently everything related with DB is getting removed from site?

[sullen phoenix]

the whole API is being removed

[cold moon]

Will this move to some async web framework?

[patent pivot]

We are still planning to use DRF as of now

[brazen charm]

Is there some convention within rediscache that should be used for namespacing the keys or should I just go with something like f"{namespace}:{key}"

[hardy gorge]

There are no other types at the moment that generate a combined namespace + identifier

There's the hashset, in which the second identifiers are completely separate

[eternal owl]

the whole API is being removed
@sullen phoenix the whole API system is being revamped?

[hardy gorge]

Then you have the queue's, of which the newer TaskQueue will always use a secondary list with a $ as separator between namespace and "pending"

Like f"{namespace}$pending"

[patent pivot]

@sullen phoenix the whole API system is being revamped?
@eternal owl it's being unlinked from the site

we will have site at pythondiscord.com and another separate django program at api.pythondiscord.com

[eternal owl]

so you wanna make the site a static website only?

[patent pivot]

pseudostatic

[eternal owl]

and then another api web server

[patent pivot]

the content will be dynamically generated I guess

[eternal owl]

ohk 🤔

[patent pivot]

based on markdown + YAML files for the most part

[eternal owl]

oh

so the DRF code is just being moved

[patent pivot]

pretty much

there will probably be some refactoring and optimising in places

[eternal owl]

got it

[cold moon]

@patent pivot I think when moving this to other app, this should have async web framework

[hardy gorge]

Why?

[patent pivot]

I'm very doubtful of that @cold moon

Our API literally has one client

If Django ASGI becomes more available then sure, but there is no appeal of switching framework for async

[hardy gorge]

I think it currently has all the downsides we're trying to avoid by staying with DRF

Namely that rebuilding our current API in a different framework was simply too much work

Otherwise we'd have gone for one of the other frameworks we've considered

So, yeah, once Django becomes fully async, we may consider it, but I don't think it has benefits for us at the moment

[sullen phoenix]

yeah, it is a lot of work to migrate our whole API and all its tests to another framework

[eternal owl]

Oh lord, the tests take most of the time

[sullen phoenix]

yeah, especially since we have 100% test coverage on site

[eternal owl]

Otherwise we'd have gone for one of the other frameworks we've considered
@hardy gorge just curious, what are the frameworks considered? 🤔

[patent pivot]

Starlette and FastAPI were the ones we looked at

[eternal owl]

ohk, same as forms backend

[hardy gorge]

We were looking into FastAPI and Starlette, yeah

[patent pivot]

lol, these 100+ commit branch updates are amusing

[cold island]

yep, I was like "oh shit"

[sullen phoenix]

lol wtf

546 commits

[eternal owl]

Can anyone have a look at these 2 PRs 😄
https://github.com/python-discord/site/pull/410
https://github.com/python-discord/bot/pull/1269

[short snow]

.issue 500

[dusky shore]

GitHub

[seasonalbot] #500 httpcat missing codes

[short snow]

for this, are we adding the codes manually to the list

or trying something else?

[sullen phoenix]

the idea was we shouldn't use prior validation for the status codes and just send a request to the http.cat

if the status code isn't valid, it'll return a 404 cat, which is perfect

[short snow]

oh ok

[short snow]

how about adding a bit regex to the bot, like when someone says hello/hi/howdy/halo then the bot says
"hello @user", and more like

[eternal owl]

That could become annoying

[short snow]

hmmm

[sullen phoenix]

i don't think we want that

[obsidian patio]

The only suggestion I’ve heard about that I initially were against, but am starting to consider, is adding a wave reaction

#community-meta message

There are some of my thoughts on this from the previous discussion

[surreal venture]

.pr 509 please anyone?

[dusky shore]

[404] #509 Issue/pull request not located! Please enter a valid number!

[surreal venture]

.pr 509

[dusky shore]

GitHub

[seasonalbot] #509 Spook Name Rate Game

[thorny obsidian]

The doc link in this case is linking specifically to the python2 version of the docs: #help-avocado message

Is that intentional or how does that work with the docs command?

[hardy gorge]

It's not intentional, but that's apparently the symbol that was matched

We used to be plagued by "2to3" docs as well

[green oriole]

It is pointing to the networkx dictionary

If you look at their page, they used a link to the py2 documentation https://networkx.org/documentation/stable/reference/glossary.html#term-dictionary

Soooo.. that’s not an issue, at least not on our end?

[thorny obsidian]

I'm dumb =] ignore that last message

[median canyon]

Can someone send me the template for this server. I am testing something

[eternal owl]

Check the pins

[short snow]

one https://discord.new/zmHtscpYN9E3

[trim cradle]

I'm getting ready to test something on the bot test server

[rapid igloo]

@obsidian patio could you have a look at this if you're free? I addressed your feedback a week ago https://github.com/python-discord/seasonalbot/pull/510 thx

[obsidian patio]

Maybe. I’ve been terribly busy lately

!remind 8h Check out PR

[stable mountain]

Yeah okay.

Your reminder will arrive in 8 hours!

ID: 806, Due: 2020-11-11T15:58:46

[clever wraith]

@cinder gorge tengo autismo

entiendo que te enfades, pero haber me rio de mi propia enfermedad

[hardy gorge]

@clever wraith Please use English in our community. It makes it easier for a larger group of people to understand what you're saying and it allows moderators to know if something you said requires moderation. That's why we have a rule that English is language of our community.

[clever wraith]

ok

[short snow]

this is quiet common, whenevr the last char of the msg is bold, even the stats go bold

[clever wraith]

Guess someone needs escape_markdown

[surreal venture]

Is there a way I can lock a Cog to be called in a certain month?

[tough imp]

probably easiest to just use cog_check https://discordpy.readthedocs.io/en/latest/ext/commands/api.html#discord.ext.commands.Cog.cog_check

you can try searching the repo for examples of how it's used

though beware that if you have listeners or tasks, those need special treatment I believe

the cog_check won't affect them

[surreal venture]

@tough imp so cog_check would be a method of the Cog I defined? And should the cog_check return a boolean of whether to run the commands or not?

[tough imp]

yes to both

https://github.com/python-discord/seasonalbot/blob/44bc2efb37702758594f7218ae998fd725394773/bot/exts/utils/extensions.py#L251

here is an example

[surreal venture]

yeah I saw that

thank you so much!

Is there a way I can raise InMonthCheckFailure? Can I put it in the cog_check?

[tough imp]

I'd imagine so, but I don't know - give it a try and see what happens

[green oriole]

this is quiet common, whenevr the last char of the msg is bold, even the stats go bold
Wait, who is u/Im_Joseph

Ah yeah, that's the Joseph

He's everywhere

[stable mountain]

@obsidian patio

It has arrived!

Here's your reminder: Check out PR.
[Jump back to when you created the reminder](#dev-contrib message)

[clever wraith]

@cinder gorge

@cinder gorge

[short snow]

@clever wraith why did u ping him/her?

[clever wraith]

because he has blocked me due to error

[short snow]

this place is for discussion on pydis projects, you are probably off-topic

[clever wraith]

oh, reasonable

[harsh veldt]

why https://github.com/python-discord/django-crispy-bulma is archived?

its master is 68 commits ahead than the original one, why stopping after working on it?

[brazen charm]

It wasn't being actively maintained by anyone

[trim cradle]

Does anyone have a copy of the config yaml that's configured for the official test server?

[sullen phoenix]

#681767410126159872 @trim cradle

although, it is a bit dated

[short snow]

how about adding a stackoverflow command to the bot

[sullen phoenix]

yeah, you aren't in the test server

a stackoverflow command?

[short snow]

yes

like

this is what i have got till now, and trying to add a way to look at the answers now

just like the wikipedia command

[sullen phoenix]

that’d be fine for seasonalbot, i think

lol

is that matt

[trim cradle]

I'm slain

I can't figure out why adding a duration to a command doesn't parse out the duration. It appears to do that implicitly for other commands.

[sullen phoenix]

discord.py uses converters to do that

it'll use the type annotations on the command's parameters to cast it to the appropriate type

you'll want either of these https://github.com/python-discord/bot/blob/36bffe9653b33c64aea21c5c31471f69d290ed37/bot/converters.py#L301-L354

[trim cradle]

let's see

[sullen phoenix]

so it's not actually implicit

[trim cradle]

Well, I'm not even seeing the log statements that I added

[sullen phoenix]

hmm well, that's an entirely different issue then

[trim cradle]

@sullen phoenix I think the discord.ext.commands.group decorator is swallowing the commands

[sullen phoenix]

hmm, how come?

[trim cradle]

    @group(name='bigbrother', aliases=('bb',), invoke_without_command=True)
    @has_any_role(*MODERATION_ROLES)
    async def bigbrother_group(self, ctx: Context) -> None:
        """Monitors users by relaying their messages to the Big Brother watch channel."""
        log.debug('hi')  # THIS IS THE ONLY CHANGE
        await ctx.send_help(ctx.command)

this debug statement works

[sullen phoenix]

that should only be called when you invoke the group without a command

so !bb

[trim cradle]

    @bigbrother_group.command(name='watch', aliases=('w',), root_aliases=('watch',))
    @has_any_role(*MODERATION_ROLES)
    async def watch_command(self, ctx: Context, user: FetchedMember, *, duration: Expiry, reason: str) -> None:  # SIGNATURE CHANGED
        """
        Relay messages sent by the given `user` to the `#big-brother` channel.

        A `reason` for adding the user to Big Brother is required and will be displayed
        in the header when relaying messages of this user to the watchchannel.
        """
        log.trace(f'LOOK: {duration}')  # THIS DOESN'T WORK
        await self.apply_watch(ctx, user, duration, reason)

[sullen phoenix]

and when you call !bb watch you should see the LOOK

but you only see hi in both cases?

[trim cradle]

bot_1       | 2020-11-12 05:03:36 | bot.exts.backend.error_handler | DEBUG | Command bigbrother watch invoked by Stelercus#6382 with error BadUnionArgument: Could not convert "duration
" into Duration or ISODateTime.

let's see

[sullen phoenix]

oh ok

well that * is in the wrong place

async def watch_command(self, ctx: Context, user: FetchedMember, duration: Expiry, *, reason: str) -> None: 
```should be like this instead

[trim cradle]

alright

isn't that the "kwarg only after this" token?

is there some meta thing going on where everything after the * gets passed as one argument?

[sullen phoenix]

yes, but dpy uses it as a "put everything else into this argument"

yep

[trim cradle]

interesting

[sullen phoenix]

so when the * was before the duration, it shoved everything after the user into duration

[trim cradle]

makes sense

idk how I feel about using the grammar like that

[sullen phoenix]

¯_(ツ)_/¯

i think it's nice, but it's being used for the "wrong" purpose

[trim cradle]

okay cool

I get the error message I wanted now

[sullen phoenix]

lol nice

for that one, just edit the docker-compose.yml

change the site aliases from api.web, etc.. to api.pythondiscord.local

[trim cradle]

and that won't mess with people's infractions on this server?

[sullen phoenix]

there's no way you'd be able to do that, since 1. you don't have the API token and 2. it's your local server

[trim cradle]

but

I wanna

[sullen phoenix]

lol

[short snow]

is that matt
@sullen phoenix yep 😄

hmmm, but i can't find a way to get the answers

all my open source bots have that pfp

[short snow]

@green oriole when thro your pr, couldn't see anything which may cuz difficulties

[green oriole]

What do you mean?

[short snow]

the help-channels name

u removed selenium

[green oriole]

Yes

We were affraid that users might think that this channel is reserved for selenium help

(Selenium as in https://en.m.wikipedia.org/wiki/Selenium_(software))

[short snow]

you wanted someone to have a quick look, so i did, and found nothing else problametic

yes i got that

[green oriole]

Riiight, thank you!

[cold moon]

Only 1 review left https://github.com/python-discord/site/pull/417