#dev-contrib
1 messages Β· Page 147 of 1
I think it is part of our test suite actually
but yeah I had failing checks because of that
@molten perch I'm taking a short dinner break, so I've posted all my comments so far
One thing I didn't mention is the newline at the end of files
Okay, thank you!
I've heard rumblings of a pre-commit config soon, so we'd probably need to get those fixed
If you have time, you could update them now, otherwise we'll just update them when we add pre-commit
Actually, it was removed in the past. I don't know the reason though.
(api#18)
Yeah, it was rediscussed here afterwards
Must have missed it :/
When ves has some more time, I'll figure out what happened and we'll go from there
@molten perch I'm running into functionality problems
Namely, sqlaclhemy is complaining because some of the tables have dependencies which need to be dropped to recreate the table, but they are not set to cascade
Hmm in that case, it's possible I messed something up while creating the new migration
It's uhh
I'll get the error in a pastebin
Auto generated
I can share the change I made
constraint api_nominationentry_nomination_id_fkey on table api_nominationentry depends on table api_nomination
Nomination_entry.. of course. The most problematic one. I'll take a look at it.
Could you please also share your migration script?
I think, I see what the problem might be.
Yeah, so
I had to recreate it, I reset my environment lmao
I basically just deleted nomination.py
(And all the changes that come with doing that)
I see. Hopefully I fixed the problem, I'll try to delete nomination and generate a migration to check if everything works.
Btw I may or may not have discovered this by deleting all the models
It ain't pretty, but you'll find stuff like that
I might have misconfigured the cascading a bit, I'll push a patch in no time. Sorry.
May I ask, how you did that? You just simply deleted the file?
I modified the cascading, everything should be working properly, but I get
sqlalchemy.exc.NoReferencedTableError: Foreign key associated with column 'api_nominationentry.nomination_id' could not find table 'api_nomination' with which to generate a foreign key to target column 'id'
Even with the version you used to test.
I don't even get to the point where the migration gets generated.
What did you do?
Deleted the nomination model, and ran an autogenerated revision π
Did you remove all the things that referenced it?
Like.. what? π I'm sort of lost here. π
If you deleted nomination.py, you'd also need to delete lines 27-31, and and line 37 of nomination_entry.py and line 8 of bot.__init__.py as they all either reference the table as a foreign key, or make imports directly from the python file
Yeah, I deleted from the bot init
But..
-_-
Got it. Thank you.
For some reason, I thought everything will be solved in nomination entry...
It seems like, there's a confusion with alembic. It can indeed detect table removals, however it has problems when it comes to references.
The order is wrong. I modified the migration script a bit to make it work. (Basically just moved the drop table expression to the end)
(https://paste.pythondiscord.com/exibemawey.py)
Alembic is different from the Django migration tool, or whatever it's called.
I managed to get it working without manual modifications to the generated SQL
It is not SQL actually. And when you generate them there is a comment "please adjust" so it's not a bad thing to modify them π
But, how?
It's fine to modify them, but I have a feeling if you just modified them in the migrations, they'll be missed in the future
Regardless, I'm trying to distill it down to the simplest method to get this running
It's a combination of a few things
Yeah, okay I give up on this
Just push your solution
It's fine as it is now, you just have to modify the migrations.
(But if you're interested: https://paste.pythondiscord.com/exibemawey.py)
I also noticed, that the cascading was fine so I reverted. So.. it's ready! π
Well, don't you have to push those modified migrations to your PR?
Or is that on the ones that drop it
Because if that's the case, we can also just add the cascade on the creation, and then we won't have to worry about it when dropping in the future
Well, I don't have to push new migrations. They're fine. Turns out that I did the cascading just fine, so it's OK.(But I'll do a final check on them later today jhust to make sure)
(And actually there's only one, an initial :D)
What I meant was if the auto-generated migrations refuse to correctly apply the cascades
We can manually add them into the initial migrations
That way if we go to drop them in the future, we can easily do so, without running into strange errors
Okay, I'll reorganise those cascades and add them correctly.
Okay, I figured out those cascading options, and pushed a new Initial migration! π
RE: sir-lancebot#842, sorry for the messy commit history, had to do some merging. Right now I have the restructure template as a multi line comment to fill out and test with. @elder belfry sorry for the late ping, but if you want to keep chugging along we should all be on the same page thanks to @clever wraith
https://github.com/python-discord/sir-lancebot/issues/913 Could I get this issue approved, I can PR a quick fix
Remember that you can also use sir-lancebot#913, not sure what's easier for you, but just a suggestion
@short snow RE sir-lancebot#919 it's resolved locally
Hence my comment before I marked as resolved
I still need to decide which suggestions I want to implement (only those of migration or also fixing up the commands).
Once I've decided and done everything that I want to I'll push it all.
Yeah, you should resolve conservations after pushing them (Hello zig π )
even i did the same earlier
zig told me that's better ^
Yeah ik, i just had the link handy so used it
Resolving just makes it easier for me to see what's still left to do π€·
But I can see both sides
Yeah
Hey @thorny obsidian! Just a message for the record for a remainder about the review for sir-lancebot#910, you can review it whenever and when youβre free since we have a lot of time, I just figured Iβd put this message out FTR
Yeah I have seen your recent commits, and it really looks good. One thing I wanted to ask, are we going to include the random subcommand too ryt?
I have a rough idea about it, but I don't know if I would be able to do it, anytime soon.
Also, I need to apologize, I have not been able to give much time (pressure from Academics mainly), although I really really want to work on it. π
No worries at all, school comes first!
I think we need to do the random subcommand as well as the name
Please review bot#1889
On my to-do
IG one perf improvement could be to move setting message_url out of the inner loop
Since it's going to be the same within each match
Ty for resolving the conflicts chris
I'm at school and doin it from my phone sucks
So the goal of this PR is to still trigger filters even if there is a user login or password that is added to the url?
No
It's to stop URLs that contain blacklisted URLs but aren't necessarily bad from triggering the filter
it's to make sure le.co doesn't filter google.com
(what happened to your other account?)
I think subdomains will be in the path too
Yeah they'd be in the netloc or path
didn't happen anything, I'm just another identity
if we support subdomains properly, we don't need to do the in() hack with www.
ok alizee
Alright
Could just do an endswith check
We do have bot#1276 but I'm not sure that requires a whole new list
I'll take a look at 400 pm CDT
Yea, we don't need that new filter list
we can just do that by default
I think we can just do ```diff
- if message_url in (f"www.{blacklisted_url}", blacklisted_url):
- if message_url.endswith(blacklisted_url):
I believe that will be a boolean on the new filter system, but I may be wrong
If we aren't careful ends with might trigger learn-python
Yea, seems non-trivial to fix properly
that discussion made me switch back 
!e ```py
if "https://learn-python.com".endswith("python.com"):
print("dang")
Can't we just strip the subdomain?
since there could be mutliple domains either end so we couldn't split on .
We'd have to to f".{blacklisted_url}" for endswith iirc
IE google.co.uk would be stripped to co.uk
similarly there could be bar.foo.google.com
Yea, could do that
I think we can just do ```diff
- if message_url in (f"www.{blacklisted_url}", blacklisted_url):
- if message_url.endswith(f".{blacklisted_url}"):
btw. Just a quick question. There's this comment https://github.com/python-discord/bot/blob/f998353ac895f59e03c0ddf636be76ecfc964f46/bot/exts/filters/filtering.py#L484
However, it seems like the regex pattern doesn't match urls without schemes. (I might be very well wrong though)
bot/exts/filters/filtering.py line 484
# If there is a scheme in the url then `β.netloc`β has what we want, otherwise its `β.path`β```
what about checking every single fragment?
individually, that is
would that work? since the blacklist could have a url like google.co.uk
so there would be 3 frags
I've got some stuff to do, I'll be back
true, it requires a scheme on the user-input, but doesn't for the filtered domain
so we could drop that when parsing the user input
but would need to do it for blacklisted_url
since a mod could add it with or without a scheme
No, that comment regards the URLs in the blacklist, not URLs sent without a scheme
Don't those sent without a scheme not trigger the regex?
No they don't
URL_RE = re.compile(r"(https?://[^\s]+)", flags=re.IGNORECASE)
there needs to be a http or https included
or it doesn't get filtered
I don't know anything about regexes
This means that it needs to start with https:// or http:// to be filtered
So if that needs to change, I won't be much help
I feel like we can convert the current blacklist to only contain the first fragment
is it even called fragment
I think name is the right term
So the blacklist doesn't allow dots at all, and if we want to filter it checks if the filtered domain contains the blacklisted frag anywhere?
that's my idea
yeah that was the plan
Yes they are
https://pypi.org/project/tldextract/ possibly?
can extract domains from the tlds
but then python is still a problem
with this we could join hostname and tld
and it seperates sub domains
>>> ext = tldextract.extract('http://forums.bbc.co.uk')
>>> (ext.subdomain, ext.domain, ext.suffix)
('forums', 'bbc', 'co.uk')
it looks like it works by making a sync request to a url to fetch all TLDs on import
or just ban .co.uk because who thought that was a good idea
with open("dump/good.txt", "r") as f:
Data = f.readlines()
f.close()
with open("dump/bad.txt", "r") as f:
content = f.readlines()
with open("dump/good.txt", "r+") as hit:
d = hit.readlines()
hit.seek(0)
for i in d:
if i != content:
hit.write(i)
hit.truncate()
seems like it is vendored https://github.com/john-kurkowski/tldextract/blob/master/tldextract/.tld_set_snapshot
why does this thing deletes every stuff from the file
rather than only the matching content
π
aha cool
Hey @clever wraith! Please see how to ask for help in #βο½how-to-get-help
i want it to delete only the matching content but it erases all the content of file
oki
There's some requests stuff in https://github.com/john-kurkowski/tldextract/blob/master/tldextract/suffix_list.py
was struggling to follow along, since I'm also in a call
I wouldn't be against adding it as a dep
I think it is used to update the file
Ah cool
# extract callable that falls back to the included TLD snapshot, no live HTTP fetching
no_fetch_extract = tldextract.TLDExtract(suffix_list_urls=None)```
Ahh cool
Hey @clever wraith just as a summary of what we've talked about, ready for when you're back.
https://pypi.org/project/tldextract/ seems to do what we want with breaking up the domain and stripping the subdomains out, so this might be best to use in your open PR. π
Why do we want it in this PR?
Is the BSD license compatible with MIT?
Please use suffix_list_urls=None so we don't pull a new cache every time
yup
Alright
So that it properly filters urls with subdomains or multi-part tlds
The current impl wouldn't filter a blacklisted url if it has a subdomain
IE if we blacklist bbc.co.uk, someone sending https://news.bbc.co.uk wouldn't get filtered
and trying to seperate sub-domains, domains and tlds ourselves would be hard, since we need a list of tlds
since tlds can be made up of multiple parts, and so can subdomains
maintaining one is
is anyone even able to start the bot?
we're not interested in making a PR each month to update the list
/healthcheck doesn't seem to be a thing on the latest site container
and why implement our own fetch and cache solution if a lib does it for us
.co.uk isn't in there
.co.uk != .uk
we changed it aroun recently, have you re-pulled your site image?
@vale ibex where does tldextract get its list
I just did
the official definition?
there's one that get's shipped with the package, and it can fetch on import too
something something root servers something something
Alright
Tldextract should work
But .co.uk is a public suffix and should be split on
I'll take a look at home
agreed, which was my point π
Wait what
Yeah I was agreeing tone is just hard kek
I mean that's what happened
ah alr lol
I mean someone did buy .co.uk and make it a tld
But it will be someone like verisign
But not verisign
I'll turn wungus.net into a TLD
Lol
it is actually so much simpler than it sounds
anyway, any idea why I cannot start the site?
well, start the bot that depends on the site /healthcheck endpoint
Did you update your config for the migration from subdomains
You cant
site_api: &API !JOIN [*DOMAIN, "/api"]
site_staff: &STAFF !JOIN [*DOMAIN, "/staff"]
Well, that would probably be why haha
.co.uk isn't technically a tld either, it's just a public domain that people can have subdomains on
pff it is my role to be pedantic like that alec
I'm the infrastructure nerd when Joe's away smh
Dude I don't have 2β¬/domain to spare
Same with ddns.net
I can host my own nameservers, easy
Cloudflare for SaaS is sick
guess it zorks
Cloudflare for SaaS is sick
works, not zorks
finally managed to convince our devops guys to migrate to cloudflare properly
we still use godaddy dns
hold on do we
we being my org
ah
It's actually cheaper for us to move to cloudflare too
since we're being shafted by godaddy on domain renewals
I'm in my CF shirt rn 
Do you ever change clothes?
Yes
i don't use their dns, but my organisation uses them to register domains
Anytime we talk CF you have it lmaooo
we pay a brand agency who registers our trademark under as many TLDs as they can
I just happen to be in my CF shirt at most times when people talk about cf
By coincidence lol
Literally the only thing in your wardrobe
I have 3 tech company shirts and they're all I wear
Do you eventually xfer renewals to CF?
nah, there isn't a point
We've got a few hundred in godaddy atm and renewals are hyper marked up
I need to be easily spottable for nerds
some of them don't even have DNS records, they literally just sit there and be registered
Ah lol
our rate hasn't changed with the brand agency so I guess they're footing whatever markup there is lol
we use them all, and we can get 2 orders of magnitude cheaper renewals by changing to CF domains
Isn't GoDaddy like the mother of toxic domain registrars
How do I make this not awful?
What on earth
store links elsewhere
we can't do this because we don't have host access
Even if I do, it is still awful
well, we do, but I don't want to install software onto the host
I kind of want to just slap a txt file
Why not make it a multiline string π€
because the links are so long you need to use line continuations
Note that this container is being run with a lot of privilege -- sharing the host's filesystem, pid and network namespaces, due to portions of the benchmark applying to the running host.
pass
Oh
we can't really change much of what it throws up
like
docker is managed by linode
Yeah I think what you have rn is good enough or separate the links like joe said
You can also stop the line before the link
So that the entire link is in its own line
oh wait
this is the wrong cluster
lmfoa
score is the same for pydis one
well, the 68% is, the actual categories are different
but i think that
posting a chart of vulns in a public channel
is not a great idea, lol
lol
i mean
nothing stands out to me looking at the output
Actually, I am. It's just halted by another PR
However, if the Alembic PR gets merged, I might give more attention to the api. So I guess you can take it π
I did! π
i-
OH I AM GONNA BE MAD FLAKE8
I mean it runs
lol
Will I really have to noqa this
try bumping flake8 in dev deps?
but it doesn't fail locally
oh wut
I mean I can try
oh, I think I know what it is
akarys@mojito ~/P/pybot (filters/autoban) [1]> git merge origin/main --no-commit
Auto-merging bot/exts/moderation/infraction/_scheduler.py
Auto-merging bot/exts/filters/filtering.py
Automatic merge went well; stopped before committing as requested
akarys@mojito ~/P/pybot (filters/autoban|MERGING)> poetry run task lint
Check for merge conflicts................................................Passed
Check Toml...............................................................Passed
Check Yaml...............................................................Passed
Fix End of Files.........................................................Passed
Trim Trailing Whitespace.................................................Passed
check blanket noqa.......................................................Passed
isort (python)...........................................................Passed
Flake8...................................................................Failed
- hook id: flake8
- exit code: 1
bot/exts/filters/filtering.py:378:33: F821 undefined name 'datetime'
That's a sneaky one
lol
Mine's very imaginative ```powershell
PS C:\Users\Chris> whoami
desktop-fec9jn0\chris
Wow
LAPTOP-EAR30CUE π
bot#1446 needs another staff/contrib approval, would appreciate a review :D
What was it?
Github Actions pulls main before running checks, and main actually removed that import
Why is your branch out of date before the reviewing starts though 
@clever wraith what @molten perch is trying to say is that there's no point doing message_url = message_url.netloc or message_url.path
since we only match on urls with protocols, the thing we want to compare will always be in message_url.netloc
We need to do blacklisted_url = blacklisted_url.netloc or blacklisted_url.path since blacklisted domains may not have protocols
Thats only on blacklisted, right?
Yea
but if we add tldextract, we won't have this at all anyway
since it'll do the splitting for us
Haha
For once I can totally blame that one of the admins since y'all wanted to discuss this PR haha
Lol
hey would it be worth making an upstream pr to improve the mocks for dpy?
clarification: upstream is pydis/bot
oops
nah if you didn't understand other ppl would've thought the same
i've taken the mocks for my own project, but improved on them to the point where I think they would be benefical to pydis as well
no, i only took the mocks
Ah right, then it's not really upstream
feel free to open an issue with your suggested improvements though
well, i'm curious if they're worth it to pydis
what i've done is made it so eg: MockTextChannel.send() returns a MockMessage
if its worth it, i can make it
No idea how that impacts personally, as I haven't worked with our tests much
That's why I suggested raising an issue with your proposed improvements, so that others can comment
!remind 8H
content
!remind [mentions]... <expiration> <content>
Can also use: reminder, reminders, remindme
*Commands for managing your reminders.
The expiration duration of !remind new supports the following symbols for each unit of time: - years: Y, y, year, years - months: m, month, months - weeks: w, W, week, weeks - days: d, D, day, days - hours: H, h, hour, hours - minutes: M, minute, minutes - seconds: S, s, second, seconds
For example, to set a reminder that expires in 3 days and 1 minute, you can do !remind new 3d1M Do something.*
Subcommands:
!remind delete <id_>
Delete one of your active reminders.
!remind edit
Commands for modifying your current reminders.
!remind list
View a paginated embed of all reminders for your user.
!remind new [mentions]... <expiration> <content>
Set yourself a simple reminder.
!remind 8H .
Your reminder will arrive on <t:1635217808:F>!
!remind 8H make issue to make content of replies optional and use message reference as content, in addition to replying to that instead
Your reminder will arrive on <t:1635217861:F>!
Hey! Can I get reviews on api#23? π€ (It's all fixed up, should work fine)
Hey guys
I want to suggest we daily have a topic to discuss so that we all know and have knowledge from each other
Ideas like that are better proposed in #community-meta. We used to have something like that previously, but it didnβt work out and we decommissioned it. Feel free to ask in meta if you want more details
I haven't reviewed in a while and I don't think Xithrius has requested my review for many more PRs. So which ones need review?
Other than DOrs3n's PR (I generally dislike ORMs quite a lot so I won't touch on that sorry).
I do have some PRs waiting in my notifications on GitHub that needs testing before approving but the only free time I have today and tomorrow morning is on my phone and maybe school laptop.. so I can do a code review
@austere hornet do you maybe want some pointers along the way on your PR?
Hi, so I am running in to one problem where when I try to add fields to an embed and update a certain field, upon editing the message, it deletes the field entirely. I haven't committed my changes, so if you don't mind, can I DM you my Python file?
Well you can send the important part of the code here, if you do want to send me the file give me 10 minutes or so to finish up my math homework and boot up my laptop
Oh, could you review sir-lancebot#910? I saw your comments and I'm implementing them, but wondering if you had any more
yea, we don't need it lol
not right now
don't see a major use for them
would rather filter outbound traffic tbqh
monzo style
lol
trust, as it gets deeper, it is not simple
it's simple to describe, hard to build
we'll probably go with some sort of egress proxy similar to how istio run it
we can get away with only supporting HTTP and SNI, and I don't expect our production traffic to move to ECH anytime soon
it's in that blog post
they inspect HTTP Host header for HTTP or SNI for HTTPS
and allow based on a set of rules
I think I'll build a Rust service that can HA on each of our nodes
need to figure out how to push traffic towards it though
One approach, used by Istio, is to run an egress proxy inside Kubernetes. This egress proxy would inspect the traffic sent to it, figure out the destination, determine whether the destination was allowed (given the source) and then pass it on. To determine destinations for encrypted traffic, it has to use the SNI (Server Name Indication) field for encrypted (TLS) traffic. For unencrypted web traffic β which is becoming increasingly rare β it can simply inspect the HTTP Host header.
yep
we'll run one proxy on each node
might even run 2 per node
with a topology key
uhh
i mean
it can be lol
but topology is also very much in computer science, and the word topology just means "how certain things are arranged"
so in our case, a topology key tells Kubernetes how to lay out pods on our nodes
if I use a topology key that has an antiAffinity to node names, then Kubernetes will try it's best never to run the application on the same node if we mark it for HA
sir-lancebot#842 is ready for you! Get some color / colour in your life
example is alertmanager: if we scale alertmanager kubernetes will place the new instance onto the node with the least alertmanager instances
namespaces/monitoring/alerts/alertmanager/deployment.yaml lines 17 to 30
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- podAffinityTerm:
labelSelector:
matchExpressions:
- key: app
operator: In
values:
- alertmanager
namespaces:
- monitoring
topologyKey: kubernetes.io/hostname
weight: 100```
our alertmanager config is probably the most complex thing we've got in the cluster
my SD solution was pretty solid https://github.com/python-discord/kubernetes/blob/main/namespaces/monitoring/alerts/alertmanager/initscript.yaml
I think DNS hijack is prooooobably the easiest way to configure the proxies
I love it
preseving unloaded cogs and modpings schedule :D
Ooh, nice work! Left a review with one or two things but it's an approval from me
Here's your reminder: .
[Jump back to when you created the reminder](#dev-contrib message)
Here's your reminder: make issue to make content of replies optional and use message reference as content, in addition to replying to that instead
[Jump back to when you created the reminder](#dev-contrib message)
bot#1911
I'm very confused as to why the tests on bot#1901 are failing. From running the test it seems I need to change'0.65 0.64 242' to '0.63 0.48 218' but I don't understand why or what these values repressent
It seems that they are HSL color values, '0.65 0.64 242' looks like the og_blurple color and the other one looks like the rebranded blurple
@static canyon we can use ctx.invoked_command to get the alias/command which was sued
I don't think that would include the subcommand (or the parent command, forgot which)
are you talking about hackstats link?
oh right ok
Actually only '0.65 0.64 242' looks like HSL https://imagecolorpicker.com/color-code/6b67df
no idea what the other value is https://imagecolorpicker.com/color-code/2d66c8
So did the autoban feature work or we broke everything hehe
I don't see any fix PR, that's a good first sign
Don't think we've actually had any triggers yet
So sad
You could always send yourself to test it :p
Haha
Wouldn't recommend though lol
I could use our alt, since it won't ping
Yeah, it's kinda confusing me
Yeah, it doesn't ping
And I can unban
My alt already has like ten infractions, half of them being kicks lol
It may be already banned though
Right yeah, if that works then +1
Aw, I don't have access to the test server so I can't give you the ID
I need to get that invite
The account is named Ishiro with a the weeknd profile picture, could you check it isn't banned?
What's the account name#discrim and I can get (send in DMs if you don't want to share publicly)
No idea what the discrim is
Ishiro isn't in the test server
Guess I yeeted it during my last testing
Is this it?
Yus
So it did work if y'all are curious
:o shouldnt the PR already be tested?
Well it has been tested locally but not in prod yet
how many alts do you have akarys π
hmm
feel free to break api#23 as well π€
I found a way to fix it. π (partial solution :/ )
@clever wraith I guess it would require patching the command rather that the code to ask for a confirmation instead, I don't know which monkey patch is the worse
I feel like patching annotations is a bit dodgy though
Also yeah I was thinking of a simple β/β
modal, asking to run ?close 0s sounds unintuivive
I would even say we should ask for that confirmation for any close, since it happens we just run close when meaning to run another command
Ultimately I just needed to update the annotation to use a new converter, I could've patched the command itself but that's pointless when patching the annotations gets me the same behaviour in a simpler manner
man, I discovered that running command.update borks the command when it's a part of a cog, not the most fun
Heh
Im not sure about that, i don't think we do that enough to justify needing confirmation - but we could ask more mods
I know Akarys does that quote often when trying to run ?cancel haha
Hi! I just updated my copy of Bot to the latest, which including moving discord.py to v2
It is now having this error (ImportError: cannot import name 'Thread' from 'discord' (/usr/local/lib/python3.9/site-packages/discord/__init__.py)) when starting, and exiting straight away:
https://paste.pythondiscord.com/gusamahata.sql
This is happening in random cogs each time, presumably the first one trying to import this.
Could someone help?
Do you run using docker?
yes - running docker-compose up to start
great, you'll need to do docker-compose build first
aah
this rebuilds the image with the updated dependancies
makes sense
Actually, there are also some site changes, so I suggest you do docker-compose pull too
then docker-compose up
π
@dim pelican You should resolve the requested changes after you push them
Also, for those site changes you will need to update your confil.yml file too @reef tinsel
Yup, I thought I did....
Running out the door now so I'll need to check everything a bit later (and away from my testing setup until the end of the day π )
Under the urls section, set site_api to site_api: &API !JOIN [*DOMAIN, "/api"] and site_staff to site_staff: &STAFF !JOIN [*DOMAIN, "/staff"]
π
yes, but it's easier for me to just say what the differences are, rather than send them a link and tell them to read the whole thing
thx chris - bot working now! the patreon additions should come by the end of half term!
Nice π
scottish half term just ended 
regarding bot#1765
with the monthly post, should we send the tier 1 supporters first, or the tier 3 first. (we decided on a seperate message for each one here: #dev-contrib message)
half term started at 12:00 last thusday (it actually started 10 mins early lol) and finishes on sunday this term
(just for my school, other schools ear here do different things)
Mine finished on this sunday π
Imo it makes most sense for tier 1 first
That way it's sequential but also the tier 3 will be the last seen when the messages "move up" from other activity
ok
i was defaulting to tier 1 first, just asked to get a sense of what other people wanted π
@clever wraith I just went ahead and tried out the functionality, yours thoughts on this.
Its how the leaderboard will interact the core leaderabord functionality
and could i get assigned to the issue
That looks good, I'd just rename that function to increment_points rather than change
ah wait, that could isn't fully updated
the funciton name is right
it would be like this
lambda before: before + 10 instead of 10
Well, I would say we should have an increment utility function, for clarity sake
I don't think we will need to do any other operation anyway
Also beware that you have a potential race condition there, I wonder if we could solve this using a Redis transaction
yeah, its not final, i doesn't even check if a cache alrady exists and just forms a new one rn
Okay, cool
and that to_dict is just for debug, i am removing it
Don't forget to also add a maximum of points earned per (user, game, day)
Morning all
I'll be doing some work on bot#1889 after school, otherwise Ill be busy today
would that be per game, or overall
Per game I'd say, so you must play different things and cannot just spam .quiz alone
Right, would you like to have another cache for that, or should i merge it with the main one (per day leaderboard)
I would say another leaderboard
π kk
Re: https://github.com/python-discord/bot/pull/1889#discussion_r735661695
I think .hostname would be good enough for message_url since the matched group contains the scheme. However for blacklist urls that may not have a scheme I generally check if .hostname is None, then re-parse with https:// prefix if so
Yea, agreed. We're changing direction with that PR anyway, due to sub domains see #dev-contrib message
is that the same message linked in the last PR comment?
Yea
π tldextract probably handles the scheme thing so that's good
Yup π
curious what advantage tldextract provides over just splitting the two hostnames by . and comparing the tuple or list? (sorry if this was discussed above the linked message since I didn't read much far before it)
you can have sub-sub domains
and tlds can have multiple parts
and web.www.google.com (this isn't real, just an example)
so there isn't a reliable way to get the actual domain without a full list of tlds
so if google.com is blacklisted, we want web.www.google.com to match? or is it the other way round like www.google.com blacklisted and wanting google.com to match?
I'm sure the BBC is trying to be a pain on purpose
www should be stripped in my opinion, it should be counted as if it was the root domain
so it's the former of which I've listed?
Yea
but we don't want to block google domain directly, ignoring the tld
since python.com and python.org are a thing
so we want to block python.com but not python.org
so we want to make sure the domains match exactly, and the tlds match exactly, and ignore any subdomains when matching
It's been a week I think when I opened my issue and I got no comment till now
Is this common π€?
What's the issue?
.src
Have made a request internally for someone to take a look π
Give it a few more days
okay that clears up a bit for me (though I still don't understand why comparing hostnames isn't enough but it's ok)
It is enough, there just isn't a reliable way to extract hostnames for all cases without a list of tlds
does anyone have any idea as to why this is happenning? (code snippets coming shortly, depending on how my internet/computer play ball π )
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/discord/ext/commands/core.py", line 167, in wrapped
ret = await coro(*args, **kwargs)
File "/bot/bot/exts/info/patreon.py", line 170, in current_supporters_command
await self.send_current_supporters(ctx.channel)
File "/bot/bot/exts/info/patreon.py", line 108, in send_current_supporters
embed.add_field(actual_tier_1_members[0]) #.mention
TypeError: add_field() takes 1 positional argument but 2 were given
add_field() requires you to use named args
not positional
add_field(name="foo", value="bar")
aah
got quite a few to change then, as i Have about 7 of these in this code atm π
good thing is i can copy paste
i assume that name is the title of the field
having done a quick test, it is
and it cannot be an empty string
how should I do the lists of members for the patreon (bot#1765) - I was planning on doing basically this: embed.add_field(value = member.mention) @vale ibex ?
Just format something pretty in the description no?
You could just do ```py
embed = discord.Embed(
title="Tier 1 patrons",
description="\n".join(tier1_patrons)
)
Since we're going with 3 embeds
that could work, guess i'd still overflow into a second emebd if limits got hit?
Fyi if you want it to be empty you can set to "\u200b" which is a zero-width space
I don't think we need to worry about that, since we're having each tier in it's own embed
and each embed description can support 4096 chars
!e print(4096//37)
@vale ibex :white_check_mark: Your eval job has completed with return code 0.
110
so we could support ~100 patrons in each tier with just 1 embed
were those messages referring to me or tizzy chris?
you
nah I wouldn't bother for now
ok
since it's not over 110 patrons, it's over 110 in a single tier
so if it does get to that point (or close) we can upgrade from there π
yes
i agree it is unlikely that it will happen π
Would it be worth just doingpy if len(tier_members) > 110: log.warning("Tier {x} has more than 110 members. May fail to generate embed due to character length.")or something?
== Raw message ==
@vale ibex
so it's an id(20 chars) +4
so 24 per name, plus one for a new line
!e print(4096//25)
@vale ibex :white_check_mark: Your eval job has completed with return code 0.
163
Don't we want to include the name?
we're gonna do a mention
That doesn't actually ping the user or send a mentions objects, meaning that for mobile client they will all render as "invalid-user"
goood point
we don't want it to ping
but the rendering point is a problem
and for the latter we don't have a better solution unless we do both
which isn't great either
we could do the mention, and if we get too many complaints change it?
Is it not? Do we really not have enough characters?
I meant on a pure looks point of view, including both the name and mention would look overkill for anyone who isn't on mobile, and may have a similar number of questions raised about it
but I guess I'm also biased since I never use mobile Discord, due to these issues and more
i will do the mentions only for now, send screenshots of it on desktop, ipad and android phone (the 3 devices I have :-)), and we can decide
whats best
sorry for the cut off
id say it looks fine, even without having had the mobile problems
does anyone have any idea as to why guild.get_role(ROLE).members is returning [], even though it clearly has a member (me)?
i can confirm that ROLE is a valid role, as the exact same variable is working elsewhere
nope, bot
Did you turn on intents for your bot?
bot has the members intent enable in the portal
i also assume that they are enabled in the mains script
Or while a cog is initializing
yes
it is inside a function called by a command function (currently useless, but for a reason)
What do you get when you do guild.members?
it is returning an empty list (it should be 2 members - myself and my copy of bot)
I'm not really a fan of how that looks to be honest.
The name#disc just doesn't look that nice and the (mention) doesn't either. Either it should be the mention or just the name.
We do the opposite in other parts of the bot, like @cold island (mbaruh#0278)
if we're doing something similar elsewhere in Bot already, then I guess we can do that, following the same pattern?
Mention in embeds make mobile users feel very bad about themselves
I prefer this format too
Yeah I'm not sure. It should be resolved by the members intent
Maybe someone over at #discord-bots has an idea
bot#1911 @vale ibex π
. @fallen patrol
Ye the point was I made it and I think I linked to the correct discussion
I'd say so, yeah
And then we'd need to figure out what the worst-case for reaching embed description char limit for the warning
Wow. Would love to see them on api#23 
Huh, interesting choice with purple closed
I guess we should start to use those tags more often CC @vocal wolf
turns out the problem was I was using fetch_guild rather than get_guild π
Both my iPad and my android phone are rendering the mention well
But that's because you're testing with yourself and bot
Aah, does it work well with myself?
Well that's you, and the bot should also be in cache
Aah, so if Iβm not in cache thats when the problems happen?
Exactly
What tags?
I suppose this also means the icons for @dusky shore linking github issues/prs should be updated
Won't fix and duplicate, as it will change the issue status icon
We don't use them that often already, and I'm looking out for those issues/PRs.
I think we should be using them more though, especially won't fix
They have relevance now
Alright, but they already don't happen that often. Usually there's something to fix, and duplicates are very rare.
I was thinking about issues we do close without submitting a PR, I think that's how the new branding is designed, you want the icon to be purple if you have a PR that fixed it, otherwise just gray?
I guess
Oh, it isn't a thing right now
That was just an example they need to figure out the details of those different closed states
Cool
@vale ibex we said we didn't want to store that in Redis lul
bot#1765
does this look good?
I think that looks decent, although I would add some wording on why Patreon is relevant for us and stuff like that
Ahhhh light mode! My eyes! ||\s|| But I agree that it looks pretty good
Store what in redis?
could you provide an idea for this?
IDs of DM bookmarks
That's not what I've stored in redis
I used redis to store the IDs of the messages you have bookmarked
so you can't bookmark the same message twice accidentally
I am currently on mobile, can't worte that right now, sorry 
sir-lancebot#826
Aah
ok π - I can't think of anything, so if someone can write anything just @ me!
This is what we say on our actual patreon page ```
We'd like to be able to offer excellent prizes for all of our events. Stuff like t-shirts, stickers, microcontrollers that support CircuitPython, or maybe even a mechanical keyboard. We believe that this would help motivate more people to participate in our events, and we believe these events are a great way to learn how to work in teams with strangers, learn about source control, and just generally get out of your comfort zone and learn something new.
While we rely on sponsorships to cover things like hosting, we will need actual hard cash if we're going to be able to pay for these prizes. But let me make this clear: We are non-profit. 100% of our earnings will be put towards improving the community in one way or another.
Patreon supporters
Here is a full list of this months Python Discord patrons!
We use the money from Patreon to offer excellent prizes for all of our events. Stuff like t-shirts, stickers, microcontrollers that support CircuitPython, or maybe even a mechanical keyboard. We believe these events are a great way to learn how to work in teams with strangers, learn about source control, and just generally get out of your comfort zone and learn something new.
You can also support us on Patreon here!
seems a bit too wordy
yeah
That text could 100% use an update
!embed Patreon supporters
Here is a full list of this months Python Discord patrons!
We use the money from Patreon to offer excellent prizes for all of our events. Stuff like t-shirts, stickers, microcontrollers that support CircuitPython, or maybe even a mechanical keyboard.
You can read more about how Patreon supports us, or even support us yourself, on our Patreon page here!
Patreon supporters
Here is a full list of this months Python Discord patrons!
We use the money from Patreon to offer excellent prizes for all of our events. Stuff like t-shirts, stickers, microcontrollers that support CircuitPython, or maybe even a mechanical keyboard.
You can read more about how Patreon supports us, or even support us yourself, on our Patreon page here!
That seems a bit better
Did we ever buy someone a keyboard lol
Lol
lol
We did π
I think a laptop at one point too
Wow
Someone did a codejam entirely on their phone and won 2nd or 3rd place, so we asked the other winners if they wanted to accumulate their prizes into a laptop for the guy, eventually sent one out.
Wow
Was it this year or in previous years?
What a god
many years ago
Ah ok
Do we have a datetime -> Discord timestamp command? should I just make one?
I see there's an issue
Hi, I'm having some weird issue with isort. I pip installed it and ran isort . in the project root (sir-lancebot\bot) but I'm still getting this error? What should I do?
Still getting same error
Well it looks like you still have a relative import
Wdym by "relative import"?
you need to run poetry install --remove-untracked @austere hornet
How do you clear the buttons of a discord ui View once the interaction is complete?
arthur/exts/cloudflare/zones.py lines 22 to 24
def disable_select(self) -> None:
"""Disable the select button."""
self.children[0].disabled = True```
that's my approach, call that from the interaction function and then do this https://github.com/python-discord/king-arthur/blob/main/arthur/exts/cloudflare/zones.py#L52-L54
arthur/exts/cloudflare/zones.py lines 52 to 54
self.disable_select()
await interaction.message.edit(view=self)```
π Thanks
π
how can i dev contribute?
!contribute
Contribute to Python Discord's Open Source Projects
Looking to contribute to Open Source Projects for the first time? Want to add a feature or fix a bug on the bots on this server? We have on-going projects that people can contribute to, even if you've never contributed to open source before!
Projects to Contribute to
β’ Sir Lancebot - our fun, beginner-friendly bot
β’ Python - our utility & moderation bot
β’ Site - resources, guides, and more
Where to start
- Read our contributing guidelines
- Chat with us in #dev-contrib if you're ready to jump in or have any questions
- Open an issue or ask to be assigned to an issue to work on
@static canyon which project had coveralls cut?
Wdym?
#dev-log message this?
Oh wait you mean that
I know what you mean
If you look at git history of the file it'll show
bot#1838
lmao
im gonna go sleep now cya
lol
!remind 16H .
Your reminder will arrive on <t:1635375667:F>!
shh
You can also just do view=None and not bother with the disabling
There's a synchronous .stop() method on the view of you are worried about hanging callback listeners
#bot-commands message why isn't this working?
There's a 2000 char and 15 line limit
Oh, it should post a message then ig
We could. It's this line here that does the check https://github.com/python-discord/bot/blob/main/bot/exts/info/code_snippets.py#L252
bot/exts/info/code_snippets.py line 252
if 0 < len(message_to_send) <= 2000 and message_to_send.count('\n') <= 15:```
Ok, want me to open a quick PR?
If it did, I'd rather it self-deletes after like 3 seconds or something
I wouldn't want it to stay
yeah
3 seconds should be enough for the message author to understand why it didn't link
Maybe 5 tops depending on the length of the message
Is it possible to send those ephemeral messages without views? Like those system messages
I'd also invert the if statement
yeah reduce one indentation
Mhm
lemme see how its implemented in views
I don't think so, but could be wrong
You can send it as a regular user message
but not a system message
Would undermine the whole system part of that
Yeah even I don;t think, but just now i got those invite people notification in the channel list and saw the ephemeral message. So thought about that
afaik ephemeral messages can only be sent in response to an interaction
Yeah
I'd still open an issue first btw
Hmm yeah, interactions without any UI component could be used though?
huh?
the point of an interaction is that its user generated by interacting (hence the name) with a ui component
Ah yeah
@gritty wind how long were the reminders you talked about?
7D1M
That's the latest one
But it varies
I also had one that was a year
let me look
Commands for managing your reminders.
24H
And one that was 10H but literally disappeared as soon as I made it
I tried to cancel it right after
How would I do that though
Not sure what you mean
I can check the DB itself, but the object is deleted
By the time I know a problem exists, it's too late
I mean after you created the reminders, did you checked they got inserted?
No, it doesn't seem reasonable to check after all my reminders π
It's why I'm making these changes
I propose we make use of the active flag
Because we do not check the return code when performing a POST on the site, so for all we know the site may be erroring and we are dropping that silently
bot#1915
cool
Well, doesn't the api wrapper throw an error on bot if the api errors
Yeah, it does
I know as much because I'm currently erroring on my test bot
For unrelated reasons
Don't you have to pass raise_for_error=True or something like that?
bot/api.py line 79
async def get(self, endpoint: str, *, raise_for_status: bool = True, **kwargs) -> dict:```
Oh cool
I suppose I haven't checked for that error
I only looked at logs for the delivery time
Maybe it's being swallowed up by sentry
Hmm no, the command error handler would handle this I think
And also I got confirmation receipts
Maybe check the site sentry too?
Nothing there
Still haven't managed to test things because it took ages to get metricity running
I've updated my build script so it just takes care of itself now
https://paste.pythondiscord.com/udejejahim.less
What PR is this for, out of curiosity?
bot#1916
It's not directly related, I just need to have metricity running to make reminders work
That's my local build script because building metricity is honestly a PITA
At least if I want to use my API's DB, which... why would I want anything else
yet still, it refuses to coperate
Why is metricity not adding my user to the DB
Misconfigured guild it would seem
You need metricity for that, huh
The API request has a user FK, the bot will try to set it to my actual user ID
It's why I wanted metricity in bot, you do actually need it if you're working with bot directly
I mean, the API is still complaining, but at least the user exists now
Right, I'm running metricity from a separate docker container, because I do mess around with metricity as well
It's why I have site in a separate compose too
It's not the "normal" workflow, so it's just me complaining. If it was, I would've gotten it fixed π€‘
because you don't want to pull latest, but locally build?
The build script does pull latest, just sometimes I want to make modifications to metricity directly, which means running it manually as well
Right now I don't have any changes
you cna just change the image name to the name of the local image in the bot's compose
rather than messing with different compose files
I see
Does the API use a different non-metricity user DB
If so, why, and WHYYYYYY
and why is my user not in it yet
Yeah, no data in that DB
Now how do I populaet it
sync?
!sync
Run synchronizations between the bot and site manually.
Subcommands:
!sync roles
Manually synchronise the guild's roles with the roles on the site.
!sync users
Manually synchronise the guild's users with the users on the site.
Why do we have two user DBs
IE in bot there is image: ghcr.io/python-discord/site:latest if I wanted to test local changes, I'd change this to site_site_1
since this is the name of my site image
wdym?
not DBs, tables bleh
in metricity?
Like we could definitely just have metricity and site write to the same DB when it comes to users
One in metricity and one in site
Ah right, yea
Right, there is nothing stopping us from accessing one from both apps and halve the size of the DB(s)
You can't do queries across dbs
We can connect multiple DBs
Also funny thing
Sync passed and there are 0 users in table :)
you can, but you can't join between two tables in different dbs
so if the users table was in pysite, metricity wouldn't have access to it sql side
Wdym
Joining user table on metricity, and on site
Metricity doesn't query pysite
what scale is suggesting is that we only have 1 users table right?
Right, I know, but there is nothing stopping us from making it do that
There is
It's a table with > like a million entries at this point π€‘
Metricity isn't used just here
If the users table was in pysite, you couldn't run a query for all metricity messages from a user sql side
you'd need to pull all messages, then all users from the two db and then do the filter in python
METRICITY_URI
USERS_URI
Set them to the same thing if you want to use it for your own purposes lol
Fair enough on that ig
Any ideas for this
That's just one example, but yea there's more use cases that would block it
is your metricity bot running?
The URI is misconfigured maybe
It is, but the metricity DB is unrelated apaprently
It's the site DB
Metricity one is populated fine
oh
Site is connected to postgres
Hmm that doesn't sound right
oh wait now, there's two syncers
the sync cog does talk to pysite
metricity has it's own sync
I ran !sync roles and !sync users after
It worked after syncing users again
And finally, reminders are working π€‘
When deleting reminders (!remind delete) do we want to keep them in DB but deactivate, or do we want to remove them
I don't think there's a reason to keep them
sir-lancebot#842 is ready for testing and reviews
Yeah, I was using ctx.invoked_subcommand instead of ctx.invoked_with trying to get the "mode" that it used
Now it shows in the embed what the user called it with
This looks really cool! One thing - could you capitalize Rgb to RGB and maybe add some sort of emphasization for the color name, like RGB information for Storm Dust or RGB information for Storm Dust
Sure thing, won't be able to test until the end of the day though (unless someone has sir-lance set up and can use gh pr checkout 842)
Yeah, I like that idea. But anyhow this is really cool @dim pelican !
Its been a wild ride for sure. My first try was a mess lol
I do have sir-lancebot set up but I'm not at home lol
That's pretty much always the case. First PR can be intimidating, even for me since I'm working on my first PR too!
I've messed up a bunch too so
We're all in this together lol
Oh the best part is that the color command will be like my 3rd (?) contrib to sir-lance
Out of curiosity, why do you have the NAME in a code block? I feel like since it's just plain text, it doesn't need to be that way.
But if it looks better, of course
Cause we make a dictionary mapping of the field names and values:
for colour_space, value in colour_conversions.items():
colour_embed.add_field(
name=colour_space.upper(),
value=f"`{value}`",
inline=True
)
I could add an if to exclude the name value
Now I'm trying to remember what I've done for sir-lance...
Realpython, Teapot, WTF Python and I guess Color now
Yeah that's where I remembered Teapot lol. Just gotta set is:closed author:brad90four
I have got 149 total PRs π - though some are test PRs
Heh, 185
Ok someone has 300, guess
My guess would be Sebastiaan
I didn't see his, it was joe
my public ones are at 234
0 for me lol - but going to go up in the enxt few hours :-)

