#dev-contrib
1 messages · Page 74 of 1
okay
thx
ops pinged the wrong mark
@tawdry vapor
😂
I don't have time to review right now
I don't know if anything was relying on the previous behaviour
okay
Let me cross check again
the surprising part is, a cog was using error.handled = True
Core Devs: Ready for merge: https://github.com/python-discord/bot/pull/1214
Created a script that would allow for identification of incorrect AoC leaderboard calculations:
https://paste.pythondiscord.com/hozunuwaxo.py
3 Relevant variables at the top have to be filled in
Is there any reason, why the bot keeps on typing even after when the command is already displayed
Seems like a quirk of d.py
I tested with this, and it just held for a while after sending the message
@bot.command()
async def test(ctx):
async with ctx.channel.typing():
await ctx.send("Test")```
ah i see
Typing events will last for 5 seconds, because of the way discord work
Hello, In this file, https://github.com/python-discord/bot/blob/master/tests/bot/test_constants.py
What is the use of line 14? - origin = typing.get_origin(annotation)
IMO the functionality will remain same i.e the assertion passes for non-normalised types too.
We need to use normalized types in order do a proper isinstance check though
`import typing as t
from collections import deque
t.get_origin(t.Deque)
<class 'collections.deque'>
isinstance(deque(), deque)
True
isinstance(deque(), t.Deque)
True`
Similar for list too, output for both is same.
!e The issue is with
from typing import List
print(isinstance([1, 2, 3], List[int]))```
@green oriole :x: Your eval job has completed with return code 1.
001 | Traceback (most recent call last):
002 | File "<string>", line 3, in <module>
003 | File "/usr/local/lib/python3.9/typing.py", line 657, in __instancecheck__
004 | return self.__subclasscheck__(type(obj))
005 | File "/usr/local/lib/python3.9/typing.py", line 660, in __subclasscheck__
006 | raise TypeError("Subscripted generics cannot be used with"
007 | TypeError: Subscripted generics cannot be used with class and instance checks
But it will work with get_origin
Thanks!
@green oriole It looks awesome! I really appreciate it
@eternal owl It'll commit them but when you try to push them it'll only send the ones you modified, not all the line ending ones
At least in my experience
okay
@nocturne hare Can you please drop a review on my pull request?
@surreal venture You haven't still addressed my comment about Redis. You can see how Redis can be used in https://github.com/python-discord/sir-lancebot/blob/master/bot/exts/halloween/candy_collection.py
Should I close https://github.com/python-discord/bot/pull/1275 because I just don't know what to do with it? I'd leave this to anybody smarter.
Yes, you can close it if you do not plan to continue working on it.
I think it's better to live with false positives and wait for a fix rather than fix that and introduce an easy way to circumvent the filter.
... Oops
We tried to keep PRs open in the past to keep the work already done, I feel like we should do the same here?
Sorry, I didn't checked this channel first, my bad
That's situational. In this case, I don't think there's much to salvage from the current diff since a more comprehensive approach is required, which means something more complex.
you never seen that
I do not know why, but I am unable to run seasonalbot. Every time I run pipenv run start, it gives me an error:
socket.gaierror: [Errno -2] Name or service not known
I have installed redis (from source) and have run pipenv sync --dev
oh wait do I need to start the redis-server?
At what port does the bot expect redis to be running?
Nevermind I got the port
Nope, it still isn't working
I am running Ubuntu 20.04.1 (AND I AM EXTREMELY EXCITED) with python version 3.8.5
Please do ping me if you know why it isn't working
@surreal venture you can enable fakeredis in the config
/etc/redis/6379.conf?
no, the bot's config
how do I do that?
so you won't need a redis server running at all
USE_FAKEREDIS in .env?
set use_fakeredis to true in your config
ok thank you!
no, that won't work
SeasonalBot have this in environment variable USE_FAKEREDIS=true
oh, i missed that this was seasonalbot
i thought you were talking about @stable mountain, sorry
@surreal venture 
Yeah, JSON have to be migrated to Redis for #509
Thanks!
Just one more question, when we initiate RedisCache, do we need to specify the namespace? Also, will it automatically load the data stored for that namespace (if there is some data)?
When you define this inside class (like in example I linked), no.
Ok, since the namespace would be class_name.var_name by default?
A RedisCache needs a namespace, as that's what tries to prevent us from having collisions in redis
Yeah, but you don't have to worry about that
RedisCache will automatically set that namespace and you never have to manually provide it
Ok thanks!
Just make sure you assign a class attribute to it
And it automatically loads the data from the namespace?
class MyClass:
some_attribute = RedisCache()
It does not load data automatically, you have to use the get and set methods (or any of the other methods)
Basically, we don't want to have a local copy of the data that could get out of sync with the state in the Redis database
Each time you want a value, you just ask for it and it will get you the value from Redis
Okay thank you!
I can't make my __init__ an async function, right?
How do I add some code in which I get something I need to await?
use property decorater
isn't that for like getter and setter?
u can use it as an attribute, so you could basically do
class A:
@property
async def something(self):
return "hello there"
async def somefunction(self):
await self.something #raises some error, but this is the basic idea u can use it as an attribute
I am not sure I understand
So like this is my code:
def __init__(self, bot: Bot) -> None:
self.bot = bot
...
self.some_attr = await something
How would I put that in my __init__?
u don't,u can use self.something as an attribute.
__init__ can't have async and await
Oh ok I see what you mean now, thanks!
I don't really see how that's relevant to running async in an init
you can set self.some_attr to None and then load it using self.bot.loop.create_task(func)
@surreal venture
Other parts of the codebase schedule the tasks for separate execution, but I don't know if that is viable in your case
That looks promising. I'll look into it thanks!
You can use a separate coro that you call manually or schedule it with the create task like above
is there HGET and HSET in async-rediscache?
(and sorry for asking so many questions)
That's what a RedisCache uses
The namespace is the main key, the key you provide is the second key
What if I want to store a dictionary in a RedisCache instance? How would I go about doing that? Normally in redis-py you could do redis.Redis().hset() and redis.Redis().hget().
json if you need it for more keys
I was using json, but ks123 asked me to use redis instead
You typically represent a dictionary by a RedisCache directly
Since that's what a RedisCache is more or less a stand-in for
A RedisCache is the hashset you're after
so I define another RedisCache for each nested dictionary?
Wait can I convert it to a string and load it back again using json.dumps and json.loads?
Right, nested dictionaries is another story. You can't really do that with Redis either.
there's only key -> key -> value, which in Python translates to "name of the dictionary" -> "key" -> "value"
A single, non-nested dictionary would already be a hashset
A non-hashset "set" is like assigning a single name to a single value (no container other than the entire database)
So basically, I need my RedisCache to be like int: Dict[int, int, str] or something
Can I do this?
Sure
That should work (but: it could create potential race conditions)
The problem with serializing and deserializing JSON is that if you get this sequence of tasks, you lose data:
That is true, json operations aren’t atomic
Could it be possible to make them atomic at the library level? Maybe using a lock or something?
-
task 1 fetches data
-
task 1 deserializes and modifies local dict
-
task 2 fetches data (OLD data)
-
task 1 serializes as JSON
-
task 1 stores new JSON to Redis
-
task 2 makes changes to OLD dict
-
task 2 serializes
-
task 2 overwrites changes made by task 1
oh... that would be bad
There is a lock you could use to create mutually exclusive callables, but you need to be careful
It's a decorator
but this would apply if task 1 and task 2 are running in parallel or near parallel right?
That’s cool
Because if task 2 runs only after task 1 finishes writing, then it wouldn't create any problems?
Well, our bots use asyncio and most of our functions await at intermediate points, meaning that, yeah, multiple tasks could run and alternate
That's correct, but it's up to you to make sure that they
Ok, thanks for your help, I'll see what I can do!
The race condition is possible with non-JSON as well, but, with JSON, there's typically more going on (more simultaneous values that rely on a single key update)
Can I just use JSON to make the data persistent instead of using redis? What advantages does redis have over JSON?
redis is a hosted service that already exists and doesn't require us to assign a specific persistent storage volume to the bot service
because of our infrastructure, if we had a file-based persistance, we'd need to add a persistent volume to the hosted container that runs the bot
that costs us money
oh oh ok
since working with it is practically the same feel as json, it shouldn't be a hassle
Except, you can't store nested dictionaries
which is a really big down for me
since I have nested dictionaries
that's just a case of reorganising your data schema tbh
you can still store id references to cross reference other data if needed
Otherwise data will get lost
But I can use JSON for one-time data right?
yep
Yes, but this will be overwritten every time when bot redeploys
it's better to use a dict tho in that case
since it's going to live/die the same way
but it's a really long file
2207 lines
what
was thinking it would be GBs worth of data
I'm trying to figure out how the verified attribute gets set on the API, as I can't find it in the bot's code, nor can I get it to be set when a user verifies
Hmm, the verified attribute on the discord objects, or on the site database @gritty wind?
Site DB
The users endpoint
it just returns null on all users
The rest of the verification process works normally and adds the role
What verified attribute?
Doesn' that just work through roles?
Do you mean the role?
api.site/users/{user.id}/metricity_data
Huh I wonder if metricity sets it
I mean, I have metricity running too
I was going to check django admin, but I cannot login with discord
Yes that is a metricity-only thing
Yeah, we don't have that on the site db
So... metricity sets it?
Well it is on the site DB. I don't think there's a separate postgres instance.
Yes metricity sets it
Oh, if that's what you're referring to, I have them set to the same DB
I'm just trying to get it to update though
It is on the postgres instance, but not on the site db, metricity is using a separate database
I'm reviewing a PR that's using it
Which endpoint would that be?
https://github.com/python-discord/site/issues/398
This specifies that this endpoint should return it
Probably the one you sent earlier
But.. wait how do I connect the site API to the metricity DB?
Sorry, I'm still figuring out my setup for the projects
It automatically connects
Huh, well it returns different values from what is stored in the DB
I'm not seeing any errors anywhere either
How are you checking what's in the DB
I was selecting the columns using SQL directly
And there is some connection I would think, because the message counts is reported
Are you using the default db connection string?
For the site itself, I'm just running docker-compose up, no changes, and for the metricity I'm not changing anything (the only change I have it getting it to connect through host network)
Metricity also has the correct channel IDs
What change exactly?
Are you sure the bot is connecting to the same db as the site?
Well, same postgres instance, but metricity uses metricity db, the site uses the site
Let me grab them
Yeah that's what i mean
Should it not be setup like that?
It should
I was asking if the site connects to the same metricity db that the bot connects to
Ah, in that case yeah
I can test to confirm
Ok update
It is definitly looking in the right place but
pythondiscord.com - A Django and Bulma web application. - python-discord/site
It selects the verified_at, which is never updated for me, even when the actual verification status is set
Did you set the right IDs in the config?
The metricity IDs?
The role ID in metricity's config
Yeah, that one is set
I can try to set it again and verify again
But I figure if it did detect the verrification...
What do you mean detect verification?
It set the verified status to true for my test account
You mean when you manually query the db?
Yes
But it doesn't show that through the api?
Is it working for you?
There is is_verified and a verified_at
the api uses the verified_at
Well...
I changed nothing
I've never tried running it so I don't know
Just reverified and
it works
I must've updated it at some point
Thanks for helping me :D
What is dump.rdb?
A Redis dump of your active DataBase cache
Here is the redis page on data persistence https://redis.io/topics/persistence
Do you have specific questions about its connection with the projects?
it shouldn't be commited right?
No, it shouldn't
Okay, that's all I wanted to know. Thanks!
@cold moon I have changed my code to use RedisCache to make the data persist, can you check if the code is good to go or if it needs some more improvements?
@nocturne hare Can you also review my pull request?
anyone help me....how to run this...?\
You'll want to follow the guide at https://pythondiscord.com/pages/contributing/bot/
Mind that running the bot locally is really only useful for development on it. If you need specific features out of it for your own use it'll be easier to just take out those features and make them work without the bot's environment
Why Core Dev approvals use checks? This isn't really nice when so much commits show ❌ even when linting success, but core dev approvals not. This should be label instead I think.
We could just have a dummy account on the whole repo to prevent prs not approved by core devs, without failing all the actions
That was one of the proposal, using the github bot for that, but we don't actually require changes
Well the check was implemented to allow core-devs to remove themselves from getting messages for the whole repo, but if we add just one account thats required for all prs, we can stick with the old system, but not have to fail commits
we're actively changing it right now
the way it's going to work is that the PR will display pending
similar to this
Does the commit status show as failed or pending when there are failed and pending tasks?
I image it would resolve lazily?
well, it's as you say there
failed shows failed, pending shows pending
oh you mean if there is still a pending task when there is a failure
good Q, I think it displays failed, but at the bottom of the PR it will always display
new system is running on forms backend now
I approved now, but that check was pending
So it is working?
forms frontend and backend have it now
if you visit https://policy-bot.pythondiscord.com/details/python-discord/forms-backend/13 you can see the status of PRs
looks like this if pending
this if approved
Hmm, the apps has some scopes I'm not sure it really needs
(these links appear on the PR checks)
Well, it would need it only for the account it is configured on, not for everyone who visit the website
A check is the correct thing to use.
a failing check doesn't mean "linting error"
that's just what you're personally used to
a failing check means "this isn't ready to merge"
a PR without a core dev approval is not ready to merge.
That is some pretty aggressive defaults then haha
lol yeah
As a counter, if I look at a PR and see failing tests, I will assume the author is working on updating the PR and wasn't using a draft
If all PRs will be marked with a fail, what use is it
policy-bot will continue to use checks, but stay in the "pending" stage (as shown in #dev-contrib message)
so you won't get the X, but you will not get the green for go
I think pending is infinitely worse and I hate this change.
If everything is setup correctly, yellow would be the new green
I get what you're trying to say, but having everything red would make everything useless for the most part
I don't really see how pending is much worse, I think that a description saying the rules are not met and then linking to a site where you can see a visual representation of why makes sense
and this allows us to have more complex rules, like "if it touches a CI file do not pass until a devops member reviews"
pending, to me, means "checks are running and haven't completed yet, but this will soon change". Having everything in pending really triggers me a million times harder than having failing checks - which is semantically correct.
a core dev has not approved. we're checking if the core dev has approved. It's a failed check.
If you are really set against pending, we can compromise and only have it run on PRs that don't have a core-dev by default
I have more seen pending than failing for such cases
That way what is pending is the assignment of a core dev to the pr
yeah, I think pending is the github norm, it's what I've used at work and on other repos where checks like this are enforced
I've never seen pending used like this
but okay
if anyone in the world has used it like this, I guess it's marginally less gross to me
What happens if the core dev requests changes?
I suppose, another alternative alternative would be to give only core devs write perms
does it then fail?
I don't think giving write permissions only to Core Devs is good idea
hmmm, I believe that by default that is a success since it has received a core developer review.
@cold moon can you open a dummy PR to forms-backend
just change a file or something
so, if it failed the check on changes requested, passed it on approval, pending if nothing, then I can sort of see how that makes semantic sense
I will change gitignore
cheers ks
actually before you do
let me just update the org review policy
to set this disapproval thing
pending what. the core dev review has happened. It was not an approval. There is nothing pending. You have to make changes.
okay @cold moon policy updated
it sends an email? good, because you should be taking action.
okay so we're in a pending stage on ks's PR now
I've totally missed that this was a thing
🎉
nice
that works for me. does the check really have to be called policy-bot main though
let's give it a very descriptive name
we can rename the first bit, I think the latter is just saying it's the review requirements for main
yeah, something like that
one second
Core Dev Police
so Core Dev Approval: main is how it will look
yeah
but yeah I can live with this. thanks for changes.
you're very accommodating
I'll send the money to the usual account
hahahha
and then that Details link will link to the page I was screenshotting above
if we ever make more complex rules about CI/devops or whatnot then that'll appear in the flow
actually @green oriole it won't generate an extra email for requested review
it'll send the usual one when someone requests changes
but since it isn't github actions you don't get a vague "run failed" email
yeah. because that won't trigger it
yep
the actual policy for this: https://github.com/python-discord/.github/blob/main/review-policies/core-developers.yml
contrib approval on this would be funky fresh https://github.com/python-discord/site/pull/431
Alright cool
Whenever I run a command, it is running twice for me, is it happening with anyone else?
The mod bot?
Huh, could you have an extra instance running without knowing?
Hmm... maybe that's the case
Try disabling one, and see if the commands still get processed
If you want to force stop all instances, reset your token
Can look at it in a few, not home yet
Sure, take your time!
@crude gyro Can you iterate keys in the Async RedisCache?
Or do you have to call .items, and index/unpack the key from index 0
You'll have to iterate items
we used to have a keys but I think it was abandoned in a later iteration
we only model the actual redis stuff now.
when it was called RedisDict, I did have a .keys for it.
@trim cradle is bot/1319 ready for review or does it still need discussion?
We are discussing it atm 
@gritty wind we're discussing it in a mod channel. Thanks for asking!
Good stuff
You can still review it though
Hi devs, I wanted to start contributing, mostly reviews to the community, can someone help to identify where and what to pick up. Apologies if this is the wrong channel.
To start reviewing, I'd start by following the guides on the website (https://pythondiscord.com/pages/contributing/) to set up all the projects (+ metricity). From there, pick one of the source repos (https://github.com/python-discord), and pick a PR. There isn't a reviewing guide yet, but keep your eyes open.
And, this is the right channel for such discussions.
Thanks I will start with the setup
Once you do, you can start looking through the PRs for all three till you find something you feel capable of contributing to (any comments, suggestions, or modifications, even a 👍 can be helpful).
the reviewing guide is coming very soon™️
i’m think hem is finished with it actually
if that’s the case, it just needs to be released
Hey @eternal owl, on sir-lancebot#540 you have a 2.2k line changes because of line endings, can you revert that please?
wait, I thought line ends are ignored :/
Only if you didnt change your default settings
ahh
You should be able to fix that by changing the autocrlf setting, and redownloading your project
sir lancebot, site, bot and forms frontend/backend are now all using policy bot
What's policy bot?
It forces all PRs to need a core-dev approval to be merged. It is used so we get pending status instead of a failed check
Ahh, okay. Was this discussed before I arrived at the staff meeting?
it was not discussed in a helper staff meeting
scroll up in this channel for discussion
nah I did mention it in the news section
here
nevermind then
it has taken action on https://github.com/python-discord/site/pull/432 now
Anyone know where I might find an expert on snekbox? I have an issue... Seems like if I send the server two requests at approximately the same time one or both gets an out-of-memory error (return code 137).
Sebastiaan was working on OOM issues recently, they might be able to work on this. I'll point him towards here if I see him
But out of curiosity, can you include some specific info like os, and snekbot version?
cc @tawdry vapor
That error may be misleading since it just assumed that SIGKILL = killed due to OOM but really it could be coming from anywhere.
NsJail doesn't log OOM termination because it's actually handled by the kernel via cgroups. NsJail only directly handles killing for an exceeded time limit and therefore you'd see a log for that.
It may be an issue with how cgroups are set up where all nsjail processes share the same cgroup. I thought it would apply separately to each process but maybe it's more like a pool.
Can you open an issue with reproduction steps so I can remember look into it later?
@tawdry vapor definitely. Thanks for being willing to look into it.
@shadow elbow @gritty wind https://github.com/python-discord/snekbox/issues/83
Issue I came across this issue when trying to run code that uses the sympy library invoked via a webserver in Snekbox. It seems that if there is a memory-intensive request, Snekbox will begin retur...
Seems like it is a memory issue based on my testing... Hm.
May be you've tagged the wrong mark.
If I run pipenv run pre-commit in powershell, it says everything is fine. But PyCharm says that flake8 fails when I click "commit".
fails in what way?
Flake8...................................................................Failed
- hook id: flake8
- exit code: 1
Courtesy Notice: Pipenv found itself running within a virtual environment, so it will automatically use that environment, instead of cre... (show balloon)
``` @brazen charm
I suppose I could try committing it using powershell
does git commit -m recognize \n?
I believe you can chain multiple -m for the subject and body
Do you have a global flake8 that pycharm could be picking up?
so git commit -m "main message" -m "secondary message"?
I don't know, I haven't used anything like flake8 outside this project.
I was able to make commits earlier today
you can just run git commit and it'll open your default editor set for git
once you type in the commit message and close the editor, it'll commit
i just use nano for git
For some reason that didn't work either
PS C:\Users\Steele\development\projects\pydis\bot> git commit
Check for merge conflicts................................................Passed
Check Toml...........................................(no files to check)Skipped
Check Yaml...........................................(no files to check)Skipped
Fix End of Files.........................................................Passed
Mixed line ending........................................................Passed
Trim Trailing Whitespace.................................................Passed
check blanket noqa.......................................................Passed
Flake8...................................................................Failed
- hook id: flake8
- exit code: 1
Courtesy Notice: Pipenv found itself running within a virtual environment, so it will automatically use that environment, instead of creating its own for any project. You can set PIPENV_IGNORE_VIRTUALENVS=1 to force pipenv to ignore that environment and create its own instead. You can set PIPENV_VERBOSITY=-1 to suppress this warning.
Loading .env environment variables...
'flake8' is not recognized as an internal or external command,
operable program or batch file.
I'll look into it some more tomorrow
Run it under the pipenv environment
^ it looks like pycharm is not using the virtual environment set in the directory for some reason (?)
Someone told me that was a bug before
(Windows specifically)
You'd either update the hook to make sure it actually uses the environment, or you change the pre-commit settings to run on system which would then use pipenv
The second solution won't work though, because you'll have to commit the modified pre-commit
So you're left with modifying the file
@trim cradle
'flake8' is not recognized as an internal or external command,
operable program or batch file.
Are you sure you have it installed?
Its probably installed in the pipenv, but the precommits don't use the env correctly on windows
(and possibly other oses)
Well, you can always try installing it...
because pre-commit is working fine for me on Ubuntu
I uninstalled my global flake8 and ran pipenv run lint and it worked well (using the virtual environment)
The thing about the global flake8
is that its caused problems for me and a couple other people, by reporting false negatives (returning no errors when the workflow fails on github)
Might've even been on one of steler's commits, i don't remember
Uhh @patent pivot https://github.com/python-discord/bot/pull/1319
It seems to just be using any disapproval, is that intended?
I think this is meant that all users with write access that require changes count to there
But only Core Devs approvals count
The check should be renamed then?
There can be cases of disapprovals, with approving core devs
Not sure how this work in such cases.
It'll still fail I'd assume
It's using a disapproval from org members. The check name is slightly misleading I guess, I'm open to any alternatives. Maybe just "Review policy"
@patent pivot Does it take core dev membership into account in the check?
What happens if a regular org member approves, but a core dev has not yet approved yet?
They are two independent checks on the website (so it'll still be pending), but any disapproval would mean failure apparently
Yes, core developer approval passes the check, disapproval from anyone in the org fails the check
that's fine
That looks like the right setup to me
Shiny: #dev-log message
Nice 😄
Just curious, do changes in the .github folder not show as a commit? Because I just updated my branch and git pulled, and it does show changes, but it doesn't show as any commits in my forked repo:
(I forked it before the name change, if it helps)
@gritty wind I hadn't yet pushed what I have locally when you gave your feedback on github, so I converted the PR to a draft to make that more clear.
It wasn’t a super detailed review anyways, I’ll check back on it when it’s ready 😄
Is it possible that flake8 is objecting to my commit message, and that's why I can't commit?
What kind of output are you getting, @trim cradle ?
...
Flake8...................................................................Failed - hook id: flake8 - exit code: 1 Courtesy Notice: Pipenv found itself running within a virtual environment, so it will automatically use that environment, instead of creating its own for any project. You can set PIPENV_IGNORE_VIRTUALENVS=1 to force pipenv to ignore that environment and create its own instead. You can set PIPENV_VERBOSITY=-1 to suppress this warning. Loading .env environment variables... 'flake8' is not recognized as an internal or external command, operable program or batch file.```
PyCharm is using the correct virtual environment, but my guess is that it's not using the pipenv that's in that environment.
if that even makes sense
Yes, I'm using a "Virtualenv environment" and not a "Pipenv environment"
pre-commit uses pipenv run to run flake8 to make sure it selects the appropriate environment
I assume that's what is not working for you
What I do is create an environment with pipenv sync --dev from the command line and then use the "existing environment" option in PyCharm, selecting the pipenv venv, as I don't like how PyCharm manages pipenv environments.
However, what happens if you try to run flake8 manually with the venv activated?
it runs fine
what does it give you for flake8 --version?
Theres a pycharm bug (possibly just on windows)
that command doesn't work, with or without the venv activated. However it might be because pycharm is doing something with that environment currently to recognize it as a pipenv.
Someone walked me through fixing it a while ago
and no, a system install doesnt work as intended
What does pipenv run lint gives you? Or you don't use pipenv at all?
everything passes
It could still be that it bypasses your environment, uses a flake8 version installed systemwide, without our plugins
I think it might be working now. One moment and we'll see
how do i make a dockerfile for snekbox
The code review guide is done!
We're a large, friendly community focused around the Python programming language. Our community is open to those who wish to learn the language, as well as those looking to help others.
you dont, it already has one
how can i use it?
but its also on docker hub
"sw" + "e" * 231 + "t"
HA
yeah
this is my entire snekbox code for my bot
I just hope it reads well. And that people benefit from it
i just threw it into a compose file
now im getting this
https://paste.pythondiscord.com/qusoboxesu.rb
There was someone asking about how to get into reviewing PyDis code a couple of days ago, could have them dry run it
what is that?
container list
Can you upload the output of running dir as well?
(btw, you don't need to build the image yourself if you are just using it)
Yeah
no its on docker hub
yeah
You only need to build it if you're modifying it yourself
Hey guys I am new
Hello
i use compose
I wanna learn python!
but its the same thing basically
It was erroring out because the copy instruction failed
please help me out
There should be a config folder there too
already ran that
what did it do
what do i put there?
anyone?
@clever wraith please check out #❓|how-to-get-help for general python help
thank you!!!!
You should have the config directory from the main repo
thank you
But this is only relevant if you still want to build yourself
why do people scroll down to the very bottom, and go to #dev-contrib instead of #❓|how-to-get-help
Should be, yeah
snekbox:
image: pythondiscord/snekbox:latest
init: true
ipc: none
networks:
- ClemBotNet
ports:
- "8060:8060"
privileged: true```this is literally the only time snekbox is mentioned in my bot
compose pulls the image, runs it on the docker network and assigns those ports
You can get the address and port through docker desktop
ideally you run BOTH yourbot and snekbox as containers
so how do i make requests to it? as in what do i include in the request
then you can network them with docker
and do i have to make requests with a specific module or
The api docs are here
oo
also here
the comment in on_post is all you need
have you ever worked with an api before?
like http stuff
or is this your first time
(Also don't forget its a POST endpoint)
i can make requests with aiohttp, right?
yes
good
this is how i did it
go wild
as you can see its dirt simple
async def _post_eval(self, code) -> t.Union[str, None]:
data = {
"input" : code
}
json_data = json.dumps(data)
async with aiohttp.ClientSession() as s:
async with s.post(BotSecrets.get_instance().repl_url,
data=json_data,
headers=HEADERS) as resp:
if resp.status == 200:
return json.loads(await resp.text())
so it returns text with the eval result?
it returns that json that you saw in the comment i just linked
its in the comment lol
what comment
the one i linked
np
@shut aspen it says localhost:8060 is an invalid url
should i replace it with my ip address
that is an ip
well no cuz its not running on your public ip
"ReplUrl": "http://localhost:8060/eval",
this is in my config
damn i even forgot the /eval oops one sec
one sec i forgot the headers
im getting http code 400
What does your code look like right now?
async def _post_eval(self, code) -> t.Union[str, None]:
data = {
"input" : code
}
json_data = json.dumps(data)
async with aiohttp.ClientSession() as s:
async with s.post(BotSecrets.get_instance().repl_url,
data=json_data,
headers=HEADERS) as resp:
if resp.status == 200:
return json.loads(await resp.text())
hm? @shut aspen
how does yours differ from mine
you used json.dumps?
yee
Ignoring exception in command eval:
Traceback (most recent call last):
File "C:\Users\vndev\AppData\Roaming\Python\Python39\site-packages\discord\ext\commands\core.py", line 85, in wrapped
ret = await coro(*args, **kwargs)
File "C:\onyx\onyx\main.py", line 54, in eval
async with s.post("http://localhost:8060/eval", data = {"input": code}, headers = HEADERS) as resp:
File "C:\Users\vndev\AppData\Roaming\Python\Python39\site-packages\aiohttp\client.py", line 1012, in __aenter__
self._resp = await self._coro
File "C:\Users\vndev\AppData\Roaming\Python\Python39\site-packages\aiohttp\client.py", line 376, in _request
headers = self._prepare_headers(headers)
File "C:\Users\vndev\AppData\Roaming\Python\Python39\site-packages\aiohttp\client.py", line 836, in _prepare_headers
headers = CIMultiDict(headers)
TypeError: CIMultiDict takes either dict or list of (key, value) pairs
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\Users\vndev\AppData\Roaming\Python\Python39\site-packages\discord\ext\commands\bot.py", line 903, in invoke
await ctx.command.invoke(ctx)
File "C:\Users\vndev\AppData\Roaming\Python\Python39\site-packages\discord\ext\commands\core.py", line 859, in invoke
await injected(*ctx.args, **ctx.kwargs)
File "C:\Users\vndev\AppData\Roaming\Python\Python39\site-packages\discord\ext\commands\core.py", line 94, in wrapped
raise CommandInvokeError(exc) from exc
discord.ext.commands.errors.CommandInvokeError: Command raised an exception: TypeError: CIMultiDict takes either dict or list of (key, value) pairs
i get this traceback
i did HEADERS = json.dumps(HEADERS)
the headers can be a normal dict
i think you can do json={dict}
but i did data=json.dumps
probably the same thing
try both /shrug
code = None
lmao
You can actually just delete lines 8 & 9
They don't do anything useful
(and as more of a sidenote, don't name your variables after builtin python ones, such as input)
yeah i know, but i thought it wouldnt affect it as im not using the input function and it's only in the scope of the command
wait but wouldn't that cause scoping issues since im defining that variable in a with/if statement?
It is, but why introduce unneeded complexity ig
^ That was to your first comment on input
Your else currently has a return
oh
Which means your variables are only used if you go into the if
Yeah
ok
!e
if True:
a = "hello world"
else:
exit()
print(a)```
@gritty wind :white_check_mark: Your eval job has completed with return code 0.
hello world
how do i install a package from github instead of pypi
A github python package? It would depend on the package, but there should be instructions on the package's repo
when is it ever not pip install git+...?
When you have a terrible programmer such as myself publishing it
Is there an issue open on making the setup for contributors easier?
If so could someone link it to me, i would like to take a look,
happen to ping me
Hey @late wolf, we are currently working on a project, smartconfig, in order to simplify the whole setup process. Once it is done, you can be sure that I'll work 'em! In the meantime, if you have any idea or suggestion for the guides, feel free to comment on meta#60
.issue 60 meta
ohh
Hey, I was wondering what people think about adding a .tldr command to Sir Lancebot.
It would retrieve the tldr-page for the given shell command: https://github.com/tldr-pages/tldr
I can vouch for the quality of these pages. I use them regularly.
I didn't know about this website, I'd 100% support this, it looks very cool
isn't that similar to man pages?
the readme sums up how it isn't manpages
Yep, but they're shorter and emphasise common examples.
alright
This might make a good first-contribution for someone?
Since I think it should be relatively straightforward.
I think, yes
I think using a line paginator will also be helpful
during the implementation
just in case
yep
Hmm, I am not sure, I usually label as first good contrib anything that would help you familiarize with the code base, that isn't really the case here
That could work, I dunno
Yeah, if it is a well written issue that should be fine
sorry thinking out loud again
self.guild = self.bot.get_guild(Guild.id) -- The expression on the right returns None when this is in the init method for a cog, I guess because it runs before the bot is fully online. I checked the docs to see if there's an async method for cogs that runs once the bot is fully online and didn't see one. Is there something I'm missing?
🔥 👌🏻
just await that method before getting the guild, and you'll be fine
I don't believe you can await in an init method?
no, i'd await in on_ready
or
you can create an async method and then create a task for it in __init__
let me find an example
def __init__(self, bot: Bot) -> None:
self.bot = bot
@Cog.event
async def on_ready(self) -> None:
await self.bot.wait_until_guild_available()
self.guild = self.bot.get_guild(Guild.id)
self.developer_role = self.guild.get_role(Roles.verified)
so I can't just do this?
class SomeCog(Cog):
def __init__(self, bot: Bot) -> None:
self.bot = bot
self.guild = None
self.bot.loop.create_task(self.get_guild())
async def get_guild(self) -> None:
await self.bot.wait_until_guild_available()
self.guild = self.bot.get_guild(Guild.id)
no, you have to wait for the guild to be available
either way works
how about now?
i'm unsure if Cog.event exists, i've only seen Cog.listener()
but other than that, yeah
it's also good practice to initialize those variables in __init__ as None
PyCharm was insisting that I do that, but I figured this wasn't the edge case they were discouraging.
imo it's not really necessary when you have a clear initializer coro like that, just adds some noise to the init
Make sure that the coro finished before going to the parts of the code that use the vars and you should be good
Hey so I had an idea.
Instead of us having to replace all the id's manually, why not make a setup command, which creates the config-yaml file, gets their id of the channels using their names, and adds that to the config-yaml file.
https://cht.sh/ a similar resource to tldr pages. but more better i feel.
ping me if u have any thoughts on #dev-contrib message
@late wolf we’re already working on a project called smartconfig
oh ok
I think https://github.com/python-discord/site/issues/406 can be closed now. Linked PR is merged.
I think one part of site is missed in dewikification: Contributing section. This should be in different app. Should I create issue?
I thought we settled on using the content application for that
Content app doesn't support bulma widgets like contributing pages have
Does anyone know why this happens?
I'm not sure.
https://cht.sh/ what do u guys think about this compared to tldr pages. it is easier to get data from this in a nice way. i have made code also for it
Hey @eternal owl, could you give me an update on the error handler PR for @dusky shore? Maybe we can work out a way to get it to a mergeable state soon so that we can merge the Advent of Code changes soon as well.
As a counter argument, tldr can be self hosted, and has a python library, while cht.sh would require extra http requests and parsing to remove the non-tldr section.
The page mentions a github page, which isn't linked, so I haven't searched too thoroughly
The tldr makes it clearer what's available and how to use it. cheat.sh doesn't make it obvious what exactly I'm supposed to do. Their github makes it slightly clearer, but I don't think I'd remember the syntax for it
There is a TLDR header on some pages which is pretty much what the tldr content is
But not all pages have it, and some have really bad formatting
Hey @hardy gorge mind sharing what might be wrong with the webhook?
Is it just authentication or something
Which webhook?
It is
It's what I feared and kinda hoped
Some secrets are not available in a pull_request run from a fork
That seems like a mostly good thing
yes
but there was no easy way for me to test it
Anyway, I have a solution ready, which also ensures that we're actually running our workflow file instead of a potentially modified one in a fork
workflow_run?
That could work, but that will make getting information about the run into the webhook embed more difficult later (which is something that's planned for later)
At the same time, it could be the safest
Hmm
You know what, I'll figure out the passing of output later. There are other ways of doing that.
Can't get everything you need from the API, or GraphQL?
Let's just fix this
@hardy gorge I think it will be better to use error.original in the local error handler (snakes cog) as the FileNotFoundError (aka OSError) does occur if the file is not found (example: https://github.com/python-discord/sir-lancebot/blob/master/bot/exts/evergreen/snakes/_snakes_cog.py#L178)
just error will be CommandInvokeError
I am not sure how the .snake video and .snake get commands would raise an OSError
but the .snake card command can raise one(check the example link above)
I'm not that intimately familiar with the code and not looking at it right now; do any of those commands use Pillow's Image?
Because Image.open raises an OSError when an empty stream of bytes is passed to it
I suspect that's why they added the handler
not empty, but even a file which is not found
We create a BytesIO from downloaded bytes, right?
If that BytesIO stream does not yield bytes, Image.open does raise an OSError
That's a weird behavior
yes
then that stream is passed into Image.open
All in all, do we even need the special case? I haven't looked into it too much, it just change the error message a little bit, right?
yes, just gotta change it use error.original instead of just error
in the isinstance
It's because OSError is raised for when I/O fails. This is usually when the file stream you're trying to read does not give you anything (or is inaccessible), but we're using a virtual in-memory stream.
I mean, OSError isn't very adapted here, nothing goes through the OS
It's because you'd normally pass it an actual file stream.
It's not that Pillow tries to work out whether or not you're using an actual file or faking it using an in-memory file object-like stream
It just tries to read from your (potentially pseudo) I/O stream
And if that fails, it raises the appropriate error
I just tested with an incorrect path and it gives me FileNotFoundError
I'm not sure if the FileIO abc group intends you to use OSError here, since the actual methods can take the actual data from anywhere
well, that's not really the point anyway
It kinda is the appropriate error, as OSError is the exception defined by the docs to be the one to be raised for I/O-related errors.
when a system function
I just tested with an empty buffer, raises OSError
And here you are just reading from memory using a non-system method
Also, you don't have any errno to fill in
should I go ahead and do this?
btw, the actual image error is UnidentifiedImageError
which comes under OSError
also, I think I can remove the error handler for get_command and video_command, I don't see how these 2 commands can raise an OS error.
Let's keep this for another PR that isn't time critical 
Yeah, I think you can do that, what do you think @hardy gorge ?
Who's bot is running on the test server, out of curiosity?
Senjan's and Hem's bots are running
Just out of curiosity, how expensive is it to have the same kinds of event listeners in multiple cogs? I assume there's one component of the bot that's actually doing the listening, and when that happens it calls the event listener in each cog.
Yeah pretty much, it calls each handler until it runs out. The overhead isn't huge (you can see the dispatcher here: https://github.com/Rapptz/discord.py/blob/master/discord/client.py#L361), but the price would depend on how long the handler itself takes.
Hey I am looking to learn the pytest library. Does anybody need some testing for a python project so that I can practice? Add me/message me if you do
Hey, this channel is specifically for discussion about the python-discord projects (github.com/python-discord/), if you'd like to talk about python testing in general, check out #unit-testing if you want general discussions or #❓|how-to-get-help for specific questions
How much of the bot's computation time is spent checking messages against the regular expressions?
It is undoubtedly you, if anyone, that would be the first to profile that.
I've always wondered about that
@cold island About snowflake command, do you mean creating custom converter? discord.py don't have in-built snowflake converter.
We have one in converters.py
Oh, I didn't know that
what's the snowflake command?
does someone know the answer to this?
I doubt that’s tracked
do you guys use some fancy tree thing when checking against the blacklisted words in the database
hmm, alright
Isn't the hastebin highlighting a little off? It happens when I copy paste and save the code very quickly: https://paste.pythondiscord.com/jevorukace.md
oh wait nevermind
It's just really, really, really bad at guessing the right language
change the extension manually and it'll be better
we have a bunch of regular expressions for terrible things people might say and as far as I know most messages just get checked against every single one.
I believe messages from staff members aren't checked.
I will go check and report back
There are currently 60 patterns.
Some patterns might be intended to capture multiple words depending on which similarly-spelled things you think of as the same word
@patent pivot didn't the saving on the hastebin default to .py?
it did — though that disappeared with the migration to pydis infra since people complained about it
Thank you Stelercus 
@patent pivot do you think adding an optional option to change language, defaulting to python, could be added?
well, we don't maintain hastebin, so we'd have to maintain a fork if we did that
Huh, wouldn't think that'd be a problem with how it's being used, saw more people confused why they were getting coffeescript
we'd either fork it or raise an issue upstream
unlimited forks
lol
Hmmm, yeah
the way we deploy it already is rather hacky
did people have an issue with automatic py?
I'd say people now have to change it to py more often than people having to change it from py to something else
yeah, I'd agree
I mean we can add the defaulting back to py, but it's just something else to maintain
It’s probably better to default to py and not maintain a fork for some features that most people probably don’t use. If you’d be interested in opening an issue in the hastebin repo about allowing users to pass the language in some sense, that would be cool though
hastebin upstream is kinda dead
I don't have the time to write up an issue, but if someone wants to open something brief on https://github.com/seejohnrun/haste-server then we'd definitely set that option if implemented
Anyone know the best way to get snekbox running on GCP? I've tried...
- Cloud Run (fails because it can't find nsjail after booting up -- maybe because no privileged mode).
- Compute Engine (adds a ton of authentication overhead / non-severless setup seems expensive. Not sure if it works, deleted the container once I read their auth docs)
- App Engine (currently trying it now but I suspect it'll fail like Cloud Run since I don't see a privileged option here either)
Compute engine should be able to run privileged containers. I'm not sure about the others.
GKE also supports privileged containers
Maybe you can find a way to configure the k8s service account cloud run uses so you can enabled privileged containers
!warn 533618406767722499 please do not post memes randomly into the server
:incoming_envelope: :ok_hand: applied warning to @ivory drift.
I wonder if I'm overcomplicating things by trying to use snekbox for my needs. I need something that runs serverless in the cloud that can run arbitrary code... And I like the memory limits snekbox provides... But maybe I could just run exec in cloud functions with a 50mb memory limit per request and call it a day? I mean, if it's severless I'm not sure there's a ton of risk there. Tried to Google it and the jury seems to be out -- someone mentioned protecting my source code would be a factor, but my source code in that case would just be exec
Not sure. I thought that setting a limit on the container will cause it to kill the container.
Sorry, you said cloud functions. I don't know about those.
Keep in mind that NsJail is built on linux kernel features like cgroups and namespaces. You can think of it as a nice wrapper for those features. There may be a feasible way to use those features with a container directly.
After all, containers are implemented using those same features
wdym?
@patent pivot
It's a PR from before the migration that had the migration recently merged into it
Shows up fine on the site though
Take your time
looking into things
well
I see the log
"error":"failed to compute approval status: failed to list commits: 3 commits were not found while loading pushed dates
I forgot you were looking into this when I approved 👀
uh oh, did something break?
Yup
D:
site ded
Well, cloudflare is probably dead
oh
Yup
o noes site is ded
Look at your pings haha
oh no
lol confirmed from at least 3 coasts
yeah looks healthy here
Its not loading for me, possibly just my shitty internet
site's dead for me
smae
Probably cloudflare's shitty internet
lol
heck cloudflare
weird, my sites haven't had any issues
hmmm
it's not a cloudflare issue
Even if its a cloudflare issue, its unlikely all off cloudflare fails at once
Now we can't blame CF :P
Getting this when trying to start up @dusky shore. Anyone familiar?
Check your site config
Either that, or redis
Actually, from that stack trace, definitely redis
Where would I find it? I don't see anything about it in https://pythondiscord.com/pages/contributing/sir-lancebot/
It isn't on that page (yet)
but set the fake-redis env var
USE_FAKEREDIS=True
If you want to, you could setup redis, but it isn't easy on a windows machine
Fairly easy through docker
excellent, that did it. I'll update the page
Actually, if you hold off for a minute
I might be able to dig through the config for all the env variables
.
I can add them myself, but I first have to figure out which ones are worth adding
A bunch of these are not so useful and would pad out the page quite a bit like the AoC ones
I think I will add the fake_redis and Tokens list for now
https://pythondiscord.com/pages/contributing/configure-environment-variables/ It's a pretty short page at the moment. You can separate them by which ones are mandatory and which ones are optional
I think having a record of them would be useful
Alright, I'll see what I can do
Although if you're actually using redis then you don't need fake-redis right? so it's mandatory, but it's either or
I've added the tokens, look over them if you have a sec
I'd also appreciate someone looking into the docs for wolfram's api
Awesome, thank you
This might even work as its own subpage where we can list all variables
Let me finish this list first
Ok, I've gone ahead and done that
Feel free to copy it to another page if you want to
I might do the same for the bot and the site tomorrow
do we have a constant for the #❓|how-to-get-help channel?
I don't think we're using it anywhere in the bot atm
looking to implement #1320
do you mean you have it, but it's not in use? will i have to create one?
If we're not using it then it's probably not in the config. I can take a look
No, the id is not stored anywhere atm as it's just a static message
I doubt we'd make that a constant
fetching the last message sent by the bot in the channel sounds wiser
making the channel a constant is what i mean
wdym, you need to know which channel
and i already am doing it that way
oh right, yeah
I assigned you to the issue@clever wraith
thank you
how should i get the latest message? i was thinking of pulling it from channel.history but i don't know if there's a better way you guys use
or is it "I assigned the issue to you"? 🤔
He’s handling the issue, so “he’s assigned to it”
?
and should i just create the message if the length of #❓|how-to-get-help is under 2?
looking at the docs, TextChannel seems to have a last_message_id property https://discordpy.readthedocs.io/en/v1.0.0/api.html#discord.TextChannel.last_message_id
hm