dev-contrib

[clever wraith]

Looks like @timid sentinel already wrote lots of good stuff on the topic https://github.com/python-discord/site/pull/773/commits/c0b16b2ce1cbd90f0342611190b97fb44909070a but I'll see if I can help push that issue forward. (Though first to figure out how to set up the site for local dev 😅 )

[thorny obsidian]

Docker! I recommend docker >_>

[outer oasis]

Embrace Docker!

[timid sentinel]

It's still sort of a brain dump of information so I'm very open to any suggestions for adding/removing/rewriting parts!

[clever wraith]

Alright, cool. I definitely think having lots of screenshots (which I can provide) of the Windows Path changing process would be good. I'll look more into it later tonight

[timid sentinel]

Yeah, some screenshots would be good. fwiw feel free to commit directly to the PR's branch if you'd like

[rapid swallow]

(in case you haven't seen this already: https://www.pythondiscord.com/pages/guides/pydis-guides/contributing/site/)

[timid sentinel]

would appreciate reviews on bot#2307

[dusky shore]

GitHub

[bot] #2307 Talentpool Cleanup

[gritty wind]

Site tags are merged :O

Thanks @last patio

[last patio]

What do you mean???

What‘s a site tag?

I didn‘t do anything

[gritty wind]

Lol I see what happened

Auto merged was enabled by you a week ago, so it appeared you merged it

#dev-log

Thanks for updating it @vale ibex :D

[last patio]

Hmm auto@merge

I found a new way to be productive at work

[gritty wind]

You can even automate updating branches for further gains

[clever wraith]

You mean natively ? Out of the box by github ?

[gritty wind]

Not natively

Though you really shouldn't anyway, merging main can sometimes break stuff, and you usually only want to do it before you merge a PR

[clever wraith]

Makes sense

[rapid swallow]

Automate the automation
???
Profit

[vale ibex]

@fallen patrol the docker steps in the build and deploy flows can be bumped to latest too of you wanted to do that in this PR

There's a few steps under the docker org than can be bumped

[fallen patrol]

i saw a comment that said there was a reason they weren't bumped

[vale ibex]

Oh, in snekbox?

[fallen patrol]

oh it was the other way around :kek:https://github.com/python-discord/snekbox/blob/375fcd9b2fedc96e0ab4d35e4806b8147f6fdf89/.github/workflows/build.yaml#L36-L38

[stable mountain]

.github/workflows/build.yaml lines 36 to 38

# The current version (v2) of Docker's build-push action uses buildx,
# which comes with BuildKit. It has cache features which can speed up
# the builds. See https://github.com/docker/build-push-action```

[vale ibex]

Ahh :p

[fallen patrol]

currently using v1

:why: my cluster won't update snekbox to 3.11

wait

why does it deploy to k8s before pushing to github packages

@vale ibex done i upgraded every action package

the versions in the deploy workflow and it's configuration is based on the versions in https://github.com/onerandomusername/monty-python/blob/main/.github/workflows/deploy.yaml

[vale ibex]

Looks good

Were you planning on doing more changes?

Or was there another reason for making it draft

[fallen patrol]

oh I forgot to undo that

did it while I checked the k8s actions

fixed

[clever wraith]

Hey, would meme images like this be appropriate on guides on the site? As a header hook to https://deploy-preview-773--pydis-static.netlify.app/pages/guides/python-guides/windows/microsoft-store/. I find it funny but it does add clutter and may be unprofessional (and idk about the rights to the drake image, though everyone uses it everywhere).

[fallen patrol]

@vale ibex i missed a package in the snekbox pr :^), an actions/download-artifact@v2 pin which should be v3
I'd also like to add a workflow-dispatch to the main.yaml workflow so it can manually rebuild snekbox eg a new point python release or other changes

[vale ibex]

Could the dispatch be added to one of the sub flows?

Or does the whole flow need to run?

[fallen patrol]

well

[vale ibex]

Iirc there's some get version script in the repo too, not sure how that would behave with a dispatch

On mobile rn so can't investigate

[fallen patrol]

build.yaml needs to run to build a new container and then deploy.yaml would need to run

[vale ibex]

Ah, might as well do the whole thing then

[fallen patrol]

so it would need to be either be a new workflow that does only those two or just the entire flow

[vale ibex]

No harm in making sure lint and test still passes on a patch version

Not that it should change

[vale ibex]

Would just need to double check this

[fallen patrol]

i mean

i can tet right now

I have a modified branch that i use for my own deployments which has said workflow dispatch trigger

[vale ibex]

Sounds good

[fallen patrol]

modified branch since I have code on there that is not in a condition to deploy upstream :^)

tldr: api controlled runtime package management

[vale ibex]

Sounds interesting

[fallen patrol]

aight, manual deployment to the package-management branch https://github.com/onerandomusername/snekbox/actions/runs/3401771988

[fallen patrol]

its currently a totally shit implementation but it works

[vale ibex]

Github mobile is so bad

Seems to have got a decent output from the get version step

Probably fine

Annoyingly still considers you a first time contributor

Need to approve flows

Also, were you going to bump a download artifact step too?

[fallen patrol]

i wonder if the setting is wrong

[vale ibex]

The message still said first time contributor, but it could just be a bad mobile message

[fallen patrol]

no lets just make that another pull instead of part of this one

[vale ibex]

😅

Padding those pr numbers

[fallen patrol]

meanwhile i've broken my vps

[vale ibex]

Lol

Apt install python3-pip is usually the fix for that one

[fallen patrol]

what

[vale ibex]

Lol

Love debian

[fallen patrol]

its ubuntu

[vale ibex]

Ahh

[fallen patrol]

on hetzner

[vale ibex]

Good choice

[fallen patrol]

I broke my package management branch upgrading it to 3.11

I can no longer manage dependencies since it shelled out to pip (in the venv tho)

[timid sentinel]

Lmao, that's great.

I'm not 100% sure, I don't mind it although if others didn't want it I would go with that.

[vale ibex]

Is there an easy way to right align it?

Just so it doesn't shove text down a page or so

Might look cool

[clever wraith]

Is there a way to make the bot sync the server's data with the db from scratch ?
I deleted my site & db containers, so all the data is gone, which results in errors like user cannot be found, and role cannot be found etc...

Ah, never mind me, I had a stupid moment there.

[clever wraith]

Yeah, you can use an img tag with align="right" (and still link it so full size can be seen)md [<img style="margin:1rem;border-radius:.5rem;" align="right" width="480px" src="/static/images/content/python-on-windows/ms_store_drake.png">](/static/images/content/python-on-windows/ms_store_drake.png) Though the text can become really skinny depending on the screen width.

[vale ibex]

looks good to me

[heavy yew]

Is there anything a beginner can work on?

[vale ibex]

bot-core#158 This is quite odd, poetry run docs is giving this error, and I don't even know where to start debugging it.

reading sources... [ 83%] output/pydis_core.utils.lock
Extension error (sphinx.ext.linkcode):
Handler <function doctree_read at 0x0000023BF28A8AF0> for event 'doctree-read' threw an exception (exception: [Errno 2] No such file or directory: 'C:\\Users\\chris\\src\\bot-core\\__init__.py')

[dusky shore]

GitHub

[bot-core] #158 Add lock utils

[vale ibex]

There are some issues marked as Good first issue in the various repos, if you weren't looking for something specific

!contributing The projects here are a good place to start

[stable mountain]

Contribute to Python Discord's Open Source Projects
Looking to contribute to Open Source Projects for the first time? Want to add a feature or fix a bug on the bots on this server? We have on-going projects that people can contribute to, even if you've never contributed to open source before!

Projects to Contribute to
• Sir Lancebot - our fun, beginner-friendly bot
• Python - our utility & moderation bot
• Site - resources, guides, and more

Where to start

1. Read our contribution guide
2. Chat with us in #dev-contrib if you're ready to jump in or have any questions
3. Open an issue or ask to be assigned to an issue to work on

[vale ibex]

Yea, you can see it in CI too https://github.com/python-discord/bot-core/actions/runs/3404034281/jobs/5660974166

not sure why it's attempting to import the repo folder

[little egret]

Hello! I'd like to suggest a tag for @stable mountain , but I'm not amazing with open source so I was wondering if someone could add it. The command is !asktoask, which just provides an embed explaining that you must ask and shouldn't wait for someone to ask. I understand that there's an embed such as that as the first message in every available help channel, but a lot of people tend to not read that, which is why I'm suggesting this.

Thanks in advance!

[clever wraith]

Hello,
That seems like something simple to implement but could also be useful.
I see that a lot as well, where people skip the embed because they don't know it exists, or they might not know what an embed is.

Maybe pointing them to the embed itself could work, but that's debatable

Please open an issue in our repo, and we'll discuss it there

[brazen charm]

a tag like that has existed in the past but was removed https://github.com/python-discord/meta/issues/136

[static canyon]

Yeah, I don't think we'll ever have that tag again given the outcome of the previous one

It ended up being more "toxic" then helpful

[outer oasis]

Hello!
We actually used to have an !ask tag, which was exactly what you were looking for, but we ended up removing it.
You can see what the content used to be here: https://github.com/python-discord/bot/blob/b27c286f2ce648d021eed0c8e07476066b86dd98/bot/resources/tags/ask.md
Overall, we felt that it really wasn't very friendly to just throw a big embed at somebody that just says "ask a question" to somebody that might just be new and confused, or even just uncomfortable asking for help.

You can see some of our comments here:
<#community-meta message>
<#dev-contrib message>
<#dev-contrib message>
<#dev-contrib message>

And some issues suggesting a new one here:
https://github.com/python-discord/meta/issues/114
https://github.com/python-discord/meta/issues/134
https://github.com/python-discord/meta/issues/136

[clever wraith]

Seems like a tough topic to address, even though it's very simple at core, or at least it should have been

[outer oasis]

For future reference, if you'd like to suggest a tag, you can go to https://github.com/python-discord/meta, click "Issues", click "new issue", and there will be a "tag" template that you can choose from.
It has a little from explaining what we're looking for, which for a tag is literally just a name and a body.
For convenience, here's a direct link to the template: https://github.com/python-discord/meta/issues/new?assignees=&labels=area%3A+tags&template=tag-request.yaml

[outer oasis]

Yeah, personally, I'm not really sure what I think would "best" action for this is

[gritty wind]

The best course of action so far has just been letting people do it themselves

It’s a lot more personal than a bot command, which reduces the hostility

And the more likely someone is to be hostile, the less likely they are to respond in the first place

[vale ibex]

It's worth nothing that I think this will be a lot better in the rewrite to use forum channels (bot#2318) since the bot sends the info embed after they open the post, and it will be the first message

[dusky shore]

GitHub

[bot] #2318 Update help channel system to use forum channels

[vale ibex]

this way, the user won't need to "look up" to read the suggestions

[clever wraith]

Aw ! Great !

Do all guilds have access to forums Chris ?

[vale ibex]

IE

[vale ibex]

no, it's at 90% rollout atm, and our test guild is in the 10%

[cold island]

Welcome to the "it will be solved in my rewrite!" land 😁

[vale ibex]

so until it's at 100% roll out we can't really test this

[vale ibex]

lol

[clever wraith]

Shoot, I see

IS there a date for the 100% rollout ?

[outer oasis]

The Big Rewrite: https://www.youtube.com/watch?v=xCGu5Z_vaps

[vale ibex]

Discord has been quite vague over the past few weeks about why it's at 90% rollout, and also hasn't said when it will do the last 10%

For those of you with access to Discord Moderators Discord <#1020283763399327775 message>

[outer oasis]

Have we considered asking Discord to move our servers up into one of the earlier rings?

[outer oasis]

o?
How do you get this?

[clever wraith]

Ah, I see

[gritty wind]

It’s actually 90% for community servers, which is not great

Just a pointless obstacle in general

[vale ibex]

we have, and they have said no

there are still some "big" servers without them too

[clever wraith]

I don't have access to this

[vale ibex]

Yea, it's int he Discord moderator discord, which I have lost the link to get access

it's currently invite only by Discord staff

there used to be a form you could fill in to get your name on the list

can't find the form link though

[clever wraith]

welp

[outer oasis]

This, right?
https://discord.com/moderation

Here
https://discord.com/blog/announcing-the-discord-moderator-academy-exam

[vale ibex]

ah yea, that'll be it

dis.gd/exam

[outer oasis]

I did that a few years ago and nothing ever came of it
Wanna sponsor me? /hs

[vale ibex]

hah don't think I have that much swing in there

[cold island]

They just have a massive backlog. I don't see many people joining

[timid sentinel]

Does anyone know why the lint & test workflow is failing here? https://github.com/python-discord/site/pull/792

[atomic ivy]

seems like it's because there's a migration that'll be applied by the PR and the failing step fails if there are any pending migrations

how do we usually handle adding migrations?

[gritty wind]

That not quite right

Django dedicated that it should generate a new migration

Looks like the one added by this PR is bugged

The migration added is 85, CI generated 86

Not really sure why, but you can try reverting, deleting the file, and generating again with the current change set in the PR

If all the migration stuff is correct, it’ll be handled by the CI completely and no one has to do anything

[atomic ivy]

oh, possibly there was another migration merged after this migration was generated?

[gritty wind]

That could cause issues, however it seems this branch is up to date

It's strictly ahead

I think it's just that the field was updated after the migration was modified

Alter field thread_id on nomination

Oh yeah that was it, check out the descriptions

+ help_text="The nomination vote thread's id"
- help_text="The nomination vote thread's id."

[atomic ivy]

a single .

smh

[gritty wind]

While we're at it, it should probably be .. vote's thread ID.

[clever wraith]

I thought I pushed that commit, that's weird

It's in my local branch, I'll take care of it

[clever wraith]

Is there a way to get the the site repo running locally (via docker-compose up) to live reload when writing a guide?

[cold island]

iirc it does

At least when you focus the terminal

[clever wraith]

Hmm, it isn't for me in VsCode

[gritty wind]

Should be IDE agonistic (also use docker compose without the hyphen)

I wonder if django doesn’t look at that folder for reloads

[timid sentinel]

you may need to ctrl-shift-r to hard refresh the browser page

[clever wraith]

Hey @timid sentinel

I've updated bot#2319 and site#792 and they should be ready for a review.
I've also made the nomination validation flow in the viewset more explicit.

I'd appreciate a review whenever possible.

[dusky shore]

GitHub

[bot] #2319 Link previous nomination threads to a user nomination's history
[site] #792 Add "thread_id" as a column in the nomination table

[clever wraith]

(Other reviews are also welcome of course)

[timid sentinel]

Nice, thanks, I can't test it right now but I'll take a quick look

Btw here's an example of a thread that probably won't show for you #874008972774998116

It's this link, (and once you've clicked it it should be cached so it would show above) https://discord.com/channels/267624335836053506/874008972774998116

[clever wraith]

So I assume there's no workaround for this ?
Since it's only a rendering issue, and not really whether the thread exists or not

I find it a bit odd, but I guess that's a "feature"

And I think coverage wasn't detecting that block because we weren't clearly defining all the other flows that we could have once we're not ending a nomination

Or updating an active one without nominating it

[timid sentinel]

Always nice to get rid of a pragma: no cover anyway so that's good

[timid sentinel]

Yeah it's pretty weird, I don't think there's really a workaround.

[clever wraith]

That's fine, a jump URL works just as well

[cold island]

hmm how do I make the news cog work on my test bot?

[timid sentinel]

I don't think you should need to do anything other than ensure the relevant constants exist

[clever wraith]

Oh, it works if I reload normally but I mean auto reload so I don't need to. By the way I've been editing a lot of the Windows guides from https://github.com/python-discord/site/pull/773 and just made a draft pull request to the add-windows-guide branch of all my changes: https://github.com/python-discord/site/pull/794

[vale ibex]

We don't have page refresh logic to auto-refresh your browser page on change

you need to refresh you page manually

The auto-load we have is your dev changes automatically get put into the docker container, so that refreshing works

[clever wraith]

This is my first PR here so just a draft atm to make sure I'm doing things right 😅 idk if it needs a long review process or what and I probably should have make smaller incremental changes rather than reorganizing/editing all the Windows guides at once? idk

[vale ibex]

Are the commits somewhat atomic?

[clever wraith]

Yeah, I think so

[vale ibex]

If so, then that's fine :P

Smaller PRs are better if you want to get some changes in sooner

but as long as the commits are atomic we can review fairly easily

I can't review myself right now since I'm abut to head into another work call, but I can later! Thanks for the PR :)

[clever wraith]

np. Should I keep it a draft for now or make it ready for review? I'm not really sure how that works.

[vale ibex]

Draft PRs are usually to indicate that you don't want them reviewed yet for whatever reason

As an example I may open a draft PR if I wanted to see how CI would react

If you want it to be reviewed, I'd make it ready

[clever wraith]

Gotcha. I'll make it ready

[clever wraith]

I've been spoiled by next-remote-watch in my nextjs projects

[vale ibex]

That force reloads all connected clients?

Would be interesting to see how it does that

[gritty wind]

I imagine it injects some code into the returned page to inject a listener connected to the server which calls reload when changes happen

Sounds like fun, but it definitely probably exists

[safe ridge]

Just await client.load_extensions('file path')
In on_ready function

[safe ridge]

U asked and I answered

[cold island]

I was asking whether there's specific setup required to run a specific cog in @stable mountain's code 😅

[safe ridge]

Oh, yes it is then.

async def setup(bot):
   await bot.add_cog(ClassName(bot))

[vale ibex]

It should just work keep in mind that it only posts "recent threads" according to the mail.python archives

[cold island]

Gives me a forbidden error when loading the Discord webhook, I checked and the ID in the config is correct

[vale ibex]

Do you have the full traceback?

[cold island]

Not at the moment

[clever wraith]

Do we have a way to generate some sort of a dependency graph (between classes/modules) of our bot ?

[vale ibex]

poetry show --tree

if that's what you mean that is :P

[clever wraith]

I actually meant the dependencies between the classes/functions in our code.

For example, if I have a class A that has needs an instance of Class B, and so on, is there a way to generate some sort of a Diagram of that for our bot ?

I'm just interested in seeing it

[vale ibex]

Ahhh right

https://github.com/thebjorn/pydeps might be the solution

Never used it myself, but seems to match the description

[clever wraith]

I'll have a go at it & let you know, thanks Chris !

[vale ibex]

👍

If it does work, could you post it here? I'd be interested in seeing it too :)

[clever wraith]

Sure

[stable mountain]

Hey @clever wraith!

It looks like you tried to attach file type(s) that we do not allow (.svg). We currently allow the following file types: .gif, .jpg, .jpeg, .mov, .mp4, .mpg, .png, .mp3, .wav, .ogg, .webm, .webp, .flac, .m4a, .csv, .json.

Feel free to ask in #community-meta if you think this is a mistake.

[clever wraith]

Well

Can't send svgs, but here's a glimpse

[vale ibex]

lol

very clean

[clever wraith]

Ahahahahaha, indeed

[static canyon]

Can you send it to me in DMs? Interested to see the full thing

[clever wraith]

I've sent you a message

[static canyon]

Thanks

[clever wraith]

Anytime 😄

[gritty wind]

Some of our codebase is egregiously organized

Though I imagine that graph includes library calls as well?

[static canyon]

Doesn't seem to actually. Think it's only our classes

Only one with "discord" in it is bot.rules.discord_emojis

[tropic pier]

!resources

[stable mountain]

Resources

The Resources page on our website contains a list of hand-selected learning resources that we regularly recommend to both beginners and experts.

[fossil veldt]

the pycharm graphing features are pretty nice as well

and you can interact with source code alongside some other graph analysis stuff

our bot class node neighbors like this apparently

[clever wraith]

And is this available in PyCharm community ? Or Pro ?

[fossil veldt]

I think it should be in community?

try right clicking the module and diagrams

oh it says professional apparently

[clever wraith]

Yep

Makes sense to have such feature in Pro

[fossil veldt]

but yeah I guess that would be one of those non-critical but nice features to have that they throw into pro

[clever wraith]

Yeah, and looks a lot better than the one I got xD

[last patio]

Welcome @clever wraith !

[clever wraith]

Thank you Volcyy ! 😄

[gritty wind]

@brazen charm and/or@vale ibex, any idea why this import exists? https://github.com/python-discord/bot-core/blob/962968fecedca3bef33ba9524d87ffedf815f16d/pydis_core/utils/cooldown.py#L29

[stable mountain]

pydis_core/utils/cooldown.py line 29

from pydis_core import BotBase```

[gritty wind]

It's gaurded by a type_checking condition, but it doesn't seem to actually be used for any typing purposes

And it doesn't seem to affect doc generation

[brazen charm]

the cast on L206

[smoky geyser]

hey @outer oasis, about python-discord/bot#2313, has there been further discussion about the tag? i was initially considering making a PR to change the code snippet so it enabled message_content instead of members, given that the example uses commands.Bot() which generally needs that intent

now alongside some paragraph refactoring in my revision, i added a short note that v2.0.0 requires intents:

[dusky shore]

GitHub

[bot] #2313 Update d.py intents tag to clarify that privileged intents require activation in d.py 2.0

[gritty wind]

Hmm, looks like it doesn't detect that as a usage

[outer oasis]

There has not
@gritty wind - could you approve?

Meta question: can we standardize on using either the s: or the status: prefix?

[smoky geyser]

i also noticed the "discord.py docs on intents" link was using the latest version of the docs rather than stable, which might include documentation changes that aren't relevant to the current release - is that worth changing back to stable?

[outer oasis]

Does the "stable" link maintain the latest stable version, or would we have to continuously update the tag to use the new link as new versions come out?

[smoky geyser]

both of them are updated dynamically, but latest is always built from the development branch while stable sticks to the most current pypi release https://readthedocs.org/projects/discordpy/versions/

[outer oasis]

https://docs.readthedocs.io/en/stable/versions.html
, so it just gets the latest tag from GitHub

Yeah, I've got no problem with that

[smoky geyser]

ah thats a better source

it appears there are two other tags (customcooldown.md and local-file.md) that also refer to the latest version, so perhaps that should be changed in another PR

[outer oasis]

Yeah, if you're changing multiple tags that should be it's own thing

[clever wraith]

@smoky geyser I was going to propose some suggestions on how I think it should be done but I see you've already made progress in it.
Do you want to take this over ? And we can discuss the content further in your PR ?

[smoky geyser]

sure, ill make one in a moment

[smoky geyser]

@clever wraith python-discord/bot#2320

[dusky shore]

GitHub

[bot] #2320 Update intents tag to better reflect discord.py's API changes

[clever wraith]

Awesome, I'll finish up something work related and have a look!

[clever wraith]

I see wookie has beaten me to it :p

[timid sentinel]

only the one updating the versions to stable, 2320 is still open

[gritty wind]

Man I wanna know why the github upload action takes over a minute to upload 30 MBs

Is the data delivered by snail mail to their storage server

[clever wraith]

Isn't there a way to have some sort of verbose logs ?
What I'm wondering is, is it only slow at the network level, or for other things that happen in between ?

[fossil veldt]

btw do we know why snekbox initially did not have file system enabled?

was it a security concern or?

[outer oasis]

They recently started allowing "debug logging" -- but it's not that specific -- you can't log the network on Actions

[gritty wind]

We should, on account of it being an in-house product 😛

We can relatively easily get file IO with nsjail (or as easy as nsjail can be), but the main issue is we have no way to isolate evals from each other

There is an issue open on snekbox for this if you want to throw your hat in the ring

[fossil veldt]

can we just allocate an unique folder in the container's tmpfs, and mount it as r/w to the eval instance.

[gritty wind]

There’s nothing in our stack that allows us to separate io access for each instance afaik

I don’t think nsjail allows it

[fossil veldt]

oh we have a single persistent nsjail?

I thought from looking at the file it makes a new instance on each eval?

[gritty wind]

Well it makes a new jail instance but it’s all running on the same container and file system. From what I could tell from nsjail, it can’t actually separate file access for each process

[fossil veldt]

https://github.com/python-discord/snekbox/blob/main/snekbox/nsjail.py#L156-L167

[stable mountain]

snekbox/nsjail.py lines 156 to 167

args = (
    self.nsjail_path,
    "--config",
    self.config_path,
    "--log",
    nsj_log.name,
    *nsjail_args,
    "--",
    self.config.exec_bin.path,
    *self.config.exec_bin.arg,
    *py_args,
    code,```

[gritty wind]

If it can, then the implementation should be relatively easy

[fossil veldt]

could we just add a mount argument on this call

I've been trying to find docs on the mount command since apparently it exists

[gritty wind]

Oh give up on docs, just jump into the source code

[fossil veldt]

like this is one of the GitHub examples

$ ./nsjail -Mo --user 0 --group 99999 -R /bin/ -R /lib -R /lib64/ -R /usr/ -R /sbin/ -T /dev -R /dev/urandom --keep_caps -- /bin/bash -i

though we have our config in the file and stuff

[gritty wind]

It’s really trivial to modify the config once loaded for each instance

[fossil veldt]

so we can just runtime generate a config file and send it?

I guess we can associate a unique temp dir with that?

I was going to have the client encode bytes via some binary escape and send it in stdout to decode later but that seems needlessly complex

the only thing with this is I don't think the file write is considered in nsjail memory limits? So technically is unlimited

[gritty wind]

If you’re going down the mount path, I see it takes a size option

[gritty wind]

Possibly, though I don’t think we have to. Can’t we pass that in as a command line argument

Does this isolate reads?

Actually that’s probably a stupid question, everything should be inaccessible by default

[tawdry vapor]

I'm not sure. It was R/O when I started working on it, so it was left that way as things were reworked.

[fossil veldt]

ah okay

I might try getting r/w going with a named unique temporary directory like I described, does anything seem immediately bad about that?

since it seems applications like matplotlib might need to write to work at all anyways

[tawdry vapor]

I think that sounds fine. Linux supports namespaces for mounts and the directory can be tmpfs.

I think NsJail can do that

I haven't played around with mount namespaces so that's something for you to research. I believe they will provide isolation between NsJail processes.

We just need to make sure other processes can't write to it. If they can still read then it doesn't really matter.

Err maybe mount namespaces arenät the right thing. In any case, research needs to be done to find a way to limit who can write (maybe just using user/group permissions?)

https://unix.stackexchange.com/questions/153665/per-process-private-file-system-mount-points

[clever wraith]

Hello folks,

Have we ever thought of a feature that allows us to track a discussion?

What I mean is, I am interested in a question or questions that people might ask, and I want to keep track of the discussions, but I can't do it now.

So I'd like to find those discussions easily later on and read through them

( we might even have something similar that I'm not aware of)

[gritty wind]

Bookmark command?

[clever wraith]

I think i fell in this case 😅
Thanks Scaleios!

[gritty wind]

Lol we have a large set of commands, you’re good

[brisk brook]

Hm I thought you were thinking of a sort-of highlight feature, like has been suggested a couple of times

[clever wraith]

You mean some sort of a star board?

[brisk brook]

Well, a personal one. You subscribe to a phrase and the bot will DM toy upon its mention. Their reasoning is being able to help in topics which are very small

[clever wraith]

Hummm, interesting

What I also realize now is that we can't search through our bookmarks

I mean, it depends on how many bookmarks one has

But if the size grows, wouldn't that be something useful to have?

[gritty wind]

You should be able to search through with discord search

[clever wraith]

Hello folks,

I'm curious, why do we have some emojis as part of the config, and some just hardcoded like the thumbs up one for example ?

[vocal wolf]

The ones in the config have IDs and are specific to this server, so we have to identify them properly if they're changed by modifying said config.

[clever wraith]

These don't seem to be server related, are they ?

[vocal wolf]

I don't think they are, no. I am unaware of the reasoning behind why they're specified in unicode.

[clever wraith]

Those are the only "errands" I see

[vocal wolf]

Although I can't type them out by :check_mark: in discord, so they're prob special

[clever wraith]

ah yes

the checkmark is white_check_mark

So if we were to use a new emoji that's not server-specific, do we need to just use its str value ?

[vocal wolf]

if it's global then yah something like the thumbs up would be represented as a string

[clever wraith]

Alright, thanks !

[vocal wolf]

:D

[sharp crag]

Anyone willing to work on one of these 2 PRs?
python-discord/site#708
python-discord/site#709
python-discord/site#701
They are PRs regarding migrating pins from #discord-bots to our site. It's mostly working with markdown files, so expertise is not needed. Should be relatively easy as well. Let me know which one you'd like to work on and I'll assign it to you. Thanks!

[dusky shore]

GitHub

[site] #708 add fix ssl
[site] #709 Migrate Setting Different Statuses to Set Your Bot Pin by Python Bot

[clever wraith]

@timid sentinel Thanks for you review of my Windows Guide changes (https://github.com/python-discord/site/pull/794). I've accepted or left comments on almost all of your suggested changes. Asking here now since I'm not sure of the next steps on GitHub 😅 - should I re-request a review or wait for more people to review or what are the next steps to keep this moving forward?

[vocal wolf]

trying to yoink my job, are you? sniff I'm so proud

[vocal wolf]

@clever wraith @sharp crag re-review thanks https://github.com/python-discord/site/pull/733

[sharp crag]

Will do, thanks! Might take a sec due to being a relatively large diff

[vocal wolf]

thank

[vocal wolf]

thanks I'll take both

[sharp crag]

Awesome! Thank you so much. Feel free to request a review from me once you're done

[vocal wolf]

you seem to have sent 3 PRs, but only 2 were shown by sir lance

#701 is this one

https://github.com/python-discord/site/pull/701

I'll take it as well

[sharp crag]

yes I edited my original message after I realized there were 3 that needed doing

[vocal wolf]

welp they're all mine now

:3

[sharp crag]

I really do appreciate it Xith. However, you are putting me out of a role >:(

Glad to see things getting wrapped up, though

I'll edit the original issue to reflect this

[vocal wolf]

yuh

[vocal wolf]

@sharp crag just for confirmation:

https://github.com/python-discord/site/pull/733#discussion_r1020837558 - Just the regular pip install -U discord.py?
https://github.com/python-discord/site/pull/733#discussion_r1020837832 - I'm gonna bold it.

also https://github.com/python-discord/site/pull/796 is now ready for review

[vocal wolf]

as I couldn't push to site#701

[dusky shore]

GitHub

[site] #701 Migrate "Keeping Discord Bot Tokens Safe" pin

[sharp crag]

Yes & Yes. Though for the former I know we usually don't recommend using pip over py -m pip or similar, so if you want to include that you can

lmao what is this

excellent youtube clickbait video thumbnail

[atomic ivy]

one of those points to x509lint

[vocal wolf]

yeah because I didn't want to go through all the different options for installing a package depending on your OS

[vocal wolf]

not my screenshot, but I shall redo

by instead putting a red box around the download link text

[atomic ivy]

I was kidding

[vocal wolf]

flameshot my beloved

ah

[sharp crag]

omg same

[atomic ivy]

though yeah, the image could be in a higher resolution

[vocal wolf]

https://github.com/Xithrius/dotfiles/blob/main/.config/i3/config#L77-L82

[stable mountain]

.config/i3/config lines 77 to 82

# +-------------------------+
# |  Flameshot screenshots  |
# +-------------------------+
bindsym $mod+p exec --no-startup-id flameshot gui
bindsym $mod+Shift+p exec --no-startup-id flameshot screen -n 1 -c
bindsym $mod+Ctrl+Shift+p exec --no-startup-id flameshot screen -n 1 -c -d 5000```

[sharp crag]

epic i3 flameshot buddies ⁉️

[vocal wolf]

arch btw?????

[sharp crag]

indeed

[vocal wolf]

are you me?

[atomic ivy]

scrot FTW

[sharp crag]

we are one

[vocal wolf]

damn

@sharp crag @atomic ivy thoughts?

[atomic ivy]

LGTM

[sharp crag]

Nice and simple. Love it

sheesh xith you really blowing up my phone today

[vocal wolf]

this is why I don't have email notifs on anymore lmao

because I'd spam myself as well

[sharp crag]

how do you stay on top of these things?

i would like to disable email notifs as well but dont want to miss anything

[vocal wolf]

the notifications thingy in github

you know

top right

[sharp crag]

OOOH

[vocal wolf]

little bell

[atomic ivy]

smol bell

[sharp crag]

thats sooo much nicer
i get to filter things and everything

[vocal wolf]

dude

filter by repo

it's so good

[sharp crag]

please
how do i disable email notifs

[vocal wolf]

profile settings

notifications or something

probably

I was right

xd

[sharp crag]

hahahha suck it github

disabled

[vocal wolf]

https://deploy-preview-796--pydis-static.netlify.app/pages/guides/python-guides/fix-ssl-certificate/

hmmmmmmmmmmmmmm

I can already see where I can make improvements

[atomic ivy]

do we really want periods in a title?

[vocal wolf]

no

it's not there in any of the other titles

I don't like how words are cut off in the screenshot

fixed

@atomic ivy what's your github username?

I'd like to request your review on the SSL certificate PR

[atomic ivy]

sid-maddy

[vocal wolf]

your review has been requested

[sharp crag]

your request has been reviewed

[atomic ivy]

your review has been reviewed

[vocal wolf]

thank

[sharp crag]

your request has been requested

[gritty wind]

Not sure if you have sufficient permissions on github to re-request a review, but if you can feel free. You can also (additionally) ping wookie here as you've done.

In terms of other reviews, you need a review from a core dev, and a review from a contributor/core dev for a PR to merge. That can be from wookie's approval, or from any other person's

[clever wraith]

Thanks for the info on how it works! I am able to request a re-review so I will.

[static canyon]

Friendly reminder that these three PRs are still waiting on reviews. They've been up for a while and could do with some love:
bot#2089 (>4 months ago)
bot-core#137 (~2 months ago)
bot-core#141 (~1 month ago)

[dusky shore]

GitHub

[bot] #2089 Feat: add reminder snoozing & show new expiry / mention count when editing reminder
[bot-core] #137 Add check_rerun_command util to commands.py

GitHub

[bot-core] #141 Migrate all message utils from bot and sir-lancebot.

[clever wraith]

Hey,
I've left a few comments here & there, mostly nit picks.

[vocal wolf]

thank

[gritty wind]

@vale ibex made a commit so good, it resolved 7 independent issues

How does one learn this skill

Lmao I love it

[vale ibex]

ez

I'd rather this over 7 separate commits that all just fix docstrings :P

since I'm going to squash them all anyway lol

[gritty wind]

Yeah that's fine

@vale ibex I believe these need to be removed and usages updated?
https://github.com/python-discord/bot/blob/7a56524a9b5780b31e0cd4aacc33f82e30e8cd4a/bot/constants.py#L390-L392

[stable mountain]

bot/constants.py lines 390 to 392

help_available: int
help_dormant: int
help_in_use: int```

[gritty wind]

Can't leave it as an independent comment because it's way outside the modified range lol

[vale ibex]

Ah yea, i did the channels, not the categories

[vale ibex]

pushed 3 new commits

[gritty wind]

How would the current structure of the server look? Would we keep the parent category with the cooldown, how to get help, and the forum?

Does the cooldown stuff actually still apply

[vale ibex]

My thoughts would be to keep the cooldown channel, and put the help forum in that category

cooldown still applies, yea

Basically, all "current" help channels would be moved to dormant, and we'd create a help forum channel in the available category

[gritty wind]

Ok, but where does the cooldown channel actually fit into this

It doesn't seem to be used in the config or code

[vale ibex]

Just lets people know why they can't make new posts

[gritty wind]

Alright

[vale ibex]

no automation or anything, just managed using the cd role

I should probably remove those constants too

[gritty wind]

Might come in handy for someone:

from discord import Intents
from discord.ext import commands
bot = commands.Bot("!", intents=Intents.all())
@bot.event
async def on_ready(*args):
    for channel in bot.get_guild(GUILD_ID).channels:
        if "help-" in channel.name:
            await channel.delete()
bot.start("TOKEN")

Just transitions from the old system to the new system

[vale ibex]

Probably useful for test servers yea

We're going to keep the channels here though

[gritty wind]

Yeah

[vale ibex]

I think sending a bot embed to all those channels saying something about how they're just there for archive and you can open a help session using #1035199133436354600

[gritty wind]

I'm updating my config right now, and if you needed proof we don't ever really update the server configs, here's a gem from mine:

        unverified:                             776836331501518913
        verified:                               776836331501518914  # @Developers on PyDis

lmfao

!sf 776836331501518914

[stable mountain]

Snowflake

776836331501518914
Created at 2020-11-13 15:50:14.351000+00:00 (<t:1605282614:R>).

[gritty wind]

That is right around when I started working on our projects

[vale ibex]

lol

[gritty wind]

I will also say, we've definitely killed off more stuff than we added, which I think is a good thing overall

[vale ibex]

for sure

how many years until this trend makes us hit 0 LoC

[gritty wind]

By my conservative estimates, -2.333 years

[vale ibex]

sounds good

[gritty wind]

I wonder how the eval features are still working given we merged 3.11 as master

!eval import sys; print(sys.version)

[stable mountain]

@gritty wind :white_check_mark: Your 3.11 eval job has completed with return code 0.

3.11.0rc1 (main, Aug 12 2022, 02:04:06) [GCC 8.3.0]

[vale ibex]

not very well lmfao

[gritty wind]

Short answer is: funky

I'll update the environment variables to make 3.11 point towards the main one, and hope no one clicks on 3.10

[vale ibex]

sgtm

Could shove it behind a feature flag too

or

I could make a 3.10 container

[gritty wind]

Making a container sounds nice, feel free to do so

!eval import sys; print(sys.version)

[stable mountain]

@gritty wind :white_check_mark: Your 3.11 eval job has completed with return code 0.

3.11.0 (main, Oct 25 2022, 05:15:45) [GCC 8.3.0]

[gritty wind]

It's kind of confusing now, we should probably automate this in a way that is more friendly to the the configuration

So have the configuration just set a list of URLs, and pull version on startup or something

[vale ibex]

I'm thinking snekbox tracks main and then we have alts for other versions

[gritty wind]

Sure, that's reasonable

[vale ibex]

currently droppping snekbox-311 and making snekbox-310

[gritty wind]

211 you can't hide

[vale ibex]

won't actually apply the manifest until pytohn changes are made ofc

[gritty wind]

For now I just pointed both environment variables at the same container, and scaled down the 3-10 one

We should definitly make the updates

[vale ibex]

ah alright cool

are you working on the python changes?

I can otherwise

[gritty wind]

No, please go for it

I'll continue review work

[vale ibex]

👍

[gritty wind]

My bot is up for the first time in possibly months 🎉

Does that not fire for threads

on_guild_channel_update

We should delete the foods.json resource file as well now

[vale ibex]

nope, there's some on_thread_* events

[vale ibex]

true

[gritty wind]

Gotcha

[vale ibex]

bot#2324 and kubernetes#144

[dusky shore]

GitHub

[bot] #2324 Make snekbox 3.11 stable default, with 3.10 as an option
[kubernetes] #144 Change the snekbox 3.11 service to a 3.10 service.

[vale ibex]

the manifests should be applied before the bot PR is merged

[gritty wind]

Woah this is cool

Try writing +:thumbs_up: or any other emoji

[vale ibex]

Yea, shortcuts are great :P

ctrl+/ to see more

[gritty wind]

I'm looking at the stat stuff

Answered/unanswered was already kind of a weak metric, but maybe we should base it on the people who've actually sent messages rather than joins and leaves

Is it feasible to fetch that info

Ah yeah, we did it with a cache previously

[vale ibex]

my plan with that is to make something in patsy that can return the answered/unanswered metric

which would start off with a basic, did someone else message in there

and could become some basic nlp at some point

[gritty wind]

My thinking is you can get people who join and not answer, and you can get people who join, answer, and leave

[vale ibex]

yup

and you can get people who send "hi" and stay

[gritty wind]

Yeah it's definitely not infallible

Though it does resolve the other two cases there

[vale ibex]

what do you think about a cache that just has the thread_id as the key if the thread has a message from any non-claimant

with the idea it will be superceeded by patsy at some point

[gritty wind]

Oh I see what you mean with patsy then

Alright this is fine as is nevermind

I'm happy to modify the bot behavior once patsy is ready

[vale ibex]

too late

already implemented

[gritty wind]

looool

[vale ibex]

https://github.com/python-discord/bot/pull/2318/commits/bed42f5747084b25d009476fb5df0961150ec3b4 wasn't too hard

I made the choice to just update the cache with every non-claimant message

rather than getting and conditionally setting

since getting and then setting would mean 1-2 calls

whereas just overwriting every time just means 1

actually

it doesn't does it

since the key is cached

[gritty wind]

Possibly, in either case I'm not too worried because we should be well within redis' throughput capabilities

[vale ibex]

hah yea

[gritty wind]

It doesn't seem to pull from cache, or the cache is just really short-lived

I tried: setting a key, then fetching it, sleeping while I shut down the redis server, and fetching again. This caused an error since it tried fetching the data from the server

So this remains the ultimate solution

This commit actually introduced a quite funny bug

I think you meant to go one indentation level deeper

[vale ibex]

oh

I thought I did an early return

lol

[gritty wind]

Use the util instead

[vale ibex]

Yea

[gritty wind]

And might as well early return on that

Alright, review done 😄

I think this was my highest review LOC/time ratio so far

Mostly because the diff size is artificially blown up

[vale ibex]

lol true

lmk when you check the recent commits

I'll squash all the fixups then

[gritty wind]

Approved

@vale ibex if you squash and do all your fancy clean up now, I'll put out an announcement asking for contribs

Oh I guess we don't actually need an announcement with
shtlrs's review

We could conceivably have the help system overhauled before the week starts, which is an intriguing thought

[vale ibex]

squashed and rebased onto main

[gritty wind]

@clever wraith are you around for a re-review of bot#2318

[dusky shore]

GitHub

[bot] #2318 Update help channel system to use forum channels

[clever wraith]

Aaand done!

[gritty wind]

Thank you :D

[clever wraith]

Did you use the pydis test server to test it ?

I suppose only core staff has access to it, right ?

[vale ibex]

our test server doesn't have forum channels either :P

I had to use Kat's, which does

[clever wraith]

Ah

Oki !

Because I wanted to have a preview: p

[gritty wind]

You can still create your own server if you like, it only takes up a minute to convert your regular server to a community server

Chris are you free to hop on a call to handle rollout

Or do we hold back for now

[vale ibex]

gonna hold back for now

going to discuss it in the admin meeting tonight

[gritty wind]

Alright

[clever wraith]

Oh

Didn't know that

[gritty wind]

@vale ibex while writing the snekbox side of the issue, I had a thought. What if we just package all the python versions inside the same snekbox server, and have it be an argument to the API (the versions can be specified with a build arg). This resolves some of the largest issues I forsee with maintaining a bunch of older branches

[vale ibex]

Yea, that could be useful

[gritty wind]

To clarify some of the specifics:

1. The docker image will simply install whatever versions of python you pass to it in the build arg, to well defined locations
2. We'll build two images in CI, one with all versions we want supported, and one lightweight for dev
3. We can expose an endpoint to get all supported versions
4. I don't have a 4th point but this seemed like a more round number

I'll make the issue about this instead.

[vale ibex]

yea, we'd need to move away from python docker images and install the versions ourselves

but that seems fine

[gritty wind]

I think you can build the images on top of each other, just keeping the binary folder at each step

Hmm, maybe not, not sure

[vale ibex]

might result in very large images

[gritty wind]

There does seem to be a way to do it without large images

https://hub.docker.com/r/fkrull/multi-python

This hasn't been updated since 3.9 though

Only 600 MBs for everything from 2.3-3.9

Time to found out how they did it

Oh it probably built it manually based on buildpack

[vale ibex]

sounds about right

[gritty wind]

I wish docker had a better way to figure out the build steps in an image

Like what the fuck is this ```docker
/bin/sh -c set -xe && echo '#!/bin/sh' > /usr/sbin/policy-rc.d && echo 'exit 101' >> /usr/sbin/policy-rc.d && chmod +x /usr/sbin/policy-rc.d && dpkg-divert --local --rename --add /sbin/initctl && cp -a /usr/sbin/policy-rc.d /sbin/initctl && sed -i 's/^exit./exit 0/' /sbin/initctl && echo 'force-unsafe-io' > /etc/dpkg/dpkg.cfg.d/docker-apt-speedup && echo 'DPkg::Post-Invoke { "rm -f /var/cache/apt/archives/.deb /var/cache/apt/archives/partial/.deb /var/cache/apt/.bin || true"; };' > /etc/apt/apt.conf.d/docker-clean && echo 'APT::Update::Post-Invoke { "rm -f /var/cache/apt/archives/.deb /var/cache/apt/archives/partial/.deb /var/cache/apt/*.bin || true"; };' >> /etc/apt/apt.conf.d/docker-clean && echo 'Dir::Cache::pkgcache ""; Dir::Cache::srcpkgcache "";' >> /etc/apt/apt.conf.d/docker-clean && echo 'Acquire::Languages "none";' > /etc/apt/apt.conf.d/docker-no-languages && echo 'Acquire::GzipIndexes "true"; Acquire::CompressionTypes::Order:: "gz";' > /etc/apt/apt.conf.d/docker-gzip-indexes && echo 'Apt::AutoRemove::SuggestsImportant "false";' > /etc/apt/apt.conf.d/docker-autoremove-suggests

lmfao

[vocal prairie]

.latex seems to be not working. Lance types for a while, and then dies. Not sure if this is the right place to mention that.

[gritty wind]

Sure, here is fine

The API we're using seems to currently be down, unknown why or for how long

I'll open an issue to better handle this scenario, but there isn't much to do right now

[vocal prairie]

got it, thanks

just wanted to let y'all know

[deft patrol]

[clever wraith]

@vale ibex Do you have a particular template you want to use upon creating an issue to keep track of a forgotten nomination ?

[clever wraith]

Also, bot#2323 is now ready to be reviewed.
In case you missed my comments, I think we should:

1. Update the Bot contribution guide to detail the new change in the config (but wanted to get an opinion first)
2. Terminate nominations of members that might have left our server

[dusky shore]

GitHub

[bot] #2323 Create an admin task when votes are 2 weeks old

[gritty wind]

If it requires creating a repo and making a GitHub token, it’s probably best to have it unset like most other external services

It only really ever matters when you’re actively working on the nomination system

[clever wraith]

True, I just thought that we'd make things easier for someone that might work on this

And this feature isn't really critical to the nomination system

So i guess it won't hurt if we don't do it

[outer oasis]

Is allowed_roles a discord.py thing, or is it something we've added ourselves?

[atomic ivy]

https://github.com/python-discord/bot-core/blob/a6760a64bcec5ba77709492ba33b6bfba1f58e84/pydis_core/_bot.py#L43
this thing?

[stable mountain]

pydis_core/_bot.py line 43

allowed_roles: list,```

[outer oasis]

Yea

It's in a few places

[atomic ivy]

yeah, it's a bot-core thing

[outer oasis]

But discord.py says no

[outer oasis]

👍

[clever wraith]

Hello folks,
Are we planning on migrating to slash commands anytime soon?

[dim pelican]

For new commands I suppose it could be done, but I'd imagine we'd keep the current commands as they are now? (I'm totally out of the loop though)

[clever wraith]

Because the new commands will stop being supported soon AFAIK

I'll have a look at one of the simple cogs that we have and fiddle a bit with it

[thorny obsidian]

What commands will stop being supported?

[wind ruin]

discord has been pushing them away, but text commands will still work - they are only applying the limitations to Verified bots iirc (which ours isn't)
the old API used by discord.py 1.7 will stop working, but that is not an issue either - is this what you were thinking about?

we probably should still migrate them to at least hybrid commands but idk if there's any github issue, discord thread or anything specifically for it

[clever wraith]

Yes exactly, I read that support for api v1.7 will be dropped and that's what text commands use

But apparently I misunderstood

[clever wraith]

I think we should, yes.
And I'd like to work on this as well

I'll check if we have this on Github and initiate the discussion there

[vale ibex]

It's dropped support of api versions below v7, which discord.py 1.7 and lower uses.

We're on d.py 2, so it doesn't apply

And yea, we're not against using slash commands where they make sense.

[clever wraith]

I'm curious to know where wouldn't they make sense?

[sharp crag]

The !e command for instance

[clever wraith]

Ah yes, sure!

[clever wraith]

Hey @timid sentinel
Could you take one final look at bot#2319 please ?
There's only one comment you left that I'd like to clarify & then we should be good to go.
Thank you !

[dusky shore]

GitHub

[bot] #2319 Link previous nomination threads to a user nomination's history

[timid sentinel]

What did you want me to clarify?

[clever wraith]

I didn't understand why you think we shouldn't use the link syntax ([]()) in the markdown file upon getting the review of a user
I'm not insisting on keeping it, I just want to know what I'm missing

[vocal prairie]

markdown links don't format in discord unless it's in an embed or from a webhook

and nomination messages aren't sent as embeds

e.g. test

[clever wraith]

exactly, they are sent as file.md, so it will render when someone opens it in an editor
Unless we're viewing the file directly from discord, right ?

[timid sentinel]

The get review command is rarely used, it's instead posted directly to the channel by autoreview or post review command

[vocal prairie]

i may be missing context, but aren't these messages sent into discord?

[clever wraith]

Yes they are, but the thing is, I thought people used it to open the file in a rich format on their editor.
That's because the history command almost does the same thing since it sends the data in a paginated embed

Well, it's not a problem anyway, I was just curious to know

[vocal prairie]

generally, i (and i presume others) just look at the plain-text message @stable mountain sends into the nomination-voting channel. unless i'm thinking of something entirely different?

[timid sentinel]

Yeah, the get review command was intended so people could edit the message before posting it so they could add extra info. With threads that isn't really necessary, but I didn't remove the feature as it could be helpful to get some information (e.g. activity)

[clever wraith]

Okito ! I'll remove the syntax thing

I think CSV will do the job, right ?

I see why I wasn't getting your points wookie & dawn

I was just looking in the wrong place

Sorry for that :3

I don't know why but I was convinced that that code block was executed in the gr command

But it's not ...

[timid sentinel]

Yeah the idea is we have a nomination-voting channel and then the bot automatically posts the message review and it looks something like this, just plain text

[timid sentinel]

no worries, more difficult for you to know when you can't see the feature in use 😅

[timid sentinel]

wdym by this

[clever wraith]

Comma separated values for the thread jump urls
So it'll be something like
List of all of their nomination threads: url1, url2, url3

[timid sentinel]

ah yeah that sounds good

[clever wraith]

Awesome, changes pushed !

& thanks for taking the time to explain ! (both of you)

[plush birch]

I did a thing lol https://github.com/python-discord/sir-lancebot/pull/1148

This is my first time making a code pr, so please let me know what I'm inevitably doing wrong

[plush birch]

I noticed a problem on line 77 of that pr, with an extra newline char. Should I do smth to fix that? Like make a new commit?

[tawdry vapor]

yes, just make a new commit

[fossil veldt]

some progress on snekbox file system

[fossil veldt]

🎉

[hoary haven]

wow

[outer oasis]

Saving the picture automatically returns it?
How do you notice that a file was saved?

[fossil veldt]

for now, it just has to be named output.*

could also implement some other ways

[outer oasis]

How many prefixes and profile pictures does Python.io have?
I think I've seen three so far

[fossil veldt]

originally thought of a .attach text file in root which would contain a list of file names to be attached

[outer oasis]

Too much effort

[outer oasis]

This provides very little friction, and doesn't actually sound too hard to implement

But yeah, really nice job, that's pretty impressive

How many file types are you thinking of supporting?

[fossil veldt]

thanks

I guess most images?

videos would work I suppose, if you can make one in the time limit

audio...?

[outer oasis]

challenge accepted

[fossil veldt]

#476196062214750219 message

audio seems fine at least

[fiery oxide]

I feel like this is starting to turn into a python discord version of colab lol

[cold island]

I'm still wondering if it's possible to use the ipython library to get all of its outputs for a code snippet

[near tusk]

Thats so cool!!!

[atomic ivy]

how do you mean?

[cold island]

jupyter uses an ipython kernel to evaluate cells. I'm wondering if you can use the IPython lib to turn the code sent to snekbox into basically a cell (or just a piece of code evaluated with ipython), and then somehow get all of the outputs you would see in a jupyter notebook

[atomic ivy]

I vaguely remember the IPython REPL supporting those outputs for matplotlib, yeah

seems like a nice idea

[cold island]

yeah, the issue here isn't running it in ipython, it's grabbing the output from inside snekbox and returning it for the bot to display

[dim pelican]

Yes please and thank you

[fossil veldt]

quite possible probably

do you think that should be another command? Or replacing eval?

or maybe we can just return the ipython eval of the last statement, a lot of eval here requires print() for the last line anyways

[cold island]

I think it might be a good replacement

For snekbox itself we could do an interactive mode

[keen yoke]

👋 Hi all, I was looking at the FAQ to see if there was a complete list of bot commands (https://www.pythondiscord.com/pages/frequently-asked-questions/#questions-about-our-bots), but after quickly skimming the GitHub repo, nothing jumps out to me. Is that resource available somewhere? Thanks!

[clever wraith]

I think we don't

You can view them with the help command though

[keen yoke]

Thanks, that does the trick!

[timid sentinel]

@fossil veldt For your snekbox PR am I correct in thinking that the tmpfs size is currently shared between concurrent evals?

[fossil veldt]

I believe so yeah

does nsjail provide an option to restrict the size of a mounted folder?

[timid sentinel]

hey I was about to ask you that :P

I'm not sure haha

[fossil veldt]

well, I don't see that from their docs

they do provide a option to mount a folder as a tmpfs in the nsjail though

so in that case the mounted folder will have a box-side imposed limit

[fossil veldt]

oh maybe it does

honestly this doc is... not great

[fossil veldt]

do you have an idea of how much we can allocate?

[sharp crag]

Alloc till ya can't

[atomic ivy]

alloc ate my baby

[timid sentinel]

Hmm

I'm not really sure

Someone from devops might be able to give an estimate

[atomic ivy]

is the tmpfs_size parameter not what we want?

[timid sentinel]

But I don't think it should matter too much, we only run snekbox with 2 workers, surely each eval shouldn't need that much

[atomic ivy]

numpy arrays can get biiiiig

imagine plotting those

[cold island]

latex OOM flashbacks

[timid sentinel]

Where are you seeing that?

[fossil veldt]

well I'm using the command line for this, so it'll have to be something here https://github.com/google/nsjail#more-info

[atomic ivy]

searched for tmpfs on nsjail README
there's a tmpfs_size parameter in the examples, but seems like it's been removed at some point

[fossil veldt]

trying to make sense of their cpp command line parser in lieu of documentation 😔

[atomic ivy]

removed in https://github.com/google/nsjail/commit/cf9850f9448af33261edbcc8301e707f0debcade

-m none:dest:tmpfs:size=...

[fossil veldt]

well that works but

the tmpfs is considered "memory" on the jailed client

so it also counts towards the memory limit

[atomic ivy]

wouldn't we be mounting some directory as tmpfs?

why would that count as memory?

[fossil veldt]

I suppose since the os thinks its memory?

[atomic ivy]

oh, because tmpfs

[fossil veldt]

I'm trying to see if I can mount a normal folder with -m

but what do I put in the tmpfs space?

so their code passes nothing

but if I pass nothing

😔

[timid sentinel]

Why don't we create a new tmpfs for each eval run?

[fossil veldt]

that would work yeah

not sure if that would incur some overheads

[atomic ivy]

that would still count towards the jail's memory limit, right?

[fossil veldt]

well no, the tmpfs would be on the host docker side

it will be mounted as a regular folder to the instance

[atomic ivy]

ahh, I see, mb

[fossil veldt]

I'm not sure what happens if you exceed size though

[atomic ivy]

try it out ¯_(ツ)_/¯

[timid sentinel]

probably shouldn't be too bad, although worth timing to give a try. We could always pre-create them, or have a pool to reuse if we really needed to.

[fossil veldt]

🥴 going well

[fossil veldt]

hm, is there some external nsjail limit I'm not aware of

I've mounted a tmpfs per instance at 32M, but writing a 16M file seems to OSError

command is

mount -t tmpfs -o size=32M tmpfs snekbox/memfs/...

[timid sentinel]

hmm, what happens if you try multiple smaller files?

[fossil veldt]

btw os shows correctly 32 megbibytes of space remaining

[fossil veldt]

seems to work?

but if writing a single file over 1MB it breaks?

[timid sentinel]

nsjail has a rlimit_fsize parameter, could try setting that

I don't think we'd be setting it so maybe there's a low default in nsjail?

not really sure though

[gritty wind]

Keep in mind snekbox is still an API I push hard for as a tool for people even outside pydis. We shouldn’t be transferring gigabytes of data over http

File limits don’t really matter too much for us beside the network concerns, you can treat storage as unlimited

If you can get proper control over file sizes and isolation, that would be ideal. Compromised could be acceptable

[fossil veldt]

nevermind I think I'm just not writing the bytes correctly

[timid sentinel]

tmpfs is stored in RAM, so it would depend on that

[fossil veldt]

if writing with os.urandom any number of files all are limited at 1MB

[gritty wind]

Ah in that case, it is a super precious resource lol

[fossil veldt]

wait no

so it seems there is a 1MB limit per file

[timid sentinel]

It does seem like the default rlimit for file size is 1mb https://github.com/google/nsjail/blob/90e285450d3f045d378ee2bada914a938d79f162/config.cc#L155

[stable mountain]

config.cc line 155

RLIMIT_FSIZE, njc.rlimit_fsize_type(), njc.rlimit_fsize(), 1024UL * 1024UL);```

[fossil veldt]

but otherwise writing multiple files will reach normally the limit of 32MB and then

OSError: [Errno 28] No space left on device

[fossil veldt]

ah I see

[fossil veldt]

I'm thinking a per instance 32MB limit overall?

[gritty wind]

Just for files?

[fossil veldt]

yeah

[gritty wind]

You can go larger than that if you want, 32 sounds good though

[fossil veldt]

or, can be anything I suppose, we control those separately now

and I'm zlib compressing those in transmission

I think the bot upload limit is 8MB anyways

[timid sentinel]

this is really cool btw, looking forwards to this

[fossil veldt]

seems to work fine now yeah

I'm mounting a new tmpfs for each run, doesn't seem to impact speed noticeably

also that might be better for security than sharing a tmpfs mount

[gritty wind]

How does the isolation look like right now

[fossil veldt]

so we make a new tmpfs on the host docker, with chmod 555 read only

/snekbox/memfs/7b85786e-c5fe-4564-8765-47f83d0ed295/

in here we create a file main.py which is the user's code, and will be called from nsjail to execute.
within here we also make a new folder home which is chmod 777 and is mounted into nsjail as a r/w directory and set as the user's cwd

theoretically they should only be able to write to their mounted home

[gritty wind]

Oooh does that enable the new error message from 3.11

What are these permissions relative to

[fossil veldt]

well, the file format has a few benefits, like previously inspect.get_source wasn't working

[gritty wind]

Are they to the process itself?

[fossil veldt]

uh, the docker process I think, which is root?

[gritty wind]

So uhh doesn’t that mean any eval can get out and modify any other eval

I guess that doesn’t matter when the folders are 777 anyway

Ideally we’d create a user for each eval and run with 700 instead for isolation, not sure how feasible that is

[fossil veldt]

when the python code executes what permissions does it have

it's the nsjail user or?

[gritty wind]

Oh it’s a config option in nsjail

We hardcode it to a fake user and group

We could modify that on every execution

[fossil veldt]

how would I limit it to only allow writes in the mounted /home folder?

[gritty wind]

How are you currently setting permissions

[fossil veldt]

like just

        subprocess.check_call(
            ["mount", "-t", "tmpfs", "-o", f"size={MemFSOptions.MEMFS_SIZE}", "tmpfs", str(tmp)]
        )
        tmp.chmod(0o711)

so Path.chmod

[gritty wind]

You’ll want to move away from chmod to something like setfacl, or whatever the utility is in Debian

And pass it whatever uid you pass to snekbox for that instance

That way the root docker user owns and controls permissions for the system, and it can grant rwx to just the jailed user (configured via the nsjail config)

You don’t really need 711 or 777, 700 or maybe even 600 should work perfectly in this case. Depends on whether or not it’s the jailed user that’s evaling the script

[fossil veldt]

I think if I don't give home a 777, nsjail refuses to mount it

[gritty wind]

Hm, is there an error message?

[fossil veldt]

mount(''/snekbox/memfs/53adbffc-2112-48fc-ae02-c3f583fd2b7b/home' -> 'home' flags:MS_BIND|MS_REC|MS_PRIVATE type:'' options:'' dir:true') src:'/snekbox/memfs/53adbffc-2112-48fc-ae02-c3f583fd2b7b/home' dstpath:'/tmp/nsjail.0.root/home' failed. Try fixing this problem by applying 'chmod o+x' to the '/snekbox/memfs/53adbffc-2112-48fc-ae02-c3f583fd2b7b/home' directory and its ancestors: Permission denied

snekbox.nsjail |    ERROR | Launching child process failed
snekbox.nsjail |  WARNING | Received error message from the child process before it has been executed
snekbox.nsjail |    ERROR | Couldn't launch the child process
snekbox.nsjail |     INFO | nsjail return code: 255

^ for 700

[gritty wind]

Do you get that just from changing the permission code in the code in your branch?

I can have a look tomorrow, it’s getting a bit late here

[fossil veldt]

should be yeah

so the folder of /snekbox/memfs/53adbffc-2112-48fc-ae02-c3f583fd2b7b/

self.path = mount_tmpfs(name)
self.path.chmod(0o700)

seems nsjail requires at least 1 for execution permissions for other users (which the error message also suggests, but I'm not really sure why)

[gritty wind]

I’d at least go with 4 if we really must

I fail to see why you’d have write only

Nsjail has never been the most friendly product

[fossil veldt]

hm? what's 4

1 should be execute only right?

[gritty wind]

1 shoooould be write no?

R (4) x (2) w (1)

Ooooook nope you’re right

R w x 4 2 1

So it somehow makes even less sense

“Oh just execute a file you can neither read nor write”

[fossil veldt]

alright I found why

https://github.com/python-discord/snekbox/blob/main/config/snekbox.cfg#L65-L70

[stable mountain]

config/snekbox.cfg lines 65 to 70

mount {
    src: "/snekbox"
    dst: "/snekbox"
    is_bind: true
    rw: false
}```

[fossil veldt]

we were mounting the entire snekbox folder for some reason

I've just unmounted that, so there should be no access outside of the mounted home folder with a main.py inside

[gritty wind]

Does that allow you to mount with 700 now?

[fossil veldt]

well I'm mounting with 777 since I assume home needs R/W

[gritty wind]

Does anything other than the server itself need that tho

[fossil veldt]

the nsjailed python process would need to write to that right?

[gritty wind]

Yes, but you wouldn’t want to set it to 777 because it allows any instance to write to any folder

[fossil veldt]

I think if I mount with 711, it would mount fine, but python-side writing would have OSError

[gritty wind]

You’d want to set it to 700 on the server itself, and give the correct uid the permissions to that folder

[fossil veldt]

theoretically you wouldn't have access to other folders no?

since only the home part of the uuid folder is mounted

[gritty wind]

What does the layout actually look like right now

[fossil veldt]

so we have a tmpfs at

/snekbox/memfs/<uuid>/

and we mount the folder

/snekbox/memfs/<uuid>/home

[gritty wind]

I see

[fossil veldt]

so since only home is mounted python shouldn't be able to go up any levels

[gritty wind]

In that case 777 is fine

I had assumed you were just mounting the entire memfs folder and using permissions within to scope it

[fossil veldt]

yeah previously we were mounting the entire snekbox folder for some reason

which I guess is not necessary

[gritty wind]

Hmm actually

Isn’t that where the python venv is

Yeah

/snekbox/user_base

Which is probably why it was mounted

Keep the snekbox folder, move memfs up to root

[fossil veldt]

huh, why'd it still work then

[gritty wind]

Did you install any extra deps in your container

[fossil veldt]

ah hm

[gritty wind]

There’s a sample command in the deployment and Readme

[fossil veldt]

also what is the deployment.yaml exactly

and how do I run it >_>

[gritty wind]

That’s the location they default to since it makes it easy for end users

The deployment is a yaml spec for kubernetes, so it’s unlikely you actually want to run it

[fossil veldt]

ah yeah deps don't load if I don't mount snekbox

[gritty wind]

The important part for us is it defines how much resources are allocated to the container (useful for dev, but not feasible to recreate), and the command to run to install deps (under postExec, which you can run by hand if desired)

[fossil veldt]

also the unit tests for snekbox don't run on windows apparently?

[gritty wind]

We can of course move the venv as well, though I’d prefer to move the fs since that’s a feature still in devleopmeng

[fossil veldt]

so I'm just... pushing and looking at the github actions for results lol

[gritty wind]

None of it does, yeah. You’d need to like mock cgroups and stuff on windows, which sounds like hell

It leaves you with two options, run the tests in wsl, or in docker

The male file includes a command to run tests in docker

Male file????? makefile autocorrect be damned

[fossil veldt]

yeah that seems to make everything work fine now

[gritty wind]

Sweet

This feature has been something we’ve needed for a long time now, thank you

Could you try out the error messages

Maybe do something like:

x = [[]]
print(x[0][0])

[fossil veldt]

oh actually I think I'm running 3.10

how do I run snekbox as 3.11?

[gritty wind]

Ah, that’s alright

If you update your branch with main and rebuild the container, you’ll be on 3.11

[fossil veldt]

did the errors not previously work?

!e

a = 5
b = 10
c = 0

result = 2 * (a / b / c)

[stable mountain]

@fossil veldt :x: Your 3.11 eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "<string>", line 5, in <module>
003 | ZeroDivisionError: float division by zero

[fossil veldt]

hm...

[gritty wind]

The specific new errors of 3.11 don’t work in these “interpreted” environments

I don’t know the specific term

[gritty wind]

Compare running the code above in idle and from a file

It’ll point out which indexing caused the issue if in a file

!e In idle (or -c) it’ll look like:

x = [[]]
print(x[0][0])

[stable mountain]

@gritty wind :x: Your 3.11 eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "<string>", line 2, in <module>
003 | IndexError: list index out of range

[fossil veldt]

yeah it shows the output fine now

[gritty wind]

I’m so glad we finally can have that

[fossil veldt]

actually... could we send back raw ansi and do a ```ansi tag to show colors?

[gritty wind]

It was proposed before, the reason we were hesitant about it was because it looks like hell on mobile

It already wasn’t a pleasant experience viewing on mobile, this just makes it a lot less so

[fossil veldt]

btw the bot is live in #bot-commands in the test server if you want to test anything

[gritty wind]

The only real thing I’m still concerned about is filtering for the file extensions, and deciding what kind of extensions we want to allow

Ideally we’d handle that on the bot side tho, the snekbox side looks perfect

[fossil veldt]

ah yeah, currently I'm just sending the file bytes and file names

it just gets uploaded as a discord attachment, so support is up to the discord side

[gritty wind]

The file type I imagine can be unreliable at times, but really all that matters to discord is the file type

(From the file name)

[timid sentinel]

There's also the potential for invalid files, which discord just shows as a normal download with no embed and we can't easily tell apart from a normal file.

Not a deal-breaker though, and it might be possible to detect that after sending it if we want extra handling for it

[gritty wind]

I’m thinking we only need to allow text files and image files, do either of those not display when corrupted?

[timid sentinel]

I think that image files don't display when corrupted

[gritty wind]

Ah yes, you are right

[fossil veldt]

audio maybe?

or I guess video

[gritty wind]

I’m not too concerned though, it’s nothing more or less than what you can already upload by hand

[timid sentinel]

Yeah

[gritty wind]

Actually we could just literally use the same filters

Genius

[fossil veldt]

like the same stdout text filter for text files?

[gritty wind]

For the content itself, wed want to run it through the same filters as the body, but for file extension we’d use the same file extension filters

The main portion of the body is already filtered

[fossil veldt]

ah hm

also I guess if you really wanted to you can render some prohibited message in an image

not sure anyone would go that far though 🥴

[gritty wind]

Funny enough I had the same thought

But it just goes back to you don’t need snekbox to do that already

[fossil veldt]

should we support multiple files at all?

[gritty wind]

Hadn’t considered that, do you forsee it being any more difficult than one file?

[fossil veldt]

it's pretty much ready to do multiple files currently, I'm sending a list of attachments

I'm currently uploading all files that begin with output

[tawdry vapor]

I haven't had time to follow this discussion closely or review your pr

But my vote is to support multiple files

[fossil veldt]

response looks like this for senkbox to bot

{
    "stdout": "Hello\n",
    "returncode": 0,
    "attachments": [
        {
            "name": "output.png",
            "mime": "image/png",
            "content": "SGVsbG8...=",
        },
        {
            "name": "output2.txt",
            "mime": "text/plain",
            "content": "SGVsbG8...=",
        },
    ]
}

[tawdry vapor]

And if we want to limit it we can do that client side on the bot

Does that sound reasonable?

[fossil veldt]

yes but I also currently have a snekbox side limit as well

since the file parsing, compression, and encoding is outside of the instance time limit, I suppose that could be a ddos route without a limit

[tawdry vapor]

That's fine. Supporting infinitely many files probably isn't a good idea

A request parameter could be added for the number of files to return as well, but that can be work for the future.

[fossil veldt]

could add parsing for multiple files as well, might be nice to explain imports and whatnot

?eval

# main.py
from lib import fn

print(fn(5))

# lib.py
def fn(x):
    return x * 2

maybe parse a format like that

[fossil veldt]

also currently it should just support them

ordering is just whatever glob("output*") finds first, not sure what else would be better

[outer oasis]

I would filenames=sorted(filenames) just to help keep the output consistent

[fossil veldt]

@tawdry vapor hm do you know what this is caused by in the unit tests?

  File "/snekbox/tests/test_nsjail.py", line 45, in test_memory_returns_137
    self.assertEqual(result.stdout, "")
AssertionError: "/usr/local/bin/python: can't find '__main__' module in '/home'\n" != ''
- /usr/local/bin/python: can't find '__main__' module in '/home'

it functions fine in normal docker but the makefile test commands results in these errors for some of them

[tawdry vapor]

So is it just a trailing newline difference?

I don't know why

But it seems fine to just rstrip and call it a day

Sorry, I realised I misread the error. It's comparing to an empty string, not to the last line

[fossil veldt]

well no

despite what's being run it's returning that in stdout

like the self.nsjail.python3("print('test')") would return

/usr/local/bin/python: can't find '__main__' module in '/home'\n"

in stdout somehow

if I pass in -c to run in -c mode it works fine

but apparently my routine for writing to home/main.py and calling main.py breaks something

[tawdry vapor]

try python -m main instead?

[fossil veldt]

seems to do the same thing

[tawdry vapor]

I'm not sure. it doesn't make sense

[fossil veldt]

test:
    docker compose build -q --force-rm
    docker compose run --entrypoint /bin/bash --rm snekbox -c \
        'coverage run -m unittest; e=$?; chown --reference=. .coverage; exit $e'

does the chown do anything special here?