PyCQA, covid, and then literally whatever (welcome to off-topic)

[topaz forge]

Ah I met @hallow crow at europython and he was like, "you I remember you, you stopped me joining pycqa" or similar

[sinful cove]

of course it was europython lol

[topaz forge]

Where did the message go?

[sinful cove]

hmm?

[topaz forge]

It says the original message got deleted

[sinful cove]

discord being discord

[topaz forge]

And I had a message about EuroPython and it's gone

Anyway I dug out the email from 2020 and I was worried about teyit being trademarked

[sinful cove]

this entire mess makes me wary of the whole organisation tbh, probably unreasonable since similar to the PyPA, PyCQA's projects are at best loosely associated with each other

[hallow crow]

lol

[sinful cove]

not that I ever had intentions to contribute to any of the projects, but I wouldn't be surprised if I had looked into contributing to isort down the road

[topaz forge]

You replied to the message but it's gone for me

[hallow crow]

just replied to your europython msg

don't know what's going on with discord tho

[topaz forge]

Yeah I see the reply like that but not my message

[sinful cove]

huh.

how weird

[hallow crow]

by the way, what's the start of discussion

[topaz forge]

Well

[hallow crow]

I am totally lost on the context (I dont even remember joining this server lol)

[topaz forge]

@dawn perch sent a tweet

Then everyone who liked the tweet got blocked by Anthony

[sinful cove]

(we're apparently pinging everyone in this thread)

all of this got started with this message #general message

[topaz forge]

Oh sorry is pinging bad?

[sinful cove]

I mean, I'm not pradyunsg so I can't comment whether it's bad, but I'm one of those people who will avoid pinging people as much as possible if it's not necessary

[hallow crow]

wow that's a lot of messages

[topaz forge]

Ah I sort of see it as rude not to ping, it's like whispering about someone behind their back

And on irc you can't not ping without being like p r a d y u n s g

[hallow crow]

(btw it is totally fine to mention me, I am actually glad to see a lot of stuff that I would have normally missed)

[sinful cove]

that's a fair argument! I guess we're just on the opposites of the spectrum

[hallow crow]

just had a very weird context switch of seeing a thread on our europython memories 😄

and was curious how I got here

[sinful cove]

I'm extremely conflict-averse so I try to avoid ticking anyone off if I can

[topaz forge]

https://github.com/PyCQA/meta/issues/54 I think is where it's all hitting off though

[sinful cove]

yeah, basically it seems like a lot of toxicity has been coming from PyCQA, specifically Sigma and Anthony. Some of it is probably justified given the heated flake8 discussion on pyproject.toml support, but the blocking seems much...?

this is totally uninformed though, I just read through the discussion here and that issue linked above early in the morning

I haven't had any direct interactions with Sigmavirus, and I've had only one encounter with Anthony and that was when he filed a bug report for Black when we introduced mypyc which caused breakage as type hints were being enforced at runtime.

[topaz forge]

I've not actually had any issues with sigmavirus24

omicronvirus22 got me though

[sinful cove]

oh my gosh

haven't gotten covid yet which is good

[topaz forge]

Highly recommend not getting it 0/10

Recovered how though

[sinful cove]

it's long covid that scares me :)

[topaz forge]

Yeah I thought that too

But short COVID is sufficiently scary it turns out

But yeah I'm sort of involved in all this because someone emailed me to get them unbanned

[sinful cove]

ah, I see

Part of me is considering setting up a public email address and/or twitter, but the reasons why I don't have them public haven't really changed yet.

There's not really any benefits to them though, just more marketing spam I suppose haha

[topaz forge]

Not having a public email address just seems unworkable to me xD

I'm also surprised you can't send email to GitHub users

There's got to be a secret Hotmail account in there somewhere that you can also use for Skype

[sinful cove]

sounds like a good thing imo, sounds like it'd be abused

I guess, it could be an optional thing though

[topaz forge]

That's what email is for?

[hallow crow]

Twitter is nice, if you don't go into any sort of drama.

[sinful cove]

Yes, but I don't think giving other people direct-ish access to someone's inbox by just creating a github account is a good default.

[topaz forge]

You can sort of use keybase

[sinful cove]

I concede it would be useful if it was an optional feature you could configure

[topaz forge]

I dunno I guess I come from irc where everyone DMs you spam constantly

[sinful cove]

I have a twitter account. I just keep it private and don't write anything on it. Basically it's my way of kinda keeping tabs on what's new in my python bubble

[topaz forge]

And you have to set a responder

[hallow crow]

Makes sense.

[sinful cove]

although I had a fun incident where someone managed to find my account when I didn't have it private yet after my mypyc series blew up

that's why it's private now :)

[topaz forge]

Rude people in the DMs?

[sinful cove]

thankfully not, it just freaked me out since you could figure out where I live (and probably other personal details by association) by who I was following

[topaz forge]

Oh dang

[loud coral]

welcome to the web in 2022, sadly

[sinful cove]

Yeah. The only way forward would be to have two twitter accounts and keep them very separate but I can't be bothered.

If you could hide who you follow on Twitter, I'd probably be fine honestly.

Eh, I'm not really concerned with developing a personal brand right now though so whatever.

[sinful cove]

nothing is really ever private <.<

if someone wanted to dig up dirt they easily could for a lot of us

[loud coral]

well, based on what I could find, Google has my age wrong XD

[stable estuary]

I want to have a public email but it's effort lol

[sinful cove]

Ditto.

[stable estuary]

I have a domain that's literally jacken.men

couldn't be more perfect lmao

[loud coral]

I have a public email, but no spam.... perks of being a nobody XD

[sinful cove]

I happen to maintain a somewhat major project so I think I'd get some spam :D

[stable estuary]

I see it as a problem that there's no way of contacting me for like security issues or stuff

getting a thank you email sounds nice but that sounds like something that would never happen lol

[sinful cove]

Black doesn't really have much that could become a security issue

The biggest concern would be supply chain attacks, directly on black or via our deps

i.e. the code itself is not risky, but the infra around that code can be attacked.

[stable estuary]

I guess also just allowing people to contact me about things wouldn't be bad, as long as I don't become someone's personal support or issue tracker lol

[sinful cove]

PyCQA, covid, and then literally whatever (welcome to off-topic)

[stable estuary]

oh no

I did it again

I off-topiced the off-topic

[sinful cove]

felt like I should've noted the fact that this thread isn't really about pycqa anymore :P

[sinful cove]

it happens a lot, don't worry about it haha

[loud coral]

yeah, that's what I meant by me being a nobody. I don't maintain (at least not in a significant part) any big projects

[sinful cove]

in the spirit of changing the topic of discussion again, what do you do when you don't really feel like reviewing PRs for a project you maintain?

they need to be worked on eventually, but like ughh

[stable estuary]

Is pretending they don't exist a valid strategy?

[loud coral]

I am at that point in my open source journey, where I either review what I can or I don't feel competent enough to review something, so can't help 😄

[sinful cove]

I would love to get black's open PR count below 20 by the end of the summer, but I don't think that's really happening

[loud coral]

you should see the issue/PR backlog we have in Poetry I know the feeling

[sinful cove]

https://ichard26.github.io/ghstats/

[stable estuary]

I'm not the best person to talk to about this, the PR count on the repo I maintain is literally at its peak

[sinful cove]

that's the current strategy haha :P

[stable estuary]

and that also applies to issue count

[loud coral]

what is the number?

[stable estuary]

112

I imagine I can get through a lot of them in few sessions

[loud coral]

https://tenor.com/8dAB.gif

[sinful cove]

last time black had that few issues open at once was end of 2018

[stable estuary]

I'm talking about PR count lmao

[sinful cove]

:O

[stable estuary]

We have 197 open issues

It's basically that I don't put much time into the project currently

[loud coral]

I worked hard to get us below 1000 issues in Poetry, now we are around 890-900

[sinful cove]

Are you a solo maintainer?

[sinful cove]

A maintainer/triager of poetry? I'd call that a major project haha

[stable estuary]

I'm doing some things that I'm hoping will either get our counts down significantly or at least make it organized

[stable estuary]

Not exactly

[loud coral]

not a core team member, just triage team 🙂

[sinful cove]

that's how I got started with black 😄

[stable estuary]

I'm the maintainer that is doing most of the work, maybe even at the current time when I don't do much

[sinful cove]

took around a year to get the commit bit

[stable estuary]

I want a triage team but I need to have triage documentation which is what a lot of my late work revolves around

😄

[sinful cove]

haha I've been slowly rewriting black's contributing overview docs

haven't finished it mostly because I was planning to switch black to nox at the same time

which itself got delayed by release 22.7.0 mypyc stuff

which was also delayed since I couldn't test it thoroughly enough in time for july

[loud coral]

contributing docs are on my TODO list for Poetry (or a rewrite of the existing one), but the days are too short

[sinful cove]

story of my life as the lead maintainer of Black :D

[stable estuary]

now 22.8.0 I guess

[sinful cove]

yup.

not even sure if it'll be released in august

It probably will, but there is no chance that all of the issues I put into the 22.8.0 milestone will be closed by the end of August.

The contributor I was counting on to take on the complex issues from that milestone is on vacation right now ☀️

[loud coral]

the pain of summer

you might think that people would have more time for project, but they just go on vacation and don't code...

[stable estuary]

yeah, what's up with people

not using your vacation time to code, who does that

[sinful cove]

lol lol

current todo list for black 😅

and this hasn't really been updated since last week, there's probably a few more things I could add

[loud coral]

was mypyc the reason black switched from Poetry to setuptools?

[sinful cove]

We never used poetry IIRC?

We used to use pipenv but we got rid of it since it kinda sucks to be honest.

pipenv isn't a build backend though unlike poetry

[loud coral]

https://github.com/psf/black/commit/6bedb5c58a7d8c25aa9509f8217bc24e9797e90d 😉

long time ago

[sinful cove]

I didn't contribute until March 2020 😅

[loud coral]

understandable

[sinful cove]

Honestly given Black is more a library from a packaging standpoint, poetry is probably overkill for us.

[loud coral]

Just thought that I might ask since you are already here

[stable estuary]

does poetry have something like setuptools-scm

[loud coral]

well, Poetry was made with libraries in mind mostly...

[sinful cove]

Would be surprised it doesn't. I'm just saying the pinning poetry offers is not that helpful to us

[loud coral]

not yet, but it is on the roadmap

[sinful cove]

the project management stuff could be nice though

[stable estuary]

that seems like a notable diff in that commit - it switches from explicitly declared version in pyproject.toml to setuptools-scm

also, it seems that the poetry config didn't use PEP 517?

[sinful cove]

lol what

so black forced people to use PEP 517/8 but we didn't use it ourselves

[loud coral]

oh, that was a long time ago...

[sinful cove]

I know I'm oversimplifying things, but that's silly :P

[loud coral]

I think it might have been caused by PEP 517 appearance

[stable estuary]

PEP 517 was supported in Poetry since 2018

so I wouldn't be sure on that

https://python-poetry.org/blog/poetry-0.12.0-is-out/

[loud coral]

I just now noticed that this commit deletes pyrpoject.lock which was old lockfile name

[stable estuary]

I wonder if poetry had some kind of editable install support back in 2019 too

I know it wasn't standardized until this year

[loud coral]

not sure...

[stable estuary]

but flit had it before standardization

so it was possible

just not through pip

[loud coral]

I think I have seen some old issues about it, so it could lack it

[stable estuary]

I guess black never really used poetry as a build backend if it didn't use pep 517 support. And it had setup.py when using poetry so

[loud coral]

I guess back then, Poetry wasn't mature enough

[stable estuary]

I guess it didn't really make that much sense to have poetry metadata if the project was focused on using setuptools anyway

[stable estuary]

yeah, that's likely the main reason

poetry is also opinionated so another reason could just be that its opinions didn't align with Łukasz's opinions

[loud coral]

yeah, we are going towards less opinionated approach

some work on introducing PEP621 support was already started

[stable estuary]

I like my simple stuff that I already know so I haven't seen the need for poetry

It has some great tech

[sinful cove]

oh really? I thought poetry was against PEP 621 support for the longest time ever

[sinful cove]

yup. I've gotten pretty opinionated with how I approach packaging for my projects haha

[stable estuary]

This is cool and it uses poetry behind the scenes
https://gitlab.com/mitchhentges/pip-compile-cross-platform

[loud coral]

partially. the way PEP621 defines dependencies, will be downgrade for us

[sinful cove]

(somehow we managed to steer this thread back on topic for the server lol)

[loud coral]

but we are trying to go towards standards than fight with them

[stable estuary]

virtualenv .venv, . .venv/bin/activate, and pip install -e .[dev] and I expect to be set

[sinful cove]

I've experimented with having a setup-env nox session that does the first and last bit

I assumed that pipx is present and is being used for nox

[loud coral]

meanwhile I can just poetry install and have it all be done automagically 😉 but to each their own

[stable estuary]

I use direnv so the first 2 commands are replaced with direnv edit . and typing layout python 3.N and saving it

I need a shell function that wraps virtualenv and also activates it after creating

That's like two line function that would make life so much easier lol

I probably need to figure out how to get completions working with that but ye

[sinful cove]

I'm crazy and do python -m virtualenv venv && actenv

[stable estuary]

This is because I often make temporary envs

[loud coral]

please don't make me... (my brain loves such challenges and makes me do stuff like this if I see it)

[stable estuary]

Why the python -m bit, that's a hassle

lol

[stable estuary]

Can't be too hard

[loud coral]

meanwhile, I go with python -m venv .env if I need a tmp env...

[stable estuary]

venv is slow

[loud coral]

I can spare 2 seconds...

[stable estuary]

I can't

and it's closer to 3

❯ time python -m venv just-no                         
python -m venv just-no  2,52s user 0,18s system 99% cpu 2,720 total

❯ time virtualenv yessss     
created virtual environment CPython3.10.4.final.0-64 in 317ms
  creator CPython3Posix(dest=/tmp/yessss, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/ubuntu/.local/share/virtualenv)
    added seed packages: pip==22.2.2, setuptools==63.4.3, wheel==0.37.1
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
virtualenv yessss  0,28s user 0,13s system 91% cpu 0,448 total

:D

[sinful cove]

I never bothered to put my main python 3.8.5 install on PATH

[stable estuary]

I'm confused

[sinful cove]

I just symlink specific scripts from /opt/python3.8.5/bin/ to ~/.local/bin/

[stable estuary]

how does that matter?

virtualenv is in the PATH, right?

[sinful cove]

no?

[stable estuary]

that's, eww

[loud coral]

well, I never had a need for virtualenv, honestly. most projects I do are on Poetry (which handles virtualenv stuff for me under the hood) or I do one python -m venv ... once every few months...

[stable estuary]

pipx install it :)

[loud coral]

pipx, my love

[sinful cove]

I've been using the virtualenv from my user wide python environment all of this time

[sinful cove]

I will

for you.

[loud coral]

LOL

[stable estuary]

then you get that sweet virtualenv dir && actenv or virtualenv dir -p3.10 && actenv :)

[sinful cove]

-p3.10 won't work either cause I don't put my other python installs on PATH

[stable estuary]

:(

[sinful cove]

I do have a shell function that does though

[loud coral]

I wish virtualenv would pull the engine out and make it into library with optional CLI interface... would be much nicer to use in other projects...

[stable estuary]

to each their own

sounds annoying lol

[sinful cove]

which I rarely use because I use fish's command autosuggestion to spare myself the typing of the full path ¯_(ツ)_/¯

I just write python -m virtualenv 3.10venv and fish suggests the rest

[loud coral]

pyenv 😉

[stable estuary]

You know what I would like virtualenv to do... Allow embedding any wheels

[sinful cove]

I've compiled every python install I have by hand 😅

[stable estuary]

not just the default pip, setuptools, and wheel

[sinful cove]

I am crazy, I realize.

[loud coral]

so, you just want templates of venvs?

[stable estuary]

I don't know lol

I just feel like the functionality is there already

[loud coral]

that's the rite of passage I have yet to go through

[sinful cove]

done!

[stable estuary]

  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/ubuntu/.local/share/virtualenv)
    added seed packages: pip==22.2.2, setuptools==63.4.3, wheel==0.37.1

it already adds these, so why couldn't it like, allow me to also add IPython

[sinful cove]

I think my logic for why I never pipx-ed virtualenv was that I thought of it the same way I do with pip.

[loud coral]

maybe I should do venvlib based on virtualenv....

[sinful cove]

I don't have my user wide Python install's pip on PATH so it's obvious to me I'm not in a virtualenv

[stable estuary]

and also have a way to upgrade embedded wheels as it can with pip, setuptools, and wheel

I looked briefly through the extension APIs in virtualenv and it doesn't seem like it would be easy for me to just add a package like that

[stable estuary]

I don't have a system pip at all

I'm convinced it is not needed

[loud coral]

@stable estuary other than it being slow (and not having nice CLI handle), is there any thing else against -m venv?

[stable estuary]

I can bootstrap pipx with itself using a temporary virtual environment and I don't system pip at all

it's nice that it's not necessary to install Python with pip

[stable estuary]

outdated pip

you can update virtualenv's bundled wheels for pip/setuptools/wheel and if I recall correctly it also just does it automatically if updates weren't checked within last 2 weeks

It might require an option, unsure

I use this tool for upgrading stuff in my system:
https://github.com/r-darwish/topgrade

[potent field]

FYI my interactions with the isort maintainer have been pretty nice!

[stable estuary]

soo I just tuck a virtualenv --upgrade-embed-wheels as a custom command to run and it's kept up-to-date whenever I run upgrade on my system.

[stable estuary]

Timothy is great

or at least seems like a great person from the interactions I had with them

[potent field]

yes same

[stable estuary]

I wonder why there wasn't an isort release for a long while

[loud coral]

does it need one?

[stable estuary]

There's a great black compatibility enhancement that's been in main for almost half a year

https://github.com/PyCQA/isort/pull/1876

it seems like the tests have been failing on main last time isort was touched

I kind of wanted to check out usort but I don't agree with one of its opinions and it makes it a hard no for me lol

Same as Black, it's meant to be opinionated

[sinful cove]

yeah, constants should be together, not mixed with functions and classes, ugh

[stable estuary]

so I won't use it

[loud coral]

there is also reorder_python_imports

[stable estuary]

you know who is the maintainer of that one though?

:)

[loud coral]

yes 😄

[sinful cove]

isnt usort like meta's thing

[stable estuary]

yes, it is

[loud coral]

just putting it on the list

[stable estuary]

I mostly found usort appealing to the ufmt tool

but isort is fine

[loud coral]

maintainer might be not that nice, but he makes good projects...

[stable estuary]

it's just ufmt that seemed nice

[stable estuary]

ye no, I use his projects

I like his projects

I watch his YT videos 🙃

[sinful cove]

I used to.

[stable estuary]

used to follow them on Twitter before they blocked me yesterday

which unfollows them automatically

I think flake8 and pre-commit are great tools

[loud coral]

I don't use twitter, but the rest of it, I also do

[sinful cove]

While writing my mypyc series, I considered linking to Anthony's video on mypyc but I decided I did not want to associate myself with him at all

[loud coral]

he made a video about it?

[stable estuary]

There are some other cool tools from Anthony but flake8 and pre-commit are the 2 main ones

there's pyupgrade...

[sinful cove]

I just linked to glyph's great article on mypyc instead as further reading

[loud coral]

that one is really good...

[stable estuary]

okay, I guess flake8 isn't Anthony's project, just maintained by him, right

[sinful cove]

yup.

[stable estuary]

he didn't create it afaik

[loud coral]

flake? no, he didn't

[sudden vine]

Solving environment...
...
...
...I use conda

Thank goodness @potent field is working on integrating libmamba to make it way faster

[potent field]

that is released already 🤣

[sudden vine]

I knew it was released experimentally but is it stable now?

Personally pre-commit is one of, if not my favorite Python development tool, which is a pretty stark contrast with Anthony being my least favorite Python maintainer. I heavily use it across all the projects I maintain. Pyupgrade is also pretty useful too

[potent field]

not yet I think

[topaz forge]

And setup.cfg it's a downgrade too

[topaz forge]

It's basically just bower that runs every commit though

[loud coral]

??

[topaz forge]

setup.cfg lets you do a DAG of extras that get linearized in the .whl

See the twisted setup.cfg

[loud coral]

You mean embedding on list in another with „pointer”?

[stable estuary]

You mean configparser's interpolation?

We use that too

Though pip now supports a package depending on its extras

https://github.com/pypa/setuptools/issues/3139#issuecomment-1203403180

[project]
name = "my-pkg"

[project.optional-dependencies]
tests = ["pytest"]
docs = ["sphinx"]
dev = [
    "my-pkg[tests,docs]",  # <-- !!!
    "pdbpp",
]

it only works with pip 21.2+

[topaz forge]

Yeah but twisted users might not be on that version

Would it violate PEP 621 to flatten a DAG of extras in a build backend?

[stable estuary]

Maybe they should update 🙃 But yeah, I kinda get it

[stable estuary]

good question

[topaz forge]

Is there a table of vendored pip version to Python version?

[stable estuary]

To me it kinda seems like it wouldn't violate it

[topaz forge]

Yeah to the point at which tools should just do it whenever they see it not just the backend

[stable estuary]

It adds onto complexity of each individual build tool tbf

when you already have pip resolving it

[topaz forge]

Anyway seems @sudden vine's thing, two different interpretations that mean the same thing but get interpreted differently

[sudden vine]

Why would it? PEP 621 (or more correctly, the canonical Declaring Project Metadata specification) only states that, for the values of the array values in optional-dependencies, "The strings of the arrays must be valid PEP 508 strings." It says nothing further about the semantics of such values; it only defines their source representation. Any restriction would be in core metadata, not in the particular metadata input format, so there doesn't appear to be a clear reason why setup.cfg would be theoretically any different than [project], unless pre-processing by the backend was involved. pip , a frontend, reads it from the built core metadata, no? So it would make no difference to pip.

Honestly, I've been rather confused by this whole discussion, sorry. I feel like there has to be something I'm missing here

Canonical spec: https://packaging.python.org/en/latest/specifications/declaring-project-metadata/#dependencies-optional-dependencies

[stable estuary]

setup.cfg supports interpolation

[options.extras_require]
dev =
    black
    isort
test =
    pytest
all =
    %(dev)s
    %(test)s

I agree that pyproject.toml is a downgrade in that regard

though it's more debatable since pip 21.2

still, it offers lower compatibility with tools

[stable estuary]

I can make one quickly but this is probably going to be inaccurate in terms of what distros ship

[sudden vine]

I had no idea you could do that, heh. I assumed the standard way was to just do myproject[extra] now that pip supports that.

IMO, I wouldn't want to rely on those sort of details of a particular custom implementation of a particular config file format anyway.

[stable estuary]

This was possible earlier than second part of 2021 though :)

[stable estuary]

I mean, it's like saying not to rely on any other feature of the format

[loud coral]

Sooner or later either TOML or just python tooling will support it if this is really needed

[topaz forge]

TOML interpolation is going to make people scream on Twitter for decades

[loud coral]

They scream anyway

[stable estuary]

i.e. if YAML was a format, why shouldn't I be using the anchors? It's part of the format (unless you're GitHub that uses a custom-written parser that doesn't support it instead of any standard implementation 🙄)

[topaz forge]

Oh yeah, it's fun when they do

[topaz forge]

It was a big missing feature for why people used setup.py

[stable estuary]

I find the constant discouragement of setup.py incredibly odd.

yes, if you don't need it, don't use it

[topaz forge]

Distributions usually have a newer pip though right?

Also I don't want people to install my software in their distribution python I want them to use a venv. And a distribution shipping an older pip than cpython vendoring is the right amount of friction I'm happy to apply

[stable estuary]

But setuptools says outright in the quick start guide that configuring new projects via setup.py is discouraged (it does have a foot note, saying it should only be used when custom scripting is necessary) but it's very strong wording without clear explanation on why it should be like that.

[stable estuary]

I'm talking about that though

Distros debundle wheels

like, they update them separately

[topaz forge]

Yeah but they're surely the same version as upstream

Or at least newer

[loud coral]

People are always sure that they have that special case to write custom scripts

[topaz forge]

hynek is doing the readme hook which is the last bit of setup.py I use

[loud coral]

Readme hook?

[topaz forge]

eg in twisted setup.py we mangle the long_description with a regex so it shows up right everywhere

[sudden vine]

Its not a feature of a file format, its an optional extended feature of one implementation of one variant of a loosely-defined family of formats

[stable estuary]

It's a feature of Python's configparser file format. INI is not defined format at all

[loud coral]

Sounds like overengineering to me. You could just put full links in README…

[sudden vine]

I can't speak for @dawn perch , but it seems unlikely TOML will support it. Part of its design goals are to avoid the feature creep and complexity of YAML. If people want that, they can just use YAML, which is perfectly fine

[stable estuary]

it's not

[sudden vine]

Another approach is a preprocessor that generates the actual pyproject.toml before build time; AFAIK there already is such a thing that some people use

[stable estuary]

you can put stuff like changelogs there as well

[topaz forge]

Dang it, I was just getting my Twitter canister ready

[stable estuary]

it makes no sense to duplicate this content

[loud coral]

I agree with that, but I guess, with enough screaming, people might get what they want some day

[sudden vine]

I'm unclear on what you're saying. What gets preprocessed in the README, exactly?

I think I missed something somewhere

[topaz forge]

https://github.com/twisted/twisted/blob/trunk/setup.py#L19

[stable estuary]

Here's an example of an equivalent plugin for hatch: https://github.com/hynek/hatch-fancy-pypi-readme

might be able to sell it better

[loud coral]

They have link that doesn’t have github URL before filename. One link. This regex in setup.py adds URL to filename

[topaz forge]

https://twitter.com/hynek/status/1555600164542468098?t=u9KP0fNs7iThN5VpntgFVg&s=19

Hynek Schlawack (@hynek)

#Python package maintainers who would like to switch to a pure pyproject.toml packaging configuration, but care too much about your project’s PyPI landing page: I have broken my no-more-projects vow just for the 3 of you!

Likes

105

Twitter · Aug 5, 2022, 5:02 PM

[topaz forge]

Wait it's only one?

Oh 😳

[loud coral]

Well, yeah… that’s why I said it’s overengineering

[sudden vine]

That seems a little silly to me. If you're not going to use actual Sphinx roles (because PyPI and its a readme, not formal docs) and just stock reST links, why not just use the full absolute link in the readme itself so it works anywhere?

[topaz forge]

I think the readme used to be worse than it is now then

[sudden vine]

There are very good reasons, its just complicated to explain to a newer user and it doesn't directly affect them so much as everything downstream

Well, a fair number of them actually do, but TL;DR its complicated. That said, the documentation could probably do a better job explaining it

Setuptools documentation has gotten better lately, especially with @wanton burrow 's help, but its still has its rough edges

Decades of baggage will do that

[stable estuary]

Not a table yet
https://gist.github.com/jack1142/f6c6aecc381f4724316d4230870f84e2

gonna do that in a sec

[sudden vine]

Hmm, interesting...but it means the description is no longer specificied staticially, deterministically and in a standardized manner. Seems useful in some scenarios, but why only generate it at build time versus instead generating it repo-side via e.g. a pre-commit hook, script, GH action, etc. such that everyone gets to benefit form it, not just people viewing the readme on PyPI (which at least my impression is only a fraction of the total viewing it in some form, e.g. GitHub, locally, other sites, etc)

[stable estuary]

precisely because it's buildable

it makes no sense to put generated content in version control

[sudden vine]

That means that the actual useful readme is only dumped into the METADATA of the wheel and installed project, rather than everywhere else people actually look at it. And it isn't blanketly true that generated files are never checked into version control; there are plenty of them in CPython's repo and in others. Sure, it should be avoided where practical, but there are use cases, in this case so people can actually see your readme, and the cost is fairly negligible as a single text file and run with your existing pre-commit suite.

(And PKG-INFO, but that's standardized and not necessarily reliable)

[stable estuary]

Duplicating changelog content doesn't make much sense.

README on GitHub can just link to the whole changelog

While you get a nice changelog for currently selected version on PyPI

[sudden vine]

Its unclear what the rationale is for doing so on GitHub but not PyPI. Either way, its either getting to see the content right there, or clicking a link and having to load another page. Sure, you can switch versions in PyPI, but equally so you can easily switch tags (or branches) in GitHub.

Its duplication, or not, either way, since generally you should be shipping your changelog in the package. You can also give it its own link, with its own icon, in the PyPI sidebar with the project_urls in the metadata.

I totally understand its convenient to have right there, but what I'm not understanding is how the rationale is so different between PyPI, GitHub and all the other places your README might be that justifies treating PyPI differently than anything else, even the built package/installed project

Also, its 4 am and I like to argue too much late at night, sorry 😂

I should probably have the good sense to go to bed now

[stable estuary]

let's just agree to disagree I guess lol

[stable estuary]

alright, table generated too now
https://gist.github.com/jack1142/f6c6aecc381f4724316d4230870f84e2

[loud coral]

I totally agree

[stable estuary]

kinda wish that GitHub's table view for CSV files also allowed sorting

so pip 21.2+ is available on Python 3.7.13+, 3.8.13+, 3.9.7+, 3.10.0rc2+, and then on any 3.11 and above it's always going to be there

not as much I thought if I'm being honest

[sudden vine]

That's very useful to have in general, thanks

I do have to ask, why all that munging to extract the distribution name and version? Since you're already using packaging anyway, why not just use packaging.utils.parse_wheel_filename https://packaging.pypa.io/en/latest/utils.html#packaging.utils.parse_wheel_filename

[stable estuary]

because I'm not a regular user of packaging and as such I don't know everything that's in there :)

for such one-off scripts, I value speed of development more

and I was not already aware of that util

so I would have needed to go through the documentation to even become aware of it

[sudden vine]

I mean, to be fair, I'm not either, I wasn't even sure if it had something like that until I did a quick check just now 🙂

But I figured that surely there was no need to reinvent the...

wheel

Ba dum tush

I'll go now

[stable estuary]

that packaging addition was kind of late thought anyway lol

I realised I need to be able to sort it and also run into an issue with naive version parsing for checking < 3.4 condition

just for you, I updated it :)

I'm also aware that csv package exists but I'm not going to use that :P

[sudden vine]

Damn, you beat me to it 😂

I guess my thought process its a few moments spent looking something up to see if its been done not only saves me time mucking about trying to implement it (it would have taken me a whole lot longer to hack that together than it would to use the existing function...that, and I'm inherently lazy) but I also learn something too. But the cost benefit is different if you're a faster/better programmer than me...which you almost certainly are, to be honest.

I'm not a real programmer, I just play one on the internet

[stable estuary]

But the cost benefit is different if you're a faster/better programmer than me...which you almost certainly are, to be honest.
I doubt I am, I just act like one 🙃

[sudden vine]

Fake it till you make it...it fooled Guido after all

[sudden vine]

<kbd>X</kbd>

[stable estuary]

speaking of packaging, I'm actually waiting for the next version because it adds __eq__ to Requirement

[sudden vine]

Its about time, lol

My "real" job isn't a programmer at all...I'm a NASA research scientist

[stable estuary]

it missed the deadline for pip though so I guess its priority lowered

the handwritten parser for requirements was merged less than 2 weeks ago

[sudden vine]

BTW, is it your Discord bot that uses our new PEP API or is that someone else I'm thinking of?

Like did Hugo talk to you about that or something?

[stable estuary]

definitely not me

I use RSS for seeing new PEPs in my bot

I'm not associated with the Python Discord

whom I imagine might be the user of the new API

I only complained about the RSS back when all the work on Sphinx support was finally getting deployed

https://github.com/python/peps/issues/2406

that got resolved quickly though :)

[sudden vine]

Ah yes, I remember now

[loud coral]

PEP API? Sounds interesting

[sudden vine]

It was this issue that was Python Discord's bot, also RSS related though https://github.com/python/peps/issues/2493

[sudden vine]

https://tenor.com/view/suicide-gif-23888293

"API"

[stable estuary]

ahh, so it was about RSS too then

but just redirects stuff

guess I wasn't affected by that

[sudden vine]

Its just a JSON dump with all the metadata, headers, abstracts, links and everything

I'm working on V2 though which will be a lot nicer, still static (no dynamic queries) but maybe also with sub-JSONs by different factors and with all the headers processed into a structured, standardized form

It relies on some non-trivial work unifying our header parsing which is currently separated between the PEP processing, the PEP 0 generation, the linting, the RSS generation and the API generation

To be honest, the RSS generation script is still a big pain, it relies on plain docutils and a bunch of hacks still, though it is still a lot better than the nightmare we had before

Honestly the original PEP building system was a fascinating time capsule that had been mostly untouched from two decades ago, in the days of Python 2.0-2.1. It was by far the oldest untouched Python code I'd ever seen.

Of course, it was a nightmare to maintain. Adam's hard work was a godsend

[wanton burrow]

We need a PEP that allows doing:

[project.optional-dependencies]
dev = [
    "black",
    "isort",
]
test = [
    "pytest",
]
all = [
    {include = "dev"},
    {include = "test"},
]

or something similar.

(Syntax courtesy, or at least inspired by, @layday, I think @henryiii would also be interested).

If any one would like to champion this, I am more then happy to contribute (this is another thing that is in the backlog of my mind).

[stable estuary]

I guess I'm wondering if, other than being able to use older pip versions, it has any benefits over my-pkg[dev]

Well, technically any installers that don't support dependency on itself, not just pip but I'm not sure if there's actually any other installer out there

[wanton burrow]

Hi @stable estuary, setuptools is committed to support setup.py. The docs might not be the best, and sometimes we get the wording wrong....
Please feel free to drop us a PR with any doc suggestion improvements. It is actually very great to have feedback about docs directly from the community, and I am trying to do that as much as I can.

[stable estuary]

sorry for hiding in this thread lol

[wanton burrow]

I usually just add a Changelog entry to the project links anyway (and I don't mind having the badges showing up in PyPI). E.g.: https://pypi.org/project/ini2toml/.

In terms of the trade-offs, this is good enough for me, e.g:

[wanton burrow]

Probably not many. The main difference is one would be a standard, the other one would be an implementation detail from pip.

I do like the syntax-oriented approach though, it looks cleaner

[stable estuary]

I do think people can get mixed signals from how it's communicated, not just due to setuptools's documentation though. There are good blog posts about deprecation of executing setup.py but from what I've seen, some people end up thinking that setup.py as a whole is deprecated (could it be because people are lazy and only read the titles? I don't know 😄). The way setuptools's official documentation talks about it doesn't make it obvious that it is entirely valid to use setup.py when you actually need it (and even if you don't, it's not like that option is going to disappear, it's just discouraged because static metadata is superior).

[stable estuary]

I'll admit that I was somewhat playing a devil's advocate. To a degree I could get behind inserting a partial changelog in the version's readme on PyPI because I have definitely found it convenient in the past. Personally I think project links are very good solution too though

Both make it easy for me to go through changelogs when I update dependencies.

Personally I'm also wanting to get my projects to have CHANGES.rst in the root of the repository (CHANGELOG, HISTORY, or NEWS in rst or md formats also work fine as names) because it is another thing that made my life easier when trying to find the changelog for a project.

it's even nicer when your tooling can find those

Refined GitHub extension can show you a link to the changelog file on the releases page if it detects one of matching files in the repo root

and then there are Dependabot and Renovatebot (I don't use either currently) that can detect at least some of these

[stable estuary]

Is it an implementation detail though? The core metadata specification doesn't say this should not be supported.

It wasn't supported until 21.2 but the question is more if it should have been supported.

[wanton burrow]

That is a good question. The specs don't say anything against or in favour of it, so I guess it is at least underspecified.
Either approaches work fine for me. Maybe we don't need to touch this problem.

[stable estuary]

Might be worth specifying it more I guess.

doing my-pkg[dev] is a new trend now that pip supports it so even if it remains underspecified in the actual spec, it's likely to become de-facto standard, especially as projects move to using pyproject.toml

related (resolved) issue: https://github.com/pypa/pip/issues/10393

[topaz forge]

So the answer is 3.10?

[topaz forge]

Yep you wrote it out here. I was also going to wish for sorting

3.10.0.rc2+ means 3.10 to me. Twisted doesn't care too much about supporting pre-releases. Especially for bugs fixed by upgrading pip

So the release date of pip 21.2 is 05 Oct 2025

@stable estuary if you're still in the mood to mess with that script can you join it with https://endoflife.date/python and sort by pip version and select only .0 releases? Then you get a nice table of pip version to when you can rely on it

[topaz forge]

Will you tweet this or can I?

[topaz forge]

https://github.com/twisted/twisted/issues/11595 look there's even a ticket and a plan

[stable estuary]

I'll look into it

[stable estuary]

I can but I pretty much don't have any audience I can reach

[wanton burrow]

Hi @stable estuary, any chance you could have a quick look on https://setuptools.pypa.io/en/latest/userguide/quickstart.html#setuppy-discouraged and check if this is enough to solve the documentation flaw that you have previously pointed out? (that would be very helpful! Thank you for the pointer!)

[stable estuary]

The foot note says:

Examples are kept in this document to help people interested in maintaining or contributing to existing packages that use setup.py.
Which I'm not entirely sure is true?

As for the note:

Setuptools offers first class support for setup.py files as a configuration mechanism.

It is important to remember, however, that running this file as a script (e.g. python setup.py sdist) is strongly discouraged, and that the majority of the command line interfaces are (or will be) deprecated (e.g. python setup.py install, python setup.py bdist_wininst, …).

This might be just be me but "It is important to remember, however, that running this file as a script (e.g. python setup.py sdist) is strongly discouraged" might not be entirely obvious to users that don't know what that is, e. g. "What's not supported, however, is running this file as a script (e.g. python setup.py sdist) and that the majority of the command line interfaces are (or will be) deprecated (e.g. python setup.py install, python setup.py bdist_wininst, …)." seems somewhat more obvious to me. It's not quite right though since it is not exactly unsupported, just deprecated, it's just that this wording makes it somewhat more clear to me that using setup.py is fine and I just shouldn't run python setup.py .... I don't quite know what exactly makes it more straightforward to me.

[sudden vine]

Correct me if I'm wrong @wanton burrow , but as I understand it it is sorta unsupported in the sense that existing issues that affect these code paths are not being actively worked on, no?

[stable estuary]

That's wrong, setup.py has full support, what's deprecated is usage of it as a script

[wanton burrow]

Thanks I will review the text to see what we can do about it (this week I have a lot on my plate though). The comment about leaving the examples in the docs is true, I swear 😜

When I edited the page I thought about removing the examples using setup.py , but then I realised it would make people that have to deal with old code bases confused.

Regarding the comment about setup.py being discouraged (beyond the minimal stub) if the user doesn’t need scripting capabilities in favour of the declarative approach. I believe that any maintainer would agree that this is the case...

The CLI interface is not completely deprecated only discouraged, but most of the commands are. The ones that are not yet deprecated will be...

[wanton burrow]

There any many tests for configuration coming from setup.py. Maybe there is no 100% coverage, but yeah... it is tested, at least partially

Probably there will be no features added to it, but I would say it is supported

[sudden vine]

I was referring to the specific "code paths" of the setup.py script commands, not having a setup.py in general

I was basing this off Paul Ganssle's well known blog post https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html

As more and more software starts to rely on modern packaging standards, expect to see more breakages from these old code paths. As mentioned in this issue, direct setup.py invocations have effectively been unmaintained for several years now. Whenever someone raises an issue with a reproducer that involves invoking setup.py, the maintainers ask for a reproducer that doesn't hit this code path, and if one can't be found the issue is closed. Put another way, direct invocations of setup.py that currently work for you do so essentially by chance — if something breaks, you are on your own.

[wanton burrow]

In that case, OK... we are trying to support things on a best effort basis, but probably if someone opens an issue about python setup.py install I would not put the same amount of energy...

If someone opens an issue about setup(name="myproj", ...) not resulting on the distribution having the correct name I would put a lot of energy there

[topaz forge]

Hmm I thought it was pending deprecation

and setuptools pep621 was experimental

Does that mean there's no supported code path?

[potent field]

what is deprecated is invoking setuptools via setup.py, not setup.py itself

so, python setup.py ... is deprecated

you should use a PEP 517 builder

[sudden vine]

Correct me if I'm wrong @wanton burrow but static source metadata in the [project] table (i.e. PEP 621) is (now) considered stable; it is the pyproject.toml-based Setuptools-specific configuration in [tool.setuptools] that's still technically experimental (unless that's changed in 65.x). However, particularly given declarative config in setup.cfg has been available and been the stable, recommended alternative to that for many years (and offers an easy migration path with ini2toml specifically designed for that purpose), that's more or less orthogonal to what happens with setup.py, aside from the fact that isn't what is deprecated anyway.

[stable estuary]

The format of pep 621 is stable. The implementation of it in setuptools is considered experimental at the time as it's a recent addition.

Okay, it's "beta", not "experimental"

[topaz forge]

Ah right so the "correct" code path is setup.cfg with a shim setup.py invoked via pep517?

[topaz forge]

But really everyone should just use the beta pep 621 support

[sudden vine]

Again, unless I'm mistaken (@wanton burrow can confirm) the PEP 621 ([project] table, equivalent to [metadata] in setup.cfg) support in Setuptools is considered stable; it is the support for non-PEP 621 Setuptools-specific configuration ([tool.setuptools] , equivalent to [options] in setup.cfg) that is still experimental/beta. Per the Setuptools docs:

Support for declaring configurations not standardized by PEP 621 (i.e. the [tool.setuptools] table), is still in beta stage and might change in future releases.

[sudden vine]

There should actually be no need for the shim setup.py shim unless you need support for editable installs and are still working with legacy tooling (now that Setuptools supports PEP 660); the PEP 517 hooks don't invoke the setup.py but rather invoke the backend directly.

[topaz forge]

Yeah I try to remove it asap

But is that beta or stable?

The messaging I see is "setuptools' PEP621 is in beta but use it anyway" and "don't use setup.py"

[sudden vine]

Sorry to repeat, but again, AFAIK the former is not (fully) correct, see my previous two messages and/or the setuptools doc I linked. It seems especially in earlier messaging around the "experimental" nature, the PEP 621 support was conflated with support for pyproject.toml-based Setuptools backend-specific configuration

[topaz forge]

Yeah sorry I realized that as you were typing

I did actually mean setuptools' messaging

[sudden vine]

Yeah, sorry

[topaz forge]

What I'm trying to get at is people are going to use setuptools' pep621 support no matter how beta it is

[sudden vine]

I can defintely understand where the confusion would arise, I thought so myself (despite being well aware of the distinction) until I looked carefully at the docs. And both were true at first before they were actually released

That said, I don't know for sure if it is supporting to put [options] in setup.cfg and [project] in pyproject.toml; the spec implies it should work, but I'm not 100% sure if that's what's actually implemented.

Using just pyproject.toml is recommended in the packaging tutorial now and there's no setup.cfg described any more , so anyone following that will use it

[topaz forge]

Pretty sure everywhere there's a loose point in the pypa specs the implementation drives the spec

But I don't even understand the edge case you're describing here

the pyproject.toml pep621 "MUST" override the setup.cfg right

I saw someone trying to write a setuptools plug-in that violated pep 621 and setuptools silently ignored the plug-in

[sudden vine]

Again, it is important to be specific here, because we are talking about two very different things, project metadata and backend-specific configuration.

[topaz forge]

Oh of course, I didn't think of it like that 🤦

Hmm the packaging guide telling people to use something that's in beta doesn't seem quite right

[sudden vine]

If [project] is specified in pyproject.toml, then backends must use it to fill the core metadata fields and nothing else, except for any "fields" (actually, implied to be[project] keys, though PEP 621 uses the actively confusing term "fields" here, which rather normally refers to Core Metadata fields which do not map 1:1 [project] keys) that are explictly listed as dynamic. Therefore, everything in the [metadata] of any setup.cfg (or equivalent in setup() options), and a number of items in the [options] table that are actually metadata, will be ignored unless listed as such. However, the same is not true for any backend-specific options, i.e. the rest of the [options] table, that would be listed in the corresponding [tool.setuptools] table in pyproject.toml, the backend would still be able to read those as normal (though it is up to the backend whether and how it handles that).

[topaz forge]

Ah yeah that's interesting

I bet it depends on the key

[sudden vine]

It doesn't, because it only talks about adding [project] metadata (which as discussed not the part is in beta), not any backend-specific configuration (which per @wanton burrow 's changes is generally no longer necessary for simpler projects using pyproject.toml for configuration, as the default behavior is smarter and more sensible). Also, Hatch is the "default" backend shown rather than Setuptools.

[topaz forge]

Ah great stuff

Hatch seems great by the way. I'm very excited to use it

[sudden vine]

@warped granite 👆

@warped granite implements new standards almost too freaky fast...first he implemented PEP 639 changes in Hatch almost as soon as I merged a PR to the PEP, then he implements it within hours of me opening a draft PR, what's next, he reads my mind and implements them before I do ? 😆

[stable estuary]

You're right

[topaz forge]

Well that's good right? In JS spaces the implementations are landed before the spec does

[topaz forge]

@sudden vine I wish I had a dedicated off topic thread in the off topic channel in every discord

[sudden vine]

Yeah, I mean its certainly better than the alternative—though implementing a draft too early can lead to backward compat problems later. cf. Chrome and WebRTC

At this point though it should be fairly stable

[topaz forge]

Ah well that's a market leader problem

The market leader in any technical space needs to take a step back for smaller faster systems to take a different approach

[sudden vine]

Though Setuptools implemented License-File IMO too early, which difers somewhat from the final version, which I've had to account for

[topaz forge]

And then copy the ideas

[topaz forge]

Yeah part of that is being able to pivot back to the spec freely

And not become so big that users get broken

[topaz forge]

@sinful cove re #1007030900489470068 message please feel free to tag me on GitHub rather than the @ graingert space whisper

Just saw this https://github.com/psf/black/issues/3169#issue-1305138566 and would have preferred to have been tagged

[sinful cove]

The intention was to ping you (and zsolt) later since July felt a bit early for determinating how to clean up our asyncio code :)

Given the next release may happen in 40ish days, I suppose it's due time to ping

[loud coral]

@sinful cove what's the base for migration to Hatch? isn't setuptools good enough?

[sinful cove]

https://github.com/psf/black/pull/3233 not with mypyc thrown into the mix :)

also hynek's fancy PyPI readme plugin for hatch is really cool (and useful!)

[warped granite]

yeah I just saw https://github.com/python-jsonschema/jsonschema/pull/983 such a great plugin 😍

[topaz forge]

setuptools is in this weird very good state

Sort of feels like once it's out of the ensurepip bundle it could have a big celebration of life and then everyone can do hatch from then on

@warped granite I think you misunderstood my tweets from yesterday. I just want a way to generate a pth from a function

[warped granite]

like a build hook that puts a .pth file in the wheel right?

[topaz forge]

Yeah well sort of

Maybe I mean hatchling?

[warped granite]

yeah that's possible

yes Hatchling

[topaz forge]

How would pytest-cov and future-fstring and future-annotations delete their setup.py?

And pthogen

I'm very sad pthogen doesn't work because entry points import too much stuff

[warped granite]

what does this do? https://github.com/pytest-dev/pytest-cov/blob/00713b3fec90cb8c98a9e4bfb3212e574c08e67b/setup.py#L79-L85

[topaz forge]

setuptools makes it inconvenient to put a pth file in a place

And so you need to bend it a bit

[warped granite]

oh yeah that'd be super simple with Hatchling