general

[woven plover]

@tidal kiln btw, whats the reason for formatting both author and author_mail fields together as list of name/email address combinations?

[golden falcon]

it seems like the obvious way to do it if you should be able to specify multiple authors

how do you think it should be?

[woven plover]

@golden falcon the metadata fileds hae the split between name and email (which is a mess in itself

pitting the combined data into all of them doesnt make sense to me however

[golden falcon]

I assume that you mean that you'd have to separate the name from the email for each author. PEP 621 says to keep them together in Author-email, omit Author

so in Author-email you'll have: Foo Bar <[e-mail]>, Bar Baz <[e-mail]>

[woven plover]

@golden falcon i see, then my understanding is, it should not go into author , only author-mail

[golden falcon]

provided that there's an email, otherwise it goes into Author

[mossy pulsar]

anyone knows a lock library? 🙂 something that can be keyed on the file path 😄

mainly thinking of replacing https://github.com/benediktschmitt/py-filelock/issues/84 🙂

I was thinking of for virtualenv/tox use case might be equivalent to have a system wide sqlite db with a lock table 🤔

perhaps could be even smarter than the current lock, e.g. release the lock when the app crashes (by having some kind of health check running in the background) 🤷‍♂️

[golden falcon]

well, you don't need a lock table for SQLite, you just need to begin a transaction 😛

[mossy pulsar]

indeed, likely could be a simple text file too, that could contain the locking process id and such; and have it a two phase lock acquire mechanism

anyone seen an implementation of this? 😄

[bleak forge]

Because name is not enough to ID the person? That's why people combine them together, and it would be nice if there was some PEP to support this scheme to avoid extra stumbling blocks when processing Python dependencies.

[woven plover]

Uh, my question was about how Trampolin processes the input data

Also I like that we don't have to worry about email everywhere, some authors don't want to publish emails as contact

[tidal kiln]

@woven plover any news on the setuptools_scm stuff?

I am gonna do some work on trampolim and would like to move the git versioning to use setuptools_scm afterwards

[woven plover]

@tidal kiln about to do a little on it now that the toddler sleeps

@tidal kiln why assign to both fields there ? https://github.com/FFY00/trampolim/blob/f3e8db8dea5607f708208b57de0d8bc49348a9d1/trampolim/_build.py#L425 and in the one below?

[tidal kiln]

yeah, I am moving to this https://github.com/FFY00/python-pep621/blob/f374fa2fe8f53fc3b24ef9e08b00870fef8fce30/pep621.py#L258-L261

I extracted the PEP 621 parser and made several improvements

[woven plover]

ah, thats looking correct ^^

[tidal kiln]

as I needed it in my meson build backend too

got to do that as part of $dayjob so 😊

[woven plover]

hmm, wouldnt it be convenient just to extend meson invocation a bit for all c gen needs?

[tidal kiln]

meson can already build python modules

projects like scipy are planning to move from setuptools/distutils to it

hence the need to have a PEP 517 backend that can take a meson project and package it into a wheel

my code FYI https://github.com/FFY00/mesonpy

[woven plover]

@tidal kiln btw, is there any documented way that a bulid backend can print issues/warnings to the user?

[tidal kiln]

you can just print

[woven plover]

btw, does meson have cython support?

[tidal kiln]

PEP 517 frontends may decide to show the output or not

pypa/build shows it

it does

[woven plover]

hmm, lovely

[golden falcon]

actually it doesn't look like Author is meant to contain multiple authors? https://packaging.python.org/specifications/core-metadata/#author

[tidal kiln]

hum

[golden falcon]

pep621 (the library) also probably shouldn't assign an empty string to Author/Maintainer when there aren't any

[tidal kiln]

that was leftover code

I should remove it now that it's not needed

if you provide an email, but not name, it will fallback to Unknown

[golden falcon]

oh and "email" should be lowercase: Author-email 😛

[tidal kiln]

if you provide a name, but not email, it will go to Author

oh, yeah

though, in practice the fields are not case sentisive

but still

should update that

[tidal kiln]

I wrote about the Debian patching of Python for the install locations

https://twitter.com/MissingClara/status/1443192988004503556

Laíns ⚡🌈 (@MissingClara)

so, I wrote a thing about the Python install locations on Debian https://t.co/o09vuHxzhR

Twitter · Sep 29, 2021, 12:36 PM

[peak ocean]

Debian patching of python

[torpid veldt]

This functionality is provided bt the site module,

Should be by

[tidal kiln]

thanks!

[bleak forge]

Is 4.0.dev0 > 4.0.24 ?

How to check?

[golden falcon]

Use packaging.version

[queen hornet]

>>> from packaging.version import Version
>>> Version("4.0.dev0") > Version("4.0.24")
False

[bleak forge]

Wow! Thanks, that helps a lot. )

[bleak forge]

Created this https://github.com/python/peps/pull/2108

[golden falcon]

if that list's meant to be in ascending order I suggest calling that out in the text

cuz it wasn't immediately obvious to me what the intention was with the PR

[bleak forge]

The chapter is called "Summary of permitted suffixes and relative ordering", although that is far away from diff context.

[golden falcon]

there's still no indication that the list is ordered...

all it says is, "The following example covers many of the possible combinations"

[bleak forge]

Need to write a function to validate the sorting.

[bleak forge]

It appears that PSF can use Open Collective for more funding options after all https://github.com/opencollective/opencollective/issues/4790#issuecomment-943792687

[lyric hedge]

What's the rationale behind removing the implicit add setup.py's dir to sys.path with the introduction of PEP 517? For context, I'm trying to answer this person's question on pypa/setuptools about this exact behaviour. https://github.com/pypa/setuptools/issues/2718#issuecomment-945994602

[lyric hedge]

Thanks @glass sand for responding!

[spiral urchin]

Another way to look at this is that importing from pwd was never a supported feature, but a side effect of setup.py being invoked as a script, so this is a good time to remove that. Pwd being importable sometimes caused unintended side effect e.g. the test directory or a local script clashing with stdlib or third party dependency

[tardy grove]

wag1 n shit homies

[lyric hedge]

So I'm experimenting with adding the PEP 660 editable hooks to setuptools, while conceptually most of it makes sense to me I'm not sure what I'm supposed to do with artifacts created by build_ext. My initial thought was to store the results of the extension build (within a temporary directory) within the editable wheel file, but the more I played around the more I had a hunch it would overshadow the source tree. An alternative is to do the in-place build in the source tree directly ... leaving artifacts :/ I guess pip's introduction of in-tree-builds means this is acceptable now (for at least setuptools)?

Hmm I suppose this would still be broken with out-of-tree builds as the extensions along with the whole temporary source tree would be deleted.

I hope I'm getting at least some of this right :)

[golden falcon]

I believe that's what https://github.com/frostming/setuptools_pep660 does, an in-place build

[merry rune]

Why is packaging version capped for pyparsing? This means packaging (which is supposed to be a tiny library behind lots of things) cannot be installed with anything that expects pyparsing >=3! That includes anything that uses packaging. Is there an issue?

[golden falcon]

https://github.com/pypa/packaging/issues/91

Actually, that was the issue that raised the min version to 2.0.2, someone then capped it at <3 but didn't update the comment in setup.py

https://github.com/pypa/packaging/pull/471

I see you've already commented there 😄

[torpid veldt]

@high stone you think worth trying a pep541 here? https://github.com/Teemu/pytest-sugar/issues/184#issuecomment-721049047

[high stone]

Yea, go ahead and file an issue.

[torpid veldt]

Always feels like getting the baliffs involved

@high stone done https://github.com/pypa/pypi-support/issues/1424

https://github.com/pypa/pypi-support/issues/1338#issuecomment-945079327 is the discovery process going to be one year from this message for suds?

[merry rune]

Possibly of interest to some here; I've written a proposal to move scikit-build to PEP 517 (and other things) https://iscinumpy.gitlab.io/post/scikit-build-proposal/ - looking for projects (mostly currently using CMake) who'd be interested in being listed as a use case (science-related ideal). Or just hearing what people think, etc.

[fallen shuttle]

Thanks @merry rune, very nice reading. I have a question: in your text you mention about setuptools that "the authors are trying to get out of supporting compiling raw extensions"; do you happen to have a link/reference for that? (As a setuptools user myself it would be nice to have a look to understand what they are planning for the future...)

[merry rune]

Ahh, that got pieced together poorly. It was originally written about CPython and Distutils. While setuptools has avoiding adding anything new to support C/C++ extensions for years, I don’t think it’s necessarily correct to say that. I’ll edit it next chance I get.

[fallen shuttle]

Thanks for the clarification!

[slow prism]

That mentions people.

[high stone]

Hey @tidal kiln, how ready-to-use is https://pypi.org/project/pep621/?

[bleak forge]

PEP621. Of course everybody remembers!

[tidal kiln]

I think it's fairly good

I am using it on two backends

the API might not be the nicest, it was born out of necessity

but I think it's usable

it has pretty good error messages, which was one of my main goals

[golden falcon]

which one is the 2nd backend out of curiosity? I know trampolim

[tidal kiln]

https://github.com/FFY00/mesonpy

I get to do that as part of $dayjob, so it's pretty nice

[golden falcon]

oh cool

I've heard generally positive things about meson

and it's nice to have another alternative for building ext modules

[tidal kiln]

yes

[bleak forge]

Some description about what is this project is about would be nice.

[tidal kiln]

which one? pep621?

[bleak forge]

Iep. I remember I've seen it already, but didn't an impression of what is was. Maybe it was another PEP..

[round rapids]

🤔 not seeing a pipenv channel... anyone know what's going on with https://github.com/pypa/pipenv/issues/4805 ?

[spiral urchin]

Basically only ReadTheDocs project admins can fix this and neither of them (there are two) are responding

[round rapids]

yikes, it's been almost a month... is there no way to get an admin added/replaced? e.g. looks like frostming has been active on the project lately

[mossy pulsar]

No way to add new

[round rapids]

wah wah... are the 2 current admins public info?

ah, yes they are: https://readthedocs.org/projects/pipenv/

thanks for the quick replies

[mossy pulsar]

Kenneth is basically inactive so you'll need to get Dan's attention

[round rapids]

Someone raised a decent point... Who controls the DNS for (pipenv.)pypa.io, and at what point do we rebuild the docs elsewhere and point there instead?

[mossy pulsar]

It's controlled by PSF @dense flint specifically

[bleak forge]

So, the bus factor is just 1?

[mossy pulsar]

Not for the PSF part 👍 but he's the primary user of these rights for now

[bleak forge]

But does he have the capacity? The bus (as in bus factor) is not only a physical thing when you have a lotsa responsibilities.

[mossy pulsar]

He does 👍

[round rapids]

is there a reason pipenv doesnt have a channel on here?

[spiral urchin]

Because no-one asked it to be created

[round rapids]

heh... how sensical :p

[lyric hedge]

TIL that egg_info and dist_info can return different versions

[golden falcon]

^_~

[lyric hedge]

yeah it's definitely fun to deal with when implementing a pep 517 hook /s

It wouldn't be too bad if the distribution version in dist-info was actually PEP 440 compliant >.<

[bleak forge]

As a person from authoritarian country I have a question - why PSF director is triaging PyPI issues? https://github.com/pypa/warehouse/issues/10283 I would really appreciate him officially answering the question why Wikipedia can accept https://donate.wikimedia.org/wiki/Ways_to_Give#Cryptocurrency donations and PSF can not? What is needed to make it legal? And also resolving disputes if PSF can use OpenCollective platform for projects it patronages https://github.com/opencollective/opencollective/issues/4790

[mossy pulsar]

He is maintainer of that project so he's entitled to that

As for the rest you want to reach out to PSF directly rather than asking your questions here

[bleak forge]

I reached, but they don't make any public comment for the reference. As you may see from OpenCollective thread I copy/pasted back and forth answers where people are basically saying "you're not right", "no you're not right" and that's not what I expect from an foundation that manages public money.

[mossy pulsar]

You're likely better off bringing the subject to PSF board

[bleak forge]

But I was writing to PSF address. Is there any other process to bring it to them?

[mossy pulsar]

The PyPA though has no direct relation with PSF so likely debating the topic here will not move the cogs much

[bleak forge]

PyPA can not accept donation because PSF is standing in the middle, and that limits potential participation. My PRs are not being reviewed for two months. This directly affects contributors experience. I could propose someone to sponsor my work, but what gives if there are no guarantees the work is even reviewed?

[mossy pulsar]

Which project PRs?

[bleak forge]

https://github.com/pypa/warehouse/pulls/@abitrolly and https://github.com/pypa/warehouse/pull/9972 in particular. I tried to keep it minimal, but then people asked to add more stuff, and now it is not being reviewed.

[mossy pulsar]

warehouse is a difficult project because only di has merge/maintainer rights there, and as you can see he's busy with other endeavours. Contributing there while welcome must be done with up most patience.

The wheels of open source are slow turning, until we get more active maintainers 👍

[bleak forge]

I would rather sit with him to plan how to growth the capacity for the warehouse, so that somebody else can do the issue triaging. The $300000 that PyPI fundraised don't get out of my head. 😄 Giving $50 a week to somebody to go through the issues and escalate them if needed is of course $2400 a year, but also only 0.8% of the budget.

My 0.02 cents about new maintainer is that it is hard to to impossible in 2021 to volunteer without any compensation. It will be either rich friends or (maybe no so rich) relatives who would support those who could be previously backed up by academia. Advances in management make sure that all time people have is spent well for the company, so the only way to get more maintainer is to build the capacity to get the out of these management practices.

I am only here, because I've got living parents and some crypto fortune. Otherwise I'd be spending hours in some outsourcing firm. And I am rather nervous that my time is limited, so if I won't merge this work, it will become abandoned, because my future owners won't allow me the privilege of spending my time on anything that is not covered by the contract.

[mossy pulsar]

There isn't someone else though

And funds raised are purpose raised meaning they are meant to achieve a given goal and not available for general spending

Also many issues require understanding the system and permission to change various parts of it, for which you can't just pull someone out of thin air for 50$ a week

[mossy pulsar]

Be that so, the entire core python and PyPA is run by volunteers, so goes against the current ethos of how Python works. I'm a maintainer of virtualenv/build/tox etc, but I'm getting 0$ to do all that.

The obvious downsides of this model is that things move slow because I do have a day to day job, and I do this in my free time, as almost everyone else around here

[bleak forge]

I am not able to maintain a full time job, that's why I am looking for an alternative way to build gameplay that compensates time for value.

[bleak forge]

That why I want PSF to use OpenCollective Projects (https://opencollective.com/opencollective/updates/introducing-projects) - so that people and companies could also openly donate to specific features they need in addition to pouring money to generic maintenance and administration fund.

[mossy pulsar]

AFAIK current fundings are for specific projects, but donated on a case by case basis by companies such as Bloomberg, Facebook or Google

[bleak forge]

I am not sure either. If warehouse had an OpenCollective, then there could be projects opened to see who donated to what specific features.

[tidal kiln]

I think you could write to the PSF, and CC di asking if there is something that could be done to fund this

it's hard though

[lusty quarry]

fyi I opened https://discuss.python.org/t/pep-610-should-metadata-be-updated-for-re-installations/11938

[lofty aspen]

https://drewdevault.com/2021/11/16/Python-stop-screwing-distros-over.html
I didn't see this posted yet on the server. (associated hn discussion: https://news.ycombinator.com/item?id=29238700)

[mortal shore]

I don't think the concerns about not caring about distros match reality

(I also think the idea that there is and will ever be a single size fits all solution to packaging is naivety

[mossy pulsar]

That post reads like a lot of venting but very little actionable proposal. It's easy and popular to thrash things, but hard to propose a solution that is actionable. And many people fall into the trap that they only care about their limited use case, and not cater for the wider community.

[spiral urchin]

I like how even HackerNews people, who are generally very resentful of Python packaging, find the author unreasonable

[golden falcon]

You could’ve omitted “of Python packaging”

[spiral urchin]

I mean you could mass replace every occurrence of “Python packaging” in the entire HN discussion page with anything and it’d still kind of make sense. The only real issue the author has is he’s using Linux distro package managers wrong, it just happens he’s using Python packaging as the scapegoat

[blazing lantern]

They are also missing the point of that XKCD comic https://snarky.ca/deconstructing-xkcd-com-1987/

[high stone]

Well, r/Python has a special kind of horrible bunch of comments. I did the reading so that you don't have to -- there's nothing useful there. :)

I shouldn've probably known.

(in context of yesterday's conversation)

[lofty aspen]

I just checked these out. I don't think they're so bad.

[merry rune]

General question. I'd really like to move the default manylinux image for cibuildwheel over to manylinux2014 (it's manylinux2010 currently). https://github.com/pypa/cibuildwheel/issues/909 Is there a community plan here for "recommended manylinux version", maybe a schedule, etc? With the demise of manylinux1 at the end of the year, it might be nice to have recommendations?

[spiral urchin]

Recommended for what? Different Manylinux versions are different platform definitions, none are not intrinsically better than others

[high stone]

Yea, the main difference is what is the oldest systems that would be supported.

I'd argue a good metric would be to stay ~1 year ahead of EoL dates for the "oldest still supported" base distribution for the image. So... I'd say the upgrade to manylinux2014 is overdue by about an year. 🙃

(2014 is RHEL 7 based, right?)

[merry rune]

I believe it's almost all based on Pip version; pip 10 removed pip.main, so distributions remained stuck on it for an unusually long time (CentOS 7, CentOS 8, Ubuntu 18.04). Yes, 2014 is CentOS 7

The problem is that we need a default image for cibuildwheel. I think 2014 makes more sense than 2010, since it can be consistent everywhere (special arches were added with 2014), it's more likely to work by a little bit, and it's much better for Python 3.10 (NumPy is shipping 2014 only for 3.10)

And I think from a packaging recommendation standpoint, if we wanted to recommend something, 2014 is pretty sensible. manylinux1 support ends at the end of the year, so it's really 2010 vs. 2014. Having a schedule might be nice (though really not looking forward to moving to 2_24, since that one is a big downgrade in terms of compiler, as NumPy found out after pushing for it)

[spiral urchin]

For this particular case 2014 seems to be the better one if you have to provide a default since CentOS 6 and Ubuntu < 14.04 have all gone EOL.

[lofty aspen]

@blazing lantern , @spiral urchin I love this proposal: https://www.python.org/dev/peps/pep-0665/
I don't track the ML so I don't know about the community sentiment, but it has a lot of sensible thoughts. e.g. using toml because pyproject is already toml (and avoids using configparser format which is gross and nasty imo).

Has anyone reached out to Poetry, Pants, or other tool communities about it yet?

How to Teach This
Nice lift from rust rfcs. great practice. 👏

[blazing lantern]

The projects I reached out to are listed at https://discuss.python.org/t/pep-665-take-2-a-file-format-to-list-python-dependencies-for-reproducibility-of-an-application/11736/10 and the projects that have shown interested are covered in the PEP at https://www.python.org/dev/peps/pep-0665/#potential-tool-support

[spiral urchin]

The How to Teach This practice dates a long time back and is listed in the official PEP template, it’s just too often ignored by PEP authors 😛

It could be worthwhile reaching out to Pants though! I should have thought of it

[lofty aspen]

Does the poetry reaction hold water? Seems weird that a lock file wouldn't be able to support sdists

[lofty aspen]

I've asked on their slack now. It's probably better to use a GitHub discussion though since it's easier to trace 🤷

[spiral urchin]

What reaction? AFAIK Poetry decided to not provide direct support way before we dropped sdist from the proposal.

Seems weird that a lock file wouldn't be able to support sdists
I’d put it this way: if we are must get sdist in, we’d still be drafting by November 2022, so I rather codify anything else first and deal with them later

[golden falcon]

the PEP's framed in terms of lockers and installers and, fundamentally, an sdist is not installable

so if sdists are accepted what we're saying is the installer has to double up as a build frontend

[merry rune]

I take it multiple platform wheels get multiple entries under [[package._name_._version_]] sorted by the filename entry. Would having an example of one such entry be helpful? Different wheels can have different requirements, which is why it's broken up that way?

[lofty aspen]

I feel that extras being in the key name isn't well defined. Maybe an example would help. I'm not sure but I suspect [] like pip uses might not work well in toml array definitions. [[package._name_[_extra1_,_extra2_]._version_]] surely isn't what is meant here

[golden falcon]

you can quote key segments in TOML, i.e. [[package."_name_[_extra1_,_extra2_]"._version_]] , but I don't know if that'd be necessary for [] mid-key

[spiral urchin]

Why do you want to write a lock file by hand in the first place? Any half decent TOML writer would handle this correctly

[merry rune]

If flit joins PyPA, then what about having a PEP 621 based setup tutorial on packaging.python.org for Flit along side the current one (or tabbed, or something)? It could even mention on the only flit specific part that other tools support this too, like pdm, trampolim, whey, etc.

[fallen shuttle]

Speaking about PEP 621, I have been playing around with the problem of adding support to it in setuptools, but it is not trivial 😛 (so I am mostly spending sometime getting used to the code base)

The proposal in pypa/setuptools#1688 is nice (in paper), but when I sat down to implement it, I found that parsing/reading configuration in setuptools is quite complicated. There is a lot of back and forth between the distutils.dist.Distribution class and the setuptools.config (which is the module that is supposed to be doing the parsing).

If anyone has suggestions about how to tackle that, I would be love to hear....

Meanwhile I have created some proof of concepts for the strategies described in pypa/setuptools#1688 and pypa/setuptools#2671:

• https://github.com/abravalheri/ini2toml
• https://github.com/abravalheri/validate-pyproject

[golden falcon]

I wonder if find/find-namespace shouldn't be an array, it's valid in setup.py to have multiple of them, right?

feel like namespace should be a flag as well

really nice work regardless

[fallen shuttle]

Thanks @layday, I agree with namespace being a flag and having a where that accepts a list as input... It feels more natural that way.

It is something in my radar. I think that since the idea is to "transpile" setup.cfg into pyproject.toml, we have room for a lot of improvements in a backward compatible way.

(For example, I am also in favour of having an implicit include_data_package=True by default going forward, now that the issue with this flag was solved. The ini2toml converter could add a include_data_package=False for the existing setup.cfg, this way we are still backward compatible)

I hope to get some time in the winter break to find a way of integrating these pieces into setuptools, and open a PR to get PEP 621 support done...

[golden falcon]

yeah, include_data_package by default would be a nice QoL improvement

[spiral urchin]

+1 to having a PEP 621 based tutorial on p.p.o (and also pypa/sampleproject as well?) Hopefully by the time that’s done, setuptools would be able to support PEP 621 to some degree and we’d be able to convert the tutorial entirely, and only starts introducing setuptools when extension is introduced.

[lofty aspen]

A lot of the current crop of packaging PEPs look to use common, easily digested config files which don't rely on pythonic customizations (as e.g. configparser would). Is there a pep for forward porting the packages on the server side? e.g. if we have the wheel metadata then surely a pyproject.toml could be generated for the package and then injected into the wheel (obviously making the hash of the package sad) - or automating PRs to the projects to make the changes? Is that a workable idea?

[spiral urchin]

A wheel is not supposed to contain pyproject.toml

[blazing lantern]

The Pants team reached out to me before we even went public with the first draft of the PR, so they are at least aware of it

Yes, that's why it's broken up per-file. And a more thorough example is possible, I just need a suggestion since I have to craft the example by hand as we don't have tools to generate the TOML yet 😁

[spiral urchin]

Lazy social-media-driven development: How can I install an egg in a virtual environment in December 2021? (zipped, not as a flat directory)

[blazing lantern]

If I remember correctly eggs can be put directly on sys.path thanks to them being a zip file, so the zipimporter will just pick it up and use it. I believe most drop a .pth file to make this an automatic thing.

[mortal shore]

Sometimes

[spiral urchin]

I should’ve provided some context. I want to test how eggs behave when being installed (IIRC they have special names and may have a different structure in their uninstalled or installed forms? I haven’t seen an egg being distributed for a long time and might be imagining things)

[mortal shore]

I'm not sure if you can actually install eggs anymore unless you use an old setuptools

[spiral urchin]

I know pip can’t, so if easy_install also can’t I’d probably just not support it. But I don’t see any mention from setuptools’s changelongs though.

[tidal kiln]

does the limited C API have an ABI tag?

I know about the d, m and u modifiers (eg. cp39m), but not the limited API

cc @quartz crystal

[blazing lantern]

Nope as it doesn't need one; the limited C API makes no guarantees about binary compatibility, just API compatibility. So you're still expected to recompile for each release, you just shouldn't have to change your code

[tidal kiln]

:/

that's a shame

[spiral urchin]

What do you need it for?

[blazing lantern]

What would it represent in the wheel tag?

[blazing lantern]

Jinx 😉

[tidal kiln]

that the wheel only uses the limited API, so it should work on all versions, no?

maybe I am getting something incredibly wrong here

😅

[blazing lantern]

The limited API doesn't mean anything at the compiled, binary level, so it doesn't provide anything for wheels.

The limited API just means we won't remove a function or change its call signature. But we can still change the order of fields in structs, remove private fields, etc. That breaks binary compatibility that binary code needs to rely on

[tidal kiln]

hum

I am getting confused here

Python 3.2 introduced the Limited API, a subset of Python’s C API. Extensions that only use the Limited API can be compiled once and work with multiple versions of Python. Contents of the Limited API are listed below.

is that wrong then?

[blazing lantern]

It's incorrect wording; that's abi3 for the limited ABI

Welcome to trying to communicate compiler details 😅

[tidal kiln]

ah!

[blazing lantern]

Admittedly we have gotten better about communicating this stuff lately due to Victor Stinner's work to more clearly separate all of this stuff out

So have the limited ABI, limited API, and the general public API

Clear as mud 😉

But you're referring to the limited ABI before we cleared this stuff up and got better at using the proper terms

I take it you're quoting the PEP? Or is this in the docs?

If it's the docs could you open an issue at bugs.python.org to clarify that's referring the limited ABI and not the API?

[tidal kiln]

https://docs.python.org/3/c-api/stable.html

how do I compile something for abi3 then? just define Py_LIMITED_API?

[blazing lantern]

Ah, key part is the next paragraph: "To enable this, Python provides a Stable ABI"

[tidal kiln]

yes

[blazing lantern]

Still wouldn't hurt to potentially clarify it in the previous paragraph

[blazing lantern]

Obviously that's between you and your build tool, but in general it should just be a normal build and then auditwheel should fix the name of the wheel based on detecting that you compiled with Py_LIMITED_API and using its value to know what to set the interpreter version tag to

But to be clear, I'm guessing since I don't build extension modules if I can help it 😁

[tidal kiln]

maybe I should rephrase, how do I manually build something against the stable ABI (ie. which options do I pass to the compiler)? does just Py_LIMITED_API enable this, or do I need to do anything else?

[blazing lantern]

Just setting Py_LIMITED_API before you include Python.h; a bunch of #ifdef in that file will take care of hiding all the APIs you shouldn't get to use

[tidal kiln]

then Py_LIMITED_API changes some declarations, right? I am just trying to make sure I get this right

because if you can change bitfield ordering and stuff from the limited API, then something needs to revert that when targeting the stable ABI

anyway, thanks for the help so far, it was really helpful!

[blazing lantern]

It at least hides some declarations

[blazing lantern]

Correct, if they were 1:1

[tidal kiln]

thanks!

[tidal kiln]

I am working on a Python build backend for Meson

[torpid veldt]

is there a table anywhere of python version to bundled pip/setuptools/wheel version for ensurepip?

[quartz crystal]

I'd welcome any clariflcations. It"s hard to explain this succinctly (and still use correct wording).
The stable ABI includes more than what's in the limited API, e.g. private functions called by public macros. Or functions removed from the limited API (nb. the limited API is not necessarily "stable")

[quartz crystal]

To build against the stable ABI, only use the limited API. As @blazing lantern said, declaring Py_LIMITED_API makes this easy by hiding the API you can't use. (Py_LIMITED_API isn't pertect, but you aren't very likely to run into the edge cases. Also, it does change some definitions, e.g. you get functions instead of some macros, but it doesn"t change behavior,)
Then put the stable API suffix in the compiled extension's filename, so all abi3-compatible Python versions will load it.

[tidal kiln]

I am a bit confused

if I understand it correctly, doesn't it conflict with what Brett was saying?

[tidal kiln]

this bit

[tidal kiln]

@blazing lantern do you know if anyone uses anything other than PEP 3149 (https://www.python.org/dev/peps/pep-3149) on Windows?

everything I have seen so far follows that PEP on Windows, but that PEP does not actually standardize the extension on Windows, it is optional

[blazing lantern]

['.cp39-win_amd64.pyd', '.pyd']

The old way is still supported, but I think most don't use it since it's more useful to users to include the details (e.g. shipping multiple versions together)

[lyric hedge]

ay welcome python discord folks!

[crisp mantle]

Hello, my first time using Discord so not use to the UI but I'm surely interested in learning more about py and anything related to. ichard26 thank you for the welcome.

[tidal kiln]

thanks

[lofty aspen]

https://discord.gg/python is a more general python server. This is the discord server for the python packaging authority (the people who make pip and pypi)

[bleak forge]

There are more people donating to unofficial PyPI page https://gitcoin.co/grants/3537/python-package-index and due to a lack of positive communication from the PSF side, we need someone from PyPA team with sufficient people skills and free time to get the issues PSF has against crypto into public discourse. And the first question - is $338 donated from 42 individuals with $493 matched by quadratic funding enough to start talking?

[torpid veldt]

Can you just donate it in dollars to the other funding methods?

[bleak forge]

Because donations in dollars that are poured in $ pools not transparent. I can't trust that $300000 donated to PyPI are well spent when I see my PRs screaming for attention. Platforms like OpenCollective (which PSF also refused to use) try to change that with Projects where you can donate to specific org and specific purpose. Otherwise, the $ pools are likely to become corrupted. It starts with the good case when they become the source of hidden funding for other small, but necessary initiatives, but then may attract causes like supporting core people that pass through hardships. I am not saying the people should not be supported. I am saying that it is a community affair, and having a pool that is easy to draw from, just make it susceptible to all kind of drainage that depletes resources for specific project goals. Crypto allows to regulate the degree of transparency and add decentralization to avoid corruption.

[mortal shore]

Crypto does none of that unless you mandate that the funds never leave crypto until they reach their final recipient. The reality is that the PSF would most likely convert to USD and then pay for things as typical, because as a non profit, there's a lot of reporting requirements, and their systems are already in place to deal with those for USD.

[obtuse holly]

how is it known that its unofficial?

[blazing lantern]

Because abitrolly himself is the only person listed on that account and he was told here that the PSF would need to okay accepting the cryptocurrency and the PSF has declined that offer. At this point I think the only option is for abitrolly to liquidate the funds (if they can) and make the donation themselves

[bleak forge]

You need to try to donate to see the warning.

[bleak forge]

That's why you don't need to convert the crypto until you need to buy a burger that would be too expensive if its price was fixed in Bitcoin. The reality is that PSF refused to discuss any doubts they have about crypto in public, including the option to accept and convert crypto to $. You can't help an authority who doesn't want to be helped. The only problem here is that you need to the authority to be helped, so that other people can get a permission to help you. This is how things are at the moment.

[bleak forge]

PSF neither accepted, nor declined that offer as there are no official and actionable statements. My strategy is to continue community fundraising until people will start questioning PSF why it is unable to manage crypto, and PSF starts talking. As I wrote to them - there are a lot of lawyers and different other useful people in crypto Python community that are willing to help, but PSF need to expose the problems in public. That's the goal. Not just converting the crypto to $, but actually decentralize development and funding efforts to be more effective.

I may, of course, die one day and then the donation will go wasted (if I rememeber how Gitcoin smart contract works), so that's why it is good to add more people into team. And also people with crypto and people skills to handle communication with PSF which I failed.

[mortal shore]

I’d personally much rather cryptocurrencies just stop existing since they solve no real problems and are contributing greatly to the further destruction of our planet. But I don’t control the PSF, so that’s just my opinion.

[bleak forge]

Then one'd rather people stop existing, because they solve no real problems and contribute to the real destruction of our planet, of which proof-of-work is only one tiny tool of many. Blaming everything on crypto is at least is unfair, given the efforts to switch to more energy efficient algorithms against human greed https://ethereum.org/en/developers/docs/consensus-mechanisms/pos/

[bleak forge]

In theory it is possible to run Python donations on a side chain, that doesn't use PoW.

[bleak forge]

Well, whatever, it is better to leave for #offtopic channel. I just want PyPI to merge my PR exposing metadata so that I get back at doing my own things.

[woven plover]

@bleak forge wtf,you basically create a unauthorised donation channel and now want to bully people into accepting it? That's highly unethical and annoying

@bleak forge if this was a bank account, then you'd likely rightfully be subject to criminal charges

[high stone]

Alright, with my moderator hat on -- folks, please keep any discussion about cryptocurrency, GitCoin and PSF fundraising stuff via those mechanisms in #off-topic.

If you think someone is behaving inappropriately, reach out to the moderators here, or [e-mail], or [e-mail].

takes off hat

Alrighty, I think most of the folks here would be the right audience for the poll in https://discuss.python.org/t/12451

Please do vote, to state whether you want a Packaging Summit at PyCon US next year.

[golden falcon]

If you are from a country without a visa free agreement travelling to America is such a hassle I personally wouldn’t bother.

It’s also on the other side of the world for many of us and prohibitively expensive to get to.

Remote sounds interesting though

[spiral urchin]

It will likely happen at a terrible time for my timezone anyway

[tidal kiln]

FWIW, PyCon should recommend you for a visa, which makes things a bit easier I think

https://us.pycon.org/2022/attend/information/

We can provide you with an “Invitation Letter” in support of an application for a United States visitor visa. The letter can be requested once you have a fully paid and confirmed conference registration. The letter confirms that as a registered delegate you are invited to attend by the Python Software Foundation (PSF) and the PyCon US 2022 committee. Once you have a confirmed registration send an email to [e-mail] with your registration ID, name, and address to request the letter.

don't know if it changes much for you

[golden falcon]

not for me personally but someone else might find it useful :)

[tidal kiln]

anyway, remote/mixed should be perfectly doable

we had a sprint on packaging, distro issues in specific, last year and it went fairly well

a PEP came out of it

[spiral urchin]

Remote should work, events less so. Mixed conference generally means on-site conference + organisers pretend they have remote attendees

[tidal kiln]

I thought it would be remote mainly with on-site PyCon attendees just working together in a room

less communication overhead between the on-site attendees

but the event would be basically organized as remote

[high stone]

I'm wary of making it hybrid -- that results in splitting and treating the two categories of people differently, especially for the sort of stuff that we'd want to do.

[bleak forge]

If I show you the link where people ask me to do first and then talk, will you drop your criminal charges?

[bleak forge]

Yea, I've got just $900 to go on, so if I find a job, I will have to abandon all the stuff I did to expose metadata for PyPI. And if I won't... well.

And now the blocker for the metadata is the zip bomb CVE problem in zipfile, which was "solved" with documentation comment. 🤷‍♂️

[woven plover]

if there was any sort of encouragement from stake-holders, then i humbly apologize, curious to see what/how was suggested to, but lets push that to #off-topic

[tidal kiln]

https://discuss.python.org/t/packaging-standards-and-peps/12503/7?u=ffy00

if anyone has any thoughts, please let me know

I would like to write up a proposal to send to the SC

[lusty quarry]

since we're all standardizing on tomli, what do we think of this? https://github.com/hukkin/tomli-w/pull/27

[bleak forge]

My thought is that the problem is communication, not standards.

[tidal kiln]

standards are also an issue

see the discussion about the editable wheels (if you are brave enough 😛)

but this is something that could also help with communication with the python core team

[bleak forge]

is there a one sentence summary of the feature? editable wheels sound like editable squashfs (.snap) and similar formats. why?

[tidal kiln]

so that you can develop things without re-installing

[lusty quarry]

is PyPI having an outage?

yeah, pkg versions are being pulled from a very old cache

oh, https://status.python.org

[bleak forge]

If you mean wheels that are installed, but not your project, then a better way is handle this is to restore the sources history given the hash.

[golden falcon]

'Editable wheel' is a bit of a misnomer. During development, especially if you are working on a library, you might want to install your package. To avoid having to reinstall it every time you make changes to the code, instead of the code being packaged in the wheel you create a reference e.g. a pth file which places your source on the Python path. The pth gets packaged in the wheel and is installed by pip.

This is all done automatically for you when you pip install -e

[shell glen]

Good evening, if anyone has a Mac M1 (i.e. arm64) & macOS 11.x, please could you run codesign --display --entitlements - --verbose=4 /usr/bin/python? It displays code signature of the bundled Python 2.x binary. I'm trying to make progress on https://github.com/pypa/virtualenv/pull/2233

[woven plover]

in wondering, are there any minimal docker container for recent twine with a recent python - i'd like to simplify the pypi upload, as well as distribution checking actions on github

[bleak forge]

pip install -e already works as described, so what is the particular problem with wheels?

[tidal kiln]

no particular problem

but editable wheels was the solution reached for standardizing editable installs

[golden falcon]

The wheel is the only installable artefact in the new packaging order. Install -e previously only worked with setuptools by running setuptools commands which are deprecated

With editable wheels any backend can support editable installs

[bleak forge]

But the editable install is a source code that needs to be built anyway, so the editable wheel becomes a task of build system where installation is just a last step with pip install --no-deps or (with deps if cache is invalidated).

[bleak forge]

https://hub.docker.com/search?q=twine&type=image&sort=updated_at&order=desc there are some, but nothing official so far.

[golden falcon]

sorry, I don't know what you mean

[tidal kiln]

yes, but built only one time

when you make changes to the code, you don't need to rebuild and reinstall the editable wheel

[bleak forge]

Nevermind. So are you saying that install -e doesn't work with wheels?

[bleak forge]

That would work only if source code that you're editing maps 1:1 to wheel contents, no?

If a person starts development process with a wheel, it may not contain all the original code that was used to build it. How it is going to work then?

[golden falcon]

have you checked out PEP 660?

It works only with wheels, the backend is asked to build a wheel for development only and pip doesn't cache it. The backend can insert a pth file or use something like https://github.com/pfmoore/editables to map files dynamically at runtime.

[potent gust]

I posted a question in the Setuptools GitHub discussion that hasn’t garnered any conversation yet so I thought I’d repost it here. Namely is there a reason why relative paths as schemeless URIs are not supported by pkg_resources.Requirement despite the grammar outlined in PEP-508 supporting it? https://github.com/pypa/setuptools/discussions/2951

[golden falcon]

does setuptools validate install_requires with pkg_resources or why does it matter what pkg_resources is able to parse?

[potent gust]

Yes it calls pkg_resources.parse_requirements which tries to instantiate each item in install_requires as a Requirement

[golden falcon]

I also don't think the intention is for relative URIs to be used as filesys paths - what is the URI relative to? there isn't a base URI to compare it to

I suppose the parser could take the base URI to be file://{os.path.dirname(of_pyproject_toml_file)} and resolve relative refs based on that but the PEP says nothing about assuming the existence of a base URI :)

[potent gust]

im trying to find it but I thought i read somewhere that schemeless URIs are assumed to be file paths and yes i feel like they should be relative to the toml file

[golden falcon]

And resolving relative to your setup.py/pyproject.toml wouldn't always be appropriate

if you tell pip to install foo @ bar/baz then should it resolve that relative to pyproject.toml, what if there isn't one? the cwd? you'd have to attach special meaning to dependency URIs depending on where they appear

[potent gust]

hmmm yea i mean i can see possibilities for edge case for sure. In this case I think it would resolve the requirement relative to the cwd and if it cant find that you raise.

I believe there is value to be able to specify dependencies using relative paths and this is something that is done in rust and other python build backends like poetry but I'd hope for a standard in how it is done. In my initial read of 508 I read it as supported but maybe path_noscheme is not well enough specified.

[golden falcon]

path_noscheme doesn't mean anything different than it does in the URI standard, the whole grammar is taken from the URI standard

[potent gust]

sorry i mean what a requirements parser should do with it is not defined

[golden falcon]

Google brought up this 2yo discussion about relative URIs: https://discuss.python.org/t/what-is-the-correct-interpretation-of-path-based-pep-508-uri-reference/2815

[potent gust]

oh this looks like a helpful discussion, surprised i missed this

ok so seems like the consensus was to disallow relative paths in install_requires.

Seems like the biggest sticking point would be distributing such packages. Like relative paths work fine when you are developing but if you want to build up all packages into wheels then your going to have a problem. So then you need to provide one requirement for an editable install and another if you wanted to install from wheels.

I think in my ideal world I could install packages editable that are referenced relative to each other for development but then build all the wheels and install from a directory of all the required wheels using no-index and find-links for distribution

Maybe it’s better to just provide a container at that point

[woven plover]

hmm, what would be necessary to make a pep to drop the version numbers off the dist-info folders and as a second level move dist-info folders from $packagename-$version.dist_info to dist-info/$packagename/

(context being wanting to enable direct lookup and faster metadata walking)

[golden falcon]

can you not pip install -e . -e ./foo -e ./bar? Bit verbose I suppose but it should work

[potent gust]

Yea this works for a relatively small number of packages. It just gets unwieldy when you are working in a monorepo where you might want to have many packages. So then you end up writing wrappers around pip and setuptools and soon enough you decide you need a custom build backend and frontend or you use a tool like bazel or pants. So really I was trying to figure out if there is a relatively lightweight way to have a Python monorepo using the Pypa ecosystem.

Rust’s Cargo workspaces are great and I’d love to have something that worked like that which is well supported and used by the community.

So my current solution is I don’t actually have multiple packages and everything is built as one. Which is ok right now for my present use case but it’s not going to scale

[spiral urchin]

Dropping the version part is likely not viable, there are too many tools depending on the format. But proposing putting all metadata in a separate directory should be possible, and dropping the version part inside that directory should also be totally doable (whether it’d be received positively is another issue)

[woven plover]

i wonder how many tools that "depend on it" are actually just either using importlib/packaging to begin with, or could use aport to those libs

[spiral urchin]

The mere fact they all need to be updated to use newer versions of those libraries is enough pain all on itself. importlib.metadata just got stable in 3.10, and there are a ton of things out there vendoring either the backported importlib-metadata or setuptools’s pkg_resources (packaging does not have functionality to read dist-infos)

And the fact that importlib.metadata got stable in 3.10 might be a problem on its own as well since you now have people depending on stdlib which can’t be upgraded on its own to receive this new layout

[woven plover]

hmm, god old stdlib deathtrap ^^

[bleak forge]

I think that's a dead end, or at least incomplete. The next thing I want to do if I decided to edit a wheel is to make modifications and track what I did with version control, so I will have to do git checkout and install editable non-wheel from there anyway.

I wish PEPs had less academic format - instead of Abstract+Motivation+Rationale just contain "The problem". There is a whole industry of startups that appeared in the last 10 years and if they did something good, it is the best practices on problem presentation.

The best description of the problem I could find in PEP 660 that it tries to solve sounds this "we need a new mechanism to install packages in editable mode". Huh? It is not a problem. It is a solution, feature, somebody's wish, but not a problem.

[golden falcon]

You are not editing the wheel

[golden falcon]

If it helps just think of it as symlinking your source code in site-packages, that installation is performed using a wheel is immaterial

[woven plover]

its important to proof that a problem is actually real, forcing people to write more details helps to wed out a lot of bad takes - writing helps to think, so taking shortcuts in writing sometimes is just taking shortcuts in thinking, thats sometimes a really bad thing

editable weels for example sove the actual real problem of having a install artifact that puts what you edit into the state of the interpreter, there are still holes (like metadata) but those holes already existed in worse and more error prone precursors, so its a direct enhancement

[glass sand]

In the latest setuptools commit, Setuptools' readme has a badge for the discord channel (https://github.com/pypa/setuptools/). Have a peek and consider doing the same for your readme.

[golden falcon]

has the IRC/Matrix channel been abandoned?

[glass sand]

I'm not familiar with Matrix and I'm out of touch with IRC. I was using Gitter, but informally for Setuptools.

I'm not even sure if PyPA is on freenode or something else.

[golden falcon]

it's moved to Libera after the whole Freenode fiasco

[glass sand]

Yeah, that fiasco ruined IRC for me.

I lost access through my relay provider and haven't had the energy to reconcile everything.

[golden falcon]

yeah, I don't go on IRC anymore either. Somebody set up a bridge on Matrix, #pypa is accessible from there.

[woven plover]

same here, alltho i wish it would go all matrix

[lyric hedge]

I really did cause all of these new members to join from Python Discord huh.

[shrewd tusk]

Haha yea

[obtuse holly]

where did you post it lol

seems like it was an internal channel

[spring hawk]

Oh yes, staff invasion!

[keen copper]

Is this the right place to ask about pypiserver?

I wonder why the default behaviour is to fall back to the global PyPI

Clients can set up a fallback themselves if they want to, so isn't this somewhat insecure?
Maybe the description could've been more explicit about it.

[lyric hedge]

Hmm, only PyPA projects are present here in somewhat decent numbers but non-PyPA projects are by-no-means expected to be here. pypiserver isn't a PyPA project and I don't see the maintainers in the members list either.

[keen copper]

Ah ok, sorry then

[lyric hedge]

Non-PyPA projects are welcome here (see the packaging related channels lower in the list) but that's up to the maintainers and there's no expectations on them whatsoever to have a presence (same thing with PyPA projects for the most part)

[lusty quarry]

seems like an incident beginning https://status.python.org

[woven plover]

@tidal kiln id there any timeline for pep621 to merge into packaging? im toying with making gumby elf use it

[tidal kiln]

I don't think it will merge into packaging

I think packaging will probably get a different, better designed, implementation

not that I think there's anything inherently wrong with the pep621 API, but it was born out of necessity, so it did not went through the same API design process as stuff in packaging

[woven plover]

i see, so i willl use it as a starting point until packaging gets a reiterated version

[tidal kiln]

👍

I think it will probably take a while until that happens, so that makes sense

[woven plover]

@tidal kiln any idea why the editables project left out support for making normal wheels, practically it just means i'll keep the hack i have in gumby elf :/

[golden falcon]

what do you mean by normal wheels?

[woven plover]

@golden falcon aka building wheel files with files added instead of map hacks

🤦 i did misread its documentation, basically its hack seems worse to me than what gumby elf did 7 years ago (as they need different runtime support)

[golden falcon]

it looks like they do more or less the same thing, run a module from somewhere else on disk? editables uses a meta path hook and gumby elf execs the original module but the result is the same

[tidal kiln]

how does editables inject the meta path hook?

[golden falcon]

it uses a pth file

[tidal kiln]

hum

that is not optimal

but makes sense

thanks

[woven plover]

it basically adds a runtime dependency that always has to run ^^ while gumby elf oly execs the code once in the required place

@tidal kiln do you have a implementation for the creation of the entrypoints.txt file somewhere?

[tidal kiln]

https://github.com/FFY00/trampolim/blob/aed1982f019fbfe1811dc0a03739a2f4dd19662b/trampolim/_wheel.py#L58

does not support entrypoints with dependencies

I mean, that bit doesn't care

the the supporting code does not implement it

[woven plover]

i see, so gumby elf is gonna keep a own for now

[golden falcon]

I use https://github.com/layday/frontend-editables for my own projects which I developed for PEP 662 (since rejected), it's simple and it works

[woven plover]

hmm, cant be used for gumby elf ^^

[woven plover]

finally after 7 years, gumby elf made its first wheels/editable wheels based on modern peps 🙂

[woven plover]

anyone aware of a attempt for autoformatters for pyproject.toml?

[mossy pulsar]

prettier can format TOML

nothing more specific

[woven plover]

hmm, it seesm the prettier plugin for toml hasnt changed in 3 years

[spiral urchin]

I wonder if a pyproject.toml formatter (not just TOML in general) is in scope of Black

[golden falcon]

you could probably hack a formatter on top of tomlkit

[fallen shuttle]

If someone manages to write the TOML in terms of the intermediate representation of ini2toml, https://github.com/abravalheri/ini2toml/blob/main/src/ini2toml/drivers/full_toml.py should be able to get it pretty far by using atoml (in the future it would be moved to tomlkit as frostming is working in porting over the enhancements)

[golden falcon]

nice!

[woven plover]

looks interesting, does it have extra code for setup.cfg 2 pyproject.toml already ? i could really use that to ship a helper in gumby elf)

(context is - gumby elf or pure python is practically working, but i want to bring back the cli and helper tools for converting legacy projects)

[fallen shuttle]

Yeap, I implemented the first version of the PEP 621 PR in setuptools using it. Of course there might be some sharp edges, but any feedback is welcome.

Those are examples of the automatic transformation: https://github.com/abravalheri/ini2toml/tree/main/tests/examples

[woven plover]

ah, lovely, thats a starting point, bascially i think i can add steop on top of it to support gumby elf that way

hmm

[fallen shuttle]

that would be awesome

[woven plover]

it needs some enhancements, parts of the toml is slightly icky

https://github.com/abravalheri/ini2toml/blob/main/tests/examples/setuptools_scm/pyproject.toml#L34-L37

[fallen shuttle]

You mean, reducing the line length in this example?

I can work on that 🙂

[woven plover]

also it might be nice to have a good well defined order

so build system first, then prpoject, then other bits

its less about line lenght, and more abotu what ought to be sections, and what ought to be inline

it mayy be a good idea to work out some pyproject.toml formatting rules, and maybe make a pep about the formatting suggestions later

for example authors arrays really ought to use multi-line for fun reading

[fallen shuttle]

I see, right now there is an heuristic for line length to decide between both. But it does make sense to force the entry-points to have their own separated table

[woven plover]

project.urls is also better as a section

and you mgiht want to ensure some oder of the sections and the keys as well

[fallen shuttle]

Currently the order in the sections are: project, build-system, tool

[woven plover]

my personal recommendation would be to start with build system, as its small and has no sub-sections, then have many sections for project.* and tool.* --

[fallen shuttle]

sounds reasonable

[golden falcon]

eh, I prefer urls.Homepage = ...

[fallen shuttle]

I will fill some issues with these items we have discussed

[woven plover]

@golden falcon thats a good and important point actually ^^

a survey of different oppinions and determining a best practice my be necessary - i do agree that urls.* is a nice way

[golden falcon]

I generally tend to avoid sectioning things off in sub-tables, it makes the information harder to consume

[woven plover]

it makes sense for entrypoints, and optional dependencies

[golden falcon]

yeah, those two it wouldn't make sense to inline

[woven plover]

also for dependencies, if the item is complex, having a newline may be nice (see https://github.com/abravalheri/ini2toml/blob/main/tests/examples/pluggy/pyproject.toml#L28)

[golden falcon]

I would write that as 'importlib-metadata>=0.12;python_version<"3.8"'

I guess tomlkit isn't clever enough to switch between quote styles

[fallen shuttle]

Unfortunately not yet 😅

It is also conservative because single quoted strings do not perform escaping.

[fallen shuttle]

tomlkit/atoml will add multiple lines for arrays with more than one line

[fallen shuttle]

Some related discussion but not direct on this topic:

People usually mention in discussions (e.g. giving the PKG-INFO/METADATA a new format) that one of the strengths of TOML is its multi-line string ability...
If the stdlib ever gains support for writing TOML, it is likely to not support this feature, by the reasons described in https://github.com/hukkin/tomli-w#why-does-tomli-w-not-write-a-multi-line-string-if-the-string-value-contains-newlines.

If people/tools want to automatically write TOML, they would have to pull a much heavier dependency like tomlkit to support multi-line strings... (which is not a problem for ini2toml of course, because it is already doing that).

[fallen shuttle]

"multi-line inline" tables would be handy here (https://github.com/toml-lang/toml/issues/781)

[woven plover]

@fallen shuttle what i meant was forcing multi line even for single element makes sense in certain cases

[fallen shuttle]

I agree, my plan is to add some other heuristics depending on the key name

Currently the heuristics are all based on the (estimated) size of the line/number of elements

[woven plover]

Btw, using the converter tool as formatter would be nice

For setup.cfg there is a pre commit hook, I think one for pyproject.toml would be nice to have as well

[fallen shuttle]

I am open to collaborate on that (I don't say I would lead this effort, because I want to focus some energy on setuptools, so we can catch up the latest standards, and unfortunately I will not have much extra time left...)

The first step though is to submit a PR to tomlkit to use 2 spaces to separate inline comments (instead of one) 😛

[queen hornet]

I've not tried them, here's a couple listed on https://pre-commit.com/hooks.html:

https://github.com/macisamuele/language-formatters-pre-commit-hooks

https://github.com/pappasam/toml-sort

[shell glen]

If anyone has a Mac M1 (or M1 Pro, M1 Max) running macOS 11 (Big Sur). Please could you try the following and pastebin the output or add it to the ticket below

virtualenv --python /usr/bin/python3 v38
v38/bin/pip install -U virtualenv==20.13.0
v38/bin/virtualenv -vvv --with-traceback --python /usr/bin/python v27
v27/bin/python -c "import sys; print(sys.version)"

I'm trying to narrow down https://github.com/pypa/virtualenv/issues/2023, I want to see if a Cpython 2.x virtualenv on arm64+macOS 11 works. I know it fails on arm64+macOS 12

[blazing lantern]

https://github.com/pypa/packaging/issues/383 is tracking a proposal for a general metadata API in packaging, but enthusiasm for the idea died out so I don't know if it will still happen. I have contemplated adding it to one of my personal packaging projects (mousebender), but I am also cognizant that I'm starting to spread myself thin

[woven plover]

I see, maybe some iteration in the pep621 project will be a help,

I also want to be able to easily generate the Metadata, fill in dynamic + update files

[woven plover]

@tidal kiln how do you feel about collaborating on having a set of simple functional/oop style tools surrounding pep-621 to deal with generating metadata targetting spcific wheel versions?

[blazing lantern]

Yep, that's what I want as well. Basically there should be an object representation of metadata that you can create from pyproject.toml, PKG-INFO, METADATA, or programmatically, and then a way to export/emit back out to those formats (plus JSON since that's is a specified in rejected PEP already and so cheap to support in case people want it)

[woven plover]

ok, im going to put that into the backlog however, my first priority will be to get gumby elf released and starting to port projejcts to it plus setuptools_scm git archive support

[bleak forge]

Is there a list of syntax comparison examples that shows where setup.cfg is bad and TOML is good and vice versa?

[fallen shuttle]

There was an investigation once when PEP 517/518 was being written... let me try to find it out

https://www.python.org/dev/peps/pep-0518/#other-file-formats

https://gist.github.com/njsmith/78f68204c5d969f8c8bc645ef77d4a8f

(of course that is a bit biased by the author's preferences specially in terms of "make humans happy", but in general it is a good sum up)

[golden falcon]

I wish we had a saner Yaml like language, I find it to be the most readable by far.

[fallen shuttle]

Yeap. Personally I think that the situation with TOML's AoTs (and general nesting/combinations of tables and AoTs) make humans cry... But on the other hand sometimes I do need to convert YAML to JSON just to make sure I am getting the nesting right

[bleak forge]

Well, there was an attempt to write a lite YAML parser without types where everything is a string or a list, or a map, but writing indentation parser is non-trivial.

I would start this as collecting a database of snippets that people would like to write, and then see at which use cases parser and format start to break or become unreadable. Because there might be some invisible fundamental problems that need a name.

https://www.python.org/dev/peps/pep-0518/#other-file-formats section on configparser kind of skips the problem with defining what is the desired format.

[build]
requires =
    setuptools
    wheel>=0.27

https://gist.github.com/njsmith/78f68204c5d969f8c8bc645ef77d4a8f#configparser write up on the other hand is straight to the point. Define how to treat comments and enable utf-8, and it will be good to go. Unless somebody wants to write deeply nested structures inside of .cfg.

[fallen shuttle]

I believe that writing deeply nested structures in TOML is cumbersome for humans anyway...
The [x.y] header kind of works as an instruction, and will do different things depending on a previous definition of x (AoT or table), so you have to mentally keep track of "the state of the parser" when writing TOML files....

[woven plover]

well, its a tradeoff, im not happy with toml, but ini or yaml would be so much more of a hell of a pain

[fallen shuttle]

Changing a bit of topic, I am reviewing some issues and discussions in setuptools, specially regarding the deprecation of running python setup.py directly...

One recurrent point (see https://github.com/pypa/setuptools/issues/2971, https://github.com/pypa/setuptools/discussions/2995#discussioncomment-1911256) is that distributions and upstream packagers still rely on it for building and installing documentation, and there is still no clear answer about that...

I wonder tackling this issue would require any standardisation effort (e.g. adding a .docs directory to wheel, another PEP517-like hook for building the docs, another docs entry in the distutils-like installation scheme mapping apart from data)

Does anyone have any ideas about how to handle this?

[mossy pulsar]

building the docs is not different than running the test suite 🙂

there's no plan on adding that to the pyproject.toml python standards as of today, when this came up the conclusion was that we should probably standardize a subset of the tox, and in the meantime use tox to achieve both

[fallen shuttle]

Thanks Bernát, I think the building part of the docs is something that could be "worked around" while the approach you mentioned is not standardised yet (personally I always resort to tox to build the docs anyway)...

The other part of the problem is that OS-distributions/upstream packagers like to treat docs (and sometime tests) as artifacts that need to be distributed and installed.
Currently for the distribution part, the common practice is to use sdist, which "gracefully" handles docs (by being opaque to the nature of the contents), but the installation part is tricky.
As far as I know the majority of upstream packagers seem to rely on python setup.py to install the docs, and that is another problematic part...

[golden falcon]

do they create manpages out of sphinx docs or in what format do they expect these to be in?

[fallen shuttle]

It seems that they create manpages using sphinx docs

[golden falcon]

do you have an example of a distro that does this?

[fallen shuttle]

The example I have in hand targets Solaris packages (https://github.com/pypa/setuptools/discussions/2995#discussioncomment-1912823) and RPM (https://github.com/pypa/setuptools/issues/2761#issuecomment-1001238283) (https://github.com/pypa/flit/issues/429)

[blazing lantern]

I have broached the subject of trying to standardize the way to discover/run the commands for testing and building docs, but pretty quickly people say "no" due to the desired flexibility that people want (you might be able to argue doc building could be simple enough, but do you handle building different output formats, linting, etc.)

[golden falcon]

I don't know, is this something that other languages support? Is it something that distros actually do? It seems fairly idiosyncratic. Do people read up on Python APIs in man pages? Do they install Python packages for development using their system package manager?

[woven plover]

@fallen shuttle the user tht posted those is kind of a menace to many projects tho,

[woven plover]

i recall the question for pytest - and it seems that if we can actually get tools to agree on a additional standard it may be possible differently

[fallen shuttle]

is this something that other languages support?
I did a quick search here so this information might be incomplete, but bundling docs together in the package and providing access to them via some command in the terminal / REPL is something (at least) the following languages do:

• ruby
• elixir

Is it something that distros actually do?
I am enquiring the user to see exactly which distributions are using this system

Do they install Python packages for development using their system package manager?
That seems to be a thing... If we look at ubuntu or arch there are hundreds of Python packages there, repackaged to be installed via system package manager

[fallen shuttle]

Thanks Ronny, I went through the other issues opened by the user in flit to check...
It does seem that there is some resistance to the proposed solutions.

I am just trying to understand here if this is a valid use case that should be tackled or something we (as a community) don't have to worry about

[woven plover]

lets wait until a valid use-case comes about (aka not that particular person)

[fallen shuttle]

Yeap, I agree with that, meanwhile I will try to get more information about this topic.
Thank you very much everyone for all the insights!

[golden falcon]

That seems to be a thing... If we look at ubuntu or arch there are hundreds of Python packages there, repackaged to be installed via system package manager

Well, typically, those packages are there because something else depends on them, a component of the system, a CLI or a GUI, something of the sort. Perhaps there's an argument to be made that generating man pages is useful to the maintainers (e.g. for debugging, for developing the distro itself) but I don't think distro-provided man pages are useful for developing Python packages more broadly.

[golden falcon]

It would be pretty cool if pip was able to install docs on demand though. pip install --with-docs setuptools and then man setuptools or man --html setuptools. It doesn't have to be man either. Granted we'd first have to get people publishing docs in wheels as data files if people are going to be able to depend on it

[fallen shuttle]

That is what rubygems does.
The documentation is automatically installed with the package (e.g. via gem install package) and then can be accessed in the terminal (e.g. via ri package) or local webserver (gem server). Ref: https://guides.rubygems.org/rubygems-basics/#viewing-documentation

(the only difference is that instead of an opt-in switch, there is an opt-out, --no-document)

[golden falcon]

That's cool, do ruby gems typically come with documentation?

Are they contained in the gem, is it a separate artefact?

I'm sorry, I should stop asking and check for myself 😛

[woven plover]

didnt pypi drop documentation upload support?

[golden falcon]

It looks like they are generated on the fly from source code with RDoc. And you can even gem serve to run a local HTTP server with all of your installed gems docs in HTML format

[fallen shuttle]

No problems 😝

That's cool, do ruby gems typically come with documentation?
AFAIK, ruby default documentation tool has support for doc files with the format .rdoc and also scan by default code comments (which would be similar to sphinx-autodoc).
So even if the authors don't write docs, there will be some amount of auto generated docs (I suppose at least the function signatures)

[fallen shuttle]

Yes, precisely... so in some sense, they are contained inside the same gem artifact, but in the .rdoc format, that gets compiled during installation (and that compilation takes AGES...)

[golden falcon]

Yeah, for Python, I'd want the docs to be pre-compiled and distributed with the wheel. You could do it with any backend which supports build steps I imagine.

[woven plover]

are there any rules/suggestions for not yet sdist supporting build backends?

[peak ocean]

I don't know, is this something that other languages support? Is it something that distros actually do? It seems fairly idiosyncratic. Do people read up on Python APIs in man pages? Do they install Python packages for development using their system package manager?
Debian generally aims to build and ship documentation with its modules. This allows a developer to access the documentation offline and provides some privacy benefit.
Debian aims to have manpages for all installed command line tools, but not Python APIs. Ideally these are built from upstream documentation source, but sometimes they are auto-generated from --help or manually maintained by the package maintainer.
Some people will do Python development with their system package manager, although I think this is less common than it used to be.
Personally, I use system packages for simple scripts where I'm using very common and stable libraries (e.g. requests), but almost everything that makes it into a git repo will have a requirements.txt and probably use a virtualenv. It's just lazyness...

[golden falcon]

Thanks @peak ocean. I imagine there's a lot of noise in those man pages unless you extract the CLI portions of the docs (if those exist) only.

[keen yew]

we banned him in urllib3, I think it was a great decision :)

[golden falcon]

There’s something slightly undignified :) about banning someone

[woven plover]

it can be a slippery slope, but it sometimes is necessary for mental health, the user in question gives-off the vibe of someone that claims 20 years of experience, when they juts had 80 repeats of the same quarter , also already drained me of the last bit of goodwil towards them

[peak ocean]

@golden falcon Yeah, it's only going to come from upstream documentation if there is a CLI page of the docs that matches manpage conventions.

[lusty quarry]

I'm curious our thoughts https://discuss.python.org/t/version-normalization/12943

[golden falcon]

Well, that's a misleading name for that function. It doesn't canonicalise the version, i.e. by stripping local version identifiers, it normalises it

The wheel spec says "Version numbers should be normalised according to PEP 440. Normalised version numbers cannot contain -." (https://packaging.python.org/en/latest/specifications/binary-distribution-format/#escaping-and-unicode). PEP 440 does not say that trailing zero segments should be stripped during normalisation (https://www.python.org/dev/peps/pep-0440/#normalization), so the function from packaging does not normalise versions according to PEP 440

Here's what flit does: https://github.com/pypa/flit/blob/main/flit_core/flit_core/versionno.py
Poetry: https://github.com/python-poetry/poetry-core/blob/af08f1ce720da467c9bf3d43eed3d9ebaf4ad7fb/src/poetry/core/version/pep440/version.py#L111

[lusty quarry]

ok ty, I'll fix it then

[tidal kiln]

anyone want to give feedback on this? https://github.com/FFY00/mesonpy/pull/24

[golden falcon]

Maybe add a simple meson.build as an example? Just to be able to get something running from the readme

[keen yew]

Has anyone else seen segfaults when testing Python 3.11 alpha 3? I know at least urllib3 and cryptography did, and this happened because pip cached Python 3.11 alpha 2 wheels and reused them for alpha 3, but the ABI changed between the two

[lusty quarry]

yes I have

[lusty quarry]

I think with coverage

[queen hornet]

I've not but there a couple on the Fedora list of 3.11 bugs https://bugzilla.redhat.com/showdependencytree.cgi?id=2016048&hide_resolved=0

[lusty quarry]

can anyone here offer advice for how this person should proceed? https://github.com/pypa/packaging.python.org/issues/1011#issuecomment-1013694487

[lapis tundra]

Hi guys,
I am running bandersnatch 4.4.0 and I would like to upgrade to the latest version. Is there any specific guide to upgrade without ruining my existing configuration?

[peak ocean]

FYI, We've made some changes to pip in Debian that bring us closer to upstream:

1. We're using the vendored dependencies, as provided by pip. (See: https://lists.debian.org/debian-python/2021/09/msg00031.html for the rationale)
2. We no longer have a separate pkg_resources wheel that virtualenv has to install. (See: https://bugs.debian.org/1003573)
   Of course this doesn't mean anything changes immediately in existing stable releases (they're stable) but we may provide some backports with this strategy.

[bleak forge]

Why not just make an argumented opt-in convention for those who care? 12-factor app is a good example. Not everybody uses everything, but parts of it are reused quite often.

[bleak forge]

Sound like a poll at least.

[blazing lantern]

Sure, but where is that opinionated opt-in supposed to live to start pushing people to adopt it? I mean I could blog about it, but I'm not prepared to do that until I have some code to help make it easier as well

[golden falcon]

The saga of people mucking around with install schemes continues: https://github.com/pypa/build/issues/215#issuecomment-1017173304

[golden falcon]

@peak ocean can you please take a look before this ships in Debian?

[peak ocean]

Having a look

The issue is that we cannot extract the venv paths if a different scheme is used for the venv because we operate on the assumption that the system default and the venv schemes are the same.
That assumption is invalid in Debian and Ubuntu, as we want pip install to /usr/local to work. While I understand the sentiment, I think it's unreasonable to call for an immediate "roll back the patches" and find a solution with upstream, because that's how we've done things since 2008, there is a lot off stuff that expects this behaviour.

We're probably going to have to find some other short-term solution here, while something long-term is figured out.

[golden falcon]

My suggestion was to add a venv entry in get_preferred_scheme. Is that something you’d consider?

[peak ocean]

I need to think about this a little more, back after lunch.

[peak ocean]

From some quick digging - the specific bug described seems to be specific to deadsnakes, not Debian python packaging, the patch referenced is very different in Debian. And I have an email from Anthony that I think describes the issues he was dealing with, that lead to this. Now, to see if I can reproduce those and maybe head him off in a better direction...

[golden falcon]

Thanks

There seems to be a similar patch in Debian which is however not applied: https://salsa.debian.org/cpython-team/python3/-/blob/master/debian/patches/sysconfig-debian-schemes.diff

I suppose he based his off of that

[tidal kiln]

that patch is not applied

would make things much better if it was

[golden falcon]

That's what I said 😛

[tidal kiln]

🤦

sorry

[golden falcon]

Lines 61-63 of that patch are especially pernicious

[tidal kiln]

I commented on the issue

using the venv scheme seems reasonable, and probably what we want to do actually on >=3.10

I have already explained in detail this issue on bpo, see that for the full details

I recommend debian to adopt the same approach of specifying a venv scheme

https://bugs.python.org/issue45413

[golden falcon]

It's ok to have a scheme for venvs but we have a mechanism to request a scheme for a particular install layout in 3.10+ and that's get_preferred_scheme - why should we refer to the venv scheme by name?

[tidal kiln]

see the bpo

the idea is that all virtual environments should use this scheme

allowing us to get the paths without running the interpreter

virtualenv already uses this approach IIRC

get_preferred_scheme is only for the current interpreter

[golden falcon]

Well you'd have to do a sort of weird dance with "venv" if "venv" in get_scheme_names() else get_preferred_scheme("prefix"), why not just get_preferred_scheme("venv")?

The scheme dict can include a venv entry so that e.g. for posix by default:

{
    'prefix': 'posix_prefix',
    'home': 'posix_home',
    'user': 'posix_user',
    'venv': 'posix_prefix',
}

[tidal kiln]

that is reasonable, but less optimal IMO as people will hardcode paths, so it would be better to have the scheme not be OS dependent

but a good proposal anyway 👍

there is also the case of pip install --user inside a venv

so I think it's cleaner to just have it as its own scheme, instead of a scheme type (I guess that would be the best name?) like that

does this make sense to you?

[golden falcon]

Well user installs are always performed on request, pip will use the user scheme no matter what, right?

[tidal kiln]

yeah, but you cannot install to the actual user install as it uses a different python executable, a couple things depend on that

also, entrypoints/scripts

you could do pip install --user build and pyproject-build might not be in PATH, which is unintuitive

I think users will often reuse commands, and this might be a footgun IMO

but yeah, that's just my opinion

[tidal kiln]

this is the main objective reason against, I think

[golden falcon]

To me it sounds like pip should forbid installing to any other location when you're inside a venv. Since schemes are all based around sys.prefix and that's gonna be the venv's prefix. I'm probably missing something (that tends to happen), but I don't see how the venv scheme being separate from preferred schemes would help in this instance

[tidal kiln]

hey, just checking, the wording of https://www.python.org/dev/peps/pep-0621/#dynamic does not prohibit us from still declaring the dynamic fields, right?

my use case: we want to make the build system pin a dependency, based on what was used for the build

[fallen shuttle]

I don't know if I understood the question, but what the spec says is that the user cannot specify a field statically and in dynamic; and that the backend cannot backfill a field that is not listed in dynamic

[fallen shuttle]

So the backend could not change the dependencies if the entire dependencies field is not marked as dynamic

(My interpretation at least)

[tidal kiln]

the use case I am targeting would be specifying dependencies statically, but mark that field as dynamic

otherwise, I have to move that field to my backend configuration section, which is not very nice

being in dynamic no static analyzer should be looking at the field, if they were I would consider that a bug

eg.

[project]
...
requires = [
  'abc',
  'def',
  'ghi',
]
dynamic = [
  'requires',
]

what I want my backend to do is to take that list and pin some of the dependencies, like ghi==1.2.3 for eg

other use-case would be adding extra dependencies for eg.

[fallen shuttle]

In the pdm project recently there was a discussiom about something similar to this approach for classifiers (I.e. having part of them statically defined and the other part backfilled by the backend). But this seems to be conflicting with the PEP.

[tidal kiln]

yeah

[fallen shuttle]

The static analyser can consider that specifying the field statically and in dynamic at the same time is a violation.

[tidal kiln]

I think in hindsight, it could be handled better

but 🤷

(still not clear if this, in fact, would be better 😅 )

anyway, thanks!

[golden falcon]

there was a discussion recently on Discord about this

I think it was with one of the Tensorflow devs

[spiral urchin]

This is against the spec, you’ll need to rename the requires array to something else with a tool prefix

[blazing lantern]

It was on Discourse and it ended with, "please make a proposal because I'm not doing it for you," which I think scared them off 😄

[candid smelt]

Good evening everyone, I was hoping someone may be able to help me with an issue i am having with RSA encryption in python

[high stone]

You'd likely find help in the Python Dicord: https://discord.gg/python

FWIW, I've gone ahead and "closed" the #warehouse channel; since the warehouse maintainers aren't super active on this Discord.

[fallen shuttle]

@high stone, quick curiosity/side question: in your comment in pypa/packaging#383 (https://github.com/pypa/packaging/issues/383#issuecomment-1024061442), you mentioned backends that only generate wheels.

Do you know if by the current standards, compliant backends can generate only wheels? I had the impression PEP 517 specified build_sdist as a required hook...

(Just asking because in my mind skipping that step for Python-only projects makes sense - I understand that your comment might be referring to backends that choose to not follow the whole PEP 517).

[high stone]

You can raise OperationNotSupported.

OK, UnsupportedOperation

That's only there for build_sdist AFAIK.

[fallen shuttle]

Oh cool! That gives me ideas. Sorry for asking, I should have read the PEP with more attention 😅

[high stone]

You have to define it, it just doesn't need to be anything more than raise UnsupportedOperation to be compliant. :)

It's a long story why we ended up that way, but... yea, that's what we have now.

[fallen shuttle]

Thank you very much for your patience replying my questions 😄

[golden falcon]

compliant backends can't not generate sdists, UnsupportedOperation exists for when a condition for generating an sdist is not met. If generating sdists was optional, the hook would've been made optional

[lean lake]

Who's planning to attend PyCon US? If work lets me, I'm going to squeeze in a visit. Almost drivable for me, still deciding if I will drive or not. Main reason I'm debating is cause the week after I'm driving though SLT to Chicago ...

[strange knot]

FWIW Ive gone ahead and closed the

[merry rune]

I'm pretty sure my talk was accepted. They didn't email me, but it shows as accepted and I confirmed it. Though it also says it is 0 minutes long... Accepting a talk for 0 minutes sounds like a rejection to me 🤣

[mossy pulsar]

We would love for you to speak, just keep it boxed in 0 minutes 🤣 the passive aggressive way to reject talks 😮

On a more serious note, mine is also 0 minutes; I'm pretty sure they'll allocate time after they have everyone accepted 🙂

I did get an email though about the one that got accepted and the one that didn't too.

[lean lake]

I've give a 0 min talk every pycon I've attended. Nice work joining that crew!

lol

[tidal kiln]

I just booked my trip, so you can count me in

[lofty aspen]

snip snip snip requests=2.26.0 not working in poetry

Shall I report this to a pypa repo or poetry? both or neither?.

can't repro with pip --require-hashes. Must be poetry. 😕

[west drift]

Are there any examples of wheels using .data/platlib or .data/purelib? I checked pypi top 100 and pip's test cases but I couldn't find any in either

[tidal kiln]

I do

https://github.com/FFY00/mesonpy

[west drift]

What's the correct way to build the purelib-and-platlib example into a wheel? With the default script pip wheel . -w . crashes, and when I change pyproject.toml to requires = ['meson-python'] i get a wheel with only a purelib (python -m build errors either way because it tries to install mesonpy from pypi)

[tidal kiln]

how does it crash

that seems like a bug

[west drift]

it complains about No such file or directory: 'requirements.txt' when trying to install MesonPy-0.0.1.tar.gz

https://gist.github.com/konstin/5bcba01837d72a601e1cad81dc598d26

maybe crashing is the wrong word, pip properly errors

[torpid veldt]

probably forgot to include the requirements.txt in their manifest.in

[west drift]

the problem is that it tried to install MesonPy from pypi in the first place instead of meson-python, especially for build which does that even if i specify requires = ['meson-python']

[tidal kiln]

ah!

yes

the test packages have the wrong dependency

sorry

I will fix them in a bit

[west drift]

thanks!

[lyric hedge]

ello!

[tidal ether]

👋

[inland gulch]

Hi!

I'm curious to know why PyPI is 400ing, saying that the name is blocked, when trying to upload a package named smartconfig. From what I can see, the only conflict is an existing package named smartconfigparser, which could trigger the typosquatting detection? Perhaps it is something that I cannot see, though.

[idle egret]

I assume they have a manual blocklist

[inland gulch]

I'm not sure why that'd be part of the blocklist, though

[idle egret]

@strange knot would know

[queen hornet]

https://libraries.io/pypi/smartconfig suggests there used to be something on PyPI called smartconfig

[spiral urchin]

Seems like the name was registered for (arguably) malicious use.

[inland gulch]

that is quite a sadge moment

thank you for looking at it

time to think about another name!

[rancid siren]

I think you can request for that name somewhere

[rancid siren]

https://github.com/pypa/pypi-support/issues/new?assignees=&labels=PEP+541&template=pep541-request.yml&title=PEP+541+Request%3A+PROJECT_NAME

[inland gulch]

oh right

I think I'll use as an excuse to gather some friends and think about a better name 😛

[queen hornet]

a new name is definitely the quicker option, the PEP 541 backlog is pretty big

[strange knot]

This is because https://pypi.org/project/smart-config/ exists. We recently added a change that prevents new project creation when the name is the same if you don't consider 'separators'.

[spiral urchin]

Maybe it’s a good idea to add a message in the 400 response saying something like “this name is too similar to smart-config” (or it already does but Twine needs to display it?)

[strange knot]

It doesn't, because a) there might be multiple projects that conflict, and b) I don't want it to become a faster avenue to direct people to bother the maintainers of the project with the name they want

FWIW, the current error message is "The name {name!r} is too similar to an existing project."

[inland gulch]

https://github.com/Akarys42/aegir/runs/5109848395?check_suite_focus=true

The error I got was

400 Client Error: The name 'smartconfig' isn't allowed. See https://pypi.org/help/#project-name for more information. for url: https://upload.pypi.org/legacy/```

[west drift]

Yes please! Some time ago someone send me an email because he couldn't register a package. It was a formerly deleted one and he had to ask on the issue tracker, but the error didn't tell him anything so he had no clue that the request-a-package issues were even a thing

[strange knot]

Sorry, I was wrong, this project name was explicitly prohibited before our 'similar name policy' was implemented (due to having a similar name to an existing project).

[spiral urchin]

Now that the similar name policy is implemented perhaps we can review the prohibition list to remove some redundant entries so they can emit the more friendly error

I assume the list is private and can only be reviewed by internal personnel though?

[inland gulch]

I wonder if there would be any issue with making it public

[rapid glacier]

I understand that the example is rather far fetched, but this could lead to some weird rejections of packages.
What happens if you disregard word separators can be seen in a few twitter hashtags ( only using twitter as example as it famously disregards word separators and had popular incidents) -> see multitudes of #analbumparty ||https://www.huffpost.com/entry/susan-boyle-anal-bum-party-twitter-gaffe_n_2171973||

Just as an example of completely different things becoming too similar, I also guess this becomes worse as amount of separators increases. Intentional unrealistic example: my_package and m_y_p_a_c_k_a_g_e where now on would typo one for the other.

[torpid veldt]

npm has been doing this for aaages

I don't think it's caused many problems

[rapid glacier]

Yeah probably me overthinking it.

Trying hard to come up with a real example and drawing a blank.

Sorry should have thought about it before posting

[peak ocean]

I notice a pattern of pep517 tools installing licenses into .dist-info, but with no standard layout. Seems like someone should write a PEP to standardize inclusion of licenses in .dist-info.

[fallen shuttle]

Isn't it covered in PEP 639?

[lusty quarry]

yes, license_files/

already supported by hatchling e.g. https://ofek.dev/hatch/latest/config/build/#build-system

[peak ocean]

Ah, thanks. I'll file bugs pointing to that in the ones that differ

[strange knot]

The PSF just published an RFP for upcoming PyPI contract work: https://pyfound.blogspot.com/2022/02/we-are-hiring-contract-developers-to.html

Please share widely: https://twitter.com/ThePSF/status/1493994827020709897

Python Software Foundation (@ThePSF)

We are hiring two contract developers to build organization accounts for @PyPI. This is a unique opportunity to flex your skills and develop next-gen features for PyPI. More details at https://t.co/Da04WUVYmV

Twitter · Feb 16, 2022, 5:04 PM

[blazing lantern]

Do note that PEP 639 has not been accepted yet, so I would hold off until it is (waiting on CAM to do a big update and then I am going to start reviewing it as PEP delegate)

[lusty quarry]

@indigo token hey! fyi I'm trying to get time at work to implement https://github.com/dependabot/dependabot-core/issues/3290

[indigo token]

Haha hi! Thank you, that’s great! Python’s flexibility with build backends might be inconvenient for static dependency analysis, but so nice for reusable build configs. The days of having setup.py + requirements.txt in an average package are numbered.

[fallen shuttle]

If anyone would like to give it a try, I just posted to Python Discourse: https://discuss.python.org/t/help-testing-experimental-features-in-setuptools/13821

(special thanks to @merry rune that have already been helping with tests in pybind and scikit-related packages)

[golden falcon]

I wonder if this wouldn't be a good time to do away with the package_dir mapping. If you're telling find to look for packages in src, why do you have to repeat src in package-dir?

[fallen shuttle]

I can analyse that in a follow up, while the feature still lives as experimental and thus can accommodate some changes. On the bright side, people using src-layout now should not have to specify any of these configs 🙂

(I think in my current implementation the other way around is true: if someone specifies package-dir = "lib" it will automatically try the auto-discovery in the lib directory)

(as a variation of the src-layout)

[golden falcon]

Is a string a valid value for package-dir?

[fallen shuttle]

ops, sorry, package-dir = {"" = "lib"}

[golden falcon]

At least with TOML you can tell that the value on the left is an empty string 😛

The setup.cfg syntax is horrendous

[fallen shuttle]

(I should probably double check to make sure I have test cases for that)

[fallen shuttle]

Would polymorphism in the pyproject.toml make this configuration more obvious (i.e. accepting a single string as a synonym of {"" = <string>}, but keeping the original behaviour of inline-tables)?
I am hesitant to allow to many ways of doing the same thing... But in some cases might make life easier.

[golden falcon]

I suppose you could break it up, package-dir for the default package dir accepting a string and package-dirs for the package dir mapping that no one will ever use

[spiral urchin]

Would package-dirs { "." = "lib" } be an acceptable workaround?

[golden falcon]

"" doesn't mean "." though?

The key is the package name and "" means any package not otherwise specified

[spiral urchin]

Eh but I think the root problem is the current package-dirs syntax is difficult to grok and tweaking the format doesn’t really solve anything; an entire reform would be much more meaningful IMO.

[golden falcon]

A better option might be the asterisk, "*"

Yeah, I don't know, my advice would be not to port either packages or package-dir to TOML config. packages will catch anyone with nested packages by surprise and package-dir makes no sense the way the default case is spelled

[spiral urchin]

Yeah simply omit the config altogether (at least for the first implementation) is probably better

[golden falcon]

Really appreciate all the work you've been putting into improving setuptools' UX @fallen shuttle, it can't be easy with some 15-odd years worth of baggage

[fallen shuttle]

I tried to make it easy for people to leave those fields out, but I did not forbid them. One of the objectives is to try being as backwards compatible as possible, considering ini2toml as a (mostly) loss-less translation.
I appreciate any suggestions as how to improve here.
I was considering for a while the "*" myself, but I decided to not go for it, since setuptools use glob patterns in other configs, and I didn't want to confuse users even more

[indigo token]

It sounds wonderful that UX improvements are happening. Is there a concerted effort to redesign setuptools’ package building experience?

I’m asking because as long as e.g. MANIFEST.in exists, or I have to look up the exclude syntax every time because it’s different from anywhere else, or a *.egg-info directory is slapped into my source tree or other things like this, I’ll continue to recommend against using setuptools (if possible), since using e.g. flit or pdm is just bliss in comparison.

One of the objectives is to try being as backwards compatible as possible

What’s the rationale behind this being a goal? Isn’t this a great opportunity to get rid of misfeatures and defaults that didn’t stand the test of time and are understood by few people, such as the MANIFEST.in DSL? I also consider options like package-dir = {"" = "lib"} to be unintuitive: who looks at this and can guess why the empty string needs to be there and what it means? Having DSLs everywhere is also not great, leveraging table syntax makes things much more intuitive:

[options.entry_points.console_scripts]
hello-world = { mod = 'spam.eggs', call = 'ConsoleInterface.main', extras = ['pretty-printer'] }

I’m sure there’s more examples.

[golden falcon]

(The entry point ship has probably sailed as that's part of PEP 621)

I'm broadly supportive of redesigning setuptools' configuration from the ground up but I think the capacity is not there. It sounds like a full-time job, something that should be subsidised by the PSF if we want to see done

[mossy pulsar]

IMHO I'd prefer to see new backends become popular, than invest more effort in setuptools 🙂 I think the PSF can spend better the money on other PyPi improvements 🙂

[golden falcon]

Doing this piecemeal is also not a great idea, like transitioning to TOML then deprecating things gradually - would just be adding overhead IMO.

[fallen shuttle]

Even more than that, the PSF would have to actively look for and hire someone... I remember (and I might be wrong here) that funding was already available for implementing PEP 621 into setuptools, but no one candidated themselves. So after a one or two years I decided to give it a try (to see it happening) 😅

[golden falcon]

You are right, I completely forgot that there was funding available for that!

[fallen shuttle]

That is one approach, but it also comes with downside:

1. I don't think you can manage to migrate all the packages in the ecosystem to other backends overnight, therefore it is important to maintain setuptools
2. If setuptools maintainers cannot add new features, experiment and in general exercise creativity, and just have to focus on fixing other's people problems with the knowledge that the project's fate is oblivion, what is the motivation they will have to maintain setuptools?
   One of the reasons I started contributing to setuptools is because I saw all the effort the maintainers were doing, but they would still be chased by the community demanding support for new and shiny things or standards (that even the optional ones like PEP 621), not being valued by the work they do.

If the consensus in the community is that setuptools should no longer used, I think we should make this clear so all the contributors and maintainers can spend their time in things with a brighter future. Then of course we also need to find a way of financing people that will make sure the ecosystem doesn’t break.

(Please note that here I don't mean to criticise anyone, the position of "let's just not spend more effort in setuptools" is a perfect valid position and have valid rationale... I am just trying to analyse things from a purely logical point of view)

[fallen shuttle]

I would really encourage starting a discussion on the issue tracker about MANIFEST.in (I agree that it is a good opportunity to make advancements in this area).

Personally, I will abstain of driving this effort, because I am a big time fan of setuptools-scm, but I would definitely help!

[fallen shuttle]

What’s the rationale behind this being a goal?
Setuptools have a base of users that is already used on how certain things are made, and there is also a lot of docs that new users may read and expect to work even if the syntaxes are different. My approach was to maintain a mostly compatible and straight forward translation between setup.cfg <=> pyproject.toml so existing users or people that don’t read the documents with attention would not be surprised.
On the other hand, I tried to make it possible (and incentivise) people to not have to type these configurations at all (so there is no confusion). It is a big goal of mine to not have to add any tool.setuptools.package.find or tool.setuptools.package-dir at all.

I believe it is a good compromise.

(BTW I am really enjoying this discussion, as soon as we have a stable baseline we can iterate and make it better, so all this feedback is really appreciated)

[indigo token]

Personally, I will abstain of driving this effort, because I am a big time fan of setuptools-scm, but I would definitely help!
Oh me too! I think it’s due for a rename / split, as it can be used perfectly well without setuptools. Not everyone likes its approach though, so a way to statically specify included files should still exist – and with that I mean inline and without using an obscure DSL.

[lusty quarry]

I am a big time fan of setuptools-scm

Oh me too! I think it’s due for a rename / split, as it can be used perfectly well without setuptools.

hatchling supports that btw https://github.com/ofek/hatch-vcs

[tidal kiln]

hey, I am helping a friend getting started with python, does anyone have good resources for beginners?

[indigo token]

that’s great! and hatch looks awesome. it passed under my radar so far, and I think the reason might be its barebones github/docs landing pages. Having a neat feature list with links would probably have compelled me to check it out earlier

I’m only speaking for myself though, idk if others tick the same

[lusty quarry]

thanks, I added links 🙂

[lusty quarry]

I basically rewrote it from scratch over the last year and half, the build backend hatchling is new

v1 of CLI is stable but still in beta

[merry rune]

I've sort of attempted to build some links https://iscinumpy.dev/page/useful-links/ in the past, but never really fleshed it out. My own material (like https://henryiii.github.io/level-up-your-python) is for intermediate (and the first two sections are advanced intermediate). Planning on providing a WebAssymbly backend at some point for that, probably after the 3.10 upgrade for Pyodide. 🙂

[fallen shuttle]

The tutorials in https://scikit-hep.org/developer are amazing! Thanks @merry rune, the team and all the contributors!

[golden falcon]

Good news everyone, the toml PEP has been accepted

[tidal kiln]

ye, just saw it

[merry rune]

Then you should comment on https://github.com/pypa/packaging.python.org/pull/1031 (prev. https://github.com/pypa/packaging.python.org/pull/1030) - I make an effort to make the packaging recommendations general to multiple PEP 621 backends but it was shot down because we can't push any other backend other than setuptools, apparently, not even Flit.

Once setuptools is no longer experimental PEP 621, I'll be making another push to get that in. IMO 1030 would have been fine today. I'd really like to keep the "sampling" of popular PEP 621 backends feature, I think that helps keep it general and not a setuptools tutorial.

[lusty quarry]

Planning on providing a WebAssymbly backend at some point for that
instead, please consider just writing a plugin 🙂

• https://ofek.dev/hatch/latest/plugins/about/
• https://ofek.dev/hatch/latest/plugins/builder/

[merry rune]

Plug-in for what? I do have some plug-in ideas (scikit-build-core if it gets written later this year), but not sure how a plug-in helps for adding a WebAssembly runner for my notebooks. Even if it did, Pyodide sadely is built on setuptools and setup.py, but I’m hoping to help them use build and maybe installer.

[lusty quarry]

Builder plugin for hatchling

[golden falcon]

I personally think it's self-evident that flit is a much, much better choice than setuptools for beginners, but if its author doesn't want it in the tutorial, I don't feel like there's much to be done.

[lusty quarry]

everything is a plugin (wheel and sdist for example are built-in plugins)

kind of like the architecture of pytest & tox

[blazing lantern]

https://www.freecodecamp.org/

[strange knot]

Here's the resources I usually share! https://di.dev/python/resources

[tidal kiln]

thanks!

[merry rune]

Does anyone know how to specify a subdirectory in build system requires? For https://github.com/pypa/flit/issues/530#issuecomment-1048247176

[fallen shuttle]

In theory pip support subdirectory doesn't it?

https://pip.pypa.io/en/stable/topics/vcs-support/#url-fragments

[merry rune]

requires = ["flit_core @ git+https://github.com/pypa/flit@i530"] but it needs to be a subdirectory flit_core

I don't think the egg syntax is supported here (at least I couldn't get it)

[golden falcon]

I didn’t try it in build system requires but this worked for me when I needed it: https://github.com/layday/frontend-editables/blob/ba49880c04cda18fc921fa0eae6739a267ff945a/noxfile.py#L53

[merry rune]

Ahh, that works (I could have sworn I tried it) edit: I was soo close. Used & instead of #

[lusty quarry]

any idea why this error is happening? https://github.com/conda-forge/staged-recipes/pull/18137

[merry rune]

Added some suggestions

[lusty quarry]

thanks! can it not handle multiple version specifiers?

[merry rune]

It can't handle as much when it's in "pure python" mode, but it's much nicer to build one package instead of dozens.

The specifiers (after the #) only affect non-pure builds.

[lusty quarry]

ohhh. thank you

[lusty quarry]

@high stone holy moly pip errors are so nice now, well done 👏

[high stone]

Thanks!

[lone knoll]

Please help steer me to the right place to address this particular problem. We have a project that has not needed to use test.pypi.org in several years. The maintaners have changed in the production version of the project, and we'd like to use test.pypi.org now, but the single maintainer at the time is no longer available. Compare https://test.pypi.org/project/svgcheck/ and https://pypi.org/project/svgcheck/. What's the best way to reset the project at test.pypi.org at this point?

[queen hornet]

you can make a PEP 541 request to claim the project and get access, see instructions at https://python.github.io/peps/pep-0541/
you won't be able to reset the old releases (though can maybe yank them?), but will be able to upload new releases

[lone knoll]

to be clear - we're all fine at pypi itself, it's just test.pypi that's an issue. Old things at test.pypi are not a problem, but it would make more sense to remove and rebuild the project at test.pypi altogether. I'll enter a 541 ticket and it can get dispatched to the right place if that's not it I suppose.

[torpid veldt]

Imho test.pypi should get wiped every so often

[lone knoll]

reading the pep and looking at the issue template, I don't feel confident that using the pep 541 request process is the right thing to do.

[lyric hedge]

Hmm? You should be able to file a PEP 541 request to transfer ownership of the package to another user controlled by your organisation. You should state "maintenance" as the reason behind the transfer and use the TestPyPI package link.

Basically a PEP 541 request is a formalized process to replace the ownership of a package with a new set of users (typically involving replacing a single former owner with another). It's formalized to keep it objective and discourage unreasonable claims.

Ah I see you've filed a request, hopefully it's actioned somewhat quickly but that's not something PyPI support is known for (understandably so).

[queen hornet]

https://packaging.python.org/en/latest/guides/using-testpypi/ says "The database for TestPyPI may be periodically pruned, so it is not unusual for user accounts to be deleted." but I've never seen it happen

apparently [in 2017] it's only happened once or twice https://github.com/pypa/warehouse/issues/2286#issuecomment-320724727

if it did happen it'd break my TestPyPI deploys when my account and its API tokens got wiped 😅

[queen hornet]

yep patience definitely required, although I did see a recent TestPyPI PEP 541 request took only 3 days

[wispy gazelle]

Latest Updates Feed¶
Available at https://pypi.org/rss/updates.xml, this feed provides the latest newly created releases for individual projects on PyPI, including the project name and description, release version, and a link to the release page. Only the latest 100,How do I get full updates?

[strange knot]

Read this as: "we make no guarantees about the availability of anything on TestPyPI", not "we will periodically clean TestPyPI".

[strange knot]

It's RSS, so you would subscribe to the feed and track new items as they are added. What are you trying to do?

[strange knot]

The PSF is hiring an Infrastructure Engineer, please help spread the word: https://www.python.org/jobs/6261/

[lean lake]

😮

[woven plover]

Is there any way to have a pep 541 style thing for when a maintainer refuses maintenance but also disallows transfer?

[mossy pulsar]

I don't think so

In that case, you're likely best of just hostile forking the repository and releasing it under the new name

[woven plover]

Yeah, I don't really want to repeat appdirs for pytest-sugar

[high stone]

Nope.

[zealous swan]

musllinux wheels have been uploaded for 283 packages already! Pretty exciting 😄

[lusty quarry]

2 are mine, entirely thanks to #cibuildwheel

[merry rune]

I wonder how many of them were made without cibuildwheel, I'd bet not many. 🙂

[woven plover]

how would people feel about sdists for pure python packages that ship a setup.py/zipapp file that just bundles the pinned build dependencies to ensure anyone can install from a source tarball without network/anything else

[merry rune]

Why does it need to be inside the SDist? I ship a zipapp for particle, but don't see why it needs to be inside/bloating the SDist. Maybe it would nice to have a place to upload them, but if you can install from PyPI, you probably don't need the zipapp.

Ah, so you want something that's not quite a zipapp, sorry, misunderstood - you do want it to install, and are targeting build dependencies. That won't work for cmake, ninja, etc. - compiled build dependencies.

[peak ocean]

I suspect that the use-cases where people prefer sdists to wheels are unlikely to be ones where bundled dependencies are helpful.

[woven plover]

i jsut had another support request on setuptools_scm because someone is running non-isolated builds in a no network environment and has no idea what they are doing

@merry rune also hence pure python, the moment you need to build something you need way to much + a clue anyway - but for a lot of pure python things, a zipapp that keeps a working setup.py may be a good idea

[high stone]

Yucky? The only situation this would help in would be when someone does not want to do the work of figuring out how to handle build dependencies; which... they should?

If this isn't a pip-installable sdist but a zipapp, that's reasonable -- I would be very surprised if this is something done in an sdist that's installable though.

[woven plover]

thing is pip install falls appart in no network situations regularly (in particular when the uninformed run non binary fetches)^^ - the idea i was thinking about was ensuring that pip works even in the no netowrk situation

[high stone]

Well, if you want to use pip in a no-network situation, you should make a bundle of wheels and install using them. This approach is documented as well: https://pip.pypa.io/en/stable/topics/repeatable-installs/#using-a-wheelhouse-aka-installation-bundles

There's a single knob that needs to be passed, to tell pip that you don't want it to use the network: --no-index.

That also means that you need to provide things to it, which you can do using --find-links .

[high stone]

IMO the solution for this situation isn't hacks in setuptools_scm -- the solution here is that user learning what needs to be done and adapting their workflows accordingly.

If they can't, then it's an organisational problem on their end -- one that I would find odd to inject hacks into setuptools_scm for.

But, anyway, you asked for thoughts, and these are mine. :)

[woven plover]

@high stone i wasnt rpoposing hacks in setuptools_scm more ike a sdist build that includes build dependencies , people get things wrong so easy sometimes - also this is not about setuptools_scm pe se, setuptools_scm would be jsut one of the payloads

[high stone]

You mean having the sdist differ from the source repository that created it?

That's equivalently "ew" IMO. :)

[lusty quarry]

sdist build that includes build dependencies

isn't that the definition? https://discuss.python.org/t/purpose-of-an-sdist/4639/19

[woven plover]

well, normally a sdist cn expect the builder to fetch the build deps

[lusty quarry]

oh you mean ship build-system.requires?

then no, the build envs should provide those imo

[high stone]

Well, I imagine folks here will have thoughts on https://discuss.python.org/t/adopting-the-concept-of-teams-from-pep-8015/14210

[spiral urchin]

Kind of like SIG was intended for?

[high stone]

Yea

Except we've lost the SIG naming convention to time, and... Well, Team is a less opaque word. :)

Plus, that's what Rust uses, if I remember correctly?

[spiral urchin]

I’m definitely not against using Team to replace SIG, after I’ve explained what that means to about a hundred people. Probably worth mentioning it in the post to pre-emptively answer the inevitable question “but this is basically SIG so why are you inventing a new thing”

[high stone]

@spiral urchin You wouldn't believe what the first response is, on that thread.

[lyric hedge]

hmmm it seems like PyPI is unstable today, bandersnatch CI has been failing with timeout errors and a local pip install run failed

[spiral urchin]

Looks like there was a CDN issue a few hours ago, seems resolved now https://status.python.org/

[lusty quarry]

I opened https://discuss.python.org/t/pep-621-normalization-of-fields/14402

[languid scroll]

I did a download count analysis on PyPi's projects, and found something interesting.

https://imgur.com/a/ZUWf28B

There is a significant jump in the number of downloads of PyPi projects from July 2018 to August 2018.

I saw that that's around the time when PyPI migrated to a new backend, Warehouse, in late April 2018. Might this affect the number of download counts? If so, how?

If anyone has any answers, that would be great. Thanks!

[high stone]

https://pypistats.org/faqs#why-are-there-so-many-more-downloads-after-july-26-2018

[strange knot]

I seem to recall there was a heroku app that could be used to browse files of distributions on PyPI, but I've lost the link. Does anyone know what I'm talking about?

[spiral urchin]

I do; I made that thing https://pypi-filebrowser.herokuapp.com

[torpid veldt]

It's just the metadata right? https://pypi-filebrowser.herokuapp.com/urllib3/

[spiral urchin]

Yes only metadata because that’s what I cared at the time; feel free to add support for other files! https://github.com/uranusjr/warehouse-filebrowser

[torpid veldt]

Ah I presume it intentionally doesn't expose all files

[spiral urchin]

It’s sort of by-design; I wanted to put links on the list page so I can click less, but the list of files is only accessible by cracking open each archive

The metadata file list is hard-coded to show METADATA setup.py setup.cfg pyproject.toml even if the dist doesn’t have them for the same reason

[west drift]

what's the evaluation order/precendence of and and or in environment markers? E.g. if i have python_version == "2.7" and os_name == "linux" or sys_platform == "win32" is that (python_version == "2.7" and os_name == "linux") or sys_platform == "win32" or python_version == "2.7" and (os_name == "linux" or sys_platform == "win32")?

[blazing lantern]

https://peps.python.org/pep-0508/#complete-grammar is probably the closest you will get to an official answer, otherwise "whatever packaging does".

[west drift]

than i'll check packaging, thanks! (i didn't find anything in pep 508 about evaluation order)

[blazing lantern]

The grammar innately defines an order based on how the parser will construct the AST

[west drift]

oh i didn't realize that part, thanks

[lean lake]

What's everyone's plans for the PyCon Sprints? I'm trying to work out what to plan for / work on

[lusty quarry]

I'm thinking about working on automating PRs to switch projects to PEP 517+621

[high stone]

I'm thinking about having a chat with people about improving the user experience of the overall tooling we have, and identifying where we have easy wins and such. Other than that, I'm kinda hoping I'd be able to contribute to Warehouse. :)

[indigo token]

Metadata browser

[blazing lantern]

Always aiming to do the small, simple things 😉

[lean lake]

Anything in particular I can look at to see if we can team up on?

[high stone]

I mean, we could talk JSON API stuff, and figure out how to move that forward.

[lyric hedge]

Is the Warehouse PR making the distribution metadata available still blocked? It'd be a great improvement to hopefully resolve dependency resolution times for good (mostly)

[high stone]

I'm unsure pip would use that directly -- unless we get checksums or something for it.

[lyric hedge]

makes sense 👍 although I was expecting the primary consumer to be pip :p

[mortal shore]

Not sure why?

Checksums don't make it more or less safe

assuming you're talking about the METADATA pr

[high stone]

Yea.

[mortal shore]

Like the checksums are going to be delivered over HTTPS

as is the METADATA file

[high stone]

True.

[mortal shore]

anyone in a position to modify the METADATA file, can also modify the checksums

[high stone]

I was thinking more along the lines of network errors, bitflips and whatnot.

[mortal shore]

HTTPS has a built in checksumming for that

[high stone]

Yup, and that's what I realised when I wrote this. 😅

[lyric hedge]

I didn't even know HTTPS had checksumming built in. I assumed it relied on the TCP checksum (although I've heard it's rarely used in practice ...?) .. or something about TCP being reliable

[mortal shore]

It may not be worthwhile to use it anyways, the one problem with METADATA, well "one", is that METADATA can be big on it's own, and you're going to be making a lot more HTTP requests

which are not free

well, not a lot more than you're currently making

but a lot more than is strictly required with a slightly different approach

[high stone]

It ensures that bits in == bits out.

[mortal shore]

but that's me just throwing random crap out there 😛

yea, HTTPS checksumming is actually a MAC

[lyric hedge]

I've reached the end of my networking knowledge 😛

[mortal shore]

and it's actually at the TLS layer

[high stone]

You can still have things get messed up on either side of the pipe FWIW.

[mortal shore]

MAC = Message Authentication Code

traditionally TLS uses HMAC, https://en.wikipedia.org/wiki/HMAC newer ciphers have it built in though

[high stone]

I love how everyone just removes the previews on the first chance they get. 😆

[mortal shore]

lol

[lyric hedge]

they're annoying!!

[mortal shore]

it depends on what the preview is, when I'm using my screenshot program I don't

https://tldr.tw/o4wdOHi

cuz I want it to embed!

[high stone]

:)