general

[high stone]

TIL about https://jumpshare.com/

[lyric hedge]

the most complicated networking thing I've coded was a local LAN chat app that implemented a custom protocol which used regex to parse ... yeah ... I know

[high stone]

Hehehe.

I'm pretty sure that's pretty similar to what I'd last coded networking wise.

Everything else has been "oh, right, someone else wrote that and I'm struggling to understand it".

[lyric hedge]

I think I tried coding a file transfer program on top of that but it was horribly slow :)

[high stone]

I think that contributor dropped off the radar, given that no one else seemed to want to embrace crypto like they'd hoped.

[lyric hedge]

Yup yup. 👍

That reminds me of my setuptools PR implementing the PEP 660 editable hooks. I think I'll just close that as it seems the setuptools folks want to implement 'em differently.

At least it triggered a good discussion :)

[fallen shuttle]

BTW @lyric hedge thank you very much for providing that. I think the PEP 660 is the next frontier... but it may take some time to get there.

I understand the concerns presented in the discussions of your PR. Personally I do like the pth approach because I like to see new files automatically popping up and it does not affect me since I test everything with tox installing the packge...

But the concerns presented are genuine and we should think about them.

[lyric hedge]

I don't have the energy or willingness to shepherd whatever discussion a PEP 660 PR would entail. To be honest, I was a bit scared off from working on anything related to PEP 660 since it's still a bit controversial. I hate conflict and I'm not dealing with that.

it's not like I have the context, knowledge, or experience to shepherd such discussion anyway -- and the discussion is certainly a prerequisite before a PR would be accepted.

And yes I'm well aware this is the healthy kind of conflict. This is a me problem :)

[high stone]

:)

[lyric hedge]

I guess I'm actually getting PEP 660 vs 662 stuff mixed with the various approaches to implement editable installs disagreement, but eitherway I'd rather not deal with that especially as the first contribution

[fallen shuttle]

I can understand that @lyric hedge. Thank you very much for proposing it anyway.

Your PR really helped and I am very sure that at least parts of it are going to be reused. And the investigation that you did was very useful!

[lyric hedge]

The code was hacky at best so I'd be nervous reusing it, but I did submit it lol

It's hard to make it clean in a legacy codebase though -- I was surprised at how hard it was to use the internal APIs from build_meta.py

[lyric hedge]

And thanks for maintaining setuptools! It's nice to see setuptools get more love

[indigo token]

This PR? https://github.com/pypa/warehouse/pull/9972

[tidal kiln]

FWIW I have released a new version of https://github.com/FFY00/mesonpy yesterday with initial cross platform support

if anyone wants to give it a try, feel free to ping me directly for feedback

[lean lake]

O lord. I've tried writing a PEP, I've had examples ready - I don't know if I have the patience to try yet again

Would love to get that PEP shipped so we can then clean up any little things to the API and then finally extend / move it / replace it etc. etc.

Would be happy to do the warehouse work if/any small tidy up is needed as I've said all along

[lyric hedge]

Mhmm

[mossy pulsar]

https://discuss.python.org/t/selecting-a-logo-for-the-pypa/14657 in case anyone has opinions 🙂

[lyric hedge]

Hmm is this not a valid PEP 508 requirement?

tomli @ https://github.com/hukkin/tomli/archive/master.zip; python_version < '3.11'

parsing this with packaging is failing

Traceback (most recent call last):
  File "/home/ichard26/programming/oss/black-deps-ci/3.6venv/lib/python3.6/site-packages/packaging/requirements.py", line 102, in __init__
    req = REQUIREMENT.parseString(requirement_string)
  File "/home/ichard26/programming/oss/black-deps-ci/3.6venv/lib/python3.6/site-packages/pyparsing/core.py", line 1134, in parse_string
    raise exc.with_traceback(None)
pyparsing.exceptions.ParseException: Expected string_end, found 'python'  (at char 60), (line:1, col:61)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "generate.py", line 24, in <module>
    req = Requirement(raw_req)
  File "/home/ichard26/programming/oss/black-deps-ci/3.6venv/lib/python3.6/site-packages/packaging/requirements.py", line 105, in __init__
    f'Parse error at "{ requirement_string[e.loc : e.loc + 8]!r}": {e.msg}'
packaging.requirements.InvalidRequirement: Parse error at "'python_v'": Expected string_end

I'm confused since pip accepts this if you put it in a requirements file, is this a pip specific extension?

oh I see, a space is necessary before the semicolon 😅

url_req = name wsp* extras? wsp* urlspec wsp**+** quoted_marker?
that + is sneaky

[lusty quarry]

do extras work on ^?

[lyric hedge]

Sorry, could you elaborate?

[lusty quarry]

pkg @ url[foo]

[lyric hedge]

❯ python
Python 3.8.5 (default, Sep  9 2020, 23:45:44) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from packaging.requirements import Requirement
>>> r = Requirement("dataclasses @ https://github.com/ericvsmith/dataclasses/archive/master.zip[foo] ; python_version < '3.7'")
>>> r.extras
set()
>>> 

the extra specifier has be after the name, unless I'm totally missing your question @lusty quarry

[lusty quarry]

ah okay nice 👍

[high stone]

It needs to be on the name side, IIUC.

[high stone]

And, repeating what someone else has said, is what I get for not scrolling all the way down. :)

[spiral urchin]

You’re not the first one having this problem. We should’ve made that space mandatory for the version-specified format as well to make this obvious

[west drift]

i'm getting non-deterministic crashes ("signal: 7 (core dumped)"/"broken pipe"/"signal: 11 (core dumped)") when running python -m compileall -l -i - for wheel installation. Is there any good way to figure out why compileall sometimes crashes?

[blazing lantern]

You could try running it programmatically through a debugger, otherwise hack up your copy to log what the code is doing?

[west drift]

I'm running multiple compileall instances on different modules in parallel and the location of the crashes varies, so i don't think extra logging would help

I'm running it as subprocess so I guess what i need something that let's me run gdb on a subprocess crash or at least print a backtrace

[blazing lantern]

What are you doing to prevent the multiple processes from stomping on each other's output?

[high stone]

And, while we can't change that, we can explore improving the error message: https://github.com/pypa/packaging/issues/529

[lyric hedge]

thanks for opening that!

[spiral urchin]

Is there any updates on the Packaging Summit? Honestly I forgot whether I signed up or not 🤦‍♂️ so not sure if it’s me not registered or otherwise

[merry rune]

I signed up a bit late (on April 1), so was afraid that no updates meant I was too late. (Maybe it still does mean that). But haven't heard anything either way.

[tidal kiln]

we were actually afraid we wouldn't get enough people, but now it seems we have a small-ish but decent number

[peak ocean]

Yeah, sorry, I'd liked to have been there, but won't be at PyCon

[strange knot]

The PSF has hired a new Executive Director! https://twitter.com/ThePSF/status/1512078490689961993

Python Software Foundation (@ThePSF)

We're super excited to welcome our new Executive Director, Deb Nicholson (@baconandcoconut)! And just in time to join us and you all at #PyConUS2022, too! 🤩🤩

https://t.co/KphRLVy0zj

Twitter · Apr 7, 2022, 2:42 PM

[merry rune]

Is there a way to get the Python bitness in environment_markers, not just the machine bitness? In the environment markers, I'd think the only thing that would matter would be the Python bitness, not the machine, but it seems due to a fix for platform.machine in Python3.2/2.7, platform_machine from PEP 508 is now useless in selecting wheels. This is breaking "oldest-supported-numpy" on Python 3.10, where 32-bit support for Windows came in 1.22 but 64 bit support was in 1.21. So to compile against NumPy (say for Cython, pybind11 avoids this), you need to put 1.21 or 1.22 for Windows based on 32/64 Python (not machine) in your pyproject.toml.

Maybe something like "x86_64" in platform_release (or whatever Windows is)? Maybe platform_version? Not sure if those are very reliable. Full issue: https://github.com/scipy/oldest-supported-numpy/issues/45

[spiral urchin]

What is the fix you referred to? I’d think physical bitness should be irrelevant (at least like 90% of the time) and we should depend on interpreter bitness. Perhaps this is simply environment markers being designed before the change and was not properly considered when platform.machine was changed.

[woven plover]

@lusty quarry are there hatch plugins for cython/mypyc?

[lusty quarry]

https://github.com/ofek/hatch-mypyc

[woven plover]

excellent, i may use that for the pluggy experiment next

[lyric hedge]

oh hi Jelle o/

[merry rune]

@spiral urchin https://bugs.python.org/issue7860

I can't think of a reason a dependency would care on physical bitness over the Python bitness - I'm pretty sure platform_machine was expected to be the Python bitness when the PEP was written.

[spiral urchin]

There are cases theoretically, but in any case, I think platform.machine is doing the right thing, and since marker variables are tied to actual Python attribute names, we should add a new marker for interpreter bitness. Is there a handy variable available in Python for that?

[merry rune]

AFAICT, not really, sys.maxsize > 2**32, struct.calcsize("P") * 8, ctypes.sizeof(ctypes.c_voidp) seem to be the common ways.

[spiral urchin]

Maybe we should propose one

The bpo mentioned platform.architecture(), did that also get changed with platform.machine()?

Note that platform.machine() and platform.processor() are not really very reliable ways of determining the machine type or processor.
Maybe it’s a mistake to introduce platform_machine in the first place

[mortal shore]

it probably was

but it's probably not worth getting rid of it at this point

adding something that gets the python bitness seems like a good addition though

[spiral urchin]

Wondering what the most useful value is. Just bitness (e.g. 32, 64) or also CPU arch (e.g. 'x86-64', 'ARM64')?

[merry rune]

NumPy backported and released, so the immediate use case is out of the way. I'd think the final build tag (minus the Python version & platform, since those already are obtainable) might be the most useful value?

I'm still not sure the 'native' values are ever useful. When you are cross compiling, you want the target arch, not the machine arch. 32 bit on 64 is basically a cross-compile-like situation, but not the only one. Defining the "platform" as the target platform would solve this without a new value(s).

On a universal or arm build for macOS could be another case where this could be currently wrong to use?

[spiral urchin]

It’s not useful for you (and probably for packaging in general), but it’s obviously useful for someone, otherwise this wouldn’t have been raised as a bug and fixed in the first place

[merry rune]

I mean for the tag platform_*, not platform.*

platform.* isn't tied to packaging, but platform_* is only used in packaging.

My thought would be to specify platform_* refers to the target platform, and not the currently running system's platform.*. It wouldn't be a 1:1 correspondence anymore if you are running a non-native Python (like 32 bit on 64 bit, arm on Intel macOS, etc)

I can't find any uses of platform_architecture on GitHub. I see some usages of platform_machine, but I think they are all wrong if it refers to the host machine and not the running Python arch if they are different. https://cs.github.com/?scopeName=All+repos&scope=&q=platform_machine+path%3Apyproject.toml

https://cs.github.com/?scopeName=All+repos&scope=&q=platform_machine+path%3Arequirements.txt

[spiral urchin]

The problem is, markers are directly tied to Python values, we can’t make platform_machine and platform.machine() mean different things

[merry rune]

Again, all of these are wrong if platform_machine is not what you are targeting, as numpy discovered. Usually, it doesn't matter, because you usually don't have a mismatch, but you can for 32bit/64bit, and I think you can for macOS too (but not sure on that one)

[spiral urchin]

If we establish platform.machine() should describe physical machine, platform_machine should as well, even if it’s utterly useless

[merry rune]

Why does it have to be 1:1, though? implementation_version is not, for example.

It's the same values, just the return value assumes you are on the "target" machine for that Python version, regardless of what physical machine it's running on while being resolved.

You should be able to run the installed packages on machine that actually is 32 bit

If I have a shared directory, and have 32-bit python on all my machines, and some of them are 64-bit, they should all be able to work with that directory. As it is now, the 64-bit machines running 32-bit Python will resolve differently and get dependencies that are not valid for the true 32-bit machines. (Of course, NumPy was the only lib I know of that were this mattered for a while)

That's my thinking, anway.

[spiral urchin]

implementation_version is based on sys.implementation.version

[merry rune]

I think there's no reason to have platform_machine be the physical machine, and all the linked projects above are assuming the target machine - which currently I think only breaks down for 32bit on 64bit systems. I don't know what macOS should report, actually, since there are two values for universal2 builds. For an ARM build on Intel, I think it's currently "wrong" for building as well (reporting Intel), but am not sure. If the current one can't be fixed, then target_machine might be a good value, with the same values as platform_machine, but always for the target platform rather than the current platform.

Maybe with special handling for Universal2? I don't know how that should work.

[mortal shore]

I'm confused, if you're cross compling then your cross compiling should emit tags that match the target platform, not the current platform

[merry rune]

I think so too! But it does not if you are on a 64-bit machine running 32-bit Python.

NumPy assumed

numpy==1.21.4; python_version=='3.10' and platform_machine!='loongarch64' and platform_python_implementation != 'PyPy' and (platform_system!='Windows' or platform_machine!='x86')
     # x86 windows wheels were only added from 1.22.3
numpy==1.22.3; python_version=='3.10' and platform_machine=='x86' and platform_python_implementation != 'PyPy' and platform_system=='Windows'

would work, but it doesn't - 64-bit machines with 32-bit Python (like CI!) all broke.

[peak ocean]

It's confusing if "machine" doesn't mean machine, but rather ABI. "Architecture", "bitness", or "platform" makes more sense. Think of somebody writing a fleet inventory tool

[mortal shore]

I don't think the problem has anything to do with cross compiling, it's just that the current tags only let you specify the machine bitness, not the python bitness

like during install, markers are not evaluated based on the build environment, it doesn't even know what the build environment is, it just knows what environment the wheel claims to support

so in that sense, all env markers are for the "target"

[merry rune]

Hmm, so this works for something like macOS, since that's a true cross compile, but there's no separation between machine and Python bitness? There is a separation between 32 and 64 bit machines

[mortal shore]

the current environment markers don't offer a way to restrict a dependency such that it cares about python bitness. Like wheels themselves that's baked into the compatibility tag in the file name in the platform tag, but for environment markers it does not

I'm not sure I understand the use case where you want to have different dependencies on 64bit python vs 32bit python

[lusty quarry]

👍 to sys.maxsize way https://github.com/pypa/auditwheel/pull/98/files

[mortal shore]

I guess this is the usecase? It looks like this is just trying to pin to a different version of numpy based on whether or not it had wheels compiled for it? I assume in both cases both versions are actually fine, you're just trying to avoid compiling from source?

[spiral urchin]

Yes. (multiple reasons why Numpy wants to avoid compiling from source in this scenario)

[mortal shore]

I guess there's a reason you can't just use 1.22.3 in both cases, assuming they have wheels available for both. Anyways, feels like a pretty niche thing, but for whatever my "from the sidelines" opinion is, seems like there's little reason not to add an env marker for the interpreter bitness, probably should get an actual API added to python first that exposes it though

[merry rune]

Compiling from source actually broke with a setuptools update - so it wasn't just wasting time, it was broken. NumPy has to provide the oldest minor version for compiling user extensions, since you can't load an extension compiled with a newer version of numpy on an older one. There's a runtime check for that.

It was a really weird case that NumPy didn't have 32-bit Windows wheels till 1.22, and it's been backported now, but it's a missing bit of functionality to be able to specify this (since there are distinct wheel files for 32/64)

A nice API for this in Python would be great, currently sys.maxsize checks are used

[spiral urchin]

fwiw that matches mine, we should get something into the stdlib, and then expose an environment marker to match that

Or we could add sys_maxsize if that’s good enough (is it?)

[merry rune]

sys.bitness? 🙂 sys_maxsize would work, though it would be annoying if you have to write out 2**32 in the marker. Either solution will be possibly be a bit messy as a marker, since you'd have to check both for 32 bit platforms and 64 bit platforms + 32-bit pythons explicitly. But it's not common.

[spiral urchin]

Let’s open an issue on cpython and continue there

[woven plover]

@lusty quarry aware of anything to make hatch build reproducible zipapps of a specific package under its control?

[lusty quarry]

like pex/shiv?

no plugin yet

[woven plover]

@lusty quarry ok, i have some larger use-cases surrounding pytest, execnet and appdirs (the idea being to ship a wheel/sdist to remote machines/vms/containers to run test-suites in specific environments)

but this is something that will manisfest very slowly

first is git archive support in setuptools_scm core, and a better configuration layout for setuptools_scm (plus first class support in hatch)

[lusty quarry]

could write environment plugins https://ofek.dev/hatch/dev/plugins/environment/

[woven plover]

that looks most promising, in particular the containers one as well

btw, could hatch be used to build speciic containers as release output as wel?

[lusty quarry]

yup, todo #2 https://github.com/ofek/hatch-containers#future

[woven plover]

@lusty quarry is there a dedicated comm channel/chat for hatch? i have quite some more nad i dont want to load them all here

[lusty quarry]

#off-topic ? unless PyPA wants to host Hatch on GitHub 🙂

[merry rune]

A hatch channel in Packaging related? Unless you are proposing moving hatch to PyPA 😉

[woven plover]

i like the idea, if @lusty quarry likes it as well we could consider adding one 🙂

[lusty quarry]

oh yes that would be nice

#hatch

[high stone]

It exists now. :)

[woven plover]

im wondering, would it be possible/sensilbe to get a standalone version package thats fast to import/parse/instantiate - context is wanting to add a version object to setuptools_scm managed packages and wanting to have comparable instances that don't have a painfull cost for cli tools
CC @lusty quarry

[lusty quarry]

not imo, I would always ship the generated version file to import. can't get faster than that

[woven plover]

@lusty quarry i want to trigger deprecations based on relation to version numbers, for that it seems beneficial to have something that behaves correct for version numbers with interesting segments

[lusty quarry]

what's a concrete example?

[woven plover]

in a number of packages i introduced a deprecation system that based on the version of the current package picks a FutureWarning, a Warning, or a WillFail style warning subclass based on warning filter added on top of that

the flow is similar to pips mechanism to declare a release in which a feature is definitively gone, and then things will break about that

i intend to add something similar to pytest to manage our deprecations/errors as well

[lusty quarry]

so in pytest itself like:

import deprecation_lib

deprecation_lib.register(...)

def f():
    if deprecation_lib.self_version_greater_than('X.Y.Z'):
        deprecation_lib.emit('f_deprecated')

?

[woven plover]

yikes - no

import warnings
import deprecation_lib
deprecated = deprecation_lib.Deprecator("mypkgname", myversion)
SOMEDEPRECATION = deprecated("something specifc is gone" gone_in="future_version")


def f():
   warnings.warn(SOMEDEPRECATION)

[lusty quarry]

what value does deprecation_lib bring, just warnings.warn being a no-op depending on version?

[woven plover]

it will pick no warn vs warn vs error, and bring testrunner integration (primarily pytest) - the main value is deeper integration of warnigns with the release management (in particular usefull with calver)

[blazing lantern]

I introduced something similar in Python: https://github.com/python/cpython/blob/efe7fd4170bb809ed46cac35c6a9007d5b794e7e/Lib/warnings.py#L488-L504

[woven plover]

Unfortunately that one is very at runtime (which prevents warning filter usage for temporary workarounds)

[indigo token]

I have actually been thinking of creating a static analysis package that warns you of deprecations. Sadly there’s no standard data on it, so I’m thinking of sourcing

1. the stdlib .rst files
2. library source code having the @deprecated.deprecated or @deprecation.deprecated decorator on a function

https://github.com/flying-sheep/deprecated-imports

[charred swift]

Is there a way i can tell python -m installer what installation scheme to use, or will it always use my default, "posix_prefix"?

I'd like to use "posix_user"

[charred swift]

this kinda works, but it's super janky

python -m build --outdir .buildwork
python -m installer --destdir /tmp/dummy .buildwork/*.whl
cp -a /tmp/dummy/usr/* ~/.local/.

[tidal kiln]

I don't think there is yet, but should be fairly simple to add that option

[charred swift]

I made an issue to track it: https://github.com/pypa/installer/issues/120

[lean lake]

Where was I ment to register for the packaging summit at PyCon tomorrow? Can I still?

Damn found it - closed. I suck.

Any owners here?

[tidal kiln]

we have space, don't worry

we will also take walk-ins as the room allows

[lusty quarry]

will it be recorded?

[merry rune]

Tomorrow as in two days from today tomorrow?

[lean lake]

Yah that too. Friday. My bad. Haha

[merry rune]

Great, I fly in tomorrow, so nearly panicked when I saw that. 😄

[lusty quarry]

Hatch 1.0.0 is finally released, I'd love any feedback! https://news.ycombinator.com/item?id=31190027

[lusty quarry]

wrote a bit more here https://discuss.python.org/t/hatch-1-0-0-is-available/15359

[jade zephyr]

Has there been any recent progress on the discussion for namespacing on pypi?

Last time I looked into it, it seemed like the general consensus was supportive, but there needed to be a sponsorship for development and I'm wondering if that's still true

[jade zephyr]

I can't speak for this year really, but I've been to a lot of pycons and the recordings eventually show up for pretty much everything, and at great quality too.

[lusty quarry]

https://discuss.python.org/t/pypi-organization-account-roadmaps/14699

[jade zephyr]

👏 Awesome. I'll volunteer for a user interview as well.

I've been using pypi since 2011 and have varied between 15 and 100 projects over 3-5 orgs (depending on who I'm working for at the moment), feel like I can provide some useful input or some developer time to contribute back.

[lean lake]

@warm isle You still discord? I want to try and kick the tires today on our JSON API PEP @ the Packaging Summit @ PyCon today ... Your fork of my fork makes it harder to work on ... Trying to workout the best way forward here ...

[warm isle]

Please take over, I'm no longer with Bloomberg and it's unclear if I will have the time to work on this soon

[lean lake]

Ahhh ok - I might just take your rewrite etc. back to my repo then

(Manually)

[warm isle]

Works for me! It's been a while 😂 time to give it some umph

[lean lake]

Ya - Once I'm done, mind deleting the fork just to reduce duplication? 😄

Also, ignore my email 🙂 Just reached out multiple ways incase you didn't discord anymore

Thanks for the repsonse

[lean lake]

@warm isle - Ok - Fork can be deleted please at your convenience 🙂

[lean lake]

I just added a Webhook for badnersnatch GitHub activity into #bandersnatch - Does anyone else want it?

https://support.discord.com/hc/en-us/articles/228383668-Intro-to-Webhooks - Was easier than I imagined.

[lyric hedge]

^^^ just a quick PSA, please do not check "send me everything", it will absolutely hose the channel with updates

[lean lake]

haha - Yeah, don't do that

[high stone]

I'm gonna hang out on the #general voice channel for most of the day, during the PyCon US sprints.

If folks wanna chat / work on something together, please feel welcome to hop in.

[lean lake]

What's the fields you wanted to kill again - I'm just trying to make a 1 page docs with all our goals etc.

I know I could search but being lazy.

[woven plover]

do we have any way yet to have the build-system consider a transitive editable install of a "current package" in some way

im brooding over how to include setuptools_scm in self-bootstrap if hatch was to be used (and hatch-vcs loops back to setuptools_scm)

[lean lake]

Welcome

[high stone]

It's getting late here, so I'm gonna drop off for today. I'll be around tomorrow, in a similar manner. Feel free to drop in folks!

[lean lake]

Thanks sir

[high stone]

Thanks for the chats today folks! ^>^

[high stone]

@blazing lantern Not sure where to flag this -- could you update the link to PEP 668 in https://discuss.python.org/t/pep-668-marking-python-base-environments-as-externally-managed/10302 (the first post) to be for peps.python.org (it's currently pointing to a specific hash on the peps repository).

[mortal shore]

man PEP 668

whatever happened to that kid

did we land it

[high stone]

Not yet.

Needs Spack + Homebrew responses.

[high stone]

Same as yesterday, I'm gonna hang out on the
general voice channel for most of the day. If folks wanna have a chat about something, or work together on something, please feel free to hop in!

[blazing lantern]

Done!

[high stone]

Thank you!

[lyric hedge]

welcome 👋

[lusty quarry]

for whenever this ends https://mail.python.org/archives/list/[e-mail]/thread/J2BL5XTTOCXUCELUE7L5P5HHJ7NXZGZ5/ do we have a doc on the repo transfer process?

[tidal kiln]

no, I don't think

just message one admin to get you in the org if you aren't already, then move

[high stone]

We don't, but we should add one.

[mossy pulsar]

The vote did end now 🤠

[lusty quarry]

this should be ready for review/merge now https://github.com/pypa/packaging.python.org/pull/1074

[indigo token]

On a similar note @merry rune what’s necessary to merge https://github.com/pypa/packaging.python.org/pull/1031 ?

Seems ready to me.

[merry rune]

I think it's been ready for a long time. 😄 It needs people saying they want it, and might need @lusty quarry stating it's okay to make hatchling the main (first) option. It was held back for months because the author of Flit wouldn't allow Flit to be the first option.

Is pyproject.toml still experimental for setuptools, @fallen shuttle ?

I'd also be fine to put setuptools first if it's considered ready. I'd be fine to add javascript to randomly select one of the backends as the first one displayed, really. 🤣

[lusty quarry]

@merry rune I'm fine with that

[merry rune]

You should say that on the PR 😉

[indigo token]

Great! I think hatch is the better default than setuptools for the same reasons as flit would have been: it’ll motivate people to get rid of MANIFEST.in and other legacy stuff, and is faster.

In addition @lusty quarry actually wants it to be specified there and unlike flit, hatch is as flexible as as setup.py once some more examples for builder plugins exist. (looking forward to a Rust one)

[golden falcon]

Is flit's author ok with it being listed as an option?

I got the impression they didn't want it there at all

[lusty quarry]

didn't read much, why not?

[golden falcon]

https://github.com/pypa/packaging.python.org/pull/1030#issuecomment-991876540

[merry rune]

Flit's author didn't want options there at all. He wanted setuptools only. Which I think is terrible, that's just setuptool's docs. It's great to show it's general and a "packaging" tutorial and not a setuptools tutorial. As well as this helped setuptools to support things like src config out of the box in this mode. 😄

So I think listing it is fine, but not highlighting it / making it default. But you could verify.

[fallen shuttle]

It's been a while that we released the feature, so I think it is safe to say we will not have to back out from pyproject.toml. We have an open PR that I have already approved (waiting to see if other maintainers are also on board) clarifying that.

However, I think we would like to keep [tool.setuptools] as experimental for a while: since we have a good opportunity to rethink some design choices, it is better if we think this one more through.

The javascript for a random choice is indeed a good (and fair) idea

[mortal shore]

Js for random choice sounds confusing to people who refresh their browser or switch computers

[golden falcon]

We could just not have a backend selected by default w/ some placeholder text

[idle egret]

Is https://pypi.org/project/wheel-inspect/ the tool of choice when I want to read some metadata from a wheel? Or are there any alternatives?

[queen hornet]

rename to .zip and look inside 🙂

[idle egret]

This needs to be done programmatically...

[queen hornet]

okay! I've not done it programmatically but at least that library looks good and has a recent release

[merry rune]

I like the placeholder + select a backend idea.

[fallen shuttle]

I think we would like to keep [tool.setuptools] as experimental for a while
Just to clarify that it should be perfectly useful without [tool.setuptools] (without this table, the flexibility is not as much as setuptools currently offers with setup.cfg, but it is a matter of organizing project layout and worflow)

[rapid glacier]

If I can chime in as someone, that recently tried to explain packaging to someone else and also tried to find the best solution for that same person:

Do not make it ambigous, having a placeholder there sounds fair and future-proof, but fails to achieve the goal of being a tutorial.

In my opinion, there should be an minimal working example and if you want to show libraries too, do it only if you also show the features of them and differences of those libraries to 1. the example 2. each other.

Having a placeholder and a straight list sounds perfect if you already have the complete knowledge, but is very confusing and contraproductive if you want to help new people. You want people to be able to get it working and only then makes it sense for them to look into option ala „what is actually the best library for me“. Answering that question without having gone through the process the „standard“ way, is impossible.

Side note: There is also the problem of quasi circular referencing if the tutorial tells you to look at the library documentation to decide and the library documentation often telling you to look in the pypi documentation. Not really circular, but the doc hopping can confuse.

Side note 2: Choices in tutorials seem weird for me, but this is most likely just my personal subjective opinion.

[merry rune]

On reflection + your comment, I think having a default but also letting a user see there are other options is probably better than having a placeholder. It is important to balance having a simple tutorial someone can follow and showing there is choice and that this tutorial is intended to be useful regardless of what packaging tool you are using. I'd like to avoid every packing tool having to rewrite the exact same tutorial; it's better to just point at this one and say "use us" - the selector makes it very clear exactly what part is specific to a backend.

You can see almost exactly what the current proposal looks like here: https://scikit-hep.org/developer/pep621#pyprojecttoml-build-system

Also the current tutorial currently has a choice (setup.cfg vs. setup.py) which is much bigger and more confusing than the two line selector here. It's much more likely to get accepted than a placeholder version that requires a user to pick a backend, I think.

[indigo token]

You can also see really exactly how it looks like: https://python-packaging-user-guide--1031.org.readthedocs.build/en/1031/tutorials/packaging-projects/

[golden falcon]

can we generate link anchors for the different backends?

[merry rune]

I just thought of that! (related to Scikit-HEP dev pages, actually, and not related to this selection). It should be, thought it depends on the JS behind this. Since we are using the tab JS from a package, might need to be added there? I probably could do that, though.

I've supported that through JS (which sends it to Pyodide) on https://scikit-hep.org/developer/reporeview?repo=pypa/build&branch=main - it's easy to do. We could have a &tab=setuptools or similar.

[merry rune]

@high stone already opened https://github.com/pradyunsg/sphinx-inline-tabs/issues/23 - that seems to already be opened, that's what we'd need here.

[rapid glacier]

That looks good, I still would think a clear Use this if you are creating your first package ever-statement would help, but the way it is set-up seems to be clear enough. Deciding which should be the default could also be impossible.

If it is ok, I would like to give a few more minor points from my recent experience (if this is not wanted here, just ignore the rest of the message, which I understand as it may be out of scope):

~~.) In my opinion, a tutorial for simple packaging should encompass the actual package creating. I understand that this is a enormous task, because most backend do it somewhat differently. ~~ saw that this is already in the second link and nicely done!

.) I don't know if this is actually bad practice, but finding out how to create and editable install was problematic when I tried to explain the packaging. From my understanding not nearly all Backends support it and the way they do it seems to be different and the command to do it seems to be different also.

To give a tiny background:

I wanted to help a friend that created a game via pygame to set up a nice structure, he already had the code. I created a pyproject.toml for him but then hit a wall as I tried to editable install it with no extra dependencies. Setuptools did not support it and I didn't want to do some pypath hack. The Project was never intended to be uploaded to Pypi so I felt like setting up flit was overkill. I tried to find out first, which backends actually are available and second what features they support. For both I was basically told to search on pypi and github and hope that their docs specify the feature I was looking for. In the end this turned out to be way to much work and I never actually finished helping him to set up the package. The same thing happened again when I tried to find out if there is any way to package cython files without a setup.py, there seemed to be no easy way to find out.

[high stone]

zipfile in the standard library?

[merry rune]

I use

# Show SDist contents
tar -tvf dist/*.tar.gz
# Show wheel contents
unzip -l dist/*.whl

But that doesn't show metadata. zipfile.Path would be easy here (wheel only, there's no tarfile.Path).

(visible at https://scikit-hep.org/developer/pep621#includingexcluding-files-in-the-sdist - just added that warning. Also that page supports ?tabs=setuptools now too 🙂 )

[idle egret]

Right. Then I'd still need to write some parsing logic for METADATA. I would assume this looks easy at first, and offers many edge cases later on 🙂

[high stone]

installer.utils.parse_metadata_file -- it's not actually that complicated honestly.

https://installer.readthedocs.io/en/stable/api/utils/#installer.utils.parse_metadata_file

[spiral urchin]

Because email.message does all the dirty work 🙂

[idle egret]

Uh. Very nice! And three familiar maintainers 🙂 That sounds like a safe bet. Funny thing - Dan is my colleague, and we worked together today. The Python world is small 🙂

[tidal kiln]

no need for any of this, the stdlib can do it all

#importlib-metadata message

[idle egret]

This is fancy!

import zipfile
import importlib.metadata as md
md.PathDistribution(zipfile.Path("test.whl", at="METADATA")).metadata

Thanks!

[tidal kiln]

you need to change to the .dist-info location, but yeah

that's it

I forgot to say it in my original reply

but I provided a link with a working code example

we should probably implement support in the the finder, so that it can do it automatically

maybe a ZipFinder or something

[woven plover]

i wish we could get a wheel 2.x format where dist-info no longer goes to version depedent folders,

[low drift]

Hello. For those who don't know me, my name is Shamika and I am the Packaging Project Manager at the PSF. I am setting up dedicated office hours to talk to anyone who would like some help in managing/planning/coordinating any project. If some issue (eg: PEP) is stuck for sometime and you would like me to knock on doors to get it moving, please use this office hour to raise the issue. You can raise anything packaging related. Please sign up using the link shared here- https://discuss.python.org/t/packaging-project-manager-office-hours/15662

[lean lake]

Hello and welcome 🙂

[echo ruin]

The project URL examples at https://peps.python.org/pep-0621/#example are incorrect, right? I took them at face value that for some reason some scheme-less URL was asked for, but PyPI seems to reject that (with a slightly hard to understand error message, though that could be coming from the thing which is doing the uploading)

https://github.com/Julian/regret/runs/6643641451?check_suite_focus=true#step:8:25 being the error message

https://github.com/python/peps/pull/2621

[golden falcon]

the error is from PyPI, not the thing doing the uploading (twine), and yeah, the URLs in the PEP are incorrect

[echo ruin]

Aha. I knew twine didn't spit out errors like that, but I was lazy in suspecting maybe that GH action was doing it. But ok thanks for confirming!

Do you know off-hand if there's a linter that'd catch this?

twine check doesn't apparently.

Ah, right, https://github.com/pypa/twine/issues/430 I always forget about

[lost thorn]

Must a URL have a scheme to be valid? I searched the relevant RFCs but couldn't find a definitive answer, and PEP 621 only says they must be a URL.

[echo ruin]

Yes

From RFC 3986:

The scheme and path components are required, though the path may be
empty (no characters).

There are scheme-relative URLs (which start with // but yeah you can't just start with the domain)

[ebon aspen]

is there a tool that verifies that the file hashes for an installed package in the RECORD file match the files that are actually installed?

[runic plank]

Hi, I was just wondering if anybody else is experiencing the issue that I filed here where a newly released package can't be found by pypi: https://github.com/pypa/warehouse/issues/11512 - given there were some hiccups with new packages yesterday I wasn't sure if they've all been resolved or not

[idle egret]

The most shocking result of the result #python developer survey 2021...
https://twitter.com/jugmac00/status/1532785826102489092

Jürgen Gmach (@jugmac00)

I think this is the most shocking result of the 2021 #python developers survey 😳

Twitter · Jun 3, 2022, 6:06 PM

[woven plover]

just another indicator that more people than acceptable are just fine with running catastrophes

[torpid veldt]

There should have been a nonsense answer

There's always a small percentage of people filling in a survey that put something absurd

[rotund gyro]

hi there!

I have uploaded a package, but I had to rename it to make use of underscore instead of hyphens. It's renamed everywhere, except in PyPI where it's still called with the original name which used dashes. Is there a way to fix it so it never uses dashes anywhere? Or the damage is already done?

it can be installed from cmd line with both names, but pip show package_name still calls it package-name

nevertheless, I see plenty other packages using hyphens in the name, and other cases of underscores in the files but hyphens in the names

I guess mine will be just another case of that

[lusty quarry]

@rotund gyro hyphenated version is the standard/normalized way. most things will represent it as such

[rotund gyro]

ok, so there's no shame in having package-name but package_name everywhere else?

[rotund gyro]

let's say I have pypi.org/project/package-name/, what should I put in GitHub? github.com/Whoeza/package-name or package_name?

[lusty quarry]

doesn't matter, but I'd choose the former (normalized)

[rotund gyro]

ok

I always liked it better with dashes (normalized)

it's unfortunate that python can't use it everywhere

[tidal kiln]

depends on which normalization you are talking about

file names and such use underscore for normalization 🤣

it's a mess

[spiral urchin]

Insert rant to whoever decidedt . should be normalised to - on PyPI but forgot to tell the rest of the world about it

[rotund gyro]

hello! I have published a couple packages on PyPI, on on 6/2nd, and one today. The one from today does not show in the package manager list in PyCharm. Does it take a bit of time for PyCharm to read it or is there a problem?

[lusty quarry]

@rotund gyro that'd be better for the Python discord

[rotund gyro]

I thought it would be a PyPI issue, but ok

btw I asked it there too

[mortal shore]

It's written in a PEP that it does that 😛

[lusty quarry]

for a spec, are we comfortable with enforcing the use of a certain library? I'm thinking of https://github.com/pytest-dev/pluggy

[mortal shore]

We've yet to require a library in a spec

[spiral urchin]

But the PEP was written after the implementation, wasn’t it?

[mortal shore]

Yea, the implementation was written in like 2005 or so 😛

[lusty quarry]

okay, raw entry point usage it is then 👍

[woven plover]

@lusty quarry whats the context?

[lusty quarry]

extension module plugins

[woven plover]

@lusty quarry as pep or for hatch specifically (for hatch specifically pluggy usage is way more open than a standard)

[lusty quarry]

pep, done implementation now just docs

only dep so far is importlib-metadata; python_version < '3.8'

[strange knot]

I decided to spin up something similar but a little more official (mostly to support the folks reporting/handling malicious distributions) and that handles all* distribution files: https://inspector.pypi.io/

• to be implemented

PRs welcome here: https://github.com/pypi/inspector

[lusty quarry]

oh neat, when did PyPI get its own org?

[high stone]

"2022-06-06T14:48:08Z", is the oldest event I can get from the GitHub events API. :)

[dense valley]

Great! (I did encounter a 500 https://inspector.pypi.io/project/pyjapc/2.0.7/packages/f9/90/1e210ef44f7dabfed93c2ec78acbe4d174a6051918e16973d290a9004784/pyjapc-2.0.7.tar.gz/pyjapc-2.0.7)

[strange knot]

We acquired it a while back based on our trademark and set it up a few months ago when the GitHub Actions queue for the PyPA org was getting really backed up, but we haven't needed to use it and haven't transferred anything there yet.

[strange knot]

Thanks! Yeah I think that's https://github.com/pypi/inspector/issues/6

[lusty quarry]

fyi for https://hackmd.io/@gaborbernat/py-packaging-summit-2022#Building-C-Extensions-without-Setuptools---Henry-Schreiner-III-and-Ofek-Lev

made #extensionlib cc @merry rune @mossy pulsar

[dense valley]

I see that this is the first project under https://github.com/pypi, what is the plan for this Github organization compared to the already existing https://github.com/pypa ?

[strange knot]

We acquired it a while back based on our trademark and set it up a few months ago when the GitHub Actions queue for the PyPA org was getting really backed up, but we haven't needed to use it and haven't transferred anything there yet.

The plan is to eventually move Warehouse and related repos there

[dense valley]

I only now see that this question was already asked, whoops 😅 . Thank you!

[dense valley]

But it is still considered under PyPA governance, right?

[lusty quarry]

I sent out the vote for #extensionlib https://mail.python.org/archives/list/[e-mail]/thread/H76Q6OO5TFHLR2IICXRRJ5ZHEPBPUVR7/

[torpid veldt]

Anyone know where I can get access to user agent stats for pypi? Eg how Darwin many users are using pip with SecureTransport?

Re #756342717725933612 message

[queen hornet]

I don't know if they're available, but for downloads, BigQuery has stats on OS and distro names (plus there's an openssl field, not sure what's in there)
https://github.com/ofek/pypinfo/#downloads-for-a-project-by-system-and-distribution

[torpid veldt]

just found out it's plaintext version string - and then ran out of credit

[queen hornet]

ah okay! it's a rolling quota thing, so it'll refill a bit in a day or two. do the OS names help?

[torpid veldt]

Yep should do

[strange knot]

The BigQuery dataset had all the user-agent details that pip provides to PyPI

[glass sand]

Cross-posting for visibility. I've added FFY00 to Gardeners for PyPA. Let me know if you have any concerns with the change.

[indigo token]

woopwoop, congrats @merry rune for getting the tutorial overhaul merged! https://github.com/pypa/packaging.python.org/pull/1031

now visible in all its glory at https://packaging.python.org/en/latest/tutorials/packaging-projects/

[low drift]

For those, who might have missed it on Discourse (https://discuss.python.org/t/updating-fundable-packaging-projects/16566), we are looking for Packaging improvement ideas that could be delivered quickly with funding. If you have an idea, please submit a PR here- https://github.com/psf/fundable-packaging-improvements. This list is open to all PyPA/non PyPA tools.

[lusty quarry]

is the test PyPI down? erroring for ~9 hours https://github.com/pypa/hatch/runs/7072766429?check_suite_focus=true

[queen hornet]

also getting 502: Bad Gateway or 504: Gateway Time-out here

[lusty quarry]

I'm so excited for this https://discuss.python.org/t/16879/14

[dense valley]

Onwards from https://discuss.python.org/t/14111/39, will the Discourse discussions of new PEP proposals be posted under the "packaging" category instead of the "PEPs" category?
The discussion of PEP-694 is under "packaging", while the one from PEP-691 is under "PEPs". I'm guessing this was deliberate?

[mortal shore]

uh, I didn't really think about it

I normally posted packaging PEPs under the packaging category

I didn't post the intitial thread on PEP 691, and I didn't really think about the fact it wasn't under there

[fallen shuttle]

It is out for testing 🙂 https://discuss.python.org/t/help-testing-pep-660-support-in-setuptools/16904

It should be at least better than python setup.py develop...

[merry rune]

Ahh, so that's what the beta release was for. Was wondering, there was no indication in the GH release or changelogs as to what the beta was for.

[lyric hedge]

I'm totally out of the loop with setuptools work but it's really nice to see PEP 660 support come (close) to fruition :)

Congrats! I'll see if I can test it out on psf/black sometime tomorrow

[fallen shuttle]

Hi Henry, I added some notes here : https://github.com/pypa/setuptools/releases/tag/v63.0.0b1.

[fallen shuttle]

Thank you very much @lyric hedge . And thank you very much for all the prior work that helped us in this process.

[high stone]

@pypa has been approved for GitHub Sponsors

Just got an email from GitHub about it! :)

@di looks like your tweet worked?

@strange knot ^

[strange knot]

Seems what actually worked is a few people reaching out internally. Still not entirely sure why it took so long. Pallets is still waiting AFAIK

[high stone]

Ooof. That's not great.

[lean lake]

O nice. 694 and upload. I need to tweak my notifications …

[lean lake]

How would this work? If someone came to bandersnatch and offered money for a feature I can’t take it under my employment contract …

Does the Money go to a PyPA pool in my case ?

Or could maintainers outsource / coach someone ?

[high stone]

I'm expecting that this goes into the PSF accounts ear-marked for the PSF's Packaging-WG.

[strange knot]

Yep, this, same as what happens with Tidelift funds, with the exception of setuptools.

[mossy pulsar]

And virtualenv 🤔

[strange knot]

Is that going to you? I'm also seeing it on the PyPA dashboard as well

[mossy pulsar]

Well it should at least 🤷‍♂️

Unless pypa called dibs on it and will be taken away 😂

[lyric hedge]

@strange knot fyi the description and sponsor amount don't match for the last one

[mossy pulsar]

Perhaps it's a hint to buy two of those 😂

[strange knot]

Thanks, nice catch!

[lusty quarry]

are we allowed to monetize/get sponsors from PyPA projects? when I transferred Hatch I removed that

[high stone]

I'm pretty sure you are -- setuptools does that.

[lusty quarry]

well that's nice, adding back now

[tidal kiln]

can someone with macos post the output of distutils.util.get_platform?

[queen hornet]

>>> distutils.util.get_platform()
'macosx-10.9-universal2'

I'm on macOS Monterey 12.4 (M1)

[tidal kiln]

thanks!

[zealous swan]

urllib3 had its first release with sigstore signatures that work E2E, details and demo here: https://twitter.com/sethmlarson/status/1545070131210059782

Seth Michael Larson (@sethmlarson)

📦 We've released v1.26.10 of @urllib3! This release drops support for #Python 3.5 and fixes an issue with a ProxyError message.

The major release highlight is this is the first release that's signed with @projectsigstore! 🎉🥳🎉

https://t.co/9ls519xQd9

Twitter · Jul 7, 2022, 3:39 PM

[blazing lantern]

Did you upload to Rekor? Did you do this offline or somehow automate w/ GH Actions? Or put another way: when is the blog post coming out? 😉

[zealous swan]

I believe so! This was all done using GH Actions. You can see the execution here: https://github.com/urllib3/urllib3/runs/7236139735?check_suite_focus=true and the configuration here: https://github.com/urllib3/urllib3/blob/main/.github/workflows/publish.yml

We'd like to do a blog post but we're super far on the bleeding edge of all of this that we wouldn't want to be the ones recommending it until the Sigstore folks are ready for that 🙂

From the output it looks like we're at these indices in Rekor:

Transparency log entry created at index: 2871035
Signature written to file sigstore-artifacts/urllib3-1.26.10-py2.py3-none-any.whl.sig
Certificate written to file sigstore-artifacts/urllib3-1.26.10-py2.py3-none-any.whl.crt
OK: dist/urllib3-1.26.10-py2.py3-none-any.whl

Transparency log entry created at index: 2871037
Signature written to file sigstore-artifacts/urllib3-1.26.10.tar.gz.sig
Certificate written to file sigstore-artifacts/urllib3-1.26.10.tar.gz.crt
OK: dist/urllib3-1.26.10.tar.gz

[torpid veldt]

I'm so glad a transparency log system is being used. I really want to be able to pass a single log root to pip and a requirements.in and have the same guarantees as pip-compile --generate-hashes

[mortal shore]

you would get different guarantees

a transparency log means that an attacker can't compromise an individual without putting the compromised artifacts on the log

You can narrow the scope some if you have trust relationships to know who is able to sign onto the log

but that's kidn of hard to do

most transparency logs implement it as monitoring, so that you get notifications the first time a new key is used, but that is an after the fact notification, not a prevention

[torpid veldt]

You can commit a historical root to a file in git

This gives you the "same as last time" guarantee that pip-compile --generate-hashes has

[high stone]

I spent 3 hours staring at my screen confused, which finally ended up being: https://github.com/pypa/packaging/commit/addd5c5699a04d07ae479438f33de7fb19b70632

I blame @mortal shore. This had basically no visible symptoms for the last 8 years.

Obviously Donald knew about this, but he committed it anyway because he wanted me to spent 3 hours on this today. :P

[mortal shore]

https://giphy.com/gifs/red-fang-3oz8xyBP22S5b6gmsw

that is an incredibly subtle bug, that's great

[blazing lantern]

"Incredibly subtle" is an understatement. The magic of regular expressions 😜

[queen hornet]

wow just looked at the whole regex and it's ~90 lines long!!

[mortal shore]

to be fair, assuming it's still mostly my oiriginal regex, a lot of those lines are to break it up to make it easier to read

[queen hornet]

plenty of comments too

[high stone]

In case anyone wants to upvote this on their end -- https://github.com/python-poetry/poetry/issues/5971

[naive geyser]

As a poetry user I’d upvote, but I’ve been curious of hatch. Would pypa support both?

[mortal shore]

pypa supports everyone, there's no one true project

pypa expends a signifcant amount of effort creating standards so that alternative projects can be created, whether that's to allow projects to make new and better ways of doing something, or just to allow projects to target different user groups with better features for that set of users

[tidal kiln]

ah yes 😅, one of the hardest bugs I tracked down was https://github.com/libratbag/libratbag/commit/8f88573a086ce8c20f150025d053441ee0351868

[lusty quarry]

I'm still most proud of this contribution https://github.com/skorokithakis/catt/pull/92

[lusty quarry]

did PyPI start sorting the project links on the left recently?

it no longer matches the metadata inside https://pypi.org/project/hatch/#files

[vital pier]

@lusty quarry likely related to the work done to make metadata part of a release, vs a project.
https://github.com/pypi/warehouse/pull/11566/files
SEE the usage of project_urls to a sorted by name.

[mortal shore]

It used to have no ordering specified, which meant it was just kind of whatever PostgreSQL decided (they officially say ordering without a ORDER BY returns rows in an unspecified order)

so I dunno, maybe in practice it returned it in order of insert most of the time?

[vital pier]

(didn't mean to call you out dstufft, but I found the change. ;) )

[mortal shore]

oh I was typing that up anyways

no worries

[lusty quarry]

ah ty. well now the first link for my projects is Funding which is a bit in poor taste imo

[mortal shore]

If we think that the ordering in the metadata is important then we'd want to explicitly track that order and record it as a column in the DB... but TOML doesn't allow you to have an ordered map, so we'd probably have to turn project.urls into an array or something

we could probably just pick some well defined URLs to put first

that already implicitly happens here: https://github.com/pypi/warehouse/blob/main/warehouse/packaging/models.py#L538-L559

with the old URL metadata for home page / download url prior the the project urls

[lusty quarry]

oh you mean JS can't do ordered maps?

[mortal shore]

TOML and JSON specs both explciitly disclaim their mapping primitives having any sort of order

[vital pier]

Time for an OrderedDict extension! 😅

[lusty quarry]

just fyi all Python toml libs read into dict as defined

[mortal shore]

I assume that's only true on Python 3.whateverintroducedordereddictbydefault

[vital pier]

3.7

[mortal shore]

that one

[lusty quarry]

3.6 without guarantee

[vital pier]

(based on insertion order)

[mortal shore]

since they'd have to go out of their way to implement TOML in a way that didn't insert keys in the order they were defined

which would be weird

[vital pier]

We could have Postgres order by row id which might be the same as insertion order

[mortal shore]

bringing me back to legacy pypi when OIDs played a non trivial part in the query pattern

[vital pier]

@lusty quarry curious, is "Funding" a standard URL prefix, or could it be "Project Funding" and solve alpha sorting that way?

[lusty quarry]

https://github.com/pypi/warehouse/blob/39daea188d2e1e6494c50753cd48cb5a8da3e8c4/warehouse/templates/packaging/detail.html#L30

[vital pier]

Thanks, missed that on my phone.

[mortal shore]

Mostly my opinion (and ofc that's just my opinion) is that given TOML and JSON both disclaim map sorting, that we can't rely on it in the specs that involve them, so that makes it hard to infer order from the metadata as it stands

[vital pier]

Agree. I think the idea of selection of a few priority keys in the dict to display first if available might make sense

[mortal shore]

which means we can either modify warehouse to put what we think are the best links first, or we can allow more keys besides "funding" in that html template

or both

or leave it as is

[lusty quarry]

actually wait PyPI gets this in the payload as an array, why talk of toml?

Project-URL: Homepage, https://hatch.pypa.io/latest/
Project-URL: Funding, https://github.com/sponsors/ofek
Project-URL: History, https://hatch.pypa.io/dev/history/
Project-URL: Tracker, https://github.com/pypa/hatch/issues
Project-URL: Source, https://github.com/pypa/hatch

[mortal shore]

Yea it does, the talk of TOML is because the direction the ecosystem is going, is that the hope is that the source of that data is coming from pyproject.toml, so conceptually our standards, when taken as a whole, are treating that data as an unordered map

just because one piece of the standards happen to be implemented in a order preserving way

I don't believe changes the overall expectation

[lusty quarry]

didn't we say no to that b/c dynamic would involve writing to pyproject.toml itself?

[mortal shore]

sorry I'm not saying that PyPI is going to read from pypyroject.toml itself, but that the "golden path" here is:

pyproject.toml -> build tool -> METADATA -> PyPI

So if the source format for that data is an unordered map, then I think the output is also an unordered map, even if the METADATA format itself happens to be implemented in an order preserving way

[lusty quarry]

ah ok 👍

[spiral urchin]

I think having a pre-ordered list of names makes sense. Although I’m surprised that Funding is listed first in this particular case… does the project not have documentation?

[lusty quarry]

Homepage is docs

[vital pier]

Let's assume Homepage is #1. What other important keys would come after that?

[queen hornet]

the order at https://github.com/pypi/warehouse/blob/39daea188d2e1e6494c50753cd48cb5a8da3e8c4/warehouse/templates/packaging/detail.html#L20-L60 is a pretty good starting point, could group the SCM hosts

[dense valley]

This would IMO be the most intuitive (using the icon names):
home > cloud-download-alt > book > scroll > bug > donate > ...other links

[lusty quarry]

also opened https://github.com/pypi/warehouse/pull/11824

[high stone]

I kinda wanna know more.

[mortal shore]

I barely remember the specifics at this point, I just remember dealing with OIDs being a thing in one of the PG upgrades, because it broke a bunch of queries

[high stone]

@tidal kiln Do you know who maintains Arch Linux's Python?

Specifically, trying to get folks to add the EXTERNALLY-MANAGED file into Arch-Linux installed Python.

[tidal kiln]

felixonmars

but you can also just open a bug in the bugtracker

[merry rune]

FYI, I've let the cat out of the bag today at SciPy, so will share here too: https://iscinumpy.dev/post/scikit-build-proposal/ has been accepted as of yesterday - I've been funded to work on Scikit-build for the next three years. 😄

[mossy pulsar]

https://c.tenor.com/GKx8If9fllgAAAAM/unai-emery-europa-league.gif

[indigo token]

That’s awesome! One question: In your blog post you said

Enscons: This is the only existing PEP 517 builder with compilation support at the time of writing.
which is wrong, since hatchling has hatch-mypyc. Will there be a hatch-skbuild? I think I prefer plugins to a million PEP 517 builders that can all just do some things.

[merry rune]

That’s the idea of exensionlib and a possible PEP. And that statement was not wrong since it says “at the time of writing”. That was written last year.

[mortal shore]

oh I didn't even notice it was written a year ago

lol

[hearty girder]

Hi everyone. I'm having trouble uploading a new project to PyPI. I get an error saying the project name is not allowed, even though an existing project with that name doesn't exist on PyPI. I'm not sure if it's a bug in Warehouse, or some other issue.

https://github.com/pypi/warehouse/issues/11872

I submitted a PEP 541 request
https://github.com/pypa/pypi-support/issues/2098

Then I noticed I'm not the only one having a similar issue with a completely different package name.

https://github.com/pypa/pypi-support/issues/2044

Any way to tell what the problem is, and get help fixing it?

[spiral urchin]

The most likely reason is your chosen name is either too similar to an existing project, or the name was previously used for malicious acts. I seem to recall that it’s unfortunately not possible to publicly publish a list of those names due to security reasons, and the only viable solutions are to either file a request (which you already did), or choose another name that does work.

[hearty girder]

Thanks. That's what I thought. Hopefully someone will look at my request soon. There seems to be quite a backlog in requests. Recreating my GitHub repository and renaming everything in my project would be a huge PITA. Plus, I have no idea what else to name it. 😄

I know there are many PyPI projects with separate names for a package name and project, but that bugs me. 😛

Anyone have any idea how often those requests are reviewed?

[spiral urchin]

I don’t really know, aside from “not that often” 😞

[hearty girder]

I was hoping I'd find a maintainer on here. Any other places to ask?

[spiral urchin]

There are some folks on here, yes. I don’t know a better place to reach out to (it’s particularly effective here either…)

[hearty girder]

I went with yara-mail instead.

What is interesting is, when I tried to upload yara-mail to Test PyPI after successfully uploading yaramail to Test PyPI, I got a much more specific error message that said the name was too similar to an existing project, until I deleted the yaramail project in Test PyPI.

(venv) sean@sean-desktop ~/d/yaramail (main) [1]> hatch publish -r test
dist/yara_mail-1.0.0-py3-none-any.whl ... failed
Client error '400 The name 'yara-mail' is too similar to an existing project. See https://test.pypi.org/help/#project-name for more information.' for url 'https://test.pypi.org/legacy/'
For more information check: https://httpstatuses.com/400

FWIW, I updated the bug report https://github.com/pypi/warehouse/issues/11872

[hearty girder]

Is there a way to update a project's PyPI README without publishing a new release?

[lyric hedge]

No, although you can use a post release version / build number (specific to wheels though AFAIK) to get across the fact this new release is a very very very minor.

[queen hornet]

The version specifier spec says you can actually end a version number in a .post component to tell installers like pip that you don't need to upgrade a package with the same non-post version, but that if this is a new install you want the post-release version. So if you just released version 1.2.3, realized you have a non-semantic thing to fix, and released 1.2.3.post1, installers will only install 1.2.3.post1 if you don't already have 1.2.3 installed. Since semantically the two version are supposed to be the same, there's no point in reinstalling a package just to fix e.g. a typo in a README file.

https://snarky.ca/what-to-do-when-you-botch-a-release-on-pypi/

(but unless it's a really high traffic package, I'd probably just do a minor)

[lean lake]

Do we have any pipenv people here?

Hi Cooper,

Yes, understandable. Thanks for having been a user of our services! Dependabot indeed does a good job, too ― for many.

Hey, if you work for PyPA, maybe you can help me get in touch with the maintainers of Pipenv, which uses our DB. We've been trying to get in touch with Dan Ryan and Kenneth Reitz for weeks, but neither are responsive. I also just wrote to Oz Tiram.

We are changing our JSON structure and would like to provide a custom endpoint for Pipenv.

Could you help us find out who we should talk to?

Thank you!

This is from people @ PyUP - Tristan Laurillard [e-mail]

I told them to just do the PR on the changes they'd want and open an issue (not sure if they had)

[spiral urchin]

Oz will probably respond to the email. Not likely for Kenneth (🤷) or Dan (day job)
This sounds like good news! It doesn’t seem like there’s something concrete we need to do right now, but once there’s something to try out I can also nudge some people about this.

[lean lake]

Cheers legend

Apparently Oz did end up replying.

[hearty girder]

Anyone know what special project.urls values PyPI does different things with? I can't find that documented anywhere. For example:

• Setting the Homepage URL to a GitHub project adds GitHub stats to the PyPI listing
• A URL named Documentation gets a notebook icon

Now I'm wondering about an issue tracker URL.

[lost thorn]

Have a look in the template: https://github.com/pypi/warehouse/blob/main/warehouse/templates/packaging/detail.html

[hearty girder]

Good call. Thanks!

[strange knot]

Related issue here: https://github.com/pypi/warehouse/issues/5947

[lyric hedge]

oh you just got here, I thought you were here already 😅

[distant briar]

hi lol

I needed to ask sth in relation to cibuildwheel and remembered that I at one point saw there's a PyPA Discord

I didn't know you maintain bandersnatch 👀

[lyric hedge]

I mean, maintain is a strong word, I don't do much but I do have the commit bit

[distant briar]

well, you must have done something if you have the bit ;)

[mortal shore]

I don't do much, but I do have the commit bit

I feel attacked

😄