#packaging

1 messages ยท Page 2 of 1

slow pagoda
#

(I assume the reason for prek.toml was to also support pyproject.toml, but don't think that happened yet. I have not used prek.toml at all)

north karma
glad acorn
slow pagoda
#

The downside to having it in prek is it wouldn't run in prek -a. ๐Ÿ™‚ But I think it should be there too. I've hand (well, with an LLM to start it) written a Python-based pre-commit check in https://github.com/pypa/packaging/pull/1133. I had to fall back on querying the GitHub API because something is weird with typo's tags, I think they get force pushed as part of the release process changelog generation, if I had to guess. The autoupdater for both tools doesn't have a problem.

mental locust
#

The moment I hit post to say something is reviewable is the moment I spot half a dozen things I don't like

distant void
#

That is my lived experience as well. Usually I spot the issues immediately after I open the PR.

slow pagoda
#

The moment someone approves it is when I usually find things I don't like. ๐Ÿ™‚

slow pagoda
#

FWIW, since we have defined __all__ and __dir__, I don't think it's necessary to start everything with underscores. I think the classes/functions are okay, to help highlight they are really not public, but the global constants probably don't need underscores unless you really want them to.

#

(though it doesn't matter too much)

#

Would it be better to make _BEFORE_LOCALS and _AFTER_LOCALS sentinel objects or enums, instead of ints?

mental locust
#

Yes, it would

#

I've got some simplifications to push, I'll include that

slow pagoda
#

I asked copilot (locally) for a review (just before posting my comments, so it wouldn't be able to cheat), and it gave five suggestions. Three of them are basically my suggestions: magic ints - it literally called them magic ints like I did), dataclasses (though it suggested frozen dataclasses), and move the type alias import. One of them was to modularize large test matrix, which I don't think matters. The last one, though, is that _get_intervals() always recomputes self._intervals, rather than returning the cached value if it is not None.

mental locust
#

You really need to tell LLMs to actually validate their conclusions

#

At least, in my experience that makes them produce 10x more useful commentry

slow pagoda
#

This is what it didn't like:

    def _get_intervals(self) -> list[_SpecifierInterval]:
        """Compute and cache the intersected interval representation.

        Returns an empty list if unsatisfiable, or the intersected interval
        list otherwise. ``===`` specs are modeled as full range (no
        constraint).
        """
        specs = self._specs

        # Intersect specs' intervals, with early exit on empty intersection.
        result: list[_SpecifierInterval] | None = None
        for s in specs:
            if result is None:
                result = s._to_intervals()
            else:
                result = _intersect_intervals(result, s._to_intervals())
                if not result:
                    break

        assert result is not None  # specs is non-empty
        self._intervals = result
        return result
#

Ahh, I see, it is never called except in a place where _is_unsatisfiable is already verified. IMO it needs a little comment explaining that. It's just as confusing to a human reader unless they know to check the places it's called.

mental locust
#

Yeah, I'll add that

slow pagoda
mental locust
#

It's been a fun exercise building the un/satisfiable Specifier lists, some real gotchas in there, can perhaps put on a Python packaging trivia quiz, e.g. why is >1.0a1,<1.0a2 satisfiable?

iron light
#

o_O

mental locust
sage marsh
mental locust
#

Nope ๐Ÿ˜‰

mental locust
mental locust
#

==1.0.post0.dev0,<1.0.post1 and >=1.0.dev0,<1.0.post1,!=1.0,!=1.0.post0 manage to trigger multiple interesting parts of the spec that I was not modelling correctly ๐Ÿ™ƒ

solar heron
#

should just raise an ThisIsActuallyStupid("you should be fired from packaging python code") exception

mental locust
mental locust
#

Ignore everything I wrote, I realized this is a specification issue

mental locust
ruby needle
#

Hi! I'm working on a tool that reads an existing wheel's METADATA file, modifies some fields, and writes it back. I'm using packaging.metadata.Metadata.from_email() for reading which seems clearly right, but I'm less sure about the writing side.
packaging v26.0 added Metadata.as_rfc822() which looks like it could work, but I've seen suggestions that packaging wasn't originally intended for writing these files. pyproject_metadata.StandardMetadata is another option but feels more oriented toward pyproject.toml โ†’ METADATA generation rather than mutating an existing file.
What's the recommended approach for round-tripping an existing METADATA file? Is packaging v26.0's serialization considered stable/appropriate for this use case?

wide herald
#

Not a packaging maintainer, but IIRC METADATA files are not supposed to be changed, but rather considered a read-only information. Why are you modifying the file?

heady prism
ruby needle
#

Thanks for the replies. The purpose is to change the metadata during building compared to what the backends wrote, rather than take existing artifacts and mutate them.

#

Maybe just change what's in the source in the first place is the better approach?

mental locust
#

To answer your question about is round tripping considered stable, I've never touched this code,but I assume not, at best packaging is just intended to read, I think build back ends have their own implementation to write. That may change in the future if we get JSON metadata.

#

I would look at your backend of choice and implement what it's using, which is probably the email module for the Python standard library

slow pagoda
#

as_rfc822 does write these files, what is it for if not writing an rfc822 file?

sage marsh
#

Trying to perfectly round trip a metadata file with packaging probably wonโ€™t work in terms of byte for byte round tripping

slow pagoda
#

Yes, it was not originally able to write these files, that's why it wasn't in before 26.0. ๐Ÿ™‚

#

It won't byte for byte, no, it has a fixed format.

sage marsh
#

It should be possible to semantically do it just fine though

slow pagoda
#

It's not really all that useful yet, since we can't read pyproject.toml's yet, but that's something I'm slowly working on.

#

But for reading, modifying, then writing, as long as sematics is what you care about, I think if it fails it's a bug.

distant void
#

You can sniff those files to figure out some otherwise opaque information about how the backend got its data. I learned this while exploring how to recover extras "accurately". Setuptools will write the fields in different orders depending on whether you really provide an extra or tried to be sneaky with the extra marker in a regular dependency. I wouldn't be surprised if there are more useful cases of looking at the ordering of the fields. Mine is just a weird thing I noticed.

ruby needle
#

Thanks very much for the helpful info. Will have a closer look at the options for my use case ๐Ÿ™‚

mental locust
#

I'm thinking of adding property based testing to packaging, in theory we should be able to translate statements in the spec to property based tests. I've been using it in a library in writing that might be a new downstream user of packaging and it's great at finding stuff that relates to the spec I implemented

mint urchin
#

with hypothesis?

mental locust
#

Yes

finite temple
mint urchin
#

ugh I messed up the git history on my PR ๐Ÿ™

#

hope itโ€™s not too annoying to squash it into one commit

mental locust
#

I think all PRs are squashed on packaging

sage marsh
#

likely the case, I tend to try to do that on all projects I can lol

robust hollow
finite temple
#

Sorry, I realized I'm not sure if you're talking about PBT in general or specifically for fuzzing. Hopefully the above is not reiterating what you already know ๐Ÿ˜…

mint urchin
#

Liam DeVoe told me heโ€™d be at PyCon if youโ€™ll be there and want to learn more from one of the experts

wide herald
mental locust
unique flint
#

in the past i've just set an env var on hypothesis tests that controls how long they go. if a change feels risky, you can have it run for much longer. hypothesis.example and experiments db also useful

mental locust
#

ai-python a now seemingly defunct project from 2021, pins most of their dependencies, but in early versions of the project instead of using == they mostly used <=V,>=V e.g.

Requires-Dist: setuptools (<=54.2.0,>=54.2.0) ; extra == 'required'
Requires-Dist: spacy (<2.3.5) ; extra == 'required'
Requires-Dist: tensorflow (<2.6,>=2.2.2) ; extra == 'required'
Requires-Dist: termcolor (<=1.1.0,>=1.1.0) ; extra == 'required'
Requires-Dist: toml (<=0.10.2,>=0.10.2) ; extra == 'required'
Requires-Dist: torch (<=1.7.1,>=1.7.1) ; extra == 'required'
Requires-Dist: torchvision (<=0.8.2,>=0.8.2) ; extra == 'required'

Found looking at Henry's search of project requirements:
https://github.com/pypa/packaging/pull/1140#issuecomment-4181987311

wicked hare
#

This is interesting. Have you considered, if it would make more sense for the spec to change and maybe making a dpo thread to discuss such possibility? <1.0.post1 causing pre-releases to be included seems strange

mental locust
#

It seems strange, but if you don't allow it you break the monotonic invariant that if v2 < v1 and v1 matches <V then v2 must match <V

#

And to be clear <1.0.post1 in packaging can already accept pre-releases of versions <1.0, and it would still not allow pre-releases of 1.0.post1, this change means it would start to allow pre-releases of 1.0 and 1.0.post0

#

The spec is complicated, there's two parts of the spec this is interacting with at the same time:

The exclusive ordered comparison <V MUST NOT allow a pre-release of the specified version unless the specified version is itself a pre-release.

And:

Pre-releases of any kind, including developmental releases, are implicitly excluded from all version specifiers, unless they are already present on the system, explicitly requested by the user, or if the only available version that satisfies the version specifier is a pre-release.

So a pre-release won't be accepted by default unless there is no other option, so this does not change the result of <1.0.post1 when the options available are 1.0.b1 and 0.9, in that case 0.9 will continue to be chosen, it only changes it if you have <1.0.post1 and your only option is 1.0.b1, prior to this change 1.0.b1 would not be accepted, now it will be accepted.

wicked hare
#

Oh

#

My bad, I didn't know the exclusion of prereleases from all specifiers is defined separately

mental locust
#

The pre-release exclusion rule has been a real pain to implement and the source of many many bugs, uv has chosen not to comply with this rule, because they use pubgrub-rs as the basis of their resolver, and it considers each version independently, and this rule requires you to have information about the entire set of versions you're considering

#

I am working on an extension to pubgrub right now, just last night I got this rule working at the resolver level, no complicated provider hacks needed

#

uv also reinterprets what "explicitly requested by the user" means compared to pip/packaging, specifically packaging interprets this to mean that is a pre-release is part of the specifier version then it is explicitly requested by the user, e.g. <=1.0beta10 includes pre-releases, and pip just inherits this on all dependencies.

uv only accepts this interpretation for requirements that user has explicitly provided (command line, requirements file, your own projects pyproject.toml) it does not consider a dependency or transitive dependency to have a requirement like that to be explicitly requested by the user, which is a fair enough interpretation. But it means some requirements won't work in uv unless the user explicitly allows pre-releases for transative dependencies, e.g. if a dependency specifies ==1.0beta1 on a transitive dependency it will fail by default in uv.

mental locust
#

Bashing the spec a little, the main problem with it is conflating two different levels: The core version specification, and tool policies for interacting with versions. Were we to rewrite it today I would want those two things decoupled, one spec would be the pure mathematics of version specifiers, and another spec would be reccomended policies for tools.

#

I guess this lesson feeds into the wheel variant PEPs, which have indeed been decoupled

mint urchin
#

also that PEPs should have test suites for their specsโ€ฆ

#

end-to-end tests with real-world packages

mental locust
#

Astral tried to provide a tool agnostic test suite: https://github.com/astral-sh/packse. But I don't think it got any buy in from other tools, and when I spent some time reviewing it certain choices are in fact not agnostic or spec complaint and I never had time to work on it.

dense yacht
#

It's more like we desperately needed a tool so I made one haha

#

I would love for it to be generalized into a standard but yeah in its current form it reflects uv's choices and nobody has the time to advance it into an ecosystem utility

mental locust
#

It's a great idea, but when it came out I didn't have enough knowledge and experience with Python packaging to be able to fully understand where it's strengths and weaknesses were, I just wanted to wite it up to pip testing and get an immediate win, and it didn't work in a lot of situations so I didn't have the time or knowledge to figure it out

iron light
mental locust
#

Yes, it is!

iron light
#

phew

#

because that er, "quiz" from the other day (I never saw an answer key) made me wonder...

iron light
mental locust
#

Honestly, I'm not always sure when something looks like it should be logically equivalent if it actually is

polar nimbus
#

@mental locust, are you perhaps the best person to query about the packaging project? In https://discuss.python.org/t/pep-825-wheel-variants-package-format-split-from-pep-817/106196/46, Paul asked about necessary parse_wheel_filename() changes due to wheel variants changing the filename. My best idea so far would be to soft-deprecate that function to avoid breakage and add a new function that accounts for the variant label. Basically, the old one will continue working for non-variant wheels and will raise an exception for variant wheels, while the new one will work for all wheels. How does that sound?

mental locust
polar nimbus
polar nimbus
slow pagoda
#

I have an open PR to add a new API, just waiting on trying to match PyPI error messages. That might be perfect to include variants.

slow pagoda
mental locust
#

I saw, I've been completely slammed this week so far, I was hoping to update my status on that tonight

iron light
#

I've been hearing exciting stuff around here lately. Been thinking about reaching out to LWN again to write up something about it

slow pagoda
#

#1145 adds a public API for markers, using dataclasses to make objects (internally it continues to use lists of strings recursively to represent nodes, no runtime cost). Opinions on the public API? I'd like a second opinion before adding new API. And/or use .operands.

distant void
#

I only read through #1145 pretty briefly, but it's an API that I would like using. It does seem a little strange that the canonical internal representation is a list of strings, but then a tree can be reconstituted from it. But maybe that's no big deal? I don't know enough about the marker parsing to say if it's cause for concern.
Just as a user giving feedback on "as_ast()", I'd say it looks good.

slow pagoda
#

It's a recursive list of strings, technically. I think using this internally would both be slower, and break the packages using the internals now. Though we could swap the internal representation and generate the "internals", but as I said, I expect it's slower than strings.

Thanks for the feedback! I think it's pretty standard for a node tree.

slow pagoda
#

All the minimum viable release PRโ€™s are in, does anyone want to review the other three or so that might go in or might not?

#

Itโ€™s really just two, the CLI one can move

mental locust
#

I might have time later today to review, I can't guarantee but I'll try

slow pagoda
#

OK, thanks! If you can itโ€™s really the first and last in that list, the CI one can wait till the next release if you want. uv team seems like theyโ€™re OK with fixing marker order to match the spec two, UV inherits some of packagingโ€™s inflexibility, but some of the cases work reverse already there.

slow pagoda
#

Actually, letโ€™s hold off on the marker order one, thereโ€™s some discussion from uv in changing the spec rather than updating the handling

#

So the new API one is the only one that really needs to be looked at before a release

mental locust
#

I added a review to the Marker API, I found a couple of issues you may or may not agree with

slow pagoda
#

I think it's fine to delay this to 26.2, there's also a followup needed to convert the trees back into Marker's. It would also give us some time to see if the projects needing it can test it out.

#

So I think we are ready for release. Do we want an RC? What sort of timing are people interested in?

mental locust
#

I don't think we need and RC, depends how risk averse you are ๐Ÿ˜‰

slow pagoda
#

We needed them last time due to the release mechanism changing, but now that should be stable. This is also about 3x/4x less development time, so should be smaller. Though I'm somewhat risk averse. ๐Ÿ™‚

#

I'll prepare the release bump PR, and we can discuss there, I guess. I'll assume 26.1 for now.

slow pagoda
mint urchin
#

thanks again for putting a release together!

slow pagoda
#

Tagged, deployment awaiting approval. ๐Ÿ™‚

mental locust
#

We need to figure out a way that more maintainers can approve deployment

#

On the pip side the release manager is responsible

slow pagoda
#

I can add people to that list (up to 6, currently at 4, or switch to a team)

mental locust
#

I think a team makes sense?

slow pagoda
#

On the pip side the release manager is responsible
That sounds like one person?

mental locust
#

Yeah, when I'm the release manager on pip I build and approve the release

#

It's all done on the CI side with trusted publishing, but I don't need to wait for someone else to sign off on release

slow pagoda
#

I could add myself to the release approval list and sign off on my own release ๐Ÿ™‚ But I like having it a team effort. If I could require 2 approvals, I'd add myself to the list.

#

Oh, I can require a different person from the one who made the release to approve, that might be useful.

#

The current list mirrors the PyPI owners. Since only those people can yank a bad release, I do want to make sure they are aware of releases, in case they have to respond and yank. That was the thought behind the current list.

dense yacht
slow pagoda
#

GH's UI around deployments is terrible. It loves to show the diff for the deployment instead of taking you to the page with the approval banner. I found if you click "..." then "view workflow run", you can get there from the normal pages. Otherwise you have to go to actions to find it.

#

I think that list are also admins on the repo on GitHub, which means if an account was hacked then they could just change the setting and deploy anyway. ๐Ÿ™‚ But yes, I like the idea.

dense yacht
slow pagoda
#

I enabled it anyway, might be a small improvement.

slow pagoda
#

Does anyone know how to just run tagged commits with asv? I thought it was possible, but didn't see an easy way and looping over with tag^ caused it to run overnight and run I think every commit from the start to the tag.

teal pivot
solar heron
#

How am I supposed to handle ExceptionGroups in a backwards compatible way? Either the built-in group is raised or a custom packaging "group" exception is raised. The latter is not documented in the API.

#

Also congrats on the release!

#

Reading the PR that added exception groups to packaging, it seems like the intention is that users import packaging.errors.ExceptionGroup despite being undocumented.

finite temple
solar heron
slow pagoda
#

It was originally there, but now it's in errors and re-exported for back-compat

#

Documentation is probably an oversight

slow pagoda
#

Missing error.rst is a mistake, as is no docs - since it's actually just ExceptionGroup from the stdlib on Python 3.11+, and we build the docs with 3.14 or so, it's empty.

#

Also, an advanced tip, if you already have an ExceptionGroup backport, you should be able to replace errors.ExceptionGroup with it. I don't think I want to document that, but I do that with pyproject-metadata in scikit-build-core to get the nicer backport.

slow pagoda
#

Ouch, mistake in the pyemscripten implementation, PYEMSCRIPTEN_ABI_VERSION -> PYEMSCRIPTEN_PLATFORM_VERSION. Though I think it actually works better as ABI, as it's not the whole platform.

slow pagoda
#

Do we want to add support for patch releases, or just call the next release 26.2? We need to fix this and get a release out, sadly, it doens't follow the PEP as is.

#

I'm guessing our release system doesn't support patch relases since it's not semver.

glad acorn
#

The packaging library uses calendar-based versioning (YY.N).

Go for 26.2

sage marsh
mental locust
slow pagoda
#

It has moved faster in the past. ๐Ÿ™‚ Is there anything else that is needed for this release? Has someone tried removing dependency-groups in pip yet? (I can quickly try if not). And maybe pylock or direct_url updates in pip?

#

We could get into a problem if we released YY.N dropping support for Python 3.X then make a new release, then need a "patch" release that fixes the old Python version... (that's why I consider dropping a Python version to always require a minor bump in semver-based packages)

mental locust
#

I don't think any PRs have been raised on pip side to use packaging 26.1, I plan to raise one later today, but it's specifier related

solar heron
#

There is one for Direct URL

#

I'd like to replace dependency-groups, but that PR will have to wait until Friday at the earliest

mental locust
slow pagoda
#

I can make it if you want (I actually already did it assisted by kimi-K2.5, but there are 13 errors locally before editing so I can't be sure it's passing (error handling is the only real change)

#

@solar wind what about pylock? Doesn't have to be a PR, just wondering if there are any possible things that might break, wondering if you've tested it yet.

solar wind
slow pagoda
#

Great!

#

If nothing comes up, maybe we can release later today?

#

My dep-group change has a matching number of errors after making it (13), so maybe it's fine. I can make a PR and @solar heron can edit/start from there if it's easier (or not, doesn't matter to me).

mental locust
#

I notice I just beat Henry on number of PRs in packaging 26.1: https://github.com/pypa/packaging/pull/1156, 24 vs 22. I keep getting nerd snipped by possible performance improvements.

Also I'm sure a couple of mine a directly based on mistakes I made in earlier PRs, so take away those and I'm certainly less.

slow pagoda
#

My goal is to always be third. ๐Ÿ™‚

#

Iโ€™m quite surprised we (which means you) managed another 2x performance boost on top of 26.0

distant void
#

I didn't realize I needed to call dibs if I wanted to do the pip PR for dep groups! ๐Ÿ˜‚
LMK if you need review eyes.

slow pagoda
#

Please do review! I mostly wanted to make sure it worked.

#

Happy to grant edit privileges if you want them to my fork

distant void
#

Don't worry about edit privs -- but I'll take a look this evening.

#

I also wanted to make sure it worked. Good opportunity to flag any issues with the exception group swap, which is the only thing I worried about at all.

#

If nobody gets to it first, I might do a PR to add the missing doc for those, per above.

#

Oh, merged 3 hours ago. Nevermind! ๐Ÿ˜†

slow pagoda
#

Tests are passing, except for changelog entry ๐Ÿ™‚ I might wait till you review to add an entry so it doesn't get merged.

distant void
#

I took a minute during my lunch break and left an approving review. It was way simpler than I expected, having forgotten everything about that code.

slow pagoda
#

Thanks! I think we are about ready for a release, then, will trigger soon.

slow pagoda
#

There are no discussions or projects, should we turn them off?

mental locust
#

I think it makes sense to turn off discussions unless someone is actively monitoring them

slow pagoda
#

Curious to see what other options we have for __reduce__ vs. __getstate__/__setstate__. I'd have gone for the latter when implementing a backward compatible fix, but storing the string and using __reduce__ is simple. Expecially if we want to throw this on some other classes (maybe without the back-compat shim if not requested?)

mental locust
#

My only concern is string is slow, if the code isn't complex I'd rather serialize via parts

slow pagoda
#

Proposed a version that's pretty significantly faster than either previous or string versions.

#

Smaller than the old version too (not as small as string, of course)

mental locust
#

I've skimmed over it, looks great to me, won't be able to sit down and give a serious review until late this evening

slow pagoda
#

Do you want to give it a serious review, or do you want me to approve and merge? The PR author might go on to apply this to other classes if it's merged. ๐Ÿ˜‰

mental locust
#

I'm happy with you approving and merging

slow pagoda
#

I've added the other core classes that someone might pickle (including me, I pickle to save time reloading when doing my analysis on packages) in three more PRs. Mostly just used a model to apply the same idea in the Version PR, along with instructions for generating the test data. Minor touchup on the logic being a bit too defensive. The last three versions (25.0, 26.0, 26.1) are supported - since movement has been pretty slow for internal changes before 25.0, versions before that probably work, but not tested or guaranteed. Eventually we could drop support for loading pickles before 26.2 - after 26.2 the format should be stable.

#

It's best to do this now, so we don't end up with too many versions to handle. After that I think we are ready for 26.2. (There's also the ExceptionGroup reexport PR).

sage marsh
#

I wonder if Metadata should have a way to go back to a RawMetadata (or if it should have some other way to turn a Metadata into a dict).

slow pagoda
#

PyPI merged the pyemscripten support. We should probably move forward. Are others fine with my pickle-safe PRs (especially the Specifier one, since Requirements/Markers depends on it), and the re-export of ExceptionGroup?

(Note: pyodide is going to also set the incorrect parameter to make packaging 26.1 work)

slow pagoda
#

Last two PRs (though I'll probably propose a documentation update too, with a note about pickling. I think we should only ensure 26.2+ pickles are backward compatible, with a potentially temporary support for 25+. Before 25+ is unsupported, but might work.). @mental locust, do you think you could review? BTW, it's intended to be much cleaner when we can use pattern matching. ๐Ÿ˜‰

mental locust
#

I'll try and do a fly by review tonight

slow pagoda
#

Thanks!

teal pivot
#

I'm out sick, but if you ping me directly I can approve any releases if necessary (I'm on the mend, but I'm not sitting on Discord either)

slow pagoda
teal pivot
#

And I happened to have taken a phone call, so I was sitting on Discord still ๐Ÿ˜…

slow pagoda
#

Nothing seems to have broken down yet. ๐Ÿ™‚ I'll merge the Python 3.8 drop on Monday if there are no objections.

mental locust
#

@slow pagoda Would you mind making a 26.1 and 26.2 announcement on DPO when you have a chance?

#

Also, I've been thinking about Paul's concern about introducing a public facing VersionRange object and I think I have a good solution, it won't be publicly constructable, it will just be a type that Specifier and SpecifierSet returns from a method like to_range, so it will be clear it is a specifier derivative

iron light
#

ah, only the major version is calendar-bound for packaging, yes?

mental locust
#

Yes (deleted previous message I thought I was in the pip channel for a second)

slow pagoda
glad acorn
#

I'm trying to add a pylock.toml to CPython's docs builds, and getting this error from pip:

WARNING: Using pylock.toml as a requirements source is an experimental feature. It may be removed/changed in a future release without prior warning.
ERROR: Cannot select requirements from pylock file 'pylock.toml': python_full_version '3.15.0a8+' in provided environment does not satisfy the Python version requirement '>=3.12'

https://github.com/python/cpython/actions/runs/25007535303/job/73234353237?pr=149058

looks like it's coming from src/packaging/pylock.py, and it's because CPython's dev builds are versioned like 3.15.0a8+ which isn't strictly a PEP 440 version

should the version check be less strict? other pip installs on dev CPython works

mental locust
#

Ah, this hits the fun situation that CPython does not follow PEP 440 Version semantics, please open an issue on pip with a reproducer

glad acorn
#

ok, to pip or packaging?

mental locust
#

Not sure, probably packaging, maybe pip

wide herald
#

I would blame CPython for using non-PEP440 versions /hj

glad acorn
#

CPython's dev version probably predates 440, so it's the PEP's fault ๐Ÿ™ƒ

solar wind
glad acorn
#

I can re-open at packaging

wide hamlet
iron light
#

clearly the solution is to just subclass PEP440Version, CPythonVersion, LegacyVersion...

wide herald
solar heron
slow pagoda
#

Do we need a release? Or could the patch just also be applied to pip (infuriating every distribution that un-vendors pip in the process... Okay, no). Or is it just something that should get in three months? (Basically, it is a pip 26.2 target or 26.1.1 target?)

mental locust
#

It's @solar wind call on whether to do a follow up pip release for this or not

solar wind
#

I personally don't feel that alone warrants a pip bugfix release. But if people think this is important and you agree to do a packaging release just for that, I don't mind doing a pip 26.1.1.

glad acorn
#

this likely only affects people using dev versions of CPython to install lockfiles, and we have a workaround of manually removing requires-python from the lockfile, so I agree it probably doesn't need a bugfix release just for this

mental locust
#

I've almost got this version range implementation and public API ready, performance is going to be good, but it was way more work than I expected to really ensure the public API acts as expected

mental locust
#

Fun fact, there is no way with PEP 440 version specifiers to express you just want the semantic PEP 440 version 1.0, as ==1.0 includes 1.0+local and ===1.0 excludes 1.0.0.

You could use Version("1.0") to represent that (that's what Poetry does in it's own stack) but that's not equivalent to a range and creates a lot of complexity when try to do algebraic operations.

My current solution is that while in general you will create version ranges from Specifier/Sets, there will be a class method to represent this specific point-like range of only being equal to 1.0.

#

Also, local versions are orderable, so the specifier ==1.0,<1.0+a would solve this, but using local parts on the version specifier for anything other than ==, !=, or === is syntactically illegal

iron light
#

It seems rather annoying that the existing PEP has these kinds of problems, and now we apparently don't want to fix it, or rather even if we "fixed" it by standard policy we would have to allow many years for everyone to get on board.

mental locust
#

Yeah, it's not ideal

slow pagoda
#

Would you have a recommended fix for the PEP, if you could make one?

mental locust
#

If I could go back in time and change the spec I would say that version specifiers are the natural specifiers on the version, e.g. ==1.0 is the same as V == Version("1.0"), and that all pre-release/post-release/local special behaviour are constraints that tools MAY or SHOULD apply

#

Having things like Version("1.0a1") < Version("1.0b1") < Version("1.0") but <1.0 matches none of those versions, but <1.0b1 matches Version("1.0a1") is what makes the whole thing incomprehensible to outsiders.

If we'd instead said that <1.0 is the same as V < Version("1.0") but tools should not provide pre-releases by default unless the user explicitly tells the tool that pre-releases should be allowed, or the tool MAY opt into pre-releases if there is no other version that matches the specifier.

Then the mathematics of versions and specifiers would be trivial to reason about, and the guards on pre-releases/sem-ver etc. would be the job of resolvers and installers, not the version spec.

sage marsh
#

== local versions were meant for downstream users to provide patches over a released version, if == 1.0 didnโ€™t match them, theyโ€™d not be able to do their job (itโ€™s possible their job is bad though!)

mental locust
#

Without understanding the use case fully if I could change time I would probably move locals out of the version spec and make them a distribution specific value, like build numbers, but I could be missing something important

sage marsh
#

If I recall correctly, part of the reason for the pre-release behavior was to make it so less things that used to work, continued to work. Probably we shouldnโ€™t have done that, but I think at the time it was a blocker for getting agreement lol

#

Yea moving it out of the version probably would have worked tbh!

mental locust
#

No doubt, hindsight vs. at the time consensus lead to very different conclusions

sage marsh
#

I donโ€™t think it occurred to me to do that. I was very mad at the time at dealing with downstream patches and the fact that โ€œ1.0โ€ meant very different things depending on where you got โ€œ1.0โ€ from

#

(To be clear, I have no ego attached to them, so it doesnโ€™t bother me thereโ€™s bad parts ๐Ÿ˜… Iโ€™m just happy the bad parts arenโ€™t blocking improvements)

mental locust
#

From my work today I've only found one actual breaking of invariants required for PubGrub style resolver , that is ==={non PEP 440 version} explicitly break De Morgan's law when you try and take it's compliment, I'm just going to leave it as an "oh well, things might break when if ==={non PEP 440 version} is involved in backtracking"

sage marsh
#

It probably wouldn't be too bad to just unsupport that particular thing

#

it was just there pretty much to support cases where old versions failed to parse completely

#

that's pretty nice that that's the only issue so far thoug

iron light
#

A concrete example would probably be useful for documentation purposes

mental locust
mental locust
#

On building a universal locker it turns out it's more or less required to have some minimal form of set algebra for markers.

Or, at least, you need to be able to tell if two markers are disjoint under certain conditions, you can effectively hack around it with regexes because the specific scenario doesn't turn up too often, but I guess I'm going to have to more deeply understand what's being proposed in the marker API and what we could do better here