pyproject-metadata

[pseudo yew]

https://discuss.python.org/t/add-pyproject-metadata-to-the-pypa/22891

I'm starting the transfer process

[agile hatch]

Pre-commit needs to be reactivated: https://github.com/pypa/python-pyproject-metadata/issues/90

[still coyote]

I can do all that, but want to get an OK from the relevant people.

[serene ridge]

Hi, can anyone push the release flow or add the trusted publishers? Thanks very much. We are eager for a new release

[still coyote]

I've set up trusted publishing configuration on PyPI.

[agile hatch]

Which looks better:

A: Field 'project.keywords' has an invalid type, expecting a list of strings (got 'some string')

B: Field "project.keywords" has an invalid type, expecting a list of strings (got 'some string')

?

[strong lynx]

Took me a while to even see the difference

[sonic monolith]

I wonder if giving the exact value here instead of the type is more confusing than helpful:

Field 'project.keywords' has an invalid type (got 'str', expecting a list of strings)

[terse mortar]

If you use typewriter quotes to signify a string, you shouldn’t also use them to signify a field identifier, especially since in TOML, 'project.keywords' is not equivalent to project.keywords/'project'.'keywords'. So either use text quotes:

Field “project.keywords” has an invalid type, expecting a list of strings (got 'some string')

or markup-like backticks:

Field `project.keywords` has an invalid type, expecting a list of strings (got 'some string')

to signify identifiers

@sonic monolith’s suggestion targets the same issue, and is an alternative way to solve it!

[agile hatch]

That wouldn’t fix nested type failures. “Expected list of stings, got list” for example. Also this is for all the messages, not just the type failures.

https://github.com/pypa/pyproject-metadata/pull/183 for reference

[agile hatch]

Support for both PEPs (#general message) are in PRs now. 🙂

[agile hatch]

If you use the classic license.file = "somefile.txt", and that file ended in a newline, would you expect the newline to show up in the License metadata? (keeping in mind the email.message format doesn't actually support newlines, but they are hacked in. But the JSON output would have it. And PEP 639 gets rid of this mess with whole files being inserted in email headers)

(Question for anyone interested)

[manic burrow]

I would say there should be a final newline even if the original file didn't have it. 😛

[forest zephyr]

I agree. Multi-line text should always end in a newline. Removing leading or tailing empty lines would be fine, though. I would not touch leading whitespace, just full empty lines. In any case, if there are sanitization steps, those should be part of the PEP.

[agile hatch]

PEP 639 gets rid of this mess, so it's only about the old version. 🙂

[agile hatch]

Opinions welcome on https://github.com/pypa/pyproject-metadata/pull/207

[agile hatch]

Yet another email-related bug for metadata to workound. Would love if we went to JSON metadata! https://github.com/pypa/pyproject-metadata/pull/232

[low oriole]

I am totally happy to have -- but not lead -- a conversation about figuring out the transition to JSON metadata (I think it's doable)

[devout monolith]

CC @subtle brook who's been thinking a lot about this too.

[subtle brook]

Oh I've been doing more than thinking 😆

I have slightly more than half a PEP drafted that I will probably return to in a few weeks. Happy to work with folks here on it!

[agile hatch]

I'd be happy to help on that 🙂

[delicate goblet]

I'm somewhat confused by some of the details in the environment markers grammar which allows weird stuff like "foo" == "bar" or "foobar" > (python_version or "bibbitybobbityboo"). Why was it made that way? And what is the intended way to use the "in" and "not in" operators? There's not a single practical example in the PEP or https://packaging.python.org/en/latest/specifications/dependency-specifiers/#environment-markers for that.

[coarse marsh]

my guess: someone wanted it to be relatively powerful to reduce the temptation for people to fall back on code in setup.py.

[ocean stirrup]

seems to be used in a few places in practice: https://github.com/search?q="python_version+in+["&type=code

[coarse marsh]

Makes a certain amount of sense for, for example, projects that were published during the 2->3 transition and supported e.g. {2.7, 3.4, ... current version} and didn't want to promise support in future versions... ?

[delicate goblet]

Trouble is, I don't see the grammar supporting brackets. Do you? https://packaging.python.org/en/latest/specifications/dependency-specifiers/#complete-grammar
After marker_op which can be a version comparison, "in" or "not in", you have another marker_varwhich means either an env_var (a predefined set of naked strings) or python_str (a quoted free-form string)

The only place where brackets are supported are in extras or inside python_str

also, none of the files at your link are pyproject.toml files

I guess I need to rephrase my question: how would one use the "in" or "not in" operators in pyproject.toml?

[delicate goblet]

If that was the goal, why is the grammar then missing the support for, say, python_version in ("2.7", "3.4")?

[coarse marsh]

🤔

[silent hatch]

Hello folks, I hope you are all well 🙂

I might be doing something stupid, but ISTM that PyPI only uses the deprecated License field, and not License-Expression, although I have seen closed issues in the pypi/warehouse project saying stating that PEP 639 was implemented.

Please see https://github.com/psycopg/psycopg/issues/1225 - to the best of my understanding we are doing everything ok and providing PEP 639-compliant package.

Is this a PyPI issue or I'm being obviously wrong?

[manic burrow]

@silent hatch Are you perhaps using an old version of twine to upload the packages?

[silent hatch]

@manic burrow version 6.0.1

I'll release psycopg 3.3.1 in a few hours, after packages are built. I have bumped twine to the latest 6.2.0, i'll check if it makes a difference, Surprising that the tool to install overrides the packages metadata tho...

[manic burrow]

Yeah, I think they added license expression support in 6.1.0.

[silent hatch]

Looks like that was the case: I've re-uploaded 3.3.0 on testpypi and the license metadata is displayed: https://test.pypi.org/project/psycopg/3.3.0/

Wouldn't it be better to refuse an upload with a broken tool?

[marsh kindle]

I don't think PyPI is actually inspecting the license-expression field in the distribution core metadata. Twine sends over the metadata through HTTP form fields (or headers...? I'm not sure)

[marsh kindle]

Depends on your definition of broken. There's nothing wrong with using an older twine if the package is not using any of the modern metadata features.

[silent hatch]

Weird design, considering that the METADATA field is in the archive uploaded

[silent hatch]

Which is my case: the package has License-Metadata in the METADATA field and the field is ignored

[marsh kindle]

It's probably a historical design quirk. Also, it's easier to not have to extract and read files from everything uploaded. Although, PyPI is now doing that to provide .METADATA files so ¯_(ツ)_/¯

Point is that PyPI really should avoid relying on the uploading tool sending over the right metadata and instead use the distribution metadata as the single source of truth.

That's not as trivial as it sounds as there is no single source of truth since almost every project will at least have a sdist and wheel each with their own core metadata.

I'm not a PyPI maintainer though so that's where my involvement ends. That's the context for why things are the way they are today.

[silent hatch]

I think this is material sufficient to open a ticket for a discussion. Is https://github.com/pypi/warehouse/ the right project?

[marsh kindle]

I strongly suspect there is already an open issue somewhere.

[manic burrow]

I think this is supposed to be fixed in PEP 694.

[marsh kindle]

PyPI is slowly getting stricter so I'd imagine this will happen at some point

[silent hatch]

I see. Thank you very much for the pointers, @marsh kindle @manic burrow !

[agile hatch]

Twine also often needs recent versions of dependenices, which it never requires. One of the new metadata features (maybe this one?) is disabled if a dependency is old, even with new twine. That's why you should always use pipx/uvx/uv tool.

[coarse marsh]

... pipx uploads stuff?

[manic burrow]

I think he meant pipx run twine.

[coarse marsh]

ah

(huh, twine dependencies are actually quite elaborate)

[agile hatch]

Yes, I did. Using pipx or uvx helps keep the dependencies fresh. You can also use uv publish

(I almost always publish from GHA with trusted publishing, so don't use either really)

[coarse marsh]

the runners would also update twine itself, yes?

[honest narwhal]

uv can do trusted publishing natively :)

[agile hatch]

If you use the pypa publish action, it's in a docker container and it is kept up to date (pinned with the action version).