#setuptools_scm

1 messages Β· Page 1 of 1 (latest)

eager flower
hot goblet
eager flower
#

Sure, just wanted to mention it here first

eager flower
#

took me a while to get to it but done πŸ˜„

wispy sapphire
# 
/private/var/folders/_8/xtbws09n017fbzdx9dmgnyyr0000gn/T/build-env-g_1okp9a/lib/python3.10/site-packages/setuptools_scm/git.py:284: UserWarning: git archive did not support describe output
  warnings.warn("git archive did not support describe output")
/private/var/folders/_8/xtbws09n017fbzdx9dmgnyyr0000gn/T/build-env-g_1okp9a/lib/python3.10/site-packages/setuptools_scm/git.py:301: UserWarning: unexported git archival found
  warnings.warn("unexported git archival found")

Would be nice if the required git version and/or current git version could be included there - guessing it's getting the system git and not the homebrew git. Also I really like always doing errors and warnings like:

msg = "some msg"
warnings.warn(msg)

so that the warning msg doesn't get repeated twice (and the raw repr one is the lowest one on the screen!)

#

I wonder if there's a flake8 check for that. Would be fairly easy to write.

hot goblet
#

@wispy sapphire i tend to eventually create modules with predeclared warnings to manage discoverability

hot goblet
#

@wispy sapphire btw, what you showed is an example of why the stdlib warnings module is bad, it's easy to use, but neither simple nor of sound ux

twilit trellis
#

The stacklevel default also does not make any sense

eager flower
#

stacklevel default is the worst thing about it

#

Almost everyone doesn't use it which makes for a hard to find source of the warning that can't be suppressed on per occurrence basis

winter ledge
#

Is there a concise primer somewhere on the best practice? (I've not looked at the warnings docs closely, so it might just be there.)

distant girder
winter ledge
#

Nod, makes sense - thanks!

wispy sapphire
#

Very true - though stacklevel=2 might be surprising in a few cases - maybe forcing the arg been better? What I (also) don't like is mostly tied to raising errors; raise MyErr("thing") prints "thing" twice, so always assigning it to a msg = "thing"; raise MyErr(msg) keeps your error strings from duplicating themselves in the traceback, sometimes with format placeholders unfilled.

eager flower
#

Very true - though stacklevel=2 might be surprising in a few cases
The problem is that in what I would think is the most common use case - deprecation warnings - stacklevel=1 is never right

twilit trellis
#

And it doesn’t help the official documentation even provides this example:

The stacklevel argument can be used by wrapper functions written in Python, like this:

def deprecation(message):
warnings.warn(message, DeprecationWarning, stacklevel=2)
but for a wrapped call you should generally set stacklevel to 3!

naive owl
#

Hi! I'm using setuptools_scm as part of hatch-vcs for a project packaged with hatch, and I'd like to know if there's a way I can control the tuple that is passed to the template. I have a NamedTuple class which has a specific style and I'd like to make this tuple backwards compatible

winter ledge
hot goblet
naive owl
#

I don't think it is, I was debugging the code and it really wasn't managing much that setuptools_scm wasn't doing

hot goblet
#

Writing version files is in hatch core

hot goblet
#

@ancient wigeon took note of your effort, I'm still down from a sinus infection, but there is a good chance i can get to releasing in the night or tommorow

ancient wigeon
#

Sorry to hear. I think the release is underway. I updated the changelog, tagged the release, and added a release in Github.

#

Is there anything else that needs to happen?

#

There's still time to cancel too, if you're concerned that's not the right thing to do.

hot goblet
#

then it's fine, thanks

hot goblet
#

@wispy leaf @broken jackal may i rope you into giving https://github.com/pypa/setuptools_scm/pull/800/files a skim, its going to be the base of setuptools_scm8 and im thinking it enables dropping the install-requires for setuptools

its aslo a bit massive as it bascially turns Configruation and a few others into dataclasses

#

@wispy leaf i am now adding a `SETUPTOOLS_SCM_OVERRIDES_FOR_$DISTNAME" envvar where one can put a toml/json

#

then one can do stuff like SETUPTOOLS_SCM_OVERRIDES_FOR_INICONFIG='{version_scheme: "foo", local_scheme: "bar"}

hot goblet
#

Oh, i just realised that setuptools_scm is older than a decade by now ❀️‍πŸ”₯

hot goblet
#

@wispy leaf im wondering, is there any nice way to have hatch/hatchl-vcs be tested against a editable install of setuptools_scm

wispy leaf
#

that's an open question, there is a way that someone described in an issue asking about that for another plug-in but I forgot what the way was

#

it's on Hynek's readme plug-in, one of the issues

hot goblet
#

@wispy leaf could hatch grow a way to pass over editable installs of other packages into a environment?

wispy leaf
#

sorry what do you mean by that?

hot goblet
wispy leaf
#

oh, that will be coming by PyCon

#

it's called workspaces, like Cargo

hot goblet
#

I have some massive need for that (like needing 10-20 editable checkouts in a shared env (about 70 when pushing it)

polar crystal
#

I'm looking at replacing versioneer with setuptools-scm in distributed but we need to expose the full git hash in distributed.__git_revision__ is there a way of getting write_to to write the full git hash in _version.py?

hot goblet
#

@polar crystal currently not, but im open to adding more details

polar crystal
#

I want the ScmVersion.node ?

hot goblet
#

yes

#

currently thats not being passed

polar crystal
#

Would you be happy to pass the whole ScmVersion into the template?

hot goblet
#

yes, but it would need a nae that indicates what it is safely (as its notthe "version" to be used but rather the version metadata obtained from scm/environment

#

also there is possibly a problem with sdists, as this would break for them

polar crystal
#

If you have node: $Format:%H$ in the git_archival.txt the node is available in sdists right?

hot goblet
#

@polar crystal a git archive is NOT a sdist as made by the standards

polar crystal
#

Oh of course the sdist gets the version from the metadata

hot goblet
#

ok, hatchling migration done, trace to logging migration done, most command invocation refactorings prepared, a 8.0 release is getting closer

polar crystal
hot goblet
polar crystal
hot goblet
polar crystal
#

Oh nice and it will write the file to the sdist?

hot goblet
#

(and correct path handling)

polar crystal
#

Where will it find the node id when building from sdist?

hot goblet
polar crystal
hot goblet
# polar crystal Fallback file?

for setuptools_scm anything that gives it scm metadata outside of the scm is a "fallback file" - so this would be something like a dist-info file and/or a added mapping

hot goblet
#

@wispy leaf @ancient wigeon sorry for the ping, but while slipping in changes and refactorings into setuptools_scm recently when i had time between the kids i may have accidentally hit a point of no return, it's nothing unsalvagable but it seems necessary to get a 3rd party reality check to decided a good course of action between next releases of setuptools_scm vs how to introduce/migrate to vcs_versioning

wispy leaf
#

is the current plan written down somewhere?

hot goblet
#

Not yet, but that's something I'll tackle next, as the current state is not sufficiently covered by current docs and I have a rat-king mr open as well

#

I just took note that I went down the wrong rabbit hole and need to decide what to correctly do about it

wispy leaf
#

what do you think you did wrong?

ancient wigeon
#

I haven't been following vcs_versioning at all, and I'm unaware of the motivations for it (over iterating on setuptools_scm). How can I help? What is the top 1-3 concerns you wish to address? I can give one piece of unsolicited advice - it's okay to throw away something or shelve it and come back at a later time.

hot goblet
hot goblet
ancient wigeon
#

I'd tried to do something similar with hgtools, providing generalized VCS tooling with just a couple of hooks for setuptools. That package has the unfortunate name of being linked with Mercurial. I've considered renaming hgtools to vcstools or similar, but all the good names are taken.

hot goblet
#

@ancient wigeon i already reserved the name vcs_versioning (by sheer luck it was still open)

#

for setuptools_scsm it will be the second name change - it started as the package hgdistver ages ago (back when pytest was still on mercurial and i wanted something like mercurial did for itself)

#

@ancient wigeon i think the more pressing matter atm is how to sort out the collected change size and ensure i can release the enhancements without causing pain

(i did a lot o public code search, but most of that turned out to be clones of the codebase or pins to old and brpken

#

for that i will first complete a listing of completed tasks, as well as the breakages/remediation that will follow of them so that i can size the release and work towards it safely

wispy leaf
#

for your own sanity I would just release the next version as a major and anything that breaks becomes the fault of those who do not set pins or ranges

strange jetty
#

defensive pins are generally not great to set unless you know the next version is going to break you

wispy leaf
#

in a production scenario the minimum due diligence is to set a version cap, one would never specify just the dependency

strange jetty
#

in a production scenario you should have a lock file, but that' for a deployed piece of software

wispy leaf
#

right I totally get what you're saying in that libraries should not set caps but I think in this case is different. I view build environments no different than applications

strange jetty
#

to make sure I understand, you think if I grab a random sdist off PyPI, the build deps should be frozen in time to when that tarball was created?

wispy leaf
#

I wouldn't expect exact pins but as a developer I don't want random things disrupting my build process

#

and I think most maintainers of packages would not want, for example, to use a new major release of a build dependency unexpectedly

hot goblet
#

@wispy leaf its going to be major, but its going to be quite a pain for people to run into it head-first - i have a rough idea for making a migration path possible,

im wondering aware of y python tool that uses the exported type-anotations of a package to see if a repalcement is reasonably compatible

hot goblet
#

@wispy leaf does hatch-vcs try to hide certain env vars, im trying to debug a setuptools_scm bug thats triggered in hatch-vcs but it seems like i cant enable SETUPTOOLS_SCM_DEBUG

wispy leaf
#

no

hot goblet
#

hmm, then i broke something else, wil ldebug after a good night of sleep

hot goblet
#

@wispy leaf so it turns out, hatch-vcs was passing a empty string, as tag regex, which worked before by accident (bool-ish) but now fails - i added a deprecation that allows it to work for now

wispy leaf
#

oh okay nice debugging! just let me know what I should do

hot goblet
teal jackal
#

Does anyone have an example of repository using setuptools-scm with mode than one python package being build? I am looking for a mono-repo approach but I do not want to be the 1st one experimenting with that kind of code layout. Basically a repo that produces multiple python packages, while using the recent full pyproject.toml setup.

hot goblet
#

@teal jackal the root key in the [tools.setuptools_scm] table should behave exactly like the root key in the dict you'd pass in setup.py

teal jackal
#

@hot goblet Thanks. I will try to make use sourcegraph.com to find some projects that make use of it, with bit of luck I might uncover how many are doing it and how well maintained they are.

teal jackal
#

Is there any way to make setuptools fails to build if some files are missing from the cloned code? I am using a git sub-module and every few weeks I endup with a new ticket from someone trying to install from source and forgetting to add the recursive cloning. I want to prevent this kind of accidents. --- note, we droped setup.py long time ago, so we are on pure pyproject,toml based packaging.

hot goblet
teal jackal
wispy leaf
#

it would be useful if there was an option to ignore tags that did not match the regex. I wanted to only consider tags that had a certain prefix and changing the pattern failed on tags that did not match which I did not expect so I ended up changing the describe command to match only on that prefix

hot goblet
hot goblet
wispy leaf
#

the use case is for Hatch since the repo uses the tags hatch-vX.Y.Z and hatchling-vX.Y.Z

hot goblet
#

Excellent,those 2 make for a perfect use case/ example

wispy leaf
hot goblet
#

PSA i severely overestimated how much opensource i can do while on paternity leave and handling both Kids

polar crystal
#

@hot goblet thanks for the support for accessing more template variables in write_to_template, eg node. How does it work?

hot goblet
polar crystal
#

No I mean how does the sdist tell the wheel what the node is

#

I'm familiar with attributes

hot goblet
#

ah, sdist currently has limited details, and the version file basically needs to come in at build time, there is a plan to migrate from the hack to a actual build step, but time is scarse

#

i beleive there is potentially a bug about writing version files wrt fallbacks, i think i needto prevent the version file writing for fallbacks

polar crystal
#

So you're saying pipx run build won't work? (Generate sdist then build wheel from sdist)

hot goblet
hot goblet
#

The docs branch is about ready to merge,then it's time to get 8.x released

Anyone up to be pedantic on the docs branch?

distant girder
#

sure, I'll have a quick check πŸ™‚

hot goblet
distant girder
#

done!

hot goblet
#

took me way longer than i wanted to get to add the overrides docs, im about to merge and release

wispy sapphire
#

But it looks like _get_version got removed, and get_version is deprecated. What's the replacement? (ideally I'd like to be able to let tool.setuptools_scm continue to work, rather than having to rewrap it like hatch does, but I can live with that if I have to)

hot goblet
#

@wispy sapphire im happy to discuss api details, in part since i'd prefer to expose something that goes from configuration to ScmVersion

that being said, the larger plan is to eventually have a vcs_versioning pacakge that superseeds setuptools_scm, hatch_vcs and others

i intend to reease this weekend and im happy to slit in sometihng that helps

#

@wispy sapphire i beleive the best option is to provide the following in a new public submodule

get_configuration(data_or_configfile: dict[str, ...]|Path) -> Configuration
parse_version(config) -> ScmVersion
format_version(scmversion) -> str

and denote is as experimental for now

#

@wispy leaf btw, should such a change be embraced/supported in hatch-vcs as well - im considering adding a integration that checks for hatch-vcs data

wispy leaf
#

hatch-vcs literally just uses your package for the most part so I don't think there would be anything for you to check?

hot goblet
#

@wispy leaf i have at leat one legacy helper function in 8.x left only for hatch-vcs ^^

wispy leaf
#

oh what do I have to change? I thought I only called get_version or something like that

hot goblet
#

@wispy leaf that one is going away as people missuse it all the time wrongly - and its quite a bit magical wrt getting the config -

another one about version files is there as well (and hatch-vcs is currently not able to pass all the needed data for full features)

wispy leaf
#

when is the new package coming out so that I can switch to that in the implementation?

hot goblet
#

@wispy leaf i left those hleper in precisely so i can flesh out the apis i mentioned above a bit better so that people can use it safely and in own builds teps without breaking the tools yet

#

plan is this weekend, just need to expose the helpers i mentioned above, in particular since that would also enable hatch-vcs to have version file templates with scm metadata like node, date & co

#

but for now bbl, late for dinner

wispy sapphire
#

That would work. Longer term, I'd love for vcs_versioning to support dynamic-metadata. It's not a PEP yet as I originally proposed at the PyCon packaging summit, as it's actually possible to get 90% there (and therefore work out the API) without a PEP (as pointed out by someone at the packaging summit), so it's a draft specification in a package for now. I'd love feedback on the draft API, though. πŸ™‚

#

I'm happy to adjust to whatever is made available and make a patch scikit-build-core release.

hot goblet
#

@wispy sapphire im also Happy to add back backward compatible functions if it helps to unbreak the lib with the new release

wispy sapphire
#

It would be nice if setuptools_scm._get_version would continue to work for a bit. Since the new module is named setuptools_scm._get_version, though, might be a little tricky/weird, though. I think this would work:

def __getattr__(name: str) -> Callable:
    if name == "_get_version":
        msg = "_get_version has been moved to setuptools_scm._get_version. Please use <...> instead if possible."
        warning.warn(msg, FutureWarning)
        from ._get_version import _get_version
        return _get_version
    raise AttributeError(f"module {__name__!r} has no attribute {name!r}")

Happy to work around whatever I need to, though. I know I brought it on myself (well, technically the person who contributed this, but I approved it) by using a private function.

hot goblet
wispy sapphire
#

Yes, you could go with the boring, simple, and failsafe solution of renaming the module. Though you could instead make the _get_version module callable. πŸ˜„

hot goblet
hot goblet
hot goblet
#

@wispy leaf would dataclasses be allowed in hatch/hatchling - i want to play around with changing the classregister from pluggy required to pluggy optional and im seeing tonns of chaces for loosing some code

hot goblet
#

@wispy sapphire i jsut merged that pr, unless you see ablocker, i intend to release 8.0.0 in a few

wispy sapphire
#

A few what? I can test it at some point today if you'd like

#

Preparing for teaching today, so likely won't be in time if it's a few hours. But if it's a few days (even tomorrow), I can test it first.

hot goblet
#

minutes ^^

#

hmm, i have one bug i ought to resolve differently , so anoter day or 2 added

wispy leaf
hot goblet
wispy leaf
#

@hot goblet omg

image.png
#

I had no idea, I'm so excited to get rid of it lol

hot goblet
distant girder
#

sometimes moving imports to lazy imports can help, if they're not always needed

wispy leaf
distant girder
# 
python -m pip install tuna
python -X importtime -c 'import pluggy' 2> import0.log
tuna import0.log
#

and it opens that thing in the browser, you can click to zoom in

hot goblet
#

@wispy leaf that one is fixable

hot goblet
#

but man, pytohn needs deferred eecution of toplevel definitions or its going to die

hot goblet
#

@wispy leaf the next release of pluggy will lazy-import importlib-metadata

wispy leaf
#

amazing thanks

hot goblet
#

Inspect as well,

hot goblet
#

TIL one can just define TYPE_CHECKING as False to avoid importing typing outside the if clause

hot goblet
#

@wispy leaf i went from aound 0.069 second to 0.004 seconds, but its a breaking release

toxic forum
hot goblet
toxic forum
#

interesting

hot goblet
#

ok, 8.0.0 is out of the window

#

@broken jackal are there any docs on how to currently setup a setuptools plugin which adds python files, data files or extras to a editable wheel, sdist or wheel (i want to move version files from the current hack to a build step + provide the scm files-list inside a sdist)

wispy leaf
#

you have to do:

from __future__ import annotations
#

also the types are wrong

hot goblet
wispy sapphire
#

FYI, I did test our usage yesterday and scikit_build_core/dynamic_metadata are happy with the new version.

hot goblet
wispy sapphire
#

I got a report that the version file isn't being generated anymore. I'll look into it soon. (scikit-build-core can generate files for metadata as well, so that's likely an easy workaround for now)

hot goblet
#

@wispy sapphire i got a inconclusive bugreport as well, that person snt providing SETUPTOOLS_SCM_DEBUG or a issue to work on

wispy sapphire
#

The file is missing when I run a vanilla scikit-build-core run with vcs config (but not if I limit setuptools_scm < 8).

hot goblet
#

@wispy sapphire _get_version is involved?

the issue is "simple" force_write_version_files=False for that by default

#

i can make it so the provided import uses true

wispy sapphire
#

I can also adapt if needed. That's a change from before?

hot goblet
#

the parameter was added after the rename, i forgot to add the default from the old when returing the alias

wispy sapphire
#

Okay, great. So 8.0.2 will fix?

hot goblet
#

lets add a issue

wispy sapphire
#

Okay

hot goblet
#

i wont be able to do 8.0.2 right now as i need sleep, im wondering about yanking 8.0.1 due to the git archival issue

wispy sapphire
#

No rush. πŸ™‚

hot goblet
#

hmm

#

actually, no yank is betterby now

wispy sapphire
#

In the README:

This feature requires setuptools 60 or later
Given that you then show PEP 621 config, don't you mean 61? Setuptools 61 added PEP 621 support. 60 just make local distutils default.

#

Do you have to have an empty [tool.setuptools_scm] section in the toml file? That's what I was trying to check in the README. I thought I remember someone noticing that.

#

(Edit: yes, you do)

hot goblet
#

@wispy sapphire setuptools_scm has its own pyproject.toml/setup.cfg reading as setuptools doesnt expose it in any way πŸ™ƒ

hot goblet
#

sup @wispy leaf still up for that chat

wispy leaf
hot goblet
#

so, lets fetch a coffe and go t a voice/video chan?

wispy leaf
wispy leaf
#

@hot goblet I'm so sorry to bother you again but your latest fix is still broken

hot goblet
#

@wispy leaf can you show where

wispy leaf
# 
# file generated by setuptools_scm
# don't change, don't track in version control
TYPE_CHECKING = False
if TYPE_CHECKING:
    from typing import Tuple

__version__ = version = '1.2.3'  # type: str
__version_tuple__ = version_tuple = ('1', '2', '3', '')  # type: Tuple[int | str, ...]

❯ ruff check .
_version.py:5:24: F401 [*] `typing.Tuple` imported but unused
Found 1 error.
[*] 1 potentially fixable with the --fix option.
#

why can't the future import be used and then just use the built-in types?

hot goblet
#

@wispy leaf that one is on the radar already, - i beleive its best to jsut add a noqa comment there, (its insane how all the different tools throw themselfes over

#

because it breaks on python3.8 - even with quotes

wispy leaf
#

can you please show me an example? what breaks?

hot goblet
wispy leaf
#

I can't reproduce that

hot goblet
#

@wispy leaf as far as i understood it happens only when running mypy on python3.8

wispy leaf
#

yes that's what I'm doing

toxic forum
hot goblet
wispy leaf
#

from the example in the issue:

# file generated by setuptools_scm
# don't change, don't track in version control
from __future__ import annotations

__version__ = version = '2.0.2.dev217'  # type: str
__version_tuple__ = version_tuple = (2, 0, 2, 'dev217')  # type: tuple[int | str, ...]

❯ hatch python install 3.8 --private -d .
Installed 3.8 @ 3.8

❯ .\3.8\python\python.exe --version
Python 3.8.17

❯ .\3.8\python\python.exe -m pip install -U ruff mypy
...
Successfully installed mypy-1.5.1 mypy-extensions-1.0.0 ruff-0.0.290 tomli-2.0.1 typing-extensions-4.8.0

❯ .\3.8\python\python.exe -m ruff check _version.py

❯ .\3.8\python\python.exe -m mypy _version.py
Success: no issues found in 1 source file
hot goblet
#

yikes :/

wispy leaf
#

also, I know TYPE_CHECKING = False is a cool trick and I didn't know that either until you discovered that this week but unless all dependencies do that then somebody somewhere will import typing, you can't escape that import I think nowadays. as such, I think it would be nice to just use the standard import

hot goblet
#

Gonna give it a try when I get back to the computer

broken jackal
wispy leaf
hot goblet
hot goblet
#

@deft vessel is there any way to undo the subtable inlining pyproject-ftm applies since version 2

#

as is i basically have to either pin or drop pyproject-fmt fo setuptools_scm

deft vessel
#

No there's not πŸ‘ because in lining is better than expanded πŸ˜…

hot goblet
deft vessel
#

The two formats are equivalent any reason why you need to drop it other than personal preference

hot goblet
#

For stuff like the urls or simple script listings its contestable

But what's done to larger listings of optional dependencies and entry points is a disservice to the reader

distant girder
#

do you have an example diff?

hot goblet
#

Latest pre-commit update on setuptools-scm

#

I can fetch the link when I get back to the computer later

deft vessel
#

I actually prefer the optional dependencies to be inline better because you can group it with other dependencies.

hot goblet
#

I consider easy finding of the name for entry point and optionals very important

In particular the entry point names for setuptools-scm are harrowing to read in the new style

#

@deft vessel I think I could reasonably concede even on the optional dependencies things

But the entry points in setuptools-scm are seriously unacceptable in a new style, just a hard no there

deft vessel
#

I can accept that, up to you πŸ‘ no hard feelings, ultimately just difference of opinions πŸ‘

#

The same way some people absolutely hate what black does in some circumstances people will also not like what this formatter does in some cases. As I said before I'm not against adding a flag to allow the user to switch between the two modes, but as I have no interest in using it it is very unlikely that I will spend the time and effort to implement it. That being said if someone does the work I'm open to review and accept it πŸ‘

#

You can also keep using version one that has the old world behavior.

distant girder
#

for those entry points

hot goblet
#

Well, the same kinda thing happened with tox 4 - and here i think the demonstration shows it's not just opinion as such I lost trust

#

And staying on old versions is also not feasible (and intentionally incompatible with pre-commit updates)

deft vessel
deft vessel
hot goblet
deft vessel
#

I know but that's kind of violates the core principle of semantic version

#

It is totally reasonable to develop version one and version 2 independently with each other

hot goblet
#

There's a reason why jinja2 has a 2 in the package name and not just in the version

deft vessel
#

Yeah, don't I don't agree with that approach. Jinja without 2 suffix just became obsolete and abandoned and for new people makes it more confusing which to use, so I don't agree with that choice they made, though I understand why some might.There's no clear winners here just trade-offs.

deft vessel
deft vessel
#

Cc @hot goblet when you get some time if you can elaborate the above point that would be appreciated thank you

hot goblet
deft vessel
#

I guess we disagree on the role of the major version in the semantic version

#

Perhaps you just don't believe in the semantic version

#

But I see it as a useful tool to not have to have tox tox2 tox2 tox4 and so on

hot goblet
#

i consider semantic versions a usefull tool, if reasonable scope is applied

#

sometimes breakeges are a accident

#

but complete replacements with large behaviour changes are a different beast that ought to be considered differently

deft vessel
#

I see, I guess we will need to agree to disagree on this πŸ‘πŸ˜Š all the best

wispy leaf
#

@hot goblet heads up, I'm trying to think of a workaround
DEBUG Ignoring existing lockfile due to mismatched version: datadog-agent-dev (expected: 0.3.1.dev1, found: 0.3.1.dev2)

image.png
wispy leaf
hot goblet
#

Locls should perhaps be slightly unbound for dev version editable installs

magic frost
#

/edit: nvm misconfiguration

dense pewter
wispy leaf
#

@hot goblet what's the progress on extracting the core logic into the separate package?

hot goblet
wispy leaf
#

reason I'm asking is I've gotten feedback from users that they wish setuptools was not a dependency when using hatch-vcs so I was considering writing a custom implementation but would prefer not to

hot goblet
#

@wispy leaf BTW any sensible way to ship dynamic version metadata with hatch based packages?

wispy leaf
#

what do you mean by dynamic?

hot goblet
#

Similar to the hacks that Versionen fork does

#

Aka dynamic version in distribution metadata and version files

#

I didn't do it with setuptools asvits insufferable

#

But with hatch it should be simpler

wispy leaf
#

perhaps explain what it would do? the word dynamic is loaded so it could mean the packaging standard/field or you could be describing some kind of specific functionality

hot goblet
wispy leaf
#

oh I see, yes that would benefit from a standard

#

but I can think of a way for Hatchling to support that

#

some kind of editable metadata implementation in the wild would be required for such a standard so that's a good idea

magic frost
#

@hot goblet if you ever wanted to make some breaking API changes, that could also be an opportunity

hot goblet
wispy leaf
#

if it's a new package you don't have to be responsible

hot goblet
#

@wispy leaf well - i want to move code to the new package without breaking the old one and i don't want the old one to just be abadonded

hot goblet
#

i finally managed to complete the next release - its time to think about the vcs-versioning migration of the codebase

hot goblet
#

@wispy leaf any recommendations for multi package repos? i'm looking towards replicating part of what hatch/hatchling does for setuptools_scm/vcs_versioning until the git repos actually split

wispy leaf
#

you mean multiple packages side-by-side in the same repository?

hot goblet
#

yes

#

it seeems i'd have to handle some bits on tag prefixes but the rest is mostly fine (with the caveat of .dev cross-poluting)

wispy leaf
#

that's the workspaces feature that I haven't pushed to GitHub because it's unfinished

eager flower
#

Is it possible to use setuptools_scm's file finders without using its versioning?

eager flower
#

I guess I wouldn't mind the automatically generated version with a commit, if it didn't require me to have a real version tag for it (like, just have a distance from repo's root commit instead or something, I don't really care). Maybe I should just create a dummy tag without version number on the repo's root commit, not sure if that would work...

hot goblet
#

The file finders always on

The version finding is opt in

#

Also no tag required

#

The default version is 0.1.dev... using distance from root commit

eager flower
eager flower
#

Or maybe the ref-names remark in the docs is relevant and I could use that

hot goblet
#

@eager flower if it doesmt work even after ading a export subst based git arcival, then its a bug

hot goblet
#

@wispy leaf within the next 1-2 months setuptools_scm will spit most functionality into vcs-versioning - im wondering whether that should ship a hatch plugin for providing version/version configuration (atm im on a close all the issues speedrun)

wispy leaf
#

I'd prefer updating hatch-vcs however you see fit

hot goblet
#

then i'l lgo with that

hot goblet
#

i released https://pypi.org/project/setuptools-scm/9.0.0/ today - its expected to work just the same for all users
its the last release with legacy python and legacy setuptools support
its also going to be the last release thats mostly standalone

future version will require modern setuptools and vcs-versioning - all build pipelines that still need legacy setuptools and/or old python had about half a decade now to update python tooling - its time to go modern

hot goblet
dense pewter
hot goblet
dense pewter
#

Oh it's a part of that same bug?

hot goblet
# dense pewter Oh it's a part of that same bug?

its more like a chain of frankenfixes - every time i terated on something, something new cropped up - but the basic origin is that simplified self-activation was in indsight way too optimistic about the ecosystem - and by now im under the impression i need telemetry to figure what to do with peoples old projects - so i'll stay in the blind

dense pewter
#

Would you mind dropping a brief comment there?

hot goblet
#

like theres people that add setuptools_scm, set dynamic=version and then let setuptools read the version from a python module

dense pewter
#

This user seems sort of hostile to the idea that there's not a problem with uv there

hot goblet
dense pewter
#

I think that setuptools-scm is escaping the cache directory when looking for git information?

hot goblet
#

hmm, i guess i'll have to verify that - its mega messy

ripe nacelle
#

is there a pre-defined versioning scheme that will let me have a distinct version number for any git commit?

hot goblet
#

also sorry for the delay - missed this while on family vacation

ripe nacelle
#

not sure what you mean by nodeid

hot goblet
#

commit id

hot goblet
#

@ripe nacelle what do you have in mind?

ripe nacelle
# hot goblet commit id

a commit id could work, or a number of commits since a tag. What did you mean by "usually its desired to have releases behave different"? Different commits will often behave differently

hot goblet
#

so as far as your question goes - as far as i understood your use case, thats the intended default behaviour

hot goblet
#

@wispy leaf i finally got around releasing 9.2.1 - im now going to focus on moving code into vcs-versioning while keeping the api compatible - the goal is that after vcs-versioning is released hatch-vcs migrates to it

wispy leaf
hot goblet
wispy leaf
hot goblet
wispy leaf
#

Thanks, sorry for the noise Ronny!

hot goblet
wispy sapphire
#

I'm looking at the failure in scikit-build-core's setuptools plugin after the recent major release, and it looks like a simple wrapper around setuptools now breaks setuptools scm unless you wrap it with setuptools_scm specfic code? Or am I reading it wrong? Here's what used to work, pretty much straight from the setuptools docs:

# my.build_meta
from setuptools.build_meta import (
    build_sdist,
    build_wheel,
    get_requires_for_build_editable,
    get_requires_for_build_sdist,
    get_requires_for_build_wheel,
    prepare_metadata_for_build_wheel,
)
if hasattr(setuptools.build_meta, "build_editable"):
    from setuptools.build_meta import build_editable

if hasattr(setuptools.build_meta, "prepare_metadata_for_build_editable"):
    from setuptools.build_meta import (
        prepare_metadata_for_build_editable,
    )
__all__ = [
    "build_editable",
    "build_sdist",
    "build_wheel",
    "get_requires_for_build_editable",
    "get_requires_for_build_sdist",
    "get_requires_for_build_wheel",
    "prepare_metadata_for_build_editable",
    "prepare_metadata_for_build_wheel",
]

But now every single one of those needs to be wrapped in GlobalOverrides from vcs_versioning, even though this has nothing to do with setuptools_scm?

#

It's happing in my test suite, so it's possible something with the setup is interfereing with it instead

#

Interesting, it's only breaking the non-isolated tests. I bet it is due to the inline tests.

#

Adding a workaround to just that one test file seems to be working, I think that's it.

#

By the way, having a def __dir__() -> list[str]: return __all__ would be nice in the generated version file. I'm getting annotations as a tab completion from this file, didn't have that before.

#

Looks like it needs to be rebased, @hot goblet do you need help with that?

hot goblet
wispy sapphire
polar crystal
hot goblet
#

It looks like I won't be able to complete it today

hot goblet
#

finally rebased, but i wont be able too work on it more

wispy sapphire
#

Hmm, suddenly started getting "no version.py file found" errors, starting about an hour ago.

deft vessel
#

There was a release few hours ago @hot goblet

hot goblet
#

the toddler is in bed now and im taking a look at why its broken

#

@wispy sapphire i yanked it for a start, thanks for the suggestion

hot goblet
#

@ancient wigeon how does setuptools make a pkg_resources.require("setuptools>=45") pass on setuptools < 42 (triggers the regression for setuptools_scm

ancient wigeon
#

I wouldn’t expect that to pass. Sounds like a bug at first blush, but given that I’m unaware of such an issue, it seems unlikely to be a general problem and is likely due to a corrupt environment.

hot goblet
#

(somehow people end up with setuptools_scm eggs that shouldn't load) i'm not sure how to replicate the issue yet

ancient wigeon
#

It’s conceivable that setup_requires or build-requirements are cause if the later Setuptools to be added but it’s somehow not overriding the older version.

hot goblet
#

https://github.com/DataDog/dd-trace-py/blob/master/setup.py#L204 somehow triggered https://github.com/pypa/setuptools_scm/issues/605

GitHub
`AttributeError: 'Version' object has no attribute 'release'` after...

We have jobs in CI that depend on the latest version of this extension and so as soon as 6.1.0 became available on PyPI we started hitting the following error: Traceback (most recent call last): Fi...

GitHub
dd-trace-py/setup.py at master Β· DataDog/dd-trace-py

Datadog Python APM Client. Contribute to DataDog/dd-trace-py development by creating an account on GitHub.

wispy sapphire
hot goblet
#

and the test env was hiding the issue

distant girder
#

hi! I just started getting ModuleNotFoundError: No module named 'tomli', using tox

#

using tox:

      File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/core.py", line 108, in setup
        _setup_distribution = dist = klass(attrs)
      File "/private/var/folders/kt/j77sf4_n6fnbx6pg199rbx700000gn/T/pip-build-env-m_s8w5_f/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 455, in __init__
        _Distribution.__init__(self, {
      File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/dist.py", line 292, in __init__
        self.finalize_options()
      File "/private/var/folders/kt/j77sf4_n6fnbx6pg199rbx700000gn/T/pip-build-env-m_s8w5_f/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 807, in finalize_options
        ep(self)
      File "/private/var/folders/kt/j77sf4_n6fnbx6pg199rbx700000gn/T/pip-build-env-m_s8w5_f/normal/lib/python3.9/site-packages/setuptools_scm/integration.py", line 52, in infer_version
        config = Configuration.from_file(dist_name=dist_name)
      File "/private/var/folders/kt/j77sf4_n6fnbx6pg199rbx700000gn/T/pip-build-env-m_s8w5_f/normal/lib/python3.9/site-packages/setuptools_scm/config.py", line 187, in from_file
        defn = _load_toml(data)
      File "/private/var/folders/kt/j77sf4_n6fnbx6pg199rbx700000gn/T/pip-build-env-m_s8w5_f/normal/lib/python3.9/site-packages/setuptools_scm/config.py", line 59, in _lazy_tomli_load
        from tomli import loads
    ModuleNotFoundError: No module named 'tomli'
#

this is with the new 6.1.1, and the old 6.0.1 is ok. ping @hot goblet

upper current
#

are you installing setuptools_scm with the toml extra?

#

it might've worked before simply because something else depended on the toml package

#

and there's nothing in your env depending on tomli (instead of toml)

hot goblet
#

another yank, i need to completely reeingineer the testsuite of setuptools scm to prevent this mess in future 😦

distant girder
#

hang on, that will have tested with the non-yanked 6.0.1, let me explicitly test 6.1.1

#

yep this worked too:

 setup_requires =
-    setuptools_scm
+    setuptools_scm[toml]==6.1.1
hot goblet
#

not having the toml extra shouldnt break the build however

upper current
#

yeah, it looks like it breaks if pyproject.toml exists and tomli is not installed

distant girder
#

yep I have pyproject.toml

hot goblet
#

Im currently out with the Toddler, it seems that someone is working on a fix, I'll pick up once I get back to the computer

distant girder
wispy sapphire
#

I suggested trying tomli and then falling back to toml in the issue specifically to protect against this.

hot goblet
#

@wispy sapphire the issue was a missconfiguration, i'm now going to do a release that always dependsd on tomli,

wispy sapphire
#

You can't ensure that the user has tomli, unless they install with the [toml] extra, so I'd suggest allowing both to work until 7.0. It's only the "load" function that's used here, so there's no harm in falling back

#

And since the toml extra always was "toml", and putting an extra in seems a little odd for pyproject.toml, I know at least a few packages have just listed toml instead of the extra.

hot goblet
#

@wispy sapphire modern setuptools does pip install for install requires, that version is required for setuptools_scm since 6.x - it will also work for build requires, so i dont see a downside in just having the correct lib all the time

wispy sapphire
#

So adding tomli to the non-extra requirements?

#

That would be fine too

#

Okay, didn't realize that was the direction you were going. Though do you really need an upper limit on tomli? Do you think tomli.load is going to break in 2.0?

#

This means that this version of setuptools_scm will become (possibly) artificially broken if tomli releases a 2.0.

hot goblet
#

@wispy sapphire good point, i'll remove the upper bound

wispy sapphire
#

Removing the extra produces a warning when directly installed, not sure if it's visible as a pyproject.toml requirement. You could leave an empty extra to keep the warning out.

#

setuptools_scm[toml]>4 would produce a warning on 6.2.0, for example, but anything less than setuptools_scm>6.2 would be broken.

hot goblet
#

true that

#

gmm, also props to fedora adding python3.10, i missed that it broke for importlib metadata selectable interfaces

#

so we also add python 3.10 support in 6.2

wispy sapphire
#

Nice πŸ™‚ Though it's also nice that GHA updates hours after each beta / RC release

hot goblet
#

@wispy sapphire i finished implementing your suggestion, will tag a release once its green

#

release pipeline started again

hot goblet
#

@wispy sapphire it seems like the empty extra is being eaten alive by something, i want to set setuptools on fire atm ^^

hot goblet
#

hmm, i made a effin typo after undoing a nuke and missed it , definitively not my day today

wispy sapphire
#

Seems to me setuptools_scm==6.2.0 works? Worked fine when I tried it. Is something wrong with it? Seems to be yanked?

hot goblet
#

@wispy sapphire its extra metadata was incomplete due to a typo

#

so it broke some buidls

#

in addition, old setuptools is just INSANE - it still buidls eggs, even if one disables it

wispy sapphire
#

It's building an egg from the SDist, not getting the wheel, if setup_requires is used, I'm guessing? (And wheel is not installed)

hot goblet
#

@wispy sapphire i made the bdist_egg command fail the build, it still builds

wispy sapphire
#

Then are you sure it's building an egg?

hot goblet
wispy sapphire
#

Ahh, .eggs/setuptools_scm-6.2.0-py3.7.egg shows up, guessing that's why?

hot goblet
#

yeah, its ludicrus

#

@wispy sapphire is pipx avaliable on github by default?

wispy sapphire
#

Yes, it's an officially supported package manager.

#

They internally use it to install other stuff, too, and do a fantastic job of keeping it up to date.

#

It's fantastic for writing composite actions (which GitHub a day or two ago finally gained support for composing other actions, but originally were shell only, so pipx was invaluable, and is still really useful)

hot goblet
#

@wispy sapphire wouldn't the cibuildwheel/build tasks be more effective as docker action against a prebuild cached image?

gray tangle
#

Ouch, psf/black felt the setuptools-scm fires. Just wanna pitch in and say it's totally fine, the issue was actually much worse on our side because of a build time mistake of our own (so I guess in a way, this mistake made our own mistake significantly more public :p). Thanks for maintaining setuptools-scm and also the fast response time!

hot goblet
#

What's the build time mistake?

#

Also once the toddler sleeps ill try another release

gray tangle
#

the build environment wasn't cleaned so the wheel ended up having multiple (and different!) versions of the source code

#

we ended up deleting the wheel

hot goblet
#

Yikes scary

#

Temporary volumes that disappear after build ftw

gray tangle
#

We've since moved our artifact building to GHA :) totally feel ya on the automatic cleanup bit tho, always a bit of a footgun

hot goblet
#

@wispy sapphire @ancient wigeon im considdering adding a packaging dependency in general - any opinion on it?

#

@hushed pelican your opinion would be appreciated as well

wispy sapphire
#

I was about to suggest @hushed pelican as a good person to answer that. I'm generally okay with safe dependencies - that's the point of packaging. But there are a few packages where you can't do that - and packaging related projects tend to be in that category, and there can be boot strapping issues. Also, does old setup_requires respect / pull in dependencies?

hushed pelican
#

from my POV it would be fine as setuptools_scm is not needed to bootstrap Python environments

#

but I am curious on why you want to pull it

wispy sapphire
#

So I don't have a strong opinion, it's going to depend on things like classic usage. I only use pyproject.toml, but others do not. I don't think it should be break classic usage. That should be integration-tested. πŸ˜‰

#

There's also vendoring, which is used in several other projects with packaging, so should be doable if you had to go that way.

hushed pelican
#

the vendoring is bad for downstream packagers who need to devendor so I'd reserve that for last case

hot goblet
#

@hushed pelican basically old setuptools will even if i set up setuptools_scm with a inteentionally broken bdist_egg command, build a actual bdist_egg and use it, then run against a setuptools with a broken version class and tople over, its not clear to me how to debug this, the easiest way is to stop using the old version class from setuptools and going for packaging

hushed pelican
#

hum

#

okay

ancient wigeon
#

I'm okay with dependencies on setuptools_scm. setup_requires does honor those dependencies. I don't believe there are bootstrapping issues. Until recently, I've had many skeleton-based projects using setup_requires=['setuptools_scm[toml]'] as a fallback for non PEP 517 builds and I never had any complaints. Excepting pip and setuptools themselves, I believe arbitrary dependencies are safe.

ancient wigeon
#

I'd really like to devendor dependencies in setuptools as well, but bootstrapping is a nightmare.

hot goblet
#

@ancient wigeon i still would like to prevent old setuptools from botching up things - it literally breaks al lthe details, leaves stuff old and is a nightmare, how can i fail setuptools_scm at build/install time for old setuptools

#

@ancient wigeon a key issue is that old messy distros ship pip+setuptools in virtualenvs that dont take pyproject.toml or build requirements into account, that messes up a number of isolation details

#

hmm, its also thinkable the vendored packaging is of the wrong version

#

hmm, i need old distro tests in the pipeline :/

hot goblet
#

@ancient wigeon i figured the roundabout issue - setuptools 40 will build eggs from wheels for all the build deps, but then fail to use the correct setuptools version/configuration, in turn topleing over, i will need to do a runtime error and suggest setup.py not to be just setup()

#

@hushed pelican @ancient wigeon would

ERROR: setuptools==40.8.0 is being used with setuptools_scm>=6.x

if you see this your build configuration is incomplete and worked by accident
please ensure setuptools>=45 and setuptools_scm>=6.2 are installed with recent non-broken tools
a bug in setuptools <45 was hiding this error

be a reasonable error (still have to research the exact versions

hushed pelican
#

πŸ‘

hot goblet
#

hmmm, even recent setuptools seems affected, yay

ancient wigeon
#

I'm a little confused. "setuptools 40 will... fail to use the correct setuptools version/configuration." What does that mean?

#

If there's a specific behavior that's missing/unexpected, describing that behavior might help me hone in on which change might have affected that behavior.

#

Regarding the error message, I'd suggest two things: (a) Use capitalization and punctuation (okay without for informal usage) and (b) avoid "a bug" and "broken" terminology; use less inflamatory/more charitable wording and links to specific issues (pypa/setuptools#NNNN) if the version changelog doesn't specifically reference the issue.

hot goblet
#

@ancient wigeon old setuptools will make a egg of newer setuptools_scm and activate it even if its conflicting with the old setuptools version

#

so a setuptools <45 will make an egg out of setuptools_scm and activate it, even if i make bdist_egg explicitly broken in setup.py

#

even setuptools > 45 will drop setuptools_scm in as a egg (tested on buster with downstream setuptools and all upstream major setuptools versions between 40.x and 45.0

#

updated the error to

ERROR: setuptools==42.0.0 is used in combination with setuptools_scm>=6.x

If you see this, your build configuration is incomplete and worked by accident!
Please ensure setuptools>=45 and setuptools_scm>=6.2 are installed with current tools.
A backward compatibility behaviour in setuptools is unfortunately hiding this issue.
ancient wigeon
#

I would not be surprised if Setuptools, during resolution of setup_requires, downloaded a different version of Setuptools as an egg and even activated it, but that still won't replace the version of Setuptools already imported and present for executing the build.

hot goblet
#

as far as i can tell it actually does, but then at the end it activates the package that is in conflict with itself

ancient wigeon
#

I can see how the presence of a newly-activated later version of Setuptools might cause that later version to be present in the metadata, causing pkg_resources.requires to pass based on the new metadata even though the Setuptools that was imported is older.

hot goblet
#

i will extend the error with a note that this particular behavior is in fact not fixable and the reason why setup_requires/easy_install is being phased out in favour of build requriements in pyproject.toml

#

+part of

ancient wigeon
#

The new error message is better. Still needs some work around "A backward compatibility behavior in setuptools is unfortunately hiding this issue." I'm not sure that's accurate, but if it is, it should point to a bug that provides an analysis of how the backward-compatibility behavior leads to an issue.

hot goblet
#

hmm, this definitively needs better wording

ancient wigeon
#

I'd be open to a change in Setuptools that would hard-fail on attempting to add a different version of setuptools to the working set - providing a fail-fast error on the known unsupported behavior.

hot goblet
#

not sure that helps, this is in distros, those are on 40.x

ancient wigeon
#

Right - the eternal challenge of stale software being unchangeable. :/

hot goblet
#

but at least i can error out sanely now

hot goblet
#

@ancient wigeon the error will now read


ERROR: setuptools==42.0.0 is used in combination with setuptools_scm>=6.x

If you see this, your build configuration is incomplete and worked by accident!
Please ensure setuptools>=45 and setuptools_scm>=6.2 are installed with current tools.

This happens as setuptools is unable to replace itself when a activated build dependeny
requires a more recent setuptools version.

Suggested workarounds if applicable:
 - preinstalling setuptools/pip/setuptools_scm as build dependencies before running setup.py
 - installing setuptools_scm using the system package manager to ensure consistence
 - migrating from the deprecated setup_requires mechanism to pep517/518
   and using a pyproject.toml to declare build dependencies
   which are reliably pre-installed before running the build tools

any further suggestions?

ancient wigeon
#

That looks pretty good to me.

upper current
#

is pip a required build dependency?

hot goblet
#

not actually, i reworked the message a bit into

f"""
ERROR: setuptools=={_version} is used in combination with setuptools_scm>=6.x

If you see this, your build configuration is incomplete and worked by accident!
Please ensure setuptools>=45 and setuptools_scm>=6.2 are installed with current tools.

This happens as setuptools is unable to replace itself when a activated build dependeny
requires a more recent setuptools version.

Suggested workarounds if applicable:
 - preinstalling build dependencies like setuptools_scm before running setup.py
 - installing setuptools_scm using the system package manager to ensure consistency
 - migrating from the deprecated setup_requires mechanism to pep517/518
   and using a pyproject.toml to declare build dependencies
   which are reliably pre-installed before running the build tools
"""
upper current
#

it doesn't explain why the build config is incomplete and 'current tools' is slightly enigmatic (I assume it means anything which isn't setup_requires)

#

will this error also be displayed when easy_install would not have been used?

hot goblet
#

yeah, as it happens for setup_requires in a number of cases

hot goblet
#

hmm, i'll ready the release tommorow, i got all regrressions sorted, but its too late, i need a bag of sleep before i can finish the pr

ancient wigeon
upper current
#

so, just to be clear, setuptools_scm v6 does not work with setup_requires, at all, unless setuptools is invoked using PEP 517

#

in which case it's not enough to say in the setuptools_scm readme that "setup_requires has been deprecated in favor of pyproject.toml"

#

scratch that, I misread the error message

hot goblet
#

@wispy sapphire im wondering, is there a proposed install/test flow thats using sdist/bdist_wheel artifacts instead of checkouts, i was jsut thinking that setuptools_scm may test better if all building rain against a first created artifact of the wheel/sdist

distant girder
#

(you can remove the redundant If you see this, to make it a little bit shorter)

hot goblet
#

@hushed pelican is there any suggested way to ensure a python environment is able/ready to build a source package (someone wants to run build commands in a own env and needs to install build requirements correctly) im documenting the breakage reason for the setuptools_scm release and want to suggest some useful commands as well (it basically boils down to ez_setup was the bootstrap that ensured a sdist/source tarball would fetch a modern enough setuptools, its no longer available and most people don't/didn't use it anymore for more modern tools)

hot goblet
hushed pelican
hot goblet
hushed pelican
#

pypa/build then? it will build in a isolated env by default, and even when you ask to build in the current environment, it will check for the dependencies

#

but I think there may be some workflows, like people using outdated pip, that would not fit into this recommendation

hot goblet
#

thx, i'l lrtfm then in a bit

hushed pelican
#

let me know if you have any questions, I know this stuff can be pretty involved πŸ˜…

hot goblet
#

@hushed pelican i cant seem to find a way for build to run the install build requirements step only (aka skipping out on the wheel building)

hushed pelican
#

yes, it can't

#

it needs to actually build something

wispy sapphire
#

prints the error on old setuptools

Do you really need to go this way? Even though I use pyproject.toml, people still may get packages on RHEL 7, or even RHEL 8 (which still ships with Pip 9 if you install the default Python 3.6), or Ubuntu 18.04 (including the python3.8 package), all these are Pip 9 (due entirely to Pip 10 breaking everyone by removing all imports from pip, including pip.main, I believe, making it a breaking upgrade for distributions). Anyone using Pip 9 will get build-from-source if packages don't ship manylinux1 wheels, which are deprecated and die out fully at the end of this year. If that breaks only because of setuptools_scm, I can see packages really wanting to pin or drop setuptools_scm instead of deal with the torrent of reports coming from default pip users. (And if you don't use pyproject.toml, this is even a wider problem, since you are stuck with the default setuptools that way)

#

Since it used to work on old setuptools, what is breaking that? Also, what is the definition of "old" here, 45?

hot goblet
#

@wispy sapphire it bascially worked by accident since no metadata broke, it s till breaks for a number of packages and bascially old setuptools hides the errors and woudl happyly run with incomplete/wrong metadata

#

@wispy sapphire its part of the reason why we suddenly got attribute errors in setuptools_scm as older setuptools simply did not have some attributes.

#

@wispy sapphire one basic reason is, that if you use pyproject toml, your packages would simply get missing/wrong metadata on those distributions with no indication of why it happens

and most of those errors can be resolved with preinstalling a reasonably working setuptools_scm version as packaged by a distr

hot goblet
#

@wispy sapphire so the basic issue is that all those setups are pretty much time bombs since years and they will eventually fail with broken/wrong metadata as required setuptools hooks are not available among other things

#

@wispy sapphire originally ez_setup would be shipped alongside setup.py to resolve it propperly, but thats not packaging practice since years, so things are half-broken since years, someone has to cast the first stone

#

even debian has setuptools_scm packages to rely on since years, people just have not installed them and so instead of a working v ersion for their env, they get a much newer version, its unfair to constraint development by dead code in distros just because people skip essentials install steps

wispy sapphire
#

You can't rely on system packaging. That would mean you have to use pip 9 on those distros I mentioned, which means you can't install manylinux2010 or manylinux2014 on those distros. That also means pyproject.toml is useless, as it pulls from PyPI, not from distro packaging. That also means users have to get admin permissions, you can't install distro packages without admin permissions. Distro packaging doesn't work with virtual environments, either.

#

I know those setups are bad, but the goal should be to keep them working as long as possible, and letting them break on something that is not optional (such as when manylinux1 dies and people have to ship manylinux2010), not on something that's completely optional and developer-focused anyway (setuptools_scm)

#

If this breaks, then users will be faced with three options: 1) Break for a significant subset of users, 2) set a limit on setuptools_scm, or 3) drop automatic versioning or use a different package, like versioneer. (2) is currently is what everyone is currently doing, but that's not a long term solution, and you can't limit based on distro. Everyone will move to (3), not (1) - saving a little developer work in versioning is not worth shipping broken packages. And (1) doesn't have a clear solution - users might not be able to sudo yum install python3-setuptools_scm, and users should be moving to PEP 517 builds, not away from them.

#

The only think that needs to work, though, is the SDist with the written-out version file - it's fine if the developer step has higher requirements

hot goblet
#

@wispy sapphire in virtualenv they can just update setuptools,

The reports I see are classically from setups that don't manage build dependencies correctly

There is no reasonable workflows for fixing sdists or providing a working setup.py that updates setuptools first

#

@wispy sapphire t
One problem is that setuptools just always creates a bad state as is, so those setups are never going to easily fix

However people can typically either update user env or install system packages

#

Im happy to wait if someone can fix it more detailed, but that seems unlike to happen soon

wispy sapphire
#

If they can update setuptools, they can probably update pip, which is better, and will both fix the wheel issue as well as support PEP 517 with 518 requirements. But if they are in that case they probably already don't know how to properly handle Python venvs and such, so the very users stuck are the ones that would not be able to fix it. But why was it working before?Was it building with version 0.0.0?

hot goblet
#

@wispy sapphire in some cases yes, exactly that

#

@wispy sapphire in addition to that for packages using pyproject.toml its sometimes completely ignored

wispy sapphire
#

If you are using pyproject.toml, then you are already getting a new install of everything in a venv. pyproject.toml is ignored if you are on pip < 10, though.

hot goblet
#

@wispy sapphire this is observed from pyproject.toml using packages whose ci pipelines failed, and they keep pulling the yanked releases

#

So I prefer error over pretending things work

wispy sapphire
#

Which ones, could you link to one?

#

I've been using it for years in dozens of projects, and so have major other projects, and we haven't been having issues (always pyproject.toml in my case). I've never had a complaint about a bad install on a legacy system, and we have CentOS 7 and even 6 systems everywhere, and no one has rights to install yum packages. (Broken builds from SDists, yes, but not setuptools_scm problems) I'm hoping all these don't start breaking now.

#

I've had the 0.0.0 version a few times, but that was for specific reasons (missing git checkouts, etc), not because pyproject.toml builds were generally broken, or because some package from a distro was not installed (I've never and would never install setuptools_scm from a distro source)

#

A pyproject.toml package installing a non-pyproject.toml package that is failing is not an issue with pyproject.toml - the environment for the sub-package may not have the requirements for that package; it doesn't declare them with pyproject.toml.

#

(Sorry if I seem to be overreacting, not intending to, just pushing people away from the mess of distro packaging should be encouraged, not pushing them towards it and away from pyproject.toml)

hot goblet
#

@wispy sapphire anyone on pyproject with propper config will just keep having a working system, anyone on distro stuff thats actually broken will get a warning that tells them to preinstall build deps as a setuptools issue prevents propper bootstrap

#

also sorry for the delay, the toddler decided to need extra help for sleep

wispy sapphire
#

No problem, I've had a toddler (now 5) and will have one again soon (6 months)

hot goblet
#

Goldilocks zone for a second

wispy sapphire
#

As long as only cases that were 0.0.0 now produce errors. The cases where it worked before hopefully will keep working, even if not "ideal"? (also would be really nice to support 42+ instead of 45+, since the standard pyproject.toml is >=42 for setuptools)

#

Goldilocks zone ends once mobility is gained, I think. πŸ™‚

hot goblet
#

hmm, as we now depend on packaging, im wondering about that

i'll play a bit with that

hot goblet
#

@ancient wigeon given that wheels may install as standalone eggs since 2017 - im wondering that if i set up the dependencies for packakging/setuptools right i may be able to support setuptools version from as early as 30.0 when using packaging>=20.0

hot goblet
#

hmm, i hope i'll find some time this evening to finish the experiment

hot goblet
wispy sapphire
#

I can try to look at it within about three hours. First day back at Princeton today after a year and a half, having to run errands and get set back up.

hot goblet
hot goblet
#

Deferred to tommorow morning as I got delayed and I'm way to tired to do it good now

ancient wigeon
#

I just don't have enough bandwidth to review on short notice. Maybe this weekend.

wispy sapphire
hot goblet
#

@wispy sapphire release is out

wispy sapphire
#

Okay, will try a few rebuilds

#

Could you go back to making github releases too? I follow those so I would get notified. Also, GitHub shows the latest release, and that's listed as "5.0.1"

#

You don't have a PyPI badge, so that's literally the only version indication on the github page

hot goblet
#

i see, i'll need a automation for that (i started to use annotated tags as git needs those for git archive support between tags

wispy sapphire
#

Got the notification, thanks! πŸ™‚ Doing a release from a tag in GHA is pretty easy. If you want to attach assets to it too, it's a bit more painful, but doable.

hot goblet
#

im considering a composite action to publish a release including towncrier details, i want to enable a pipeline that proposes a next release pr, will upload releases to test-pypi and finishes approved releases with a annotated tag ad a gh release

wispy sapphire
#

I've never mastered automated release notes (partially because I really care about getting great release notes), but the hypermodern cookiecutter uses release-drafter, which I plan to look into at some point (heard about it on Python Bytes).

#

Composite actions are great, especially now that they can actually compose actions (instead of of just shell commands)

hot goblet
#

yeah, once i figure why pytests regendocis currently messed up i'll spend my next pytest spike on adding pytest-dev/actions as repo and having default actions/tools there

#

hm, release-drafter cant be used for my purposes, i need towncrier support

#

hmm, god i hate dependencies

wispy sapphire
hot goblet
#

pr with tomli dep incoming, not going to add a toml fallback

wispy sapphire
#

Are you yanking 6.3.0? Looks like you already have 20 thumbs up on the PR to add tomli as a requirement. (Edit: looking at the issue, not the PR, but similar)

hot goblet
#

new version is up, having issues with my password manager now the new version is out as well

#

fixed and yanked

#

@wispy sapphire the transitive issue from black is due to black being pinned to a version not using [toml], i perhaps need to have better toml error handling anyway

wispy sapphire
#

The pin isn't in a release yet, I think, but it was not using the toml reading portion, so didn't need toml or tomli before, but now tomli really is required just to import setuptools_scm (I believe)

upper current
#

Is it intentional that it requires setuptools>=42 with the extra but an unqualified setuptools without it?

wispy sapphire
#

It's really >=31 without it.

#

But @hot goblet didn't want to make that explicit 😒

upper current
#

But why is it different? If tomli is baseline now

wispy sapphire
#

Setuptools >=42 is required for pyproject.toml support (for any package). The reason you'd have the toml extra is if you are using pyproject.toml.

upper current
#

Are you still meant to use the extra?

wispy sapphire
#

Without the toml extra, the "classic" usage really only depends on 31+ (somewhat unofficially, the recommended minimum is still 45+, though the reason for that is gone now that packaging is used)

#

You still need the extra to support older setuptools_scm's, and it's still recommended in the docs (I think)

hot goblet
#

setuptools 45 as baseline is nice as its py3 only, and has bugfixes for the finalizedist hooks

#

the initial intent was just to have the baseline but then old setuptools decided to pull new setuptools into the workingset without actually importing new setuptools

wispy sapphire
#

setuptools 45 is irritating because it's py3 only, so you can't put setuptools>=45 in a project supporting both. πŸ˜›

upper current
#

I don’t know, the extra is used to convey something different now, instead of just install toml, now it’s opt into PEP 517 support

wispy sapphire
#

(Setuptools >=42 is also very common as a recommendation for pyproject.toml)

gray tangle
hot goblet
#

right now i just wanted to ensure the regressions in the ci pipelines stop, i can work out more details on setuptools versions later

wispy sapphire
#

Lots of projects were pinning, and that was a bad idea, IMO, it was better to yank (which was done)

hot goblet
gray tangle
#

I'm doing a hotfix release hopefully later today..., shall we remove the pin and specify the toml extra? Basically what is the least bad fix out of all these fires πŸ˜…

hot goblet
#

yeah, please add the toml extra, also by now it should work again ^^

gray tangle
#

for context: our os native binaries were broken too so it wasn't setuptools-scm being problematic making us do a hotfix release

hot goblet
#

hm, maybe we should aim for loosing the toml extra in normal usage

wispy sapphire
#

Do you need to? You didn't need the extra before, and you don't now. To be really sure you could block out the bad releases, but not that big of an issue because they are yanked.

#

You are using it in pyproject.toml, but you aren't placing any configuration in pyproject.toml, so... Not sure if that should conceptually have the toml extra or not.

hot goblet
#

quick upvote/downvote pull - should we loose extras - use πŸ‘ and πŸ‘Ž

gray tangle
#

my reasoning: while it'd be great if people packaging their projects were actually paying attention and put the toml extra when necessary, seems like that's unfeasible to assume :/

wispy sapphire
#

As long as adding the extra doesn't cause an error, or cause setuptools 42-44 to blow up, I'm fine with either.

#

People were paying attention, because you had to. There were two problems:

  • Some people (including me once in a while) were adding toml explicitly instead of the extra. Maybe because of the extra looking a little weird in pyproject.toml requires (don't know why)
  • Some packages didn't need the extra (black) but it suddenly because required even if you didn't use it in 6.3.0.
hot goblet
wispy sapphire
hot goblet
#

the regression shitshow with setuptools_scm around te last few releases was pretty indicative that a deeper testsuite is necessary

wispy sapphire
#

Some integration tests were needed (looks like some have been added), yes.

gray tangle
#

although we have a healthy userbase of 20.8b1 and that doesn't have a wheel so that's where most of the reports are coming from

#

I'm considering doing a 20.8b1.$insert-build-tag release to add a wheel so whatever packaging flaws are present in that release hopefully less people hit them.

wispy sapphire
#

But historically, not adding toml or the [toml] extra failed, so everyone that was using the toml interface found out they had to add one of the two. It's just the two issues listed above that hit people. You can always be strict at the start, but it's always painful to become more strict later.

hot goblet
#

since everything seems to work atm, i'll be out a bit and play with the toddler

wispy sapphire
#

FYI, Monday many users upgrade pre-commit checks (due to pre-commit.ci), so that's when people are likely to go to the next version if there's been one in the last week.

#

Don't worry, there are tons of packages that are either building in CI or have compiled extensions or something else, so not being notified is not a problem @hot goblet has to worry about πŸ˜†

wispy sapphire
#

The conda-forge recipe will need tomli added as a dep when it updates, FYI. And packaging. (all downstream recipe's will be wrong when adding toml manually, but shouldn't hurt now with it in the main recipe)

#

Actually, tomli is already there, nice, so just packaging.

gray tangle
#

I've removed the setuptools-scm pin, but I've also kept the lack of the toml extra since the setuptools[-scm] bump apparently makes that OK? https://github.com/psf/black/pull/2475

To be clear I wouldn't mind adding the toml extra as I'm just happy to see the pin go. Thoughts on how this fix lands on the "worst kind to ok" scale?

hot goblet
hushed pelican
#

just making sure this is intended

hot goblet
#

Om currently out with the toddler and will fix later

hot goblet
#

@hushed pelican whats the actual error, a integration test of setuptools_scm i added works

hushed pelican
#

just set the version to something not PEP 440 compliant, aka legacy version

#

see the CI

#

I looked at the code briefly, you are instantiating packaging.version.Version directly

#

IIRC

hot goblet
#

ah, dump_version ^^

#

i think i can fix that bit

#

oh DeprecationWarning: Creating a LegacyVersion has been deprecated and will be removed in the next major release as error ^^

@hushed pelican are you aware of a replacement for legacy versions? -

hushed pelican
#

yes

#

hence my PR moving to PEP 440 versions

#

I thought you had just remove them before packaging

hot goblet
#

(i meant as an api, because removing legacy versions will break some old bad stuff i think ^^

#

but yeah, i best dont bring them back into setuptools_scm as well, however i should handle the warning

hushed pelican
#

I think you essentially just keep the string

#

as is

hot goblet
#

i did some release field extraction bit

hushed pelican
#

this is the implementation

#

so yeah

hot goblet
#

hmm, all None

hushed pelican
#

yup

#

there's no meaningful data to be extracted

#

as there's no spec

#

for setuptools_scm shouldn't matter as you don't actually operate the version, just use the string as is

hot goblet
#

@hushed pelican i will put 0.0.0 in the outputted version tuple and print a traceback for now

hushed pelican
#

left some comments on the PR

hot goblet
#

@hushed pelican i lost track of where you asked for the status on git archvies for setuptools_scm, im a bit swamped atm, i got a pipeline of about 4 pytest prs and then i'll get back to the git archives

hushed pelican
#

no worries

#

let me know if there's anything I can help with

spark plinth
#

hi! i want to repackage a library downstream for internal distribution and am trying to use 1) setuptools_scm and 2) https://www.python.org/dev/peps/pep-0440/#local-version-identifiers. tagging a commit e.g. "1.0+bla0" drops the "+bla0" part from the resulting version and distribution files, though... does this work or should i rename the package instead and use normal version tags?

Python.org
PEP 440 -- Version Identification and Dependency Specification

The official home of the Python Programming Language

broken jackal
spark plinth
#

oh, okay... thanks, will look into that

broken jackal
spark plinth
#

i think this is more trouble than its worth 😬 i'll just rename the package

broken jackal
# spark plinth i think this is more trouble than its worth 😬 i'll just rename the package

Have you had a look on the available built-in options to see if they solve your problem? node-and-date, node-and-timestamp, dirty-tag, no-local-version
I have never used node-and-date or node-and-timestamp, but they have a a local node tag, which could help to distinguish from the upstream... or you could always use a fixed custom local_scheme together with a post-release version_scheme, instead of parsing the local tag from the SCM...

spark plinth
#

hm... version_scheme may do the job, but i prefer to have no setup.py... oh well, thanks for your time πŸ™‚

hot goblet
hot goblet
#

@ancient wigeon any thoughts on having setuptools_scm use a non-setuptools build backend?

ancient wigeon
#

No concerns.

broken jackal
#

Hello,

I noticed recently that setuptools-scm code seem to use UTC in some places: https://github.com/pypa/setuptools_scm/blob/088c652ec755f0982b3705272a1e6aa3e07c71ef/src/setuptools_scm/version.py#L116

but the local date in others: https://github.com/pypa/setuptools_scm/blob/088c652ec755f0982b3705272a1e6aa3e07c71ef/src/setuptools_scm/version.py#L336

Is that something intended (or am I struggling to see something)?

GitHub
setuptools_scm/version.py at 088c652ec755f0982b3705272a1e6aa3e07c71...

the blessed package to manage your versions by scm tags - setuptools_scm/version.py at 088c652ec755f0982b3705272a1e6aa3e07c71ef Β· pypa/setuptools_scm

hot goblet
#

@broken jackal different devs at different times doing different things ^^ should be fixed

broken jackal
#

Perfect, I can find time later to submit a PR

wispy leaf
#

@hot goblet hey! does setuptools_scm.get_version consider a fallback_version? it doesn't work for me

hot goblet
#

@wispy leaf it should, im also preparing for a release that will include a Configuration using version of it

hot goblet
#

@wispy leaf 🀦 fallback currently cannot work for your case, it hacks into a detail of project expectations, the next release of setuptools_scm will have the bugfix for that (namely when there is neither setup.cfg nor setup.py it has no ancor to attach to)

#

@wispy leaf im about to release v6.4.0 ading this among some other enhancements

#

@wispy leaf another feature release will land somewhere this month to complete Configuration apis (so you can better pass the pyproject.toml data from build tools, as i need it for gumby elf, and git archive support tools)

#

@broken jackal

#

whops

#

it includes sanity for guess-no-dev plus a number of forward support enhancmenets for setuptools/tooling - among other things handling of projects without setup.py/setup.cfg

broken jackal
#

That is very cool! Thank you very much Ronny!

hot goblet
#

@wispy leaf btw, any opinion on how to pass in the config options of pep-517/518? (to enforce ignores/overrides)

wispy leaf
#

@hot goblet

  1. version 6.4.0 fixed it, thanks!!!
  2. I'm not using those (yet) as I haven't encountered a case that isn't better solved by config or env vars
hot goblet
#

@wispy leaf true, i had gotten some requests for setuptools_scm to be able to take them

broken jackal
#

I think currently only build has the -C option to pass configuration (missing in the case of pip)

hot goblet
broken jackal
#

(the -C is not very ergonomic, though...)

broken jackal
hot goblet
#

pip has some build options stuff in place already

#

for the global/local build options (its a pita, i recall creating part of the mess)

broken jackal
hot goblet
#

harrowing, even 11 years later

#

bbl

broken jackal
#

Thanks Ronny, it seems to be --global-option. Very good to know!

upper current
#

I don't think that option works with PEP 517 builds

hot goblet
#

hmm, oh yay ^^

wispy leaf
hot goblet
#

@wispy leaf in terms of code there are not

#

@wispy leaf in a brakign release i can remove the dependency line in install requires, none of the core logic depends on setuptools

#

@wispy leaf so bascialy removing the in code deps is already done, and once i drop python 3.6, i will also drop some of the not packaging support code

wispy leaf
#

cool! might be nice for the name to not be associated with a specific backend though

hot goblet
#

@wispy leaf when it was start, setuptools was practically the only game in town one could use ^^

wispy leaf
#

right! but now that it's not... πŸ˜‰

hot goblet
#

in the earliers olden days, it was called hgdistver and targetted only mercurial

wispy leaf
#

hmm, would a rename be acceptable then? sounds easier since the de-coupling work is done

hot goblet
#

yes, but tis is something i would tackle sometime in Q2 at earliest, (renaming, exposed apis, expected apis of the new thing) after all some mistakes have been made

wispy leaf
#

awesome, I'm excited!

#

btw I did figure you'd refactor which is why my plugin only exposes the basic options

hot goblet
#

i'll go to try and fold stuff like path determining/specific plugin finding and some more details

this will likely need all new entrypoint names + backward compat to the old ones for a while

#

(this is tech debt incurred from creating and overcomplicating the configuration)

#

the whole topic about overrides wil lbe a pain

wispy sapphire
#

You could rename setuptools_scm to "scmversion" or something, then make setuptools_scm a really simple package that just depends on scmversion & setuptools and reads/adapts the config as needed. Then it's not breaking, and you have some refactor chances too. Up to you. πŸ™‚

hot goblet
#

yeah, also i want some integrations into the editables package ^^ (to auto-sync the version metadata)

#

the most atricious part of the editable wheels thing is, that it creates N copies of the metadata all over the place ^^ so updates are hell

upper current
#

I don't understand, how does it do that?

hot goblet
#

@upper current editable wheels are wheels with the metadata static + some import hacks, so if you have like 10 tox envs, then all of those get the copy of the outdated metadata

wispy sapphire
#

I just happily released boost-histogram 0.0.0; nothing in the entire release chain (setuptools, setuptols_scm, cibuildwheel, twine check, or pypi) restricts a default guess of 0.0.0 from going through (with a red CI X, not just a logging). 😠 - This is after ~3 pre-release wheel builds where I didn't notice that everything was 0.0.0.

#

Did something change recently that made checkout@v1 stop working for versions?

#

I also set SETUPTOOLS_SCM_PRETEND_VERSION: "${{ github.event.inputs.overrideVersion }}", which is just "" on releases, but it's always been happy with that before - following the convention that an empty string envvar should be treated like an undefined one.

eager cargo
#

Honestly, I think this check belongs in setuptools-scm or setuptools. I've seen way too many people release 0.0.0, because they ran setuptools-scm with the wrong args.

upper current
#

Maybe setting the default version to something invalid would work, or a version with a local version identifier which you wouldn't be able to publish

wispy sapphire
#

It really hurts that twine check is practically useless - I don't think I've ever had it fail. It really, really should error out for 0.0.0! And other missing things. And forgetting to remove bdist_wheel.universal=1 after dropping Python 2 (which I forgot about on a package just recently), etc. Maybe we need a new checker tool? I know most people might not run twine check, but I do, and I still wasn't saved by that.

#

I do understand the benefit of 0.0.0 not breaking local installs, as that sort-of-works (I prefer fail and fix over sort-of-work, especially for from-git installs, but, fine), but I see the possible point. But not inventing a name (UNKNOWN) and injecting it, or letting it pass twine check, etc.

hushed pelican
#

setuptools just should not be defaulting the version or name fields

wispy sapphire
#

That's why this is special (somewhere I saw "where would this stop" type argument - but only these get "defaulted"

eager cargo
#

What @FFY00 said, but you might have some luck trying to get the twine maintainers to add the check. :)

upper current
#

Name is required by PEP 621 so that’s another reason not to backfill it.

#

I see that was mentioned by Henry in that thread πŸ˜›

hot goblet
#

Oh fun

hot goblet
#

@wispy leaf i decided to stop trying to revive gumby elf and want to make sure hatch will have a first class setuptools_scm integration
Sometime mid April

wispy leaf
hot goblet
#

the idea being that setuptools_scm wil lbe more like hatch-vcs for setuptools , and the new package having a own tool key and hatch-vcs/setiptools_scm recommending those names

wispy leaf
#

version -> __version__

#

please πŸ™‚

hot goblet
#

hmm, im not sure its worth it, anyone can just import it as version

#

please log an issue tho, else im oing to forget

#

@wispy leaf that being said, i strongly prefer people not using a extra python file, but rather getting the version from the package metadata

#

@wispy leaf i wonder if there ought to be a package that jsut provides a version object thats insanely fast to obtain from metadata

wispy leaf
#

getting the version from the package metadata will always be slower, a no-go for CLIs

hot goblet
#

Hmm

#

@wispy leaf i just realised that it's perfectly valid to just assign both dunder and non dunder names, which makes it a freebie

distant girder
#

pkg_resources is much slower

from pkg_resources import get_distribution, DistributionNotFound

try:
    __version__ = get_distribution(__name__).version
except DistributionNotFound:
     # package is not installed
    pass
unknown.png
gray tangle
#

30ms is still not negligible

wispy leaf
#

30ms for me is slow

distant girder
#

fair enough!

wispy leaf
# 
Benchmark 1: pipenv
  Time (mean Β± Οƒ):     856.1 ms Β±  40.1 ms    [User: 775.2 ms, System: 80.3 ms]
  Range (min … max):   786.8 ms … 965.6 ms    100 runs
 
Benchmark 1: poetry
  Time (mean Β± Οƒ):     557.3 ms Β±  22.4 ms    [User: 515.8 ms, System: 41.3 ms]
  Range (min … max):   524.2 ms … 617.4 ms    100 runs
 
Benchmark 1: flit
 
  Warning: Ignoring non-zero exit code.
  Time (mean Β± Οƒ):     240.2 ms Β±   9.2 ms    [User: 212.7 ms, System: 27.4 ms]
  Range (min … max):   226.9 ms … 265.9 ms    100 runs
 
Benchmark 1: hatch
  Time (mean Β± Οƒ):     151.2 ms Β±   6.4 ms    [User: 132.6 ms, System: 18.6 ms]
  Range (min … max):   142.3 ms … 180.4 ms    100 runs
distant girder
#

hyperfine is a really nice benchmarking tool

polar crystal
#

importlib.metadata.version is actually very slow

#

It breaks dask distributed CI when using it :(

twilit trellis
#

pkg_resources is very slow since it eagerly loads the entire environment in import (and it gets slower the more packages you install)

#

But on the other hand importlib.metadata does not cache metadata so it may be slower in certain scenarios

hot goblet
#

i wish we could finally drop the version numbes in dist info folders

hot goblet
#

@wispy leaf any thouhts on using sometihng like eliot for logging/tracing (intending to use it in setuptools_scm instead of the trace hack )

wispy leaf
#

unfamiliar. describe?

hot goblet
#

@wispy leaf basically a type of logging that allows to trace the tree of actions that unravels, even across processes

wispy leaf
hot goblet
#

@wispy leaf hmm, those are semi orthogonal (ux vs logging)

wispy leaf
#

what's an example of what you want to do?

hot goblet
#

primarily i want to replace the current trace mechanmism with sometihng that tracks the tree of actions/iterations, so that i can debg them bettern , practically anything in that direction would be a poor reimplemenation of eliot

wispy leaf
#

would this require changes on Hatchling? when/how would you want to display stuff?

hot goblet
#

no, this is about logging, not ux

wispy leaf
#

oh then up to you entirely πŸ™‚

#

my first impression is I'm not a fan

hot goblet
#

@wispy leaf both of them lack a trace tree /multi process concept, so to me not better enough than logging

hot goblet
#

@wispy leaf any opinion on setuptools_scm being python 3.7+only soonish,

wispy leaf
#

nope, hatchling plugin on py2 will pull old versions so np

hot goblet
#

@wispy leaf ok, then im going to bump things, im adding typing details as i set up the git archive handling, and the legacy type specs are a major pain however it finds me a lot of edge cases missed in code thats run by minor subsets of the users

wispy leaf
hot goblet
#

hmm, oh, goot call, let me just add the dunder variants

wispy leaf
#

cc @deft vessel I'm sure you'll like ^, no more manually setting template in config πŸ™‚

deft vessel
#

Huh!

hot goblet
#

I hope to complete this later today

hot goblet
#

ffs, mercurial no longer has the tag null 🀦

hot goblet
broken jackal
#

I received a bug report recently (https://github.com/pyscaffold/pyscaffold/issues/649) that seems to indicate that setuptools-scm fails when the repository has a tag similar to draft1 (i.e. word+number).
I saw that currently that is an open issue in https://github.com/pypa/setuptools_scm/issues/449, but it seems like it has already been solved (only pending regression tests).

Is this failure intentional (e.g. the format of this tag is not supported), or should I open a new issue? (The weird part of me is that first setuptools-scm seems to be OK with it and just display a warning, but then in a later point it becomes stricter and fails)

hot goblet
broken jackal
#

Thank you very much for the clarification Ronny! We are planning to add more detailed instructions for this in PyScaffold, but yeah, definitely a migration warning would help those that don't manage to fully read the docs.

hot goblet
#

anyone experience with homebrew - it seems te last setuptools_scm release finally tipped their fragile bootstrapping over

hot goblet
#

fix is out

upper current
#

I think any pacman which builds from source would’ve hit the same issue, homebrew just tends to be updated quicker

twilit trellis
#

Also they tend to care less about how packages interact with each other

gray tangle
hot goblet
#

@gray tangle a number of homebre pacakges triggered a recursion with the 7.0 series for importlib metadata, i now removed the dependency

gray tangle
#

Ah okay, it doesn't seem related then since the crash was an ImportError

hot goblet
#

its still lucky this was fixables swiftly

gray tangle
# 
Collecting black==20.8b0
  Using cached https://nexus.jx.cicd/black-20.8b0.tar.gz (1.1 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  Γ— Preparing metadata (pyproject.toml) did not run successfully.
  β”‚ exit code: 1
  ╰─> [51 lines of output]
      Traceback (most recent call last):
        File "/private/var/folders/0t/f4fb584j0d9d7bb4g6f05vy40000gp/T/pip-build-env-pbooiros/overlay/lib/python3.7/site-packages/setuptools_scm/_entrypoints.py", line 72, in <module>
          from importlib.metadata import entry_points  # type: ignore
      ModuleNotFoundError: No module named 'importlib.metadata'
#

I have no idea why this broke Β―_(ツ)_/Β―

hot goblet
#

is this right now?

gray tangle
hot goblet
#

ah, thats already fixed in 7.0.1

gray tangle
#

ah, so this was a reported issue? I should probably let them know

hot goblet
#

7.0.0 was yanked for that one

#

as for 7.0.2 - it allows to run without the importlib_metadata, but its still a install_required to ease bootstrap

gray tangle
#

I got sidelined into thinking somehow pip's build isolation or importlib.metadata was messing up the environment. Didn't think it would actually be setuptools-scm since I couldn't reproduce it at the time

hot goblet
#

problem is that pip install --no-binary :all: is a footgun, and homebrew is using it extensively

gray tangle
#

This is kinda on us though since we deleted the wheel for black==20.8b0

#

Thank you for figuring it out either way!

hot goblet
#

i now have an extra test cycle with build to build the artefacts isolated ^^

gray tangle
#

I'd have to check the old messenger (yes we used FB messenger originally) logs to confirm, but I'm not doing that :)

gray tangle
#

(I got fed up with dealing with new releases breaking black without warning, so I made a CI workflow that installs black's dependencies from source from github / gitlab daily)

#

Apparently colorama can no longer be built from source on Python 3.6 since the latest version uses hatchling (πŸŽ‰) as its backend which depends on editables 3+ which is Python 3.7+

hot goblet
#

hmm, anything wrong with going 3.7+ by now (setuptools scm just did as well)

gray tangle
#

not sure, probably worth filing an issue in the psf/black tracker about dropping 3.6 support

wispy leaf
wispy sapphire
#

Ouch, so you can't use hatchling (or is it hatch-scm?) + Python 3.6? Python 3.6 will still be around till mid 2023 via Ubuntu 18.04 & manylinux. Though that's only build-from-source, which isn't a huge deal breaker.

gray tangle
#

Yeah, hatchling 0.6 and higher doesn't support Python 3.6

#

Surprisingly most releases don't seem to specify requires-python metadata though

#

Only starting with the 1.x.x series is Requires-Python: >=3.7 in the metadata

wispy leaf
#

b/c < v1 supported py2

hot goblet
#

@wispy sapphire dont older versions of setuptools_scm work there still? the 3.6 drop is recent

wispy sapphire
#

I'm going with what @gray tangle said about colorama, haven't tried 3.6 myself recently. Not sure I want to bother trying to set up 3.6 on Windows to test that. πŸ˜„

broken jackal
#

Just learned that %(describe) in export-subst needs git 2.32.0 and that %(describe:tags=true) needs git 2.35.0.

It seems that github servers are a bit behind...

#

So it seems that the support for git archives fixes the problem with unknown versions, but projects would still not be able to rely on include_package_data=True, because the information about list of files is missed on the archival process, right?

hot goblet