off-topic

[desert verge]

Oh. What happened there?

[rigid sphinx]

PEP 751 - A file format to record Python dependencies for installation reproducibility

Status

Final

Created

24-Jul-2024

Type

Standards Track

[long knoll]

nothing; pep 751 describes pylock.toml, which is what the GH issue relates to

[long knoll]

... I notice that DPO commonly gets posts from beginners on Windows who are confused about what to do after running the installer.
I wonder if it would be worthwhile to add a hyperlink on the last page of the installer, to the tutorial in the docs

[desert verge]

is there a standards or a blog post about pyproject metadata and requires-python? just saw a project stop testing on python 3.9 (and only test 3.10-3.14) but leave requires-python>=3.9 intentionally....

[marsh kite]

Are you thinking of https://peps.python.org/pep-0621/ ? The specification for the requires-python is https://packaging.python.org/en/latest/specifications/pyproject-toml/#requires-python

[desert verge]

I'm thinking of some blog post or github comment I once saw about requires-python needing to be properly set

where setting requires-python to >=3.8 but only testing 3.11, 3.12, 3.13 is useless, and if requires-python>= 3.8 then the application should test 3.8 to 3.12

because when it inevitably stops working on python 3.8 then that release is just... perpetually broken on account of what amounts to incorrect metadata

somewhere someone had strong feelings about this and already explained it

[dreamy hatch]

This one? https://iscinumpy.dev/post/bound-version-constraints/

[desert verge]

Not quite, I don't think

it was how requires-python's minimum should be what you test on

if you aren't testing on python 3.8 but you test 3.10 and up, your requires python shhould be >=3.10

[dreamy hatch]

Okay, well it's generally good advice. If you're not testing it, you don't know if you accidentally broke it.

[lapis solstice]

Testing on every supported version may be overkill, but yeah, testing on the lowest version with declared support, the latest version with declared support, and maybe some of the versions in between is a good idea.

[junior narwhal]

Finally announced today

https://squidfunk.github.io/mkdocs-material/blog/2025/11/05/zensical/

[kindred hound]

Thanks for the ping!

[silk jungle]

Ah crap, I think I lost my security key. It broke off the (admittedly thin) metal clip off my keychain. Fortunately, I have a backup, but now I got to go order another one 🙃

[kind moon]

Here’s your biannual reminder to make sure your recovery codes are up-to-date too!

[silk jungle]

They are.

[desert verge]

Shit.

[desert verge]

And then rotate every credential..... Yikes

No seriously that's the double edged sword of passkeys IMO

[silk jungle]

I keep track of my credentials. I can rotate them easily. I review my accounts somewhat regularly anyway.

[kind moon]

... don't you just remove the old passkey?

[desert verge]

Much better organized than me lmao

[silk jungle]

Well I kept track of them because I wasn't using a password manager.

Then I began using a password manager, but I kept the spreadsheet and now I have both

[desert verge]

I'm using a password manager and not keeping track of mine lol

It's a problem

[kind moon]

I'm still being lazy about it

[desert verge]

Also hi Richard! It's nice to talk to you again

I think it's been almost two years since we last had a conversation

[kind moon]

But I do at least have https://landing.google.com/intl/en_us/advancedprotection/

[desert verge]

Hm does https://flying.google also exist?

[long knoll]

(I've never owned a security key. Wouldn't know where to get one, and banks don't offer the option to use them, so what's the point)

[silk jungle]

For most individuals, I agree that security keys are not a good fit.

TOTP 2FA or passkeys through their password manager/Windows Hello/macOS keyring is much better.

[long knoll]

I think I would prefer the world where they were, though.

many businesses are still only offering 2fa via SMS etc.

[silk jungle]

The benefit of the security key is that it is literally a separate hardware device. Nowadays, I'm pretty sure most smartphones can act as passkeys. And for most people, their phone is probably a better choice.

For GitHub and my most critical accounts, I don't manage them through my password manager. Those accounts I memorize an unique password and use security keys for 2FA.

[long knoll]

if you run an authenticator app on the phone, I suppose so. I don't have one, though. I want to be able to use a desktop auth client and supply the 2FA to the usual web-page desktop login. The alternative is to have the smartphone and also obtain the company's app, further enhancing their control of my computing devices.

[silk jungle]

Are you talking about corporate 2FA?

[long knoll]

mainly banks

[silk jungle]

Ah, yeah, banks are just behind on times.

I'm still stuck with SMS 2FA there.

[long knoll]

but probably some other sites as well.

Oddly enough: the only place I know of that I can auth properly, that's actually relevant to me, is PyPI.

(actually, GitHub uses my computer's keyring, so that also counts)

[long knoll]

(Ugh. Discord seems to have decided to, in effect, limit my effective typing speed to an unknowable and unreasonably low value)

[desert verge]

This is the context, fwiw https://github.com/timvink/mkdocs-git-revision-date-localized-plugin/compare/e156f445c517ea659a6abe5d237af0b15f72629c...54dc89bac68702da00a8f3ee1e063e4d98a975b6

[dreamy hatch]

fixed: https://github.com/timvink/mkdocs-git-revision-date-localized-plugin/pull/198

[desert verge]

Ah nice, thank you Hugo!

[junior narwhal]

What's our process for inviting a new co-maintainer of a project to the org so that they have write access?

[silk jungle]

You should be able to add them as an outside collaborator with your repository admin rights in the interim, but you'll need to contact one of the PyPA admins to have them invite the individual to join the organisation.

You can contact Pradyun, he should be around (barring timezones, of course).

[silk jungle]

Hi!

You've undergone so many changes in username that I can hardly keep track of them!

[desert verge]

Hey hey the username hasn't changed, it's just the display name! I'll always be the same ol' qt I've always been!

Your name has lengthened too, has the rest of your name in it now, I see 🙂

[mighty flower]

lol: https://github.com/easy-install/easy-install

[silk jungle]

Clearly not a python user, ha

[desert verge]

Could someone possibly weigh in on https://github.com/sphinx-doc/sphinx/issues/8709? I've found it is indeed in docutils, but I'm unsure how to proceed (the first step of course would be to make a docutils issue, but not sure if its a bug or a feature request and other concerns.

[silk jungle]

What is up with GitHub sending so many emails? They first of all send me an email about GitHub copilot and now they're sending a ton of marketing emails.

[long knoll]

I don't experience this. Could be something in settings I fixed a while back and forgot about... ?

[timber sphinx]

Same experience, been mashing the Unsubscribe button.

[lyric quiver]

Can't relate

They must think you are rich or they figured out I am broke and no amount of discounts can convince me

[hexed briar]

I reported as spam coz I was like... Nah, GitHub ain't gonna market like this

[kind moon]

Speaking of GitHub marketing

[long knoll]

I mean, I'm not in the US, but Canada doesn't exactly have GDPR-level protections afaik

[kind moon]

Well, Canada isn’t the US

[long knoll]

yeah. I meant: maybe that could explain why I don't get the emails, but I half expect GitHub to send them to me anyway, because I'm not in Europe.

[silk jungle]

I am Canadian and I still get the emails. The Canadian anti-spam laws are effectively non-existent since they aren't enforced.

[silk jungle]

I should do that, too. I just created a filter to autodelete the emails since I just. do. not. care.

[long knoll]

(maybe I set up a filter and forgot about it?)

[robust sandal]

Just wrote a long idea for PEP810, then reread the (final) PEP and realized one part was already fixed, and the other was discussed and rejected (for good reasons). Quite happy with it now. Wish I'd reread it sooner, though. 😉

[mighty flower]

It's a long PEP, I have no idea how anyone keeps these long specifications in their head

[frank shore]

That's nothing compared to being able to solve Einstein's field equations in his head for @solemn fulcrum

[desert verge]

ah yes. PSF sponsors PSF 😂

[silk jungle]

infinite money glitch!

[desert verge]

(don't worry, this is the test.pypi instance)

[solemn fulcrum]

To be honest I can only solve two cases of Einstein Field equations in my head and one is when everything is 0, 1 or -1 🤣

[silk jungle]

I've been noticing an uptick in wholly AI-generated issue "analyses" and also PRs. It's frustrating 😦

[long knoll]

it's hard to understand the mindset of some AI users

[lapis solstice]

There was a post (since removed) in the venv-info thread that I reported to the mods mostly out of confusion. I think the poster had just fed my original post into a chatbot with a request to summarise it, and then quote-posted the whole thing back to the thread together with the corp-speak summary. It definitely wasn't a helpful post, but none of the existing reporting categories really covered it.

Using an AI to reframe a proposal to check your own understanding is correct seems reasonable, but actually posting the reframing back to the thread was just very odd.

[torn fjord]

Is anyone following along with the Shai-Hulud stuff? It got me thinking if PyPI has protections in place to prevent an attack like this?

context: https://www.sysdig.com/blog/return-of-the-shai-hulud-worm-affects-over-25-000-github-repositories

[dreamy hatch]

https://blog.yossarian.net/2025/11/21/We-should-all-be-using-dependency-cooldowns

[wind pewter]

Though I agree with dependency cooldowns, I think there should be some level of protections put in place to avoid the same fate as npm. I am not sure what those protections would look like.

[kind moon]

yossarian
Why do I know that name?

woodruff
oh, that's uh... PyPI guy?

[long knoll]

yes, very active on HN too

[lyric quiver]

I just learned today that you can't even technically enforce your own license on yourself

So say I got a library with gpl, i make a tool that relies on that library

And I don't open source it

What am I supposed to do, file a lawsuit against myself?

[dark anchor]

your users would file a lawsuit against you, if you release binaries with no source code; assuming you’re using gplv3 and not agpl

the gpl is A-OK with private closed-source code, the trouble comes when you redistribute binaries derived from GPL code

[lyric quiver]

O

I derived this from AGPL

I thought it also applied to gpl

[dark anchor]

and for AGPL it would only matter if you were operating a web service that others can use, purely private closed source uses are fine

[fierce horizon]

Also if you own the code, you can just give yourself a license to do whatever the fuck you want.

So I doubt a user suing you would go anywhere.

[lyric quiver]

It's weird, I was always under the impression that licenses apply to everyone including the owner of said license

Well I guess that's not the case for gpl

[mild pollen]

You can't license something to yourself. You already have all the rights.

[lyric quiver]

The more you know

[shadow zealot]

Well technically the license can apply to yourself if you want it to.

[fierce horizon]

And now we're get back to what I said: it can, but it can't meaningfully restrict you, as you still own the code and can therefore at any point in time decide to e.g. create a one-time special license to anyone you want giving them any rights you want them to have.

[lyric quiver]

Hmm, after a quick back and forth with chat gipity

It seems counterintuitive

It says that I can still sell my software that relies on a gpl/AGPL licensed library which I own therefore because licenses only apply to the recipients of the code

I know I should not base my research on chatgipity, I just got curios

[fierce horizon]

People are already really bad with license stuff, I’m sure LLMs can only increase confusion here.

First: you can do whatever. Worst case, if you end up in breach of license terms, then you can end up in legal trouble. Nobody can force you to do anything, so don’t believe any nonsense you hear about the GPL “forcing you to open-source your proprietary code”. This is a remedy you can choose in case of breach, but you can’t be forced to do it.

[boreal bramble]

Any (*) code you write is yours and you can do whatever you want with it. You can choose to distribute it under OSS (or other kind of) license which gives certain rights to people that you distribute it to. That doesn't restrict your (copyright owner's) rights, it just gives the person you distributed to certain rights provided that they fulfill certain conditions.

(*) excluding code that you sold your monetary (economic) rights to, as is common as part of employment. You're still the author of that work in such case but you can't license to others nor use it yourself unless the entity you sold it to gives you that right

I'm not a lawyer so the terminology might be off, especially with the usage of "copyright owner" and "economic rights" terms

[fierce horizon]

It says that I can still sell my software that relies on a gpl/AGPL licensed library which I own therefore because licenses only apply to the recipients of the code
The last bit makes no sense. The GPL doesn’t apply to “recipients of the code” lol, getting the code is the purpose of the GPL. You can’t just let statistical slop generators loose on precise legal language and expect anything but getting more confused. Talk with people who know their stuff instead, they might be inaccurate in wording, but at least they’ll know what the gist is supposed to be.

The GPL comes into play when you (as e.g. a customer) exercise your right to execute something built from GPL code, and decide you want the source. You can then ask for it, and if you don’t get it, whoever sold the software to you is in breach of GPL and can be sued.

Regarding your scenario: you can provide access to software under multiple alternative licenses. So implicitly, of course you can use a modified version of (usually GPL) library code you own in your software (that you sell), licensed from you to you under the “I can do whatever the fuck I want license”.

[dark anchor]

you can sell code under the GPL, you just have to give people the sourcr code and let people redistribute the source code

you can also sell code under a different, proprietary, license with restrictive terms

tbh the main use of the AGPL these days is to do that, making your software AGPL-only is a little anarchist in that it means basically no company will touch the software with a ten foot pole because it’s easy to accidentally breach the AGPL.

[lyric quiver]

Oh this went downhill

I guess one can just say:

for my deploymet, I grant myself an exception

and screw them all 🤣

[lyric quiver]

Now that I think about this, this would be extremely hard to expose.

How can I the end user even know what runs on their cloud machine.

the best way to conceil the code is to never give the code after all

[fierce horizon]

It's hard to prove, but if someone blows the whistle, suddenly many customers have a legal claim against Amazon.

[lyric quiver]

Been a huge OSS advocate since the moment I started coding

but the whole Meta situation with the books and what not

I've been losing trust

• I am pretty sure that anthropic and all others are alr training on GPL licensed code

[lyric quiver]

https://www.cbc.ca/news/canada/newfoundland-labrador/meta-ai-local-authors-1.7502766

https://www.theguardian.com/technology/2025/jan/10/mark-zuckerberg-meta-books-ai-models-sarah-silverman apparently it wasn't just rumors, it's actually true, I wonder if they got fined with pocket change ( for them ).

[fierce horizon]

They always do, if at all.

[long knoll]

Apparently my local 3.6 release build will segfault when importing ctypes; debug build works. I think the same is true for my older builds as well, which might explain some other things I encountered in other experiments...

It was not very pleasant figuring that out.

[floral oar]

The licensing thread above reminds me of a funny wrinkle I've come across, on account of the fact that I'm employed by a US based university. There's some paperwork I was required to sign which basically gives the university some broad IP rights to anything I produce "using university resources". I can't remember the exact terms, but it was clearly aimed at people creating profitable products, and probably thinking about the physical sciences and wet lab stuff. I do most of my dev work on a laptop which technically is "a university resource", so I think they could claim ownership of my code. It's not a real, practical issue (esp. since I'm not trying to turn my projects into $$$), but it's funny to think about.

Notably, my employer's claim to ownership doesn't take the form of copyright, so it highlights how copyright/licensing concerns can be orthogonal to practical ownership and use concerns.

[lapis solstice]

It's not uncommon for a lot of "standard" employment contracts to have overreaching IP clauses, as the notion folks might have work-related copyrights that they actually cared about retaining as personal copyrights is unusual outside the open source licensing context. Two main ways I've seen of dealing with it are explicit copyright disclaimers (e.g. my current employer made a written commitment not to interfere with my open source work before I was willing to sign their employment contract), or outright amending the IP claim clauses in their employment contracts to take open source participation into account (a previous employer did that).

As far as licensing affecting the copyright owners goes, it depends on how they accept contributions to the project:

• no external contributions -> no restrictions
• copyright assignment to the project IP owners -> no restrictions
• contributor licensing agreement to the project IP owners -> restrictions as per CLA (e.g. the PSF is permitted to relicense CPython contributions from our contributing licenses to another open source license, such as the PSF license, but not to a proprietary license)
• project license (aka "license in = license out") -> even the project IP owner must abide by the project license for all external contributions, changing the license requires removal of code provided by any contributors that don't explicitly agree to the license change

In the absence of an explicit copyright assignment or contributor licensing agreement, projects generally fall into the last category.

(caveats: this is a simplification that ignores a bunch of jurisdiction specific details, and even though I've run a corporate open source licensing regime before, I'm still not actually a lawyer of any kind, let alone specifically an IP lawyer)

[floral oar]

Never having had to handle a major project license change, I've long wondered how you actually (mechanically) deal with the removal of people's work. As a concrete example, if someone contributes a helper that adds a method User.get_id(), whose content is return self._id, and they don't agree to the new license for their contribution... I remove the method, but then I'm free to reimplement it? Identically, since it's trivial? And that's even a simple case, since there's a specific singular thing to remove. vs many cases where reverting a physical diff is no longer even possible, but someone's ideas, renamed and restructured, live on.

[lyric quiver]

Simple, one liners are impossible to copyright

Like you can't copyright the function: return print(text), too simple and boilerplate

Even in the worst case scenario,you could just remove it and then "rewrite" it from scratch

[undone sinew]

But the reality is that major license changes are incredibly rare. And individuals objecting even more so. The usual problem is just getting enough people to reply, that you can make the call to do it.

[dark anchor]

and this is also why it’s important to get things like license choices correct the first time, otherwise it’ll be a major headache to change

[cyan kestrel]

In this specific case, in US copyright 'merger doctrine' may apply: where if an idea and its implementation are inseparable, copyright protection does not apply. In the broader case, consider, what if someone proposes some novel (and copyrightable) solution that you reject but later implement?

[dreamy hatch]

https://cloudisland.nz/@xssfox/115648821542637298

[long knoll]

... Did we get a spammer again?

[onyx spindle]

seems like it

[silk jungle]

Yes.

[silk jungle]

I'm setting up my new Ubuntu install and decided to upgrade my Python toolchain to 3.14. I didn't realize that the ecosystem is still not quite ready for that 😅. Pydantic is building from source

[marsh kite]

Are you using a free-threaded or GILful build?

[silk jungle]

no idea. I'm just building a shared build with PGO and LTO. That's all.

I guess BOLT support is still experimental. I still haven't been able to build Python with Bolt at all yet.

[dreamy hatch]

Pydantic has supported 3.14 since 3.14.0 release day

[marsh kite]

The default build is not free-threaded (yet!). BOLT is still somewhat experimental I'd say, but if you encounter a bug/issue please do report it!

[onyx spindle]

is BOLT another speedup/perf thing?

[marsh kite]

Even "I tried enabling it but it didn't work and I don't know why" is useful.

[marsh kite]

"BOLT is a post-link optimizer developed to speed up large applications. It achieves the improvements by optimizing application's code layout based on execution profile gathered by sampling profiler" https://github.com/llvm/llvm-project/blob/main/bolt/README.md

[onyx spindle]

thanks

[silk jungle]

¯_(ツ)_/¯

I got it sorted out using an alpha build of pydantic. I only need for personal scripts and it works now.

[silk jungle]

I strongly suspect that the default bolt toolchain on Ubuntu is still not new enough. I'm on LLVM-bolt 20.

I'll search the CPython issue tracker, but I'm wary of opening issues that people aren't going to be able to action.

[marsh kite]

Well we should probably check the version if old versions break

[silk jungle]

hahah what the heck vim

Overall, ubuntu 25.10 is still quite buggy. Although I guess that's to be expected since it's only a few months old.

[marsh kite]

Yeah I only run LTS releases of Ubuntu these days

and usually 6+ months after they are released

[silk jungle]

I'm stuck since Ubuntu 24.04 is too old for my hardware (Intel Lunar Lake) and Ubuntu 25.04 is going to be EOL in a month.

I'm waiting for 26.04 LTS and then I'll stop upgrading my OS :)

[marsh kite]

Ah yeah that's tricky

[silk jungle]

Seems like it was already fixed in https://github.com/python/cpython/pull/128511

[long knoll]

I'm on Mint, so I just get their patches to the Ubuntu LTS. So even keeping on top of Mint updates, the system Python is a couple versions back. Which is fine for day-to-day use, and I have my own local source builds for things where I want to try the newest versions.

[marsh kite]

Hmm, is your checkout out of date then?

[silk jungle]

I was building 3.14.2 as a shared build.

FWIW, BOLT works if I build a static build.

[silk jungle]

I don't use the system Python. I compile Python from source. I could probably use the deadsnakes PPA or something else, but I'm used to it, hah.

[marsh kite]

sure, but seems like a regression, because that PR should have been in 3.14.0

[long knoll]

yeah, building from source is the way honestly.

but I don't really see value in keeping my "toolchain" on the leading edge. Older versions of, say, Pytest run my tests just fine, etc.

I'd rather look for new tools and evaluate them.

[silk jungle]

I'm mostly upgrading out of interest.

[long knoll]

ah

[silk jungle]

I also reinstall my OS a lot

[long knoll]

... yeah I've had issues with the PGO stuff too, so I kinda just stopped trying

[silk jungle]

PGO has always been rock solid for me. It's just BOLT that's finicky.

TBH, Python is fast enough. BOLT is just a curiosity.

[long knoll]

(more that I don't understand it)

the lesson is, computers are really fast. CPython can be terribly inefficient in a variety of ways with it not really mattering. But it's still nice to write well designed architectures that don't compound the overhead of Python

[marsh kite]

Well, I'll try to repro in a container and file an issue if it is a regression

[long knoll]

(I've been banging a drum this year about where uv's performance comes from, you see....)

[marsh kite]

there's also https://github.com/python/cpython/pull/140250 which should resolve the computed-goto issues entirely

[jaunty marlin]

We use BOLT in python-build-standalone

And I've done some work to stabilize it upstream, there's really no reason it should be considered experimental anymore

I think the last reason was that we wanted to choose optimal default BOLT flags, but that's not really critical

What error are you encountering?

[silk jungle]

Hmm, this bug is happening fairly often. I can't tell what's causing it yet.

Switching terminal emulators doesn't fix it, so I suspect it's either an issue with the theme or vim-airline (or a bad interaction between the two).

[onyx spindle]

Is it vim or neovim?

[silk jungle]

Vim 9.1

[silk jungle]

Aha, I think I found the issue. https://github.com/vim-syntastic/syntastic has been deprecated for years now.

Disabling syntastic fixes the issue. Time to migrate to ALE I guess

I suppose at some point, I should probably get around to setting up a proper LSP since I don't use VSCode anyway, but I'll deal with that later

[silk jungle]

Any benefits of neovim over vim as an already happy vim user?

[onyx spindle]

Lua for starters 🤣

Native LSP support

Tree-sitter

But I might be biased, Since I have never used the classic Vim for more than some online config edits

[long knoll]

I actually have recently been thinking about things I would do differently in a vim-like editor

[onyx spindle]

try helix, it's another take on vim-like editor

[silk jungle]

who ever implemented exit for the new REPL, it's great ❤️ ❤️

While I've mostly learned to ingrain Ctrl-D as part of my muscle memory. I still sometimes use exit and to have it work in the new Python REPL was a pleasant surprise.

[silk jungle]

I'm probably going to scribble this down as a future project for me when I have more disposable time.

[onyx spindle]

I think it's 3.13+, right? or was it 3.14?

[silk jungle]

who knows ¯_(ツ)_/¯

[onyx spindle]

3.13 :)

[marsh kite]

Yeah it was added by Pablo in 3.13 based on code from PyPy, it's fantastic!

[floral oar]

I migrated from vim to neovim this year, and carried my ALE config across. That setup was quite pleasant. But then I tried to setup LSP support, and that may have driven me slightly (more) mad, as nothing worked quite how I wanted. I decided to break from my vim user identity and try Helix. I've been using it now for several months very happily, as a 10-year vim user. So I recommend nvim+ALE and/or helix -- but I had trouble with nvim+pylsp and can't fully recommend it (presumably I did something wrong, of course, and there's a "right way" which would have worked better)

[silk jungle]

I've never used a LSP with vim TBH.

The only fancy feature I'd I have used is syntastic for syntax checking because I am very typo prone.

[floral oar]

My experience of LSP usage is that it works much better and is more useful if you have installed packages available to it, typically in a project dev venv. It let's you do things like "jump to definition" into site-packages, which is a really cool navigational tool. But I couldn't get neovim to invoke pylsp the right way to make it aware of my venvs, and under helix it picked up on an activated venv out of the box.

It's been super long since I made the jump from syntastic to ALE... I think you're about to enter the world of potentially running fixers like black on the current buffer? It's a big adjustment, but I've come to like it. I get to see the formatted code much earlier, so I know right away if it gets folded in funny ways.

[silk jungle]

Does anyone here have any experience in using https://ghost.org? I'm trying to figure out how it compares to Wordpress.

[kind moon]

PyDis migrated away from it in July 2024: https://github.com/python-discord/infra/pull/415
You might ask Joe/Chris

[silk jungle]

I am finally learning to properly leverage Docker. I'm rebuilding my VPS and I've realized that I don't remember how I configured the services running on it. I could use systemd services + bash scripts as I have done before, but this is a good chance to learn to write docker images.

[kind moon]

Good luck

Once you Docker you never go back

It’s fun

[onyx spindle]

Until you learn about Nix 😉

[kind moon]

I’ve been meaning to try Nix

Tried to spin up a VM a while back but Windows wasn’t having it

[silk jungle]

OK, migrating volumes from one host to another is miserable.

[marsh kite]

yeahh they're not really designed for that

[silk jungle]

so tough luck if you have to change your server

[silk jungle]

Is there a proper way to handle persistent data that may need to be moved around? I guess bind mounts?

[marsh kite]

how frequently do you wish to move it around, and in a single direction or syncing?

[silk jungle]

Just curious in general. I'm moving these volumes as an one-off (migrating from DigitalOcean to Netcup).

[marsh kite]

Volumes are the de-facto way to store data, but I use bind mounts a fair bit (maybe too much 😅)

[silk jungle]

I ended up using a bash helper script someone else wrote to export and import snapshots of the volumes and let docker compose use externally managed volumes.

[marsh kite]

interesting. Yeah I think the best way I've seen is to mount the volume in a container and copy the contents as you normally would to a new container/volume pair

Not many great solutions here I'm aware of :/

[silk jungle]

I'm surprised that docker itself doesn't have a nicer official method

[frank shore]

At least Docker works for you! macOS desktop Docker is no fun: https://github.com/docker/for-mac/issues/7783#issuecomment-3520918134

[silk jungle]

I've never used docker desktop even on my daily driver

[dark anchor]

orbstack works pretty well on my mac, it's only free for personal use though

[timber sphinx]

Works so well, we pay for it for work, too!

[frank shore]

brew install orbstack 😄

[silk jungle]

happy new years to those on the east coast! 🎉

[torn fjord]

orbstack is the best

It's really awesome that you can also use it to do linux vms as well, somewhat of different way of doing something like proxmox.

[torn fjord]

volume mounting is the recommended solution.

You could also use container volumes (like virtual disks) but I generally recommend against those unless you're a more advanced user.

There's not a whole ton of benefit that you get out of those if you're just a home user.

[dark anchor]

FYI, the ruaml.yaml maintainer decided to stop distributing wheels, this broke pre-commit among other things. If you see CI breakage around this package that’s what’s happening. https://sourceforge.net/p/ruamel-yaml/tickets/558/

[fierce horizon]

decided to stop distributing wheels

Huh yeah, they specifically mention that in ruamel.yaml.clibz’s PyPI readme that the lack of wheels is intentional.

It's also funny to me personally because I recently got banned by Anthony for daring to suggest he switch to a more correct yaml parser “like prek has”. (For people unfamiliar with Anthony: yes, I really just said that)

[kind moon]

(Did ruamel give a reason?)

[fierce horizon]

The intention is to only distribute this package as .tar.gz, allowing for optimised compilation according to your machine's specific architecture and capabilities, instead of some (low performance) common denominator.

[dark anchor]

https://github.com/pre-commit/pre-commit-hooks/issues/1229#issuecomment-3704873830 yeah this doesn’t seem very helpful for pre-commit users. PyYAML is also kind of a mess. It’s too bad YAML in python is such a pain.

[dreamy hatch]

The motivitation for this change is the availability, and easy of use, of Zig as the toolchain (in the form of ziglang on PyPi), so lenghty, non-optimized, pre-compilation and uploading to PyPI, is no longer necessary. The time spent on creating ~60 wheels and even more time wasted on dealing with CI providers (Appveyor not being updated to support 3.14, Github CI being slow, and charging for the use of your own computer, etc).

The split out of ruamel.yaml.clib after the 0.15.100 release, was also motivated by the time spent on generating .whl files even if only Python code was changed. The use of ziglang and setuptools-zig does make re-integration of the C sources into ruamel.yaml feasable, but there are no plans yet to make this happen.

https://pypi.org/project/ruamel.yaml/0.19.0/

one-time build of ~60 wheels at release time vs 118,749,267 builds/month at install time

[floral oar]

Oh, great... I tried to help ruamel.yaml last year, but I couldn't even get the information I needed from the maintainer. Plus, sourceforge is just "out of sight, out of mind" for me.

Trouble is, if I want roundtripping or to get the position of a value in a file (line, column), I only know how to do that with ruamel.yaml. Do I just pin my projects to <0.19? Anyone have ideas in this space?

[dark anchor]

If you want to avoid from-source builds with zig at install-time. The builds should work with recent setuptools and wheels versions, apparently they didn’t specify the build dependencies in a way that would cause those to be updated before the build and that’s now been fixed, I guess?

[floral oar]

Last I recall, the project didn't have a pyproject.toml file. Is it now using one to specify setuptools-zig usage?

[dreamy hatch]

still no pyproject.toml: https://sourceforge.net/p/ruamel-yaml/code/ci/default/tree/

[dark anchor]

I haven’t looked closely, I only saw they did a couple quick releases an hour ago.

[floral oar]

Oh, setup_requires=.... ... 😐

[dark anchor]

the clib also includes bundled generated C sources from Cython code for similar reasons which is also a bad idea from way way before python packaging was working well

[kindred hound]

They added it in the dependency https://inspector.pypi.io/project/ruamel-yaml-clibz/0.3.6/packages/7c/1e/60393fed27bd79fcf8a70c776b25cc78e6d2f5950d769e23b4db612c0b35/ruamel_yaml_clibz-0.3.6.tar.gz/ruamel_yaml_clibz-0.3.6/pyproject.toml

[floral oar]

My one request, 2026. My one request was "could things just stop happening, for a little while?" Already, not getting a year off 😂

[onyx spindle]

I guess we just have to wait for a ruamel-yaml-binary project or a fork

[silk jungle]

or just universally agree to delete YAML from existence ¯_(ツ)_/¯

[onyx spindle]

+1 to that, but all CI but Jenkins would probably die

[long knoll]

... Anthony?

[floral oar]

Anthony Sottile, the pre-commit (and deadsnakes and other-such) maintainer. He has a very "my way or the highway" way of handling online/community interactions, which is often feels hostile and is generally problematic.

[dark anchor]

they could have released abi3 wheels and saved a bunch of trouble but they seem to be checked out of this community

zig can build wheels too 🤷‍♂️

[floral oar]

they seem to be checked out of this community
100% that seems to be the issue. And I mean that not in a catty way. This is someone who is interested in working on YAML but I think has near-0 interest in packaging.

FWIW, I don't think this maintainer is bad or toxic based on my interactions. But being on the sourceforge+Mercurial island is definitely doing some harm in terms of how easily anyone can contribute.

[dark anchor]

they did another release that removed the compiled dependency completely

there’s a bunch of discussion in the readme https://pypi.org/project/ruamel.yaml/

[long knoll]

However, after uploading version 0.18.7, I got an email from PyPI, about having to change the project name to ruamel_yaml to comply with PEP 625, sometime in the future. The email doesn't say if namespace packages are no longer allowed, or how to deal with the very real clash with the pre-existing package ruamel_yaml.

I don't know what to make of that. If I try modifying the URL to ruamel_yaml it just redirects to ruamel.yaml

[silk jungle]

The actual problem is that the sdist name needs to be ruamel_yaml. The actual project name can still include the period.

We are not banning namespace packages.

[long knoll]

right

also there doesn't appear to actually be any clash? or is there some other project that has ruamel_yaml as a top-level import name

[silk jungle]

https://github.com/termim/ruamel_yaml ? which is quite literally a redistribution or fork

[long knoll]

(... but also, is "Anthon van der Neut" really the same person?)

it does seem like the README commentary is motivated by a confusion between package names and import names

[silk jungle]

It is confusing to non packaging experts, but I really have no interest in interacting on sourceforge (I don't even have an account there).

[floral oar]

I don't think Anthon is on GitHub at all. I would be suspicious of anything which looks like him on that platform. And there are some junky forks of the project.

[dark anchor]

from having gone through the setup.py recently, I think they’d probably benefit from getting a link to this discord; they just need somewhere to ask questions and not get outdated advice or advice based on bad opinions of modern python packaging tools and standards

[floral oar]

Speaking of that readme note, there is this sourceforge ticket, but I admit that I gave up

[silk jungle]

The takeaway is that you should let your build backend handle the standards for you. If they don't tell you or help you when you violate the standards, the backend is not doing a great job (I know that setuptools is like super old, but for most people, they really would be better served by modern backends).

[long knoll]

I mean, https://github.com/AvdN appears to be the top contributor to https://github.com/termim/ruamel_yaml . I don't even see the repository owner in the contributor list. But I do see both Anthon and Anthony. 😕

[silk jungle]

Well that doesn't give me hope.

[long knoll]

I really have no idea what's going on there.

[dark anchor]

I think they are on github

[long knoll]

but Anthony is active on GitHub and also streams development on Twitch regularly

[dark anchor]

they mentioned in a recent issue about free-threading that they were building wheels on github

[silk jungle]

Anthon and Anthony are two different people.

[floral oar]

You mean asottile? (Anthony) vs Anthon? ☝️ yeah that

[long knoll]

Yeah, that much seemed clear enough... eventually

[floral oar]

And yeah, I guess AvdN does look like Anthon. So I'm wrong and he is on GitHub? I thought he was fully staying off the platform, as some people choose to do.

[dark anchor]

https://sourceforge.net/p/ruamel-yaml-clib/tickets/44/#6658

[long knoll]

anyway, there are still wheels for now

and they're none-any wheels at that, so I assume there's a Python fallback for the parsing?

[marsh kite]

Yaml is kinda a mess. I worked on https://github.com/emmatyping/ryaml to try to have a non-pyyaml base for yaml parsing in Python, but a lot of people actually need yaml 1.1.

[dark anchor]

and so we come back to ^

[marsh kite]

I've been considering trying to write something based on https://github.com/simonask/libyaml-safer, which might provide a more reasonable base

[long knoll]

I know there's also PyYAML which I've seen as a dependency a few times, and which distributes platform-specific wheels

[dark anchor]

having dealt with pyyaml as an upstream to add free-threading support, it’s not much better

[long knoll]

I wonder if there are compiled/accelerated TOML options out there

[marsh kite]

There are rust parsers for toml

[dark anchor]

also pyyaml as a codebase makes some questionable decisions

[kindred hound]

https://github.com/samuelcolvin/rtoml

[dark anchor]

while testing pyyaml on free-threading, lysandros found that if you define a subclass of some of the public API meant for that, it mutates state in the base class, so the whole API is fundamentally thread-unsafe

[kindred hound]

Actually, looks like tomli started shipping mypc-compiled wheels too https://github.com/hukkin/tomli#mypyc-generated-wheel

[floral oar]

If you want to load YAML purely as data, and don't care about the source text as such, I think the rust wrapper makes sense. My trouble is that my use cases are all "auto-fix this github workflow" and to do that, I want to read the data and do stuff like "get the line number where this string constant starts". And then I can do fun regex stuff.

As far as I know, ruamel.yaml is the only parser which lets you get information about the source text like that

[dark anchor]

https://github.com/yaml/pyyaml/pull/852 for context about that

[marsh kite]

I think I could add support for this to ryaml

[dreamy hatch]

... or add a Rust-based YAML library to the stdlib?

[floral oar]

🤫

[marsh kite]

In due time Hugo!

[floral oar]

Would that require that we have a Rust implementation owned by CPython? I had been under the impression that the Rust-for-CPython proposal was one in which we'd have no crate dependencies. But also the DPO thread got... long.

[marsh kite]

Crate dependencies are almost certainly going to be a thing but I think like vendored C dependencies we need to be judicious

[floral oar]

And serde-yaml is dead / stable / 🤷‍♂️ , right? I feel like it wouldn't make the cut.

[marsh kite]

yeah it wouldn't. I think the current best path for yaml landing in CPython is:

1. make libyaml-safer Yaml 1.2 compatible (I feel much more comfortable doing this type of refactor in Rust rather than C)
2. Move ryaml to libyaml-safer
3. Implement spans and pyyaml-like features in ryaml
4. Get people to move to ryaml
5. Propose for inclusion in the stdlib

[kindred hound]

There's also https://github.com/saphyr-rs/saphyr which is YAML 1.2

[long knoll]

(it'd be nice if the last step could be parallelized a bit...)

[marsh kite]

True! I've actually evaluated it. I think it is a bit too unstable for me to want to use as a dependency at the moment, but another possibility is adding Yaml 1.1 support to saphyr, that would likely be much more pleasant to work with

[marsh kite]

proposing for inclusion to the stdlib?

[long knoll]

yeah

like, start the pep process before the candidate is ready (but after the choice is reasonably final I guess)

[marsh kite]

This is definitely something where people will want the library to be widely adopted before adding to the stdlib

So I think the best place for parallelization would probably be step 4

[dark anchor]

one request for ryaml: cp314t wheels?

i’ll open an issue…

[marsh kite]

Yes, I have a number of deferred TODOs to take care of to revamp the project. I should really get to those...

I can probably make cp314t wheels easily enough though

[marsh kite]

I'm publishing v0.5.0 which will have cp314t wheels now :)

FWIW I would be careful to recommend ryaml right now. I think changing the parser could cause breakages, so I want to be careful about getting a lot of people to adopt it right now

[dark anchor]

ok I won’t do that then 🙂

i guess there’s yaml-rs as well

[marsh kite]

yep

[marsh kite]

hm, the CIBW example I built off of tries to upload pypodide wheels to pypi, I'll need to fix that :/

[silk jungle]

The only positive about Outlook, it correctly identifies GitHub marketing spam

[long knoll]

lol
My fun story here: sometimes I report YouTube ads as scams, and then they'll automatically send me a confirmation that they received the report.
Which is promptly rejected as spam.
By GMail.

[fierce horizon]

The ruamel.yaml situation is solved: https://github.com/robotpy/semiwrap/issues/310#issuecomment-3705927128

[dark anchor]

one teeny correction: cp314t wheels are needed still, hopefully there’s a way to build stable ABI wheels for the free-threaded build in 3.15

[onyx spindle]

noob question: does using abi3 limit what can be done/used on the C/rust/whatever side?

for example: I have a rust ext with pyo3, do I have to change my code to be able to produce abi3 wheels?

[dark anchor]

depends on what you’re doing; there’s definitely quite a bit of conditional compilation in pyo3 to support the limited API

some features aren’t available but most are

extensions built using the limited API might also be a bit slower, there are certain performance optimizations that aren’t possible because they rely on low-level details

grep the PyO3 source code for Py_LIMITED_API to see what I mean

[silk jungle]

Has anyone noticed absolutely absymal network performance on the Windows/macOS GHA runners? pip's functional tests that involve spinning up a local HTTP server are taking >35s when they should at most take 10s

It's also strangely consistent at 35s which makes me suspect that there's some bad interaction somewhere in the network/server stack. The Linux runners are unaffected.

Also apparently the Windows-2025 images are removing the D: drive which is going to be bad for our Windows CI times.

[hexed kindle]

Again? They did it during the beta but had to revert due to breakage and slownes

[silk jungle]

¯_(ツ)_/¯

All I read that is they did remove it once. I guess they brought it back?

[hexed kindle]

Yeah, it was brought back

[normal ibex]

tomli now ships mypyc accelerated wheels

[onyx spindle]

but I don't think it is accelerated if it comes from stdlib, right?

[normal ibex]

yes, that is tomllib not tomli

[long knoll]

(hmm, I should play with mypyc some time. Would give me a reason to care about the type annotations I usually don't bother with, anyway.)

[mighty flower]

Randomly speculating, could be security tools on the runners

[lyric quiver]

https://github.com/meta-pytorch/torchcodec/issues/1149

Kings I require your expertise

On a scale of 1-10 how stupid is my proposal

[quartz yew]

not stupid

we do that for several native dependencies already

in the past it was prone to be problematic, but not much anymore

[lyric quiver]

I don't really understand how it would be

" I think that'd be OK in terms of licensing. But what you're describing looks a lot like a full-fledge package manager, "

Running a subprocess and downloading a .zip file isn't on par with what choco, winget or you name it does.

it seems like a bit of an exaggeration

I'm a bit frustrated cuz I spent 1-2h figuring out why the library wasn't working yesterday despite meeting the requirements ( except installing anything through Conda, I refuse to have that installed on my system )

[quartz yew]

you can package it yourself, and add the libs to the linker search tree via https://github.com/pypackaging-native/dynamic-library

which reminds me I have a lot of documentation work to do there

[lyric quiver]

will check it out

Gonna see if him or the team are actually interested in a migration, otherwise I am forking the repo and just on every official release generate my own .whl

[silk jungle]

And I managed to injury my middle finger while playing badminton so now I can barely type 🙃

Fortunately, I don't have to type too much, although no OSS tonight :P

[cursive gale]

hope it gets better soon!
I'm curious how you injured it though

[lyric quiver]

I have made it work relatively easily with a simple .zip shared file download from gyan.dev and unpacking somewhere + add it to env

took me maybe 1h tops

it's pretty slow so I will keep coping with my own library for now but maybe one day

[silk jungle]

Not sure either. I probably whipped the racket in a weird way that left a cut/flap of skin. Fortunately, it didn't bleed.

[timber sphinx]

Only 10 hours left https://hachyderm.io/@itworldcup/115960578331633434

[lapis solstice]

Final tally: 50/50, with Python winning by 3 (915 to 912)

[fierce horizon]

I got fed up that creating reproducers when reporting Sphinx bugs required creating TWO WHOLE FILES, so I made a playground: https://flying-sheep.github.io/doc-play/

[dark anchor]

I wish discussions on discourse didn’t feel so contentious ☹️

[floral oar]

Agreed. It's a real downer. I try to only contribute positive vibes, and some threads make that really tough.

[timber sphinx]

not for nothing, anytime I see a thread with triple digit responses, my brain simply says "probably not worth the time needed"

[mighty flower]

I think it's a somewhat inevitable outcome of structure of proposals being rejected or accepted at the end, authors are very invested, most proposals will have some detractors, conversations then start to focus on the most contentious parts as that's motivates people to reply.

I often wonder if a two-phase structure would work better, at lease for bigger proposals, that first phase accepting or reject the idea in principle, with only an outline, and if consensus is agreed it is accepted "in principle", then the second phase can proceed with the assumption it will eventually be accepted, once all contentious points are resolved or voted on.

Perhaps in the furture this is something I will submit to a packaging council (or run on that idea).

[dark anchor]

I think having a PC at all will help a lot with a lot of this

[timber sphinx]

I think that's the hope! Some days I wish for "threading" in a thread, similar to old BBforum or SMF - so if there's a tangent about "you misquoted me! no, you misquoted me!" we could collapse all of that away if we so desired

[onyx spindle]

or discussion going of course of the original topic and you end up reading through people arguing X vs Y while the thread was about Z

[long knoll]

this is fundamentally why stack overflow exists, but everyone hates being expected to make sure up front that the question enables keeping on track and doesn't enable extended discussion

[marsh kite]

Yeah this is one of the areas I highlighted as an issue when discussing problems with the PEP process with the steering council last year. I am excited to see what they come up with to improve things.

[clear wigeon]

new fear unlocked

[fierce horizon]

Were there any proposals where making that call early would have changed the outcome? I guess when Guido was BDFL, he could be swayed, but today it’s harder to sway a whole group of people.

[fierce horizon]

For sure. I don’t like Discord because it’s not searchable on the open web, but having that one level of “thread spin-off” feature together with moderation tools that allows to move stuff into there would be perfect. Would also dissuade trolls as they can’t derail things as easily anymore. They’ll just be shunted into their little containment threads and debate with other trolls, which is less satisfying for them.

[foggy elk]

one of the big problems is that it's just one big thread, so the attention an aspect gets is determined by the volume of responses

this is different for github review or zulip, where you can start separate threads and track them separately

it's also a big problem that this makes replies go lost: if someone posts an important concern or a specific solution and then there's 20 comments repeating basically the same things that were said before, that important comment is completely lost

it also means that people (who may not even be stakeholders or understand the pre-PEP state) can effectively hijack a thread by insisting on something unattainable or irrelevant, burning out everyone else in the process

it also makes it hard to say what the outcome of a thread is: we have no way of tracking individual concerns and alternative proposals, and whether/how they were addressed

i'm trying to invite other stakeholders to discussions, but if you have to 200 responses first to know if something you want to say has come up before that's a non-starter for many people

[dreamy hatch]

It's not easy. One person's pet debate in the lazy imports thread was extracted to its own thread by mods. I think that worked well, they could work through their idea that in the end went nowhere, and the main thread was all the better

I think we could do things like this more often. And especially large PEPs could have official sub-topics for different aspects. The challenge is predicting what those may be ahead of time. Or else ask a mod to extract certain messages.

[foggy elk]

for PEP 817 specifically, feel free to reach out on other channels if you have feedback to share, we collect all the feedback we get, DPO or not, and will integrate it into the PEP revisions

[frank shore]

I agree that Discourse makes it more difficult for the signal to rise above the noise, and it's way too easy to amplify the noise. So part of the problem is the tool, and I wish we could subthread more easily. From an SC perspective, we are trying to give early signal not just on the overall PEP but certain details, but that's also difficult because we're all volunteers and only sync once per week. That's also harder to do for packaging PEPs, at least until we have a Packaging Council.

That all said, this is on the SC's agenda to discuss and hopefully address. Feel free to reach out to me or us with any concrete proposals, or if you'd like to attend an SC Office Hours.

[fierce horizon]

Here are discourse people taking about it: https://meta.discourse.org/t/how-to-avoid-an-explosion-of-sub-topics-in-a-topic/279257

[silk jungle]

Yeah, honestly, I have almost zero desire to weigh in the PEP 817 discussions even though I've been invited to do so because the thread is so unwieldy already.

I honestly think that the current model of a core group works on a draft PEP -> presents it all at once to the entire packaging community for comments two-step process is not tenable. As others have stated, if there was a in-between stage where the certain core parts of the PEP could be debated and revised in public, without necessarily concerning PEP approval, that would go a long way in making the discussions more productive.

Also, IMO there is a problem of trust. Since the signal-to-noise is so bad, it's hard for the wide spectrum of concerns to be addressed, so there are going to be stakeholders that feel like they've been totally excluded, which then leads to resentment and/or spamming of a single concern. Furthermore, IMO, just dumping a PEP draft for comments and presumably approval is putting way too much on the community feedback process at once. I get the impression that certain individuals in the thread were surprised and taken aback by what's in the proposal.

It's too late for this now for PEP 817, but I'd like to see future PEP authors provide more regular updates to stakeholders before reaching out to literally everyone on DPO. That way, at least a large chunk of the discussion participants will be on the same page and we don't have to rehash the same "what is this PEP even about?" discussion endlessly 🙃

I guess I'm proposing that a "media package" of sorts is presented beforehand.

FWIW, I am not someone actively involved in the packaging discussions so my suggestions are with limited context and background, but as an outsider, the process does look and feel dysfunctional.

PEP authors provide more regular updates to stakeholders before reaching out to literally everyone on DPO
I realize that this is hard since even if you do provide this sort of information to stakeholders in advance, you can't make them read it, but IMO, as the PEP author you have the right to firmly ask people to read a summary before engaging in the discussion.

So basically, I agree with @mighty flower's points with my pip hat on.

[frank shore]

I believe 810 followed a similar trajectory, although I don't know the details and could be wrong. In the sense that a core group of folks spent a lot of time and effort in working out the specification, building reference implementations, and extensive testing on a wide range of use cases. Then presented it as -- not a fait accompli -- but in a well thought out and researched approach that was still subject to feedback and adjustment.

There are differences of course. 810 evolved from the previous rejected 690, so it had a starting point where I think there was predominantly positive feedback on the idea but not so much on the shape of it. Also, 810 falls within the domain of the SC to decide, whereas 817 falls to the packaging delegate, which is one person (PEP 772 Packaging Council still not approved).

But to be clear, 817 development took about a year, through many iterations, and completely in the open, although admittedly, not in the "normal" Python forums. 817 today looks a lot different than where the idea of variants started, and that was through much hard won experience, much experimentation, and actual implementation on real-world packages. It was also shaped in many respects by feedback from the packaging community. 817 also isn't a fait accompli and I think while the PEP authors are in their right to make a decision about this or that aspect, they are very open to feedback, which is what I see the current DPO thread trying to get at (successfully or not).

The problem is fundamentally design-by-committee, in a committee that is large with a huge diverse set of voices, some very loud, some inexperienced, some quiet, some extremely knowledgeable. I don't think that leads to better design honestly, just a mass of compromises that offends no one but also doesn't solve the problem.

I'm not saying the way 817 was done was perfect, but neither would be DPO-lead design with semi-engaged participants. How do you avoid overwhelming stop energy that just leads to everyone giving up?

[mighty flower]

Also, 810 falls within the domain of the SC to decide, whereas 817 falls to the packaging delegate, which is one person (PEP 772 Packaging Council still not approved).

Yeah, I'm increasingly agreeing that this is a major issue for the packaging community, the SC can weigh the pros and cons rather than waiting for 100% community consensus with no objection, as well as, if needs be, step into a discussion when it gets contentious and direct it with some level of authority

[silk jungle]

Sorry, I should've clarified. I 100% agree that the draft PEP process should be entirely up to the PEP authors, and if they deem it better to work on it independently outside of DPO, that's cool. It'd be unworkable to draft PEP 817 on the DPO. My problem is that the friction between draft PEP -> RFC period -> consensus -> approval is quite high.

The thing is that the PEP 817 thread seems to be operating on assumptions which evidently is going to drag out the thread. I don't mean to single out Paul, but he did say that he was providing feedback primarily on assumptions since he couldn't find the answers to his questions in the PEP. I have not read the PEP yet, and I do not intend to pass any judgement on the quality of the PEP. The problem it's trying to solve is large enough is that no PEP is ever truly going to be comprehensive and easy to read.

This problem (amongst the many others in the current packaging PEP process) seems somewhat fixable by providing summaries that are relevant to the stakeholders. I was going to suggest that the authors reach out to the stakeholders for early feedback, but TBH, I don't think you'll be able to get much feedback at this stage since most of us are volunteers.

For example, if there was a short(er), more focused summary on what I need to know as a pip committer, that would make it much easier for me to engage. Even if I read the PEP (which I'm planning to do at some point), it is easy to miss what's relevant in these massive PEPs for me as a pip committer. Bonus marks if there are leading questions where feedback is wanted.

TL;DR, yes the PEP process is broken, but I do think we can do better even before the PC is a thing.

Although honestly, it may just be better to wait for the PC since I can't think of an easy way to improve things that doesn't involve pushing even more work onto the PEP authors :(

[wind pewter]

I agree with that sentiment and the one that Paul has stated. The spec was unclear beyond a lot of this is what we expect your tool UX to be after this PEP is approved. Which left me thinking i dont even know how i implement this in hatchling. I know that there was a lot to boil down into that PEP but after reading it I feel like it should have started with a more spec and interoperability focused approach with a smaller section on what the UX could look like. I dont think this is on the authors, the previous thread on the topic had a lot of voices asking what the UX should look like. I do think the PC here would have helped a lot.

[frank shore]

I will say having served 6 terms on the SC, that having 4 other minds, filling in your gaps, challenging your biases, confirming your opinions, relentless seeking consensus, and really focused on what's best for Python and its community has been a wonderful model that's worked very well. I wouldn't say always gets it right because that's impossible, but almostly nearly so. And I cannot imagine how Guido ever did it all himself! Which yes, means I really don't know how Paul does it. Packaging is way way too complex (in someways even more challenging than stewarding Python-the-language) for one person, as good a job as Paul does.

[vast wren]

I think sometimes "going off into the wilderness" (even if that wilderness is a 30s bike ride away on GitHub) and riding back into town with TheSolution(tm) is sometimes the right answer. Sometimes you need to step back and ingest the feedback you've gotten or do some research or some experimentation or something.

I also think sometimes doing that makes it a lot harder for people to follow what's changed from one iteration to the next, forcing them to basically start over or be unable to track how a concern of theirs was resolved (or attempted to be resolved, maybe it wasn't actually resolved!). Which can lead people to feel overwhelmed.

I think it can be really hard "in the moment" to know if this time it's going to be beneficial or harmful (and probably the real answer is that it's always both, so ultimately you're trying to figure out how much benefit for how much harm).

I've half followed along on 817 while it's in the "going off into the wilderness" phase and half followed along while it's in the "ride back into town" phase. I think 817 has made improvements that probably wouldn't have happened without the authors going off to experiment and try things out. I also think it's pretty clear that it's at least contributed to folks (on both "sides") getting testy with each other.

Human communication is hard!

[lyric quiver]

Pep 817 really reminds me of my hardships with pip

I currently have 5 requirements.txt files, one for necessary dependencies, 2 for windows, named full ( pytorch cuda + tensorrt + onnxruntime ), one for AMD/Intel systems and 2 for Linux ( same as the one for windows )

I always wondered if there is a better solution than what I got right now but the innate limitations of being on the blazing edge is that some hardware say the gtx1000 series get deprecated and I need so much logic to wire those Nvidia users to get only the CPU wheels + DirectML and what not..

Maybe uv is onto something with their implementation but as a c++ to pybind "dev" myself, I start hating this enshitification of python where everything is basically a wrapper around lower level languages

I guess the primary fault is that python has became the defacto standard when it comes to AI all while never being suited for the task imho

[clear wigeon]

pyproject.toml with deps groups solves this in a neat way imo

[frank shore]

https://github.com/python/peps/pull/4801

[lapis solstice]

There is a process for that, which is to write an Informational PEP that defines the problem to be solved, and then a Standards Track PEP that proposes a concrete solution. I don't think it's explicitly documented anywhere though, it's more a matter of having seen it done before, and realising (potentially after the fact) that a given proposal falls into a category where it will be useful. I don't think it actually results in less overall work as a PEP author, but it can potentially help keeping discussions more on track (assuming participants actually read the informational PEP, which unfortunately isn't necessarily a warranted assumption).

[dreamy hatch]

PEP 1:

However, other parts of the Python community may also choose to use the process (particularly for Informational PEPs) to document expected API conventions and to manage complex design coordination problems that require collaboration across multiple projects.
...
An Informational PEP describes a Python design issue, or provides general guidelines or information to the Python community, but does not propose a new feature. Informational PEPs do not necessarily represent a Python community consensus or recommendation, so users and implementers are free to ignore Informational PEPs or follow their advice.

[lapis solstice]

Packaging is way way too complex (in someways even more challenging than stewarding Python-the-language) for one person, as good a job as Paul does.

It was slightly easier several years ago (both before Paul took over and during the earlier part of his stint), as the standards were still new enough that there were a known handful of people & projects that could function as a de facto council (not officially, but in the sense of "if these folks are all in favour, it's potentially a decent idea, and unlikely to be a disastrous one"). Setting up the interoperability standards worked, though, so now the tooling landscape is far more diverse than it used to be, and that's on top of ongoing spectacular growth in the popularity of Python itself (and its expansion into ever more fields of endeavour).

[vast wren]

Setting up the interoperability standards worked, though, so now the tooling landscape is far more diverse than it used to be

Alas, we've been hoist upon our own petard

[vast wren]

I always feel weird because DPO PEP discussions have never really bothered me (even though I've mostly been doing other things for awhile now ). I'm not sure if that means I'm making them bad or if starting in ⁨distutils-sig⁩ broke my brain .

[floral oar]

I suspect that you aren't engaging in the bad behavior which makes DPO feel bad (for me, at least): trying to read every message. I've stopped doing that unless I really care about a topic.

The PEP threads, in particular, can move at a mile a minute. When it moves fast so that it's hard to keep up, and it gets heated, it becomes hard to engage unless you tune out the thread and address comments back to the document itself. I assume the mailing lists had some similar dynamics but I suspect not quite as much volume.

[vast wren]

Non jokingly I think part of it is just that it still feels to me like there is a way to make decisions, and a very frustrating part of distutils-sig in the beginning of my involvement was that discussion would happen but pre PEP process any single person could basically veto a change, and someone always did. So it still feels hopeful to me lol

I think my brain also just likes it tbh. Early on there were some spirited distutils-sig threads where I was, on my own, half of the posts… and those emails were not any shorter 😅 I got some feedback at one point back then that it felt exhausting to argue with me, because I’d fill a thread without really trying.

So since then I’ve tried (with varying levels of success ) to slow myself down lol

(We will not be auditing what % of the latest 817 thread is me)

[foggy elk]

you're not the only one expressing this sentiment, but i admit i was surprised that seemingly didn't reach people: jonathan reached out to a lot of people (that's how i ended up working on this), we did a first DPO thread (where many things of the current PEP are already in there), we wrote blog posts, we have the #wheelnext channel, all the text changes and issue tracking happened on github, and we did get input from a lot of sides, but somehow others weren't reach

[foggy elk]

there's a lot of valuable feedback, but the demand that we should have developed this on DPO (not from you) is the only one i reject, it's functionally impossible to develop anything of significant size on DPO

people got burnt out on DPO on way smaller proposals

[foggy elk]

the part that's unclear, is it how to expose this to users or which changes to make where to the wheel writer?

[lapis solstice]

This "information firehose" problem is actually one of the things I consider a paradox of open source sustainability: for complex problems with a lot of people actively working on them, the key challenge with being actively involved is just keeping up, rather than not having access in the first place. The result is folks participating on their own time may still feel excluded, when the PEP update is the information summarisation and consolidation step for the folks that don't have the time or inclination to track every step that was taken to get there. (And I agree, basically no complex PEP is developed solely on discuss.python.org, especially ones with multiple authors that need a venue to establish their own consensus that they're willing to advocate for).

Separating out genuinely useful feedback from "Why wasn't I consulted?" complaints that can be reasonably ignored is both time consuming and harder than we might wish, though 🙁

[wind pewter]

What the expected metadata is that needs to be in format for wheel writing, some of this might have been a bit of reading fatigue by the time I got to that section.

[foggy elk]

i see, dedicated sections what changes for different audiences is something we need to work on

[floral oar]

This "information firehose" problem is

[wind pewter]

I think that would be super helpful! I had to sort of dig through information that was not as relevant to build-backends to get to the meat of what is it that hatchling needs to potentially change at some point if at all.

[silk jungle]

Sorry, I said a lot, but it was exactly this that was my main suggestion. I'm glad you got that.

If I had more time, I'd write a shorter letter is definitely applying to this situation 😅

[coral rivet]

Hello, everyone. @foggy elk has been passing on some feedback on PEP 817 from here.

I've started working on a PEP update, and I'd like to hear your feedback, in particular if you feel that it addresses your points. The commits are at https://github.com/wheelnext/peps/pull/34 but please note that they're not "official" yet, as in approved by other co-authors. However, I think that it'd be helpful to know if I'm walking in the right direction.

Particularly, I was trying to make things clearer for different audiences. However, I don't really want to split the "Specification" section since there's going to be quite some overlap. For example, the rules on how labels are named apply to multiple groups, and I don't think it's a good idea to have them repeated multiple times. Instead, I've focused on providing outlines at the end of rationale section, that roughly list what steps particular tools take.

[wind pewter]

I took a read through it and I think this is walking in the right direction. I really appreciate the outline of tool behavior! That makes things much more clear about the steps of the behavior. I would agree that the Specification would not make sense to split out, as long as we have that outline of the behavior that helps us then understand the Specification section in terms of which pieces apply to the particular behaviors. Also I want to say thank you to the authors, you have all been very engaging in the feedback process!

[vast wren]

I wonder what dpo thread has the most replies

And can we get a high score

[silk jungle]

https://tenor.com/view/some-men-just-want-watch-the-world-burn-dark-knight-batman-christian-bale-gif-6083967

there should be a separate high score for longest thread with a productive outcome (e.g., PEP acceptance)

[vast wren]

To make it fair, another high score for PEP acceptance with golf rules

[dark anchor]

https://discuss.python.org/search?q=min_posts%3A400

only 4 have gone over 400

[vast wren]

Neat

Only 140 to go

[foggy elk]

according to my script, the PEP with the shortest discussion is PEP 687 in https://discuss.python.org/t/pep-687-isolating-modules-in-the-standard-library/14824, with a total 4 posts

[lyric quiver]

I had a goal

Type shit

[lyric quiver]

Very interesting

I will take a look

[vast wren]

I love that you actually went and found this. A+

[lapis solstice]

Now I'm mildly curious as to the shortest creation-to-acceptance times. That's around 2 months (but was retroactively PEPifying already in progress work, so it was more backfilling missing paperwork than a genuinely new proposal). Matrix multiplication was around 3 months (and was a genuinely new proposal, albeit a meticulously researched one).

[dreamy hatch]

https://peps.python.org/pep-0417/ "Including mock in the Standard Library" was created and accepted the same day. It had already been discussed and agreed at the Language Summit.

[dark anchor]

I just managed to lose a talk outline in pretalx

twice

grumble grumble

[long knoll]

https://pypistats.org/packages/six

still trending up 🙁 but not as much as overall PyPI volume

(I think?)

[mighty flower]

It's so hard to know what download volume on PyPI actually means, like maybe it's LLM scrapers or misconfigured CIs

[vast wren]

I know there’s been at least one case of someone just downloading their thing in a loop to raise their stats too lol (not that six would do that)

[marsh kite]

Or accidentally deploying a bad image that tries to install on start such that you try and fail to install infinitely: #pypi message

[clear wigeon]

has happened to me a few times

[fierce horizon]

If you care: https://github.com/dateutil/dateutil/issues/653#issuecomment-1396351075

By the way, https://github.com/dateutil/dateutil/pull/1130 is really not that hard for a motivated individual to finish, I just have approximately 0 time for any of this stuff. I've said there's no problem if someone wants to take it over.

[long knoll]

... It looks like someone else is interested already. and I don't immediately see how #1130 relates to dropping 2.x support

It is indeed, but we are waiting for #1130 to land first, so we can split tzdata from this package and allow users to continue taking new tzdata releases without having to reinstall dateutil.

ah

there's a fair bit of discussion to read here but I think I get the picture now

[floral oar]

Okay, pip-tools v7.5.3 is out (hooray 🎉 ), and now I have a question for anyone who wants to weigh in.

We had a bug which failed to cleanup a tempfile, so if you ran while true; do echo 'six' | pip-compile -r -; done, you'd run out of disk space.
It was reported as a security bug (resource exhaustion, DoS), and I just... don't see that as a security bug? But I can sort of tilt my head and squint and see it as one.

What do the folks here think? I think we can retroactively assign a CVE and so forth, but this just feels like "an ordinary bug" to me.

[royal dirge]

Context https://sirosen.net/2026/02/11/pip-tools-753.html

[royal dirge]

Probably best to ask @valid rover

[silk jungle]

My gut tells me no.

[floral oar]

Yeah, I was debating asking Seth directly.
(Also, for anyone who wants it, the PR with the fix, which therefore demonstrates exactly where the bug was.)

[silk jungle]

If you run any tool that produces persistent logs a trillion times and that leads to resource exhaustion, then that's on you.

[floral oar]

Yep, that's my feeling as well. But getting a security report immediately makes me question my priors. Like, someone else looked at this and thought "security vuln!" But also if they're actively looking for security issues, they'll tend to see things through that lens... 😵‍💫

[valid rover]

Definitely not a security issue.

Thanks for the ping, I welcome them if you're not sure 🙂 Happy to be added to GHSAs or reports of PyPA projects to lend a hand in determination or "what to do next".

[floral oar]

I will remember to ask directly next time I'm unsure. I'm always too hesitant to ask people for help. It's a work in progress! 😁

[vast wren]

I could probably construct something resembling a justification, but in reality a lot of people just want CVEs under their belt or err on the side of assuming it’s security sensitive

[royal dirge]

And those who manage to get a CVE end up causing spam upstream because some people saw a "vulnerability" while in reality it may be utter nonsense...

[mighty flower]

The one bug in pip that I dealt with that I genuinely consider a serious security issue doesn't have a CVE assigned to it 🙃

I will leave it as an exercise to the reader to figure out which one. Hint: ||It was referenced in the security implications in a recently accepted PEP||

[royal dirge]

That's understandable if you yourself can justify that. My flashback was https://nvd.nist.gov/vuln/detail/CVE-2022-33124 / https://github.com/aio-libs/aiohttp/issues/6772 that was being picked up by scanners and driving humans to us, despite neither the CVE, nor the issue having any details whatsoever. We ended up having to beg MITRE and a bunch of other registries to stop showing it as something valid with reaction time of over a year. And the CVE still references some mysterious error in a mysterious context claiming that people dispute said context while in reality the context itself does not exist in the first place and so those claims are based on nothing...

[valid rover]

If you'd like we can publish a CVE retrospectively, it is not much of a lift. Just a small description and the commit that fixes the issue. I can handle the rest.

[royal dirge]

Sounds like a good topic for a note in your blog — some people might not know that this practice exists (and is a good idea often) + the implications.

[valid rover]

Agreed, I’ll make a note to write about this when I’m back from vacation!

[mighty flower]

Accessing an @property method seems to have significantly sped up in Python 3.12, but I'm not sure which of the listed performance benefits caused that

[marsh kite]

Probably https://github.com/python/cpython/pull/93912

[onyx spindle]

I keep forgetting how much I hate reading C code (one with tons of macros that is)

[marsh kite]

Well maybe some day it can be Rust :)

But yeah, I don't love heavily macro'd C code either

[onyx spindle]

I was actually meaning to ask, how is Rustifying the CPython different from RustPython (code-wise, not maintainership/management wise)

[mighty flower]

It's noticeable, testing performance of avoiding property access (https://github.com/pypa/packaging/pull/1083) has a drop in effectiveness on Python 3.12+ , to where not all scenarios I'm benchmarking are statistically significant any more

[onyx spindle]

Also, did anyone ever looked into switching the build system of Cpython from autotools to something else (like Zig for example)?

[marsh kite]

Rustifying CPython will almost certainly keep the core interpreter in C for a long time. The Rust code in CPython will have to work with the existing model of thread states, reference counting and garbage collection, etc. RustPython has implemented a few things in their own way which makes total sense, they don't need to be C API/ABI compatible

[marsh kite]

This is awesome!

[onyx spindle]

thanks. Been using Python for most of my programming career, going into compiled languages now and it's so confusing...

[marsh kite]

Yes, in fact rewriting the build system was my next project instead of Rust. But both that and introducing Rust are going to take 5+ years, and I didn't want to wait to introduce Rust

And I certainly don't have bandwidth to do both!

[marsh kite]

Totally reasonable, I remember struggling to wrap my head around C when I first learned it

[onyx spindle]

Interesting. Would love to help, but I have like 0 knowledge in that stuff 🤣

[onyx spindle]

C itself is not the problem for me, it's how the macros make it less readable and all the stuff around compilers, optimization, debugging, build systems etc

[marsh kite]

Well, there's a lot more to a project like this than just the code going into CPython. I was chatting with someone who didnt know Rust well about working on a Rust for CPython website

[marsh kite]

Ah yeah build systems are really complicated

So many corners

One of the things I want to do before I rewrite the CPython build system is document how it works thoroughly

[onyx spindle]

That is why I like Rust (despite being still an noob). I can just cargo build and it works (or rather, doesn't most of the time, but hey, progress)

[dreamy hatch]

see also https://discuss.python.org/t/what-do-you-want-to-see-in-tomorrow-s-cpython-build-system/28197

[dark anchor]

it’s hard because of bootstrapping 🙁

As a big meson user these days I’m a fan though. It’s so easy to get up and running for most things I do. And my brain fits with the meson language a lot better than cmake (let alone autotools: m4 lmao)

[marsh kite]

Unfortunately I think meson may not be the best choice. Whatever replaces the current build system pretty much needs to be ubuquitous and I don't think meson has quite that reach

[long knoll]

RustPython just gets to use Cargo I suppose...

[marsh kite]

I'm hoping we can also use cargo, at least for the Rust bits

but that may or may not be possible

[dreamy hatch]

🧊 only two more alphas until feature freeze 🧊

[marsh kite]

Yeah Rust unfortunately won't be making it into 3.15 I don't think

[vast wren]

I hope so too!

Being able to use rust makes me a lot more likely to contribute speed up thjngs at least

[marsh kite]

That's good!

[floral oar]

Yeah Rust unfortunately won't be making

[coral rivet]

We've just posted PEP 825: Wheel Variants: Package Format as first part split from PEP 817; and opened a DPO thread. I'm grateful for your past feedback on this, and hope the new PEP is clearer and doesn't turn into another 300-post DPO thread :-).

[kind moon]

@steel crane is there a reason that an "invalid" Dependabot package-ecosystem is a fatal error for Zizmor?
it just dies every time a new ecosystem comes out (not that that happens all that often)
For example, today's is https://github.com/zizmorcore/zizmor/issues/1636

heh

Same braincell

[steel crane]

no good reason 🙂

i'd be fine with changing it, the only downside is that the schema error message becomes slightly less nice. but that's honestly minor since it's only shown on the error path, zizmor itself doesn't use the JSON schema at all

[kind moon]

Is that the right PR diff?

Is it pulling the value from the doc comment? I didn't know Rust could do that

[steel crane]

yep, that looks right to me

it's getting the value from this: https://github.com/zizmorcore/zizmor/blob/8434a1069aa64f68ac2e45f36bb33c5c0a3c53d7/crates/github-actions-models/src/dependabot/v2.rs#L350

(the doc is just for reference on my part)

so PreCommit should get ser/de'd as pre-commit

[kind moon]

oh right

I see I see

[steel crane]

@kind moon thanks! if you can add a changelog line for it too that'd be great, otherwise i'll get to it tonight

[kind moon]

The copy/paste demon hath struck again

[silk jungle]

Did GitHub remove the feature where sorting issues by reactions would actually show the reaction count in the issue list view?

If I'm not crazy, the GitHub issue view list used to show that information but now it doesn't.

hmm, apparently github never implemented this feature....?? maybe it came from Refined GitHub but I also can't find a trace of this behaviour in the RGH repository

OK, I feel like I'm going crazy. It definitely used to show up for me, but I have no idea from where.

[marsh kite]

It could be that it was a feature test they didn't do?

[silk jungle]

like A/B testing?

I had the reactions count pretty consistently for a long time until recently

[marsh kite]

hmm I do vaguely feel like I saw this too

[silk jungle]

somehow searching the internet returns no trace of this feature either

[marsh kite]

okay no I agree I think this was a thing

[strong blade]

I find it only shows up if I refresh the page after sorting. That's how it is on mobile, at least. Don't know if you already tried that, but pointing it out just in case.

... until just now, when I tried it and it doesn't work at all. Never mind, I guess

They can't have removed it that long ago.

[silk jungle]

This is when I wished I had a GitHub contact, but alas, I don't

[royal dirge]

The UI was indeed implemented by Refined GitHub. Not sure if the official API has that sorting or this is just a full scan on the client side...

[kind moon]

Do you have a link to the Refined GitHub feature?
I scrolled through their feature list and didn’t see it either

[royal dirge]

No, but I was sure I remembered something

[kind moon]

Not sure if the official API has that sorting or this is just a full scan on the client side...

[royal dirge]

Maybe I was thinking of https://github.com/refined-github/refined-github/issues/2190

[kind moon]

I have gotten nerd sniped

You can sort by reactions

But you can't do that on the REST API

Apparently it's a custom GraphQL thing

[royal dirge]

And this is on Chrome for Android which doesn't support extensions, meaning GH has some built-in sorting support.

[kind moon]

Yeah see above, the sorting is built in but they use a custom query on their end

[royal dirge]

https://docs.github.com/en/issues/tracking-your-work-with-issues/using-issues/filtering-and-searching-issues-and-pull-requests#sorting-issues-and-pull-requests

https://docs.github.com/en/search-github/searching-on-github/searching-issues-and-pull-requests#search-by-number-of-reactions

[timber sphinx]

That's super weird, since some projects show them and others don't! https://github.com/pypa/packaging.python.org/pulls?q=is%3Apr+is%3Aopen+sort%3Areactions-%2B1-desc shows them

oh wait, that's pull requests, not issues - super odd!

[kind moon]

Can you screenshot the difference in what you see?

[timber sphinx]

here's incognito windows (isolating from refined github and others) showing that packaging.pytohn.org has a different issues experience than pip

[vast wren]

that's PRs vs issues, is that the difference?

[silk jungle]

IIRC the issues and pull search lists are different. They're starting to roll out the new issues UI for pulls too, but I'm not sure if that's finished

[timber sphinx]

dammit I did it agaion

I was excited to see reactions

[silk jungle]

time to create a script that uses the GitHub graphQL API, I guess

sigh

[lyric quiver]

GitHub and API is not a good mix

[mighty flower]

Seeing a lot of this when I look at recent contributors

Sigh

[kind moon]

Is that AI spam?

The sheer amount of PRs those things manage to turn out impresses me.
I couldn’t reasonably find that many repos to work on.
Though I guess that’s exactly the problem, they can’t either.

[mighty flower]

Yeah, it seems to be people setting bots to find any and all possible contributions they can make in the hope some will be accepted

[long knoll]

people got the idea that anything can be automated now

including the search for repos and issues to work on

[onyx spindle]

Yeah, I hand out 30 day bans for contributors like that

[mighty flower]

It's not that fly by PRs can't be helpful, but they need to be correct, small, simple, obvious, and complete, and so far none of them are.

[clear wigeon]

ouch, my timeline probably looks something like that due to the nature of my work

[mighty flower]

That's usually from a few days

[clear wigeon]

ok not too bad

that's this year alone

[mighty flower]

That screenshot was from someone who started posting PRs less than two weeks ago

[clear wigeon]

oof, ouch

i just reloaded the page and now it says this

oh well, at least most of them got merged

[floral oar]

I came across one recently with a "CV" repo with a bunch of links to hallucinated issues "solved" by that person. It was... Not good. I think that if your activity is genuine, it shows pretty quickly in a cursory check. I wouldn't worry about it unless people start asking "are you a bot?" a lot.

[silk jungle]

What's crazy is that so many people legitimately think they're doing a service to the world by contributing LLM-generated PRs to open-source projects, often with little care or oversight.

Back in time (old days, 🙃) you get PRs submitted by inexperienced students, likely contributing as part of an assignment, but those people were usually nice and easy enough to reason with. Perhaps their PR wasn't of much value, but you could probably turn it into something worth merging without too much back and forth.

Now, with LLM-generated PRs, the whole review-iterate cycle is so much more annoying and frustrating, especially when they let the LLM write their replies and you get walls of text.

</rant>.

[mighty flower]

The Airflow project is having to write a copilot set of instructions to review PRs that are obviously LLM generated

https://github.com/apache/airflow/pull/62442

[silk jungle]

Talk about not being in the end-times

[long knoll]

"fabricated diffs" ??? how, even?

[silk jungle]

I wouldn't mind if a PyPA GH admin banned haosenwang1018 on github for sending duplicate automated PRs to "fix bare excepts":

• https://github.com/pypa/pip/pull/13824
• https://github.com/pypa/pip/pull/13822
• https://github.com/psf/black/pull/5008
• https://github.com/psf/black/pull/5007

[mighty flower]

I've reported them as spam

[kind moon]

What does this mean?
https://github.com/pypi/warehouse/issues?q=state%3Aclosed label%3Aclaude-code-assisted

I know what it means in the literal sense, but I normally see it in the PR description, not as a label
And this being is added after the PR is merged -- if its going to be disclosed, shouldn't it be being disclosed up front?

[timber sphinx]

I am most of these, I think? I don't know how I feel yet on the whole "disclosure" front. I don't need to disclose if I used emacs or vim, do I? I often use Claude during my coding to either call out things I missed, but rarely do I ever ask Claude to produce something completely that I ask it to commit it for me.

So maybe it's a "Claude was consulted for this PR" label

[kind moon]

Well vim doesn’t write code for you
But I agree on principle, given that no one is disclosing that GitHub Copilot autocompleted some random function bodies.

I am curious about how the label got added though
How Claude knew about this PR

[timber sphinx]

I have noticed that Claude Code does seem to understand when I've opened a PR for a specific branch and adds it to the bottom toolbar.

[kind moon]

I’ll have to try all these new fangled apps one of these days
I haven’t used anything outside of GitHub Copilot’s interface yet

[wind pewter]

https://claude.com/contact-sales/claude-for-oss I just applied for this with Ofek

[dreamy hatch]

it also shows this if you've checked out someone else's PR, for example via gh co 123

[dark anchor]

let us know if you hear back, I also applied and haven’t heard anything

[mighty flower]

Be aware, as this is currently worded, this is nothing more than a glorified six month free trial. I previously got directly emailed by Anthropic with a shorter glorified free trial and I was personally not interested in that.

That said I've put my name down for this one and their recent security tool that they announced as free for open source maintainers and also received no response

[dark anchor]

oh huh I also got an email about that last august

suffice it to say: the hype and fomo around claude code has certainly gone up quite a bit since then - i'm more interested now

[mighty flower]

The security tool https://www.anthropic.com/news/claude-code-security

[long knoll]

https://github.com/dateutil/dateutil/pull/1130

👀 we might soon see six finally dropped from python-dateutil

there's been a whole bunch of recent activity seemingly out of nowhere

[onyx spindle]

wow, maybe I won't have to vendor and maintain my own version after all

[long knoll]

(I was thinking about doing it too...)

[long knoll]

https://github.com/chardet/chardet/issues/327 oh boy...

[onyx spindle]

Oh boy indeed

[lyric quiver]

https://github.com/NevermindNilas/TheAnimeScripter/issues/270

Try to explain to people that open source needs funding in order to get developed

And maintained

[mighty flower]

I thought most the ecosystem was now on charsert-normalizer, in part because apache airflow pushed requests to switch over due to licensing issues, and helped by the fact that in the moment charset-normalizer was better maintained and they got rid of dependencies and fixed things to accommodate requests and apache airflow

[dark anchor]

fun fact: charset_normalizer is the #1 most downloaded package on PyPI that ships compiled code in wheels (via mypyc)

[long knoll]

I thought it was like, almost entirely based on what requests has as a dependency. :/

[mighty flower]

These days chardet is only an optional dependency:
https://github.com/psf/requests/blob/main/pyproject.toml#L19
https://github.com/psf/requests/blob/main/pyproject.toml#L54

[long knoll]

the extra name still references py3, huh o_O

[onyx spindle]

changing extras is pretty breaking and no way to do it safely

[long knoll]

(you can at least add redundant ones I guess, but)

(sometimes it feels like every rock you turn over in the packaging ecosystem has more technical debt underneath it... to mix a few metaphors)

[floral oar]

I prefer that we call these things "fun little discoveries". 😉

Removing or changing extras is the sort of thing that I think people treat too seriously as a compatibility concern in many cases. Which is somewhat ironic since I think much of the community is a little too blasé about compat. But weirdly this case gets treated as a big deal, when usually it's just fine to remove/rewrite extras in a major release.

Not that I think requests should do a major just to clean up extras. But it has been a long time for that library without a breaking release, which is basically what you're seeing in the extras.

[mighty flower]

LLMs are such lying liars, playing around with this Claude Max 20x free trial Anthropic have handed out to open source maintainers:

Opus 4.6: I have carefully planned everything out, do you want to implement?
Me: Yes
Opus 4.6: There, all done all tests pass
Me: I ran tests and there are 4 failures
Opus 4.6: Let me check... that's an environment issue and unrelated to your changes
Me: Prove it by fixing the environment and having the tests pass
Opus 4.6: Oh, actually this does change the behaviour with respect to these tests

[dark anchor]

if claude's a liar in a situation like that then I am when I'm wrong about something

[mighty flower]

Well lying implies intent, and LLMs don't have intent, so it was a bit of hyperbole

But it neither checked tests actually all passed when it claimed it did, not actually validated the presented failing tests were just environment related when it said they were

The problem I presented is pretty complex, and I wouldn't have expected an LLM to be able to solve it without significant intervention from a dev who knows the codebase really well, and Claude expectantly was not up to the challenge

[dark anchor]

as a guy who often seems confidently incorrect about things, it makes me feel nervous when people call that “lying”. Mostly it’s me thinking out loud and sharing suppositions based on a hypothesis that I think is right but turns out to be wrong. Communication is hard.

[mighty flower]

I don't mind being hyperbolic about LLM failures, I owe them no empathy, humans are a very different story

[onyx spindle]

@mighty flower what was the verification process for getting Claude?

[mighty flower]

I have to give my email address, GitHub username, and an open source project I'm a maintainer of that met certain criteria (or an alternative free text reasoning why you think you should count)

[dark anchor]

I submitted that form but never heard back.

[mighty flower]

It took some time for me to get an invite, I think they're sending them out very gradually in waves

[mighty flower]

Just used claude code to search pip pull requests for things I'd promised to review but haven't, and arrange them as a checklist. Honestly there was almost no chance of me remembering to go back to most of these.

Not sure how complete the results were, but was happy to find at least some I'd forgotten about.

[long knoll]

Sounds like a great use of the tech to me. The consequences of hallucination are low, as you'd immediately notice that the PR isn't relevant

[kind moon]

Huh, I never would have thought about doing that

Neat

[mighty flower]

It seems to really like using gh and not the rest API or scraping the HTTP page, so I did need to set that up first

[kind moon]

Is there something wrong with the API?

[mighty flower]

No idea, I've used the rest API a bunch in the past, but claude seemed unhappy and kept trying to use gh

[floral oar]

Apparently GitHub rate limits authenticated calls more leniently, so if you're doing batch actions against the APIs, gh can behave better than unauthenticated curl/whatever

I wouldn't be surprised if that's why.

[long knoll]

...does the API not involve authentication?

[timber sphinx]

It does, but naive AI tools will try to visit webpages and scrape content, so Claude is primed to be better

I have an idea of setting up a triager/verifier for issues and PR s to see if they have already been solved somehow, and give me enough details to confirm and close things out.

[long knoll]

https://github.com/astral-sh/uv/pull/17643

https://news.ycombinator.com/item?id=47295014

mattip

PyPy core dev here. If anyone is interested in helping out, either financially or with coding, we can be reached various ways. See https://pypy.org/contact.html

Hacker News · Mar 8, 2026, 6:17 AM

[vast wren]

TFW a non zero number of people thought PyPI was shutting down

[fierce horizon]

the remaining core devs (me among them) don't have the capacity to keep up with cpython.

Interesting, I wonder what the whole mission was then. Why painstakingly reimplement something without caring much about staying compatible?

Speeding up legacy systems? Research into JIT building?

Not a criticism, just wondering.

[dark anchor]

I think there was hope that cpyext, pypy wheels, and conda-forge packages would cause more users to try it and push for improvements. In practice that never materialized.

[foggy elk]

i'd love to have more data about this, but my understanding was that the lack of C API support prevented people from migrating to pypy, while also (and if there's a cpython dev who knows this better please correct me) cpython's performance is constrained by the object model and GC API (refcounting) being exposed in the C API

[vast wren]

You could often use the same C API for PyPy but it was super slow if you did

Slower than CPython iirc

Also PyPy was always slowish at new python versions (which is reasonable ofc, at a mimimim they can’t release until CPython does so they’re always playing catch up) but it meant unless you really needed it you tended to want CPython anyways for some new feature

It was great during the 2/3 days when the language was effectively frozen for libraries though

[foggy elk]

do you know why it was so slow?

[vast wren]

I’m not an expert, but my understanding was a combination of:

• The C API was very CPython focused and exposed a lot of CPython intervals that PyPy didn’t have, so it required emulating them.
• What happened inside the C-API was opaque to PyPy, so the JIT couldn’t optimize across the C-API boundary.
• The actual design of the C-API and the internals it exposed were not well suited to optimization.

The second and third one is becoming problematic for CPython now that it’s trying to do JIT stuff, and is causing them to rework the c-api

cffi was made by the PyPy folks to solve this problem, but I’m not sure how much uptake cffi ever really had. I suspect to get maximum performance on CPython you needed to use the actual c-api but tbh I’m not really sure.

I’ve used cffi and it was a much nicer experience for binding some random C code into Python lol

[foggy elk]

at least from a rust/pyo3 perspective, i was missing the python object model on the native side in cffi

this part of its goal

Keep all the Python-related logic in Python so that you don’t need to write much C code (unlike CPython native C extensions).

[dark anchor]

cpyext is a very impressive bit of engineering, but just getting numpy running on pypy doesn’t make numpy performance-competitive on pypy

but also matti came on as a numpy maintainer as a result of his work on cpyext, so it benefits me greatly to that extent

[foggy elk]

we have support for pyo3, uniffi and cffi in maturin, but pyo3 was always the by far most popular choice

HPy sounded it could solve the problem of exposing the Python data model natively with exposing CPython internals, but i haven't heard anything about it in a while

https://github.com/hpyproject/hpy/issues/489 :/

[dark anchor]

I have a branch of PyO3 that can build extensions for 3.15 and newer with an opaque PyObject layout. This uses a bunch of incremental additions and evolutions in the C API. So I think a lot of hpy’s goals can be achieved by evolution.

[foggy elk]

that sounds great!

[coral rivet]

Does anyone happen to know how the PEPs repository got the pull request template dropdown working? Is there some secret file to enable it, setting or perhaps only available to select orgs?

[dreamy hatch]

it's some experimental/alpha thing that GH enabled for that repo: https://github.com/python/peps/pull/2956
also as you discovered, it's not that useful, because the option to use a template only shows up if you open a PR from the repo homepage, and more often than not it's usually missed...

[coral rivet]

Yeah, but still better than nothing :-). In conda-forge, we've introduced a combination of split templates with default template that forwards to them, but having a drop-down would be better for users: https://github.com/conda-forge/admin-requests/pull/1919

[quartz yew]

I wrote a bit about getpath in my devlog this week, in case anyone happens to be interested, https://ffy00.github.io/devlog/2026-W10/

[long knoll]

as a heads-up, the text is low contrast and hard to read in dark mode, especially section headings

I didn't know about getpath but this is interesting stuff. I had been thinking of doing some more general writing about the import system and this is a formerly-unknown-unknown I'll need to slot in somewhere, thanks

Would it be possible to point at where the code gets run from? Is it invoked directly from C?

[quartz yew]

thanks for the feedback! and apologies for the typos and missing words, my dyslexia makes it difficult to proof-read without waiting a long time.

I didn't mean for it to be a full write-up on getpath, but rather to just to write down my thoughts, so that I can refer to them later without having to re-read the code and try to remember all the exceptions and little caveats (like the warnings being skipped on sub interpreters)

similarly, for the /proc/self/exe case, I just wanted to write down my thoughts at the moment, as it is incredibly complex, and I'll surely forget part of it 😅

but yeah, regarding the question about getpath getting called in C, it gets called by the function that initializes the interpreter from the provided PyConfig

have a look at Modules/getpath.c for the private function that actually calls the Python code (Modules/getpath.py)

the rest of the initialization is in Python/pylifecycle.c, and the config specific bits in Python/initconfig.c IIRC

[long knoll]

ah, that sounds familiar now

I've been redoing my own blog CSS lately along with implementing some other stuff for the page templates (nothing pushed yet but I think it will look a ton better)

[quartz yew]

there's a PEP that introduced the new config API, might be a good read too

[long knoll]

741, it looks like?

[quartz yew]

I don't remember from the top of my head, but probably

[long knoll]

I'll keep it in mind

[quartz yew]

yeah, I got fed up with how annoying sphinx is to theme, and how (subjectively) bad the other static site generators were that I just wrote a script that renders rST to HTML, which I then patch for my specific style

that's why the interlinking functionality is pretty scarce

though it uses docutils underneath, which is also annoying to add roles and stuff

it's all global state 🙃

I might end up building something better structured at some point

[long knoll]

ah
(what's your experience of Bulma been like btw?)

it occurred to me that my project ideas list already included a markdown-alike language, a simple HTML templating system and a lightweight build-orchestration tool, so in the long run I might as well just package those things up into an SSG.

I've been using Nikola with a bunch of my own hacks; it helped me get started but it's ... just way too much, overall

[quartz yew]

I'm really not a frontend person, bulma seems okay enough for what I wanted

though it's basically a single maintainer project, so not great in terms of maintainability

if you find something similar, but better maintained, I'd recommend going for it instead

otherwise bulma is fine

[long knoll]

ah
I actually have been hand-writing my css but I hadn't heard of the tool and figured I'd get your thoughts on it

[quartz yew]

yeah, I don't enjoy that part, so I was looking for a CSS framework that was relatively clean looking without me having to do much customization, and that did not require JS

[fierce horizon]

CSS has changed so much, if I had the time to stay up-to-date, I feel like I wouldn’t need a framework anymore.
Maybe slap on a normalization stylesheet to make sure what I see is close to what others see, and just use plain CSS.

Last thing I learned is light-dark: https://developer.mozilla.org/en-US/docs/Web/CSS/Reference/Values/color_value/light-dark
so today for perfect theming, just do the meta tag, set :root { color-scheme: light dark }, and use --bg-color: light-dark(#fff, #000) to define your colors. No media queries, no framework-specific way to override the the theme (just set color-scheme: light to override it for a component), just one line of global CSS and one declaration per reused color.

[long knoll]

Yep. It's not quite considered "widely available" and I honestly don't find the corresponding media queries difficult or tedious; but it does look quite neat.

[lyric quiver]

Not gonna lie

I jumped on the UV bandwagon

because so many people / repositories have migrated to it

it's really fast.. like too damn fast

it's a big pain in the butt having to manage the uv.lock and everything inbetween, I also didn't use a pyproject.toml up until now

but the installation of pytorch and tensorrt that took multiple minutes is now done in more or less a minute.

is simply the leverage of Rust as its backend that significant that it can outperform pip by such a huge margin?

[silk jungle]

I mean, yes and no. Fundamentally, uv is architected with performance in mind.

It makes different design decisions to maximize performance, like heavy caching and doing the bare minimum to install packages, etc.

[dreamy hatch]

https://nesbitt.io/2025/12/26/how-uv-got-so-fast.html is a good overview

[lyric quiver]

interesting, will give it a read

I wonder if pip could leverage the new path that Python is taking with the no GIL in order for it to also gain some speed improvements.

[silk jungle]

The GIL is really not a concern, like at all.

[lyric quiver]

"""
No bytecode compilation by default. pip compiles .py files to .pyc during installation. uv skips this step, shaving time off every install. You can opt in if you want it.
"""

I didn't know this honestly

[lyric quiver]

""" Parallel downloads. pip downloads packages one at a time. uv downloads many at once. Any language can do this. """

Having proper concurrency, with threading module for example should help. I'd reckon

[silk jungle]

I'm still not particularly convinced that parallel downloads is a major bottleneck? For large download, your internet connection is ultimately the bottleneck. For many small downloads, yeah sure, concurrent downloads may help with the per-request latency, but HTTP 2/3 should bridge that gap by a decent amount. (* I'm aware that requests probably doesn't support either)

On high-latency connections. I concede that concurrent downloads would probably be quite helpful, but even then, it's actually the network requests from dependency resolution that are slow, not the package downloads (which happen at the end). It's harder to parallelize dependency resolution.

[lyric quiver]

I would also add another dumbed down idea, you could also spawn a thread that downloads and another that installs the already downloaded wheels.

[lyric quiver]

As you already mentioned, small packages would heavily benefit from parallelziation, since they are more I/O bound than Network bound.

[silk jungle]

I wasn't talking about local file I/O at all

[lyric quiver]

you're right, I misinterpreted

[silk jungle]

Honestly, we (the pip team) has had this conversation many times before.

The answer is generally we know how to make pip faster, but we don't have the resources to do so.

If you want something fast, use uv.

[lyric quiver]

maybe I unknowingly brought up a sensitive subject

my bad

[silk jungle]

It's fine. I get that for 99% of people, they don't know or care about the development of the tools they use. And you know what, they shouldn't have to care because if they do, the project probably isn't doing a good job at maintaining their software.

[lyric quiver]

To be honest, back in 2020 ( when I started coding ) up until late 2023 I took pip for granted as a part of python

[silk jungle]

And you should!

[lyric quiver]

I wasn't aware of the active development behind it.

iirc, I started digging into pip back when some wheel didn' want to install due to a subprocess error and there was a " this isn't pip's fault but... " or something along those lines

[boreal bramble]

Huh, I didn't know uv doesn't support upper bounds on python version, that's good to know.

Tbf, I could probably get a bit more helpful message, if I had a package that always fails on install and has a python_version > 3.x marker on it anyway

• avoids backtracking which means that older versions without upper bound won't get installed instead

(to be clear, this is for app distribution use case, not for libraries)

[long knoll]

I also have some notes: https://zahlman.github.io/posts/oxidation/

[long knoll]

I haven't been convinced either, but others have assured me it does matter. I think there was something about complex resolutions potentially involving "many small downloads" of metadata. But I haven't done any kind of detailed testing or benchmarking

[long knoll]

yeah, basically people end up getting sdists for things that they aren't set up to build locally, and then pip has to bear the bad news

[long knoll]

I think it's more that they deliberately ignore it

[boreal bramble]

That's a better way to phrase it, yeah

It's a design choice and feature requests to adhere to it have been explicitly rejected

[silk jungle]

I've put very little thought into this, but reading this and following the general discussion on uv vs pip, I just wonder if the end-goal should be just to have pip be obsoleted.

[long knoll]

:( I haven't gotten a lot done with PAPER either. I really do want to have the "set up from a wheel and/or cache" part done, at least, along with the API (for projects that want to do that kind of thing programmatically, including other tooling)

and also prototype some ideas that uv is still missing ;)

[silk jungle]

I dunno. It is only natural for things to be replaced. I used to be part of Black, but it's been largely replaced by ruff format these days.

[long knoll]

I don't like these all-in-one tools, like I say in the post, but it's hard to argue with this kind of popularity.

A post about uv last year rocketed up in popularity (currently topping a search for python) out of nowhere on HN

er I phrased that weirdly but you know

(I also don't really use linters, type checkers etc. at all)

(anyway I feel like Rust shouldn't become a de facto dependency of Python, at least not until it's mandatory in the interpreter anyway)

[mighty flower]

Both pip and uv's resolver are single threaded, but uv prefetches data ahead of resolving allowing it to take advantage of multi threaded download

uv can do this because it's collector and resolver are built around the idea of intervals rather than version iterators. I'm about to refactor packaging's internals to be built around intervals, but it might take me years to come up with an API that exposes that and rework pip's resolver to take advantage of it

[long knoll]

hmm, it sounds like there's more to those architectural improvements than I knew about...

[foggy elk]

the conclusions in this article are all wrong i'm afraid, this article is an enumeration of misinformation on uv and python packaging performance more broadly

[dreamy hatch]

ok, I won't share it further. would be great to see an accurate writeup!

[foggy elk]

i can't speak for pip, but for uv the roundtrip time for requests is noticeable

you can test this out with UV_CONCURRENT_DOWNLOADS=1, which in effect turns off concurrent downloads

$ hyperfine 'echo "transformers[torch]" | UV_CONCURRENT_DOWNLOADS=1 uv pip compile --no-cache -' 'echo "transformers[torch]" | uv pip compile --no-cache -'
Benchmark 1: echo "transformers[torch]" | UV_CONCURRENT_DOWNLOADS=1 uv pip compile --no-cache -
  Time (mean ± σ):     661.1 ms ±  34.5 ms    [User: 239.7 ms, System: 74.5 ms]
  Range (min … max):   616.0 ms … 727.0 ms    10 runs
 
Benchmark 2: echo "transformers[torch]" | uv pip compile --no-cache -
  Time (mean ± σ):     213.1 ms ±  19.8 ms    [User: 132.4 ms, System: 50.5 ms]
  Range (min … max):   188.4 ms … 248.7 ms    13 runs
 
Summary
  echo "transformers[torch]" | uv pip compile --no-cache - ran
    3.10 ± 0.33 times faster than echo "transformers[torch]" | UV_CONCURRENT_DOWNLOADS=1 uv pip compile --no-cache -

Benchmark 1: echo apache-airflow | UV_CONCURRENT_DOWNLOADS=1 uv pip compile --no-cache -
  Time (mean ± σ):      1.502 s ±  0.088 s    [User: 0.512 s, System: 0.157 s]
Benchmark 2: echo apache-airflow | uv pip compile --no-cache -
  Time (mean ± σ):     383.6 ms ±  28.6 ms    [User: 246.8 ms, System: 95.0 ms]
Summary
  echo apache-airflow | uv pip compile --no-cache - ran
    3.91 ± 0.37 times faster than echo apache-airflow | UV_CONCURRENT_DOWNLOADS=1 uv pip compile --no-cache -

an extreme example with pictures: https://github.com/astral-sh/uv/pull/2452

[foggy elk]

upper bounds and what to do about them is a painful problem: https://docs.astral.sh/uv/pip/compatibility/#requires-python-upper-bounds, https://docs.astral.sh/uv/reference/internals/resolver/#requires-python, https://github.com/astral-sh/uv/issues/4022, https://discuss.python.org/t/requires-python-upper-limits/12663

[mighty flower]

Oh yeah, didn't have time to read it yesterday, but the main issue is the author represents static metadata as coming from PEP 658, rather than PEP 427, 10 years earlier, PEP 658 was just about distributions, once you have wheels you no longer need to execute, and you can be clever about HTTP requests and extract the metadata quickly, and even if you don't that you could cache the metadata more efficiently than pip did.

Other things, pip.conf does not make pip slow, and at this point uv.toml has more complicated hierarchical structure, the main reason to not use pip.conf is not to couple to a different applicaiton, especially when we make different choices about options.

Requiring virtual environments also doesn't speed things up, there's no "category of permission checks and safety code" slowing pip down.

"Stricter spec enforcement" again this isn't a performance issue, and actually pip is now stricter than uv about certain things, such as specifier compliance.

Given that's the section which I have a good handle on I imagine the other sections are equally fraught.

[mighty flower]

Some people have done some benchmarks on making a multi-threaded version of pip and found no significant speed up, I don't know the quality of this benchmarking and this could be a result of the GIL so we may see different results in the future. Also pip doesn't prefetch metadata in batches, something we are a long way from being capable of doing.

[foggy elk]

it could be a lot of reasons! many optimizations in these systems are path dependent, they are only noticeable in profiles or benchmarks after you remove previous bottleneck(s)

[lyric quiver]

if it was done with the threading module, you will not see much of any benefits

you'd want multiprocessing for that I am pretty sure

[dark anchor]

https://labs.quansight.org/blog/free-threaded-python-halfway

I wrote a blog post commemorating the halfway point on free-threaded Python support. Thought it might be of interest here.

[fierce horizon]

Whoa, I'm doing performance stuff in numeric code using numpy, scipy, numba, … and I hadn't even considered that it might be far enough to try it out. Let's see.

[dark anchor]

numba support isn’t there yet, it’ll hopefully be in the next release

[long knoll]

I think I managed to point out some errors in this, but I may well have made some (and certainly omissions) of my own. An inside perspective would be highly valued (including on HN) I'm sure 🙂

[long knoll]

it could be that a pubgrub-like algorithm is also required to realize the benefit... ?

[mighty flower]

No, thinking on it it should be pretty dooable with pip's current resolver, but it would require introducing a new phase and decoupling some things

[boreal bramble]

legacy versions sure are something lol

[mighty flower]

In theory it's still selectable with ===F, but it's effectively unsupported and untested in pip

[vast wren]

https://gist.github.com/dstufft/c24bcacef202a3837600

That may not have been the final list, but that’s one of the list of versions I made when we were trying to determine what rules to add to pep440 to try and capture more versions without allowing nonsense

https://gist.github.com/dstufft/5c291d6ae03eaa67e96d

that’s the list of the projects that no longer had any pep440 versions available

I think some of those are wrong though. Inthink all the X.Y-N ones got fixed after I made that list

Oh yea, and the projects where the “latest” version changed between legacy (pkg_resources) and pep440

https://gist.github.com/dstufft/412717a3435342ad125f

I’m still pretty happy how compatible we were able to be without adding a ton of crazy. Obviously still some though 🙁

[long knoll]

I would assume the -main-.-VLazy.object.at.0x1006edf10- versions resulted from a bug in someone's build pipeline...

... wow, 11 years ago...

[vast wren]

yea a lot of them felt like bugs

I think pytz was the one I felt worst about not being able to support

they used a YYYYL formater, like 2026a, 2026b, 2026f, etc

which was just the version number from the olson database that pytz wrapped

[long knoll]

yeah, there's a definite logic to that

a ton of them look like people just not having a standard for the a/b/c/dev/rc suffixes

[vast wren]

I think some of that came from setuptools itself

IIRC if you produced a sdist from a branch, it would try to guess a suffix for you

or something along those lines

1.1.-svn-unreleased- is a great version number for something released to PyPI

[long knoll]

heh, yeah.

[vast wren]

I think if someone wanted to, it wouldn't be unreasonable to produce an update to PEP 440 that just acts as a restriction of syntax for new versions. They'd probably have to do similar number crunching as I did back in the day for 440 originally to decide what those rules should be.

You'd probably end up with 3 categories of rules:

• Supported on Write and Read.
• Supported on Read, but not Write (enforced by PyPI and build backends).
• Unsupported.

Maybe there's nothing that would fall into the third category, but from what.. I think @mighty flower found, maybe epochs could just be moved to unsupported (though the cost of continuing to read them is probably small?).

[mighty flower]

The technical cost of epochs is close to zero, it's the cognitive load and inter ecosystem compatibility where there are costs

[dark anchor]

I’m going to pretend all discussion of epochs in this server are about the ship in chrono trigger, makes the discussion a lot more entertaining

[long knoll]

I don't really see why we'd need further restrictions.

(or benefit from)

[vast wren]

I think the main benefit would be that it'd remove some confusing cases that can happen currently and/or remove some features that nobody uses. In theory if you wait long enough after restricting them, you might even be able to drop technical support for them all together

[fierce horizon]

Does anyone know how to get in contact with @raw lantern regarding my Sphinx playground? I tried the Sphinx bug tracker and writing him here, but no response. (https://flying-sheep.github.io/doc-play/)

[dreamy hatch]

He's recently moved and started a new job, so I think he's pretty busy with all that

[raw lantern]

Does anyone know how to get in contact

[silk jungle]

@undone sinew thanks for correcting me. Sorry that it took too many words to get there.

I'm happy to work with downstream. It's just hard since we (or at least, I) don't have great visibility into what y'all are doing and it fits or doesn't fit into our stuff.

[undone sinew]

It's always very nice when an issue turns out to be an non-issue 🙂

[vast wren]

The #debian-python IRC channel is typically pretty low traffic, as is the mailing list... sadly they're still on IRC and mailing lists, but such is life in many (for lack of a better word) 30+ year old projects 😉

[undone sinew]

All our main IRC channels are bridged on Matrix, these days. https://matrix.to/#/#debian-python:matrix.debian.social

[vast wren]

oh hey

maybe that'll be enough to get me to try out matrix, even though I don't participate in #debian-python much it's basically the only reason I keep an IRC client around anymore

[undone sinew]

Matrix is nowhere near as polished as Discord... But it is open and distributed

[vast wren]

it's bound to be better than HexChat + ZNC that I probably haven't patched in 8 years

[dark anchor]

i pay $5/mo for irccloud instead of futzing with a VPS, irccloud is great

[undone sinew]

If you're comfortable with IRC, IRC is a better IRC client than Matrix is...
But having access on your phone, not having to deal with bounces, etc. are all valuable

[dark anchor]

irccloud also has a great phone app

[long knoll]

honestly a lot of the "polish" to Discord doesn't really matter to me

there's stuff IRC doesn't do that I'd miss... I've thought idly about what it would be like to build a protocol on top

[lyric quiver]

https://github.com/peakoss/anti-slop

🤔

[kind moon]

https://github.com/jazzband/help/issues/196

cc @dreamy hatch

[silk jungle]

For pip-tools, I'd imagine PyPA would be a suitable new home for it. cc @floral oar

[floral oar]

Yeah, I've been meaning to get my act together on "how do I transfer a repo between orgs when I'm not an admin of either" 😂
I guess this lights a fire under me to figure it out

It will actually be nice to remove some of the defunct integration with the jazzband publishing pipeline. We don't use it, but we're still pushing artifacts over there and stuff. Lots of little cleanup to do.

[silk jungle]

Let me know if/when you want to start the transfer process to PyPA. I can be the PyPA committer sponsor for the PyPA vote.

[lyric quiver]

imagine trashing OSS devs for a living

[mighty flower]

Today's news is weirdly making me feel inspired to go back to resolution problems, and get pip's resolver up to speed with uv, using a CDCL equivalent. I think all the ground work in terms of spec compliance and understanding is almost in place.

[long knoll]

CDCL?

[vast wren]

Conflict driven clause learning

[mighty flower]

Yeah, I got some good results already with an associated optimization, conflict prioritization, I thought I'd tried this before and not seen a difference, but I must have missed something, because testing now I see about half my saved ResolutionTooDeep scenarios are solved. Should have a PR out this weekend.

The good thing about this optimization is it only requires a change on pip's side.

[mighty flower]

Walla: https://github.com/pypa/pip/pull/13859

[floral oar]

I love how small that is for such a big apparent win. 🚀

[long knoll]

and this works within the existing resolvelib framework? nice
Worth a PR back to resolvelib then?

[mighty flower]

No, resolvelib relies on the provider to do a bunch of work, including pick what package to choose when faced with a choice

This is leveraging that design

[hexed briar]

@mighty flower Have you considered adding a "how many candidates looked at" counter to see resolver "efficiency"?

[mighty flower]

I think that's my total visited packages in my tool that measures resolver scenarios and outputs statistics on them, e.g. https://github.com/notatallshaw/Pip-Resolution-Scenarios-and-Benchmarks/blob/main/summaries/problematic/kedro-test/26.0.json

[hexed briar]

It might be useful to have that per-package, like uv does in their tracing logs.

[mighty flower]

Yeah, I can do that, I used to store the whole resolution, but that was 100s of MBs, so I couldn't store it, so I summarize it down to what's useful enough that I can save the results on GitHub and check the results across multiple approaches

My main goal at the moment though is to get rid of all the known ResolutionTooDeep results, I'm working my way up the ladder of optimizations that can be: applied in pip, applied in resolvelib without an API change, applied in resolvelib with an API change, require a new a brand new resolved library

PubGrub can't be implemented without the resolver having some concept of version ranges, so that needs to be implemented on the packaging side if that's going to be the winning optimization, so I'm also working on a packaging public API to express Specifiers in terms of intervals

[mighty flower]

Some mkdocs drama apparently: https://fpgmaas.com/blog/collapse-of-mkdocs/

[boreal bramble]

They've also disabled issues and discussions on httpx repo a few weeks ago

[silk jungle]

Is there a connection between httpx and mkdocs?

[dark anchor]

the same original author / current maintainer

[silk jungle]

Yikes

I didn't realize that the encode organisation has uvicorn and starlette, too

[boreal bramble]

I don't think uvicorn is there anymore

(but Mia Christie is still listed on the pypi project, at least)

[mighty flower]

I was musing about this on my walk, I think open source can't help but be messy, either organizationally or socially. Until you build a foundation and that foundation has enough resources and policies in place to manage people and maintain the project.

But a lot of valuable projects are built on one person having a really good idea and just putting out there, not having to think about people and policies and foundations. As much as we might disagree with the project owner I think try to instigate a coup is the wrong approach, forking is also messy but it's the only way you can really take a project in the direction you want.

[onyx spindle]

not anymore, those were transfered to Kludex

[long knoll]

forking is 100% in the FOSS spirit and it should generally more often be the response to SHTF

[boreal bramble]

Everyone hopes someone else does it first so that they don't have to step up and take on maintenance of a fork

[silk jungle]

I tried to push a change to GitHub only to be met with an error "Invalid username or token". It turns out that my PAT is almost an month expired.

My time for open-source has been so limited lately 😅

[silk jungle]

I'd had participated in the 2021 Tidelift open-source maintainer survey. It's surreal reading my own quote.

[robust sandal]

FWIW, it seems like -X lazy_imports=none might not work. I'm using my flake8-lazy based additions of __lazy_modules__ and there's no time difference between none/normal in cibuildwheel. -X importtime shows the same thing. While if I remove the __lazy_modules__, then I see way more in importtime.

Also, does anyone know if lazy is going to be added to stdlib modules? I wasn't able to speed up build as much as I'd like because of non-lazy imports inside modules, I think (I also fixed some things in flake8-lazy, need to retry to be sure!)

[dreamy hatch]

thanks for the report, here's an issue and fix

[mighty flower]

I thought we were dropping lazy=none?

[robust sandal]

I'd hope not, it's really handy for testing. I can't tell if it's working as well without it.

I'm not sure why it's not never instead of none (lower case) which is a bit odd. As is normal instead of default. 🙂

[mighty flower]

I would add your use case to this discussion then: https://discuss.python.org/t/concerns-about-x-lazy-imports-none/106203

[robust sandal]

I like the suggestion I made there; none should only affect __lazy_modules__ and not the lazy keyword. That way it's perfect for my use case and doesn't break the standard library from adding lazy, keeping it working with circular imports and such.

[dreamy hatch]

or go full granular? PYTHON_LAZY_IMPORTS={normal|all|keyword|dunder|none}

[robust sandal]

I think there shouldn't be a way to turn off the lazy keyword, because you should be able to rely on that always being lazy (such as for type checking and circular imports). But asking Python 3.15 to treat the backward compatible __lazy_modules__ as if it was 3.14 seems handy.

PYTHON_LAZY_IMPORTS={normal|all|dunder} that is. 🙂

Although just making none only disable the dunder form would also be fine, I think.

https://github.com/pypa/cibuildwheel/pull/2797

And https://iscinumpy.dev/post/flake8-lazy/

[robust sandal]

It's around 4x faster with lazy imports to do cibuildwheel --help or cibuildwheel --print-build-identifiers. Since I can't disable lazy imports, that's 3.14 vs. 3.15 + lazy.

[dreamy hatch]

3.15 with the fix, optimised non-debug build:

$ hyperfine --warmup 10 \
   "./python.exe -m cibuildwheel --help" \
   "PYTHON_LAZY_IMPORTS=none ./python.exe -m cibuildwheel --help"
Benchmark 1: ./python.exe -m cibuildwheel --help
  Time (mean ± σ):      56.2 ms ±   1.5 ms    [User: 46.5 ms, System: 8.6 ms]
  Range (min … max):    53.6 ms …  62.3 ms    50 runs

Benchmark 2: PYTHON_LAZY_IMPORTS=none ./python.exe -m cibuildwheel --help
  Time (mean ± σ):     173.9 ms ±   2.3 ms    [User: 151.8 ms, System: 19.9 ms]
  Range (min … max):   169.9 ms … 177.5 ms    17 runs

Summary
  ./python.exe -m cibuildwheel --help ran
    3.09 ± 0.09 times faster than PYTHON_LAZY_IMPORTS=none ./python.exe -m cibuildwheel --help

[robust sandal]

Nice. 🙂

FYI, I can't get build significantly faster with lazy imports (flake8-lazy --apply src/**.py (fish syntax) followed by hyperfine ".venv/bin/python -m build -h"). It's around 50ms without, 45ms with, but 35ms if you turn on all lazy imports, which makes the stdlib imports lazy too. I'd love the stdlib to get lazy, but it seems lazy_imports=none is a blocker for the syntax version of lazy imports :(.

Ah, though I did forgot about from a import b being ambiguous. Maybe I'm still missing some imports. Tried a quick pass, doesn't seem like it helped.

[fierce horizon]

Spooky supply chain attack: https://github.com/BerriAI/litellm/issues/24518

• trivvy security scanner is compromised
• in turn compromises the owner of BerriAI/litellm
• which compromises a lot of people apparently
  Things are automated and standardized enough these days that attacks can spread a few links far before being found out …

[kind moon]

Same people also got Checkmarx
https://www.sysdig.com/blog/teampcp-expands-supply-chain-compromise-spreads-from-trivy-to-checkmarx-github-actions

Let this show that third party scanning doesn't necessarily make you any more secure, but it does increase the attack surface

[vast wren]

I’m not sure that property holds

Anything you depend on definitely increases the attack surface, but you’d have to figure out if those scanners had a positive influence independently of that to determine if they made you more secure

[long knoll]

...why does the scanning have access that would allow a compromised scanner to compromise the scanned code?

[robust sandal]

It was reading /proc/*/mem on the worker.

[lapis solstice]

I sometimes wonder what people writing scanning services are smoking when it comes to their overall system design, since basic ideas like "least privilege" get thrown out the window (Snyk requires Admin access on its service account if you want to integrate it with BitBucket server): https://docs.snyk.io/developer-tools/scm-integrations/organization-level-integrations/bitbucket-data-center-server

[floral oar]

A few years back, when I was less aware of how contracts can twist your technical requirements, I tried to argue against some "tools" at work because one of the limitations was that we couldn't update to the latest Linux kernel versions -- because the tool would break. This argument did not result in us rejecting the tech, and probably didn't make me any friends. But I learned a lot about where not to spend effort!
I think many security services are like this. 🙃

[lapis solstice]

Yeah, I asked my boss about it and his response was essentially "Yup, this is why we don't set it up until we have the executive override of our security concerns in writing"

[lyric quiver]

https://www.youtube.com/watch?v=mx3g7XoPVNQ

oh yikes

[silk jungle]

@silk crypt I deleted your message as this isn't really the place you should try to find jobs. You should use actual job boards.

[onyx spindle]

Those job seekers are a plague recently. All tech discords get those. And every one of those is about being master AI dev

[fierce horizon]

The job seeking part is understandable, it’s not like we live in Fully Automated Luxury Gay Space Communism.

The “master AI dev” part not so much.

[mighty flower]

I mean a lot of job applications are looking for "AI devs" now...

I'm job seeking right now, and I'm very grateful I'm in a position to lean on my networking to find positions to apply to and I have open source work I can point to before the age of AI to show I was solving interesting problems before outsourcing to the machine

[robust sandal]

Lazy imports make --help in repo-review go from 113 ms to 66.7 ms. If CPython used lazy imports internally, it would go to 45.7 ms. https://github.com/scientific-python/repo-review/pull/311

[long knoll]

nice, that may cross a threshold of noticeability

[robust sandal]

@steel crane Does zizmor handle .github/workflows/copilot-setup-steps.yml? I noticed pedantic mode triggers concurrency-limits, but I don't think that makes sense for copilot-setup-steps? I'm not even sure it's allowed, as there's a pretty limited subset in that file from what I understand. Also I don't think you need to give copilot-setup-steps a name. Not even sure you need to give it tightened permissions.

Ah, no, I think it does support permissions, good: https://docs.github.com/en/copilot/how-tos/use-copilot-agents/coding-agent/customize-the-agent-environment

[steel crane]

it doesn't special-case that workflow yet, i haven't looked at it closely: https://github.com/zizmorcore/zizmor/issues/1481

[robust sandal]

Great, will drop this idea in there

[robust sandal]

Got the speed of repo-review --help down to 35 ms. Also, it think I was wrong before, if rich used lazy imports inernally, that probably would have been much of that extra time. I was importing rich.traceback. (Dropped the custom traceback handler, but I think that's fine, the default one is pretty good these days.) https://github.com/scientific-python/repo-review/pull/317

[tool.ruff.lint.flake8-tidy-imports.banned-api]
"typing.TYPE_CHECKING".msg = "Use TYPE_CHECKING=False instead"

is a handy trick if you want to use the TYPE_CHECKING=False trick everywhere. 🙂

[onyx spindle]

does setting the TYPE_CHECKING=False do anything significant actually?

[marsh kite]

the constant TYPE_CHECKING is treated as true by type checkers https://mypy.readthedocs.io/en/stable/config_file.html
You can save an import from typing by defining it yourself

[floral oar]

Oh great, now I ran hyperfine on a couple of things and I want to figure out a move off of pyenv because I saw how much overhead I'm paying. Thanks Henry, there goes my weekend! 😆

Startup on 3.14 for me seems to be ~10ms . Faster boxes may do better. So 35ms seems like it's getting pretty near the limit for a nontrivial program?

Nevermind; that 10ms was still going through some extra jazz. Still. 35ms seems pretty fast for a Python package to get loaded and actually do something.

[dreamy hatch]

By default pyenv builds on your machine without PGO and LTO, so you're losing a 30% performance improvement right there

[dark anchor]

does turning them on make builds a lot slower?

i’m rebuilding Python a lot

[marsh kite]

Yes, lto/pgo are not incremental and significantly slower in general

[jaunty marlin]

yeah that's a major benefit of using python-build-standalone instead

we do a bunch of performance tuning

[dreamy hatch]

yeah, PGO is profile-guided optimisation, it does a build, then generates a profile by running some "realistic" code (running tests), then rebuilds using the profile to make optimisations. on this machine:

• 41s without
• 2m45s with
  for developing CPython itself, without is usually fine, unless you're benchmarking performance improvements!
  for normal use, use PBS or the official installers that do all this slow optimisation for you.
  (see https://github.com/pyenv/pyenv/pull/2579 for the pyenv flags)

[boreal bramble]

Very insignificant thing but someone accidentally swapped the #installer / #packaging breaking the alphabetical order

[dark anchor]

TIL —enable-optimizations

[dreamy hatch]

https://devguide.python.org/getting-started/setup-building/#optimization

I think —enable-optimizations was added first for PGO, and then LTO came next as --with-lto 🙂

[silk jungle]

ooo, I've never used -march=native -mtune=native

I doubt it leads to a huge uplift in practice, but it probably helps by a % or two.

[mighty flower]

Finally going back to my secret project! I've had a vague idea of what I wanted out of it for years, but today is the first day it has working code.

[robust sandal]

@steel crane

 WARN collect_inputs: zizmor::registry::input: failed to validate input as workflow: input does not match expected validation schema
 WARN collect_inputs: zizmor::registry::input: failed to validate input as workflow: input does not match expected validation schema

That doesn't tell me which files failed, by the way. (.github/release.yml and .github/zizmor.yml, by elimination, but it would be really nice if the message mentioned it!)

[steel crane]

Mind filling an issue for that? So I don’t lose track of it. Thanks!

[robust sandal]

Done

[frank shore]

Is there going to be a Packaging Summit at PyCon? I don't see one listed under Events & Summits.

[timber sphinx]

I don't think anyone stepped up to make it happen

[robust sandal]

I was planning on going to PyCON specifically for the packaging summit

Is there something I could help with to make it happen? Or is it too late?

[dreamy hatch]

@mighty flower alsp offered to lend a hand: did you hear back from people? (re: #general message)

[mighty flower]

Filipe told me CAM and Janis are organizing, but I haven't heard from them. @hexed briar also say he might accept a helping hand. I meant to chase up again, if there's nothing as formal as a summit I think we should try to sort out something less formal

[marsh kite]

I'd be happy to help with either!

[dreamy hatch]

ping also @muted whale and @calm horizon

[muted whale]

Thanks @dreamy hatch , @calm horizon has been doing most of the work, I haven't been able to help much on this one unlike previous due to working 12-16+ hours a day for much of the past few months thanks to the NASA Artemis II launch tomorrow (it's 4 am for me and I gotta drive 11 hours tomorrow to Cape Canaveral tomorrow, er, today 😵 ). Hopefully I'll have more time after that's all over soon (assuming it doesn't get delayed another month, fingers crossed...)

[calm horizon]

I’m waiting for the go ahead from Pradyun

IMO it’s ready to launch 🚀

I don’t know why we need more hands right now if the forms and page are ready tbh