off-topic

[lyric quiver]

Figured out how to package with a portable python.exe and download Pytorch and TensorRT during runtime on the user's system.

this breaks all my github actions, but it's better long term since new updates will only require a 90mb download instead of 2GB ( zipped for github limits )

I am not sure exactly how I will update dependencies tho'.

[serene fulcrum]

Do you know uv python install ?

[lyric quiver]

never tried it

[serene fulcrum]

You should it's freaking dope

[lyric quiver]

please don't give me ideas

[serene fulcrum]

https://docs.astral.sh/uv/guides/install-python/

Little player 😄 Not even midnight

[lyric quiver]

I have work at 8am 😭

[serene fulcrum]

It's like early afternoon for dedicate people

😄

[lyric quiver]

I have to be up at 7am and still didn't figure out the whole dependency debuckle

[serene fulcrum]

Just install uv and install python you will enjoy trust me. Best thing is that it will probably take you under 5min end - 2 - end

[lyric quiver]

https://tenor.com/view/pepe-leaving-gif-24551619

Yeah, I don't know exactly how I would upgrade the dependencies

say pytorch 2.8 releases and the user has 2.7 installed, how can I 'force' the user to upgrade 🤔

I'd need to probably read the requirements file, look for torch and compare the currently installed version and the one I force set in the .txt file

I hate this 😭

[long knoll]

if they already have your code (at a certain version) and a working set of dependencies installed, why do you want to force an upgrade?

especially if you haven't bumped your own version?

[lyric quiver]

Well in the sense that I bump the version too

I release a new version of my software that requires a new release of pytorch or tensorrt or any other dependency

I'd have to compare the currently installed version and the version found in the requirements.txt file and if they don't match run an update.

that's probably a more naive approach, maybe there's something better that I am missing

[torn fjord]

oh the reason for the size increase is because of blackwell support

https://tenor.com/view/crying-the-office-sad-steve-carrel-gif-6504535541019570333

Generally the reason why PyTorch wheels grow in size is if we need to add architecture support here or there.

For this particular release we added 10.0, 12.0+PTX which you can observe here: https://github.com/pytorch/pytorch/blob/dccb7a9cb28836d280d6cd5d8e9a3956306d52ff/.ci/manywheel/build_cuda.sh#L57

Representative windowws file: https://github.com/pytorch/pytorch/blob/dccb7a9cb28836d280d6cd5d8e9a3956306d52ff/.ci/pytorch/windows/cuda128.bat#L40

[lyric quiver]

you ruined me

set TORCH_CUDA_ARCH_LIST=5.0;6.0;6.1;7.0;7.5;8.0;8.6;9.0;10.0;12.0

I wonder, could I simply yeet out some of those? and build the wheel myself 🤔

especially since I heard 2.7.0 no longer supports Pascal or no longer works on pascal at least.

https://github.com/TNTwise/REAL-Video-Enhancer/issues/104

[torn fjord]

Yup! Generally we recommend for people who don't want these fat wheels to build from source and modify TORCH_CUDA_ARCH_LIST to their liking

[lyric quiver]

5.0; seems to be

yeah these are like what... 10yo now?

are people even using them anymore 🤣

[torn fjord]

my 970 is still kicking in my nephews computer 😛

But yes I generally agree

Will approve a PR to reduce the list if you submit one!

[lyric quiver]

O

so I can call myself a maintainer of Pytorch????

sure will do!!!

[torn fjord]

Feel free to ping me on the PR, github handle is the same as my handle here!

[lyric quiver]

@torn fjord I've been checking and it seems like 6.0 is no longer present in Geforce products but only in one single Nvidia Quadro , specifically the Tesla P100 6.0 a gpu from 2016, would you say that both 5.0 & 6.0 could be removed?

[torn fjord]

I think it's reasonable to do that for CUDA 12.8 considering we also removed everything up to 7.5 for linux

[lyric quiver]

oh, so I should only propose changes to 12.8 🤣

I already removed 5.0 and 6.0 from 12.4, 12.6 and 12.8

[onyx spindle]

So, pytorch is shipping that much data in the wheels or is it all just code and/or compiled libs?

[torn fjord]

mostly compiled libs

[torn fjord]

I think it makes the most sense for CUDA 12.8 since those arches were deprecated in 12.8

[lyric quiver]

https://github.com/pytorch/pytorch/pull/152069 well I did it for both 12.4 and 12.6, I guess I could revert those changes?

[torn fjord]

I think it's fine, pinging nvidia folks who might have an opinion

[lyric quiver]

Thanks a lot ❤️

was attempting to self build the wheel just now to see how much smaller it would be, turns out I am quite unlucky today

[onyx spindle]

do you think something like PEP 725 could help with this issue?

[torn fjord]

I think it could be helpful!

[long knoll]

Every time you release a new version, it gets to have its own metadata which can specify its own constraints on the version of dependencies.

but you can't really make anyone upgrade anything. You can have code within your project that shells out to an installer to tell it to upgrade, if it's really that critical, but that runs after your project is already installed

and automatically upgrading things can be seen as user-hostile. after all you don't know what other reasons the end user has for having the current versions of things in the current environment; you don't know what else might be installed there besides your project; maybe your new version introduces a bug for the other user's workflow; maybe the user is in a corporate environment where the upgrade would be against policy....

[lyric quiver]

I use a portable local python

Everything is self contained

I always " force " the end user to be on the latest release due to bugfixes and new features I add.

The intent isn't hostile, it's just that I used to get so many false alarms with issue X which was already fixed 3 releases ago

[lapis solstice]

My current job essentially exists due to the size of AI libs: https://lmstudio.ai/blog/venvstacks

Feels like a similar use case to the one discussed here: we don't want to be using whatever the user has lying around on their system, but we don't want to be redeploying PyTorch and the CUDA libraries all the time, either. Since LM Studio is a desktop app, we can't really make the supported arch list any smaller than the upstream project defaults, and we want everything to be self-contained inside the app so external dependency declarations likely won't make a major difference to us.

What does make a difference is being able to cleanly separate the high churn "app" layers from the less frequently updated framework and runtime layers. It doesn't make it easy, by any means, it just gives us deployment chunks to work with that split the difference between individual packages and monolithic application environments.

(Happy to answer questions in the #venvstacks channel if it seems like this might be useful to you)

[lyric quiver]

I like this nickname more 😎

[lyric quiver]

Crazy stuff, so I guess it was pretty much a universal issue. I wonder how solutions like Nuitka haven't been more popularized nowadays, having everything compiled would mimmick the feel and workflow of any C program and updating / maintaining software would be significantly streamlined.

[lapis solstice]

@mighty flower When it comes to static site generator hosting, it's possible to put things like GitHub Pages behind proxy services like Cloudflare to host them on a custom domain (for example, curiousefficiency.org is a Nikola site published on GitHub pages with the TLS endpoint provided by Cloudflare)

[dreamy hatch]

you don't even need a proxy, hugovk.dev is directly on GitHub Pages
https://docs.github.com/en/pages/configuring-a-custom-domain-for-your-github-pages-site/managing-a-custom-domain-for-your-github-pages-site

[junior narwhal]

same for ofek.dev

[long knoll]

(one of these days I should get a domain. I guess first I have to be properly set up for paying a provider over the Internet)

[undone sinew]

Watch out, they're addictive. Next thing you know you're paying for 20 domains you don't really use but don't want to give up...

[junior narwhal]

also be prepared to continuously check your registrar of choice every few months to see if it yet supports that one TLD it's been missing, but has announced future support for, so you can't consolidate completely

[ionic tulip]

i consider venv stacks the worst way to implement share-able wheel installs - the granularity is all messed up - while its still arguably better than multi version wheel working sets its still sriclty more limiting than being able-to just link immutable pre-installed packages/dist-infos into a venv without incurring cost

[silk jungle]

this is amazing: https://github.com/hxu296/tariff

TARIFF is a fantastic tool that lets you impose import tariffs on Python packages. We're going to bring manufacturing BACK to your codebase by making foreign imports more EXPENSIVE!
My god, hahaha.

(For the Americans living through this madness, I'm sorry, but as "not an American" it is pretty funny—if not a bit fear-inducing—the absolute chaos that is the current trade situation).

[kind moon]

This is unalivejoy (from pydis)
https://github.com/hxu296/tariff/issues/37

They were playing with it the other day

[boreal bramble]

Honestly the issue tracker is the best thing about this project

[lapis solstice]

Yeah, it's definitely an app component distribution tool rather than an env management tool. That said, it's theoretically possible to add an alternative distribution mode that adds the lock files to the shipped archives and builds the environments on the target system instead of shipping the installed wheels in the layer artifacts.

Hmm, you know, that would also be a super useful mode for speeding up the artifact generation tests.

[lapis solstice]

Moved that idea to the venvstacks channel: #venvstacks message

Static hosting with custom domains

[ionic tulip]

from that perspective it also can be interpreted that venv stacks is a crippled variant of container layers

[lapis solstice]

Which works cross platform, as it doesn't require cooperation from the operating system kernel.

[lapis solstice]

It's very directly inspired by the way Flatpak works on Linux

[ionic tulip]

is wsl already able to run containers?

[marsh kite]

Yep, has been for a while

I run docker in it all the time

[lapis solstice]

Sure (my main dev environment is Fedora in WSL2), but making WSL2 a hard dependency for a Windows desktop app isn't a great user experience.

Beyond that, the launch use case for venvstacks was Apple's MLX libraries, which only run on macOS

I'd still like to support conda as a base runtime provider in addition to python build standalone, and the virtual layer archive idea may provide a practical way to achieve that. (Work don't need the conda support, though, so it's unlikely I'll get to it myself any time soon)

[marsh kite]

I'm curious, how did you set up your Fedora WSL instance?

[lapis solstice]

I use the Whitewater Foundry remix from the Windows app store. Set it up years ago, and IIRC it was around $10AUD at the time.

[marsh kite]

Gotcha!

[lyric quiver]

Figured it out

Import torch, if torch not found then pip install otherwise compare requirements.txt version torch cuda version Vs torch.version

If they don't match, trigger a full requirements upgrade

Was a bit painful, it adds about 0.1-0.2s to the initial startup but not having to package with Pyinstaller means i mostly negate the start up

And the 30mb updates are so much nicer than a whole 6.5GB

[torn fjord]

It's the unfortunate side effect of windows vs. linux (we don't have as much expertise with windows unfortunately)

[lyric quiver]

no worries @torn fjord

as an official Pytorch Maintainer I will help maintain the windows side of things

😄

gridsample TRT is still borched, hoping for a fix

[lyric quiver]

Linux wheels used to be even larger

[lyric quiver]

not so sure if that still applies today with the whole sm removal

[raw lantern]

Is there a resource for the current best-practice on deprecating/removing project extras? A contributor has suggested that we simply delete them, and that hard-breaking projects is better than making the current extra names install nothing, but I'm less sure.

For context, we had extras that were functionally project-internal, so we now have duplicated information across dependency groups and extras.

[silk jungle]

Missing extra is a hard-error? I thought at least pip would only issue a warning if an extra is missing.

[raw lantern]

I had assumed a hard error

[silk jungle]

pip install pip[missing]
Requirement already satisfied: pip[missing] in /home/ichard26/.local/share/vem/envs/pip-20250428-171940/lib/python3.9/site-packages (25.1)
WARNING: pip 25.1 does not provide the extra 'missing'

[raw lantern]

If it's only a warning, it might be less bad to simply delete them.

[silk jungle]

I'll confirm that pip doesn't error out if it's requested in a dependency. You should also double check uv pip does the same thing (I'd imagine it will, and deviating here would be surprising IMO).

[raw lantern]

I'm just very averse to complaints that X has broken and it's my fault...

[silk jungle]

requesting a missing extra as dependency seems fine too

Given the existence of https://github.com/pypa/pip/issues/9736, it seems safe to assume to pip has never (in the ~past few years) raised an error for a missing extra.

Uv may transition to erroring, but they haven't done so yet @raw lantern https://github.com/astral-sh/uv/issues/2828

[onyx spindle]

@raw lantern you could in theory duplicate what urllib3 did some time ago. they made a special package https://github.com/urllib3/urllib3-secure-extra that was installed from that extra and added some code to the lib that detected it and if so, they showed a deprecation warning

[dreamy hatch]

the framing I like to keep in mind: is this a public API?

"extras that were functionally project-internal" don't sound like they are, so you can do what you like

to be on the safe side, you could have a period with both (like you have now), and mention in some release notes or docs that they'll be removed in the next (or a named) release, but no need for a full deprecation process

[raw lantern]

I’ve tried to define a public API for Sphinx and comprehensively failed several times!

[dreamy hatch]

I should have said "public interface" in this case 🙂

[jaunty gust]

dang, OSUOSL is up against a wall 😦 https://osuosl.org/blog/osl-future/

PyPI ran there for a hot minute

from two lil vms with a drbd mirrored volume and no CDN

[hexed kindle]

it's the only mirror service in the US that has been reliable for me over the years 😦

[lyric quiver]

Ever since I started coding I've been working with videos and I/O alongisde that I've been an FPS chaser but I always lacked the resources necessary to figure out which library shall I use for decoding videos. For that reason I made a repo just to track the raw performance of each library and make smarter choices.

https://github.com/NevermindNilas/python-decoders-benchmarks

I will eventually add more libraries as I go and set up a GH Action to keep the data up to date, but this will hopefully help others who kept on using some heavily abstracted library of another library.

[torn fjord]

This is awesome btw! We actually recently released a new library that might have better performance than torchaudio!

https://github.com/pytorch/torchcodec

[lyric quiver]

I develop primarily on windows

I will need to set up wsl and have it implemented

[torn fjord]

Let me check in with the team to see where the windows support is

[lyric quiver]

https://github.com/pytorch/torchcodec/issues/639

I alr opened an issue

[torn fjord]

Ah I see that, yeah the library is relatively new so I'd expect for it come sooner rather than later, but will check into this for you to see if we can get a better sense on timeline!

[lyric quiver]

I want to add it

it's just that I hate working with WSL

also I can't test cuda decode on gh actions for obvious reasons so I will have to run all of the tests myself

I plan to add 480p, 720p, 1080p and somehow keep track of previous results vs current and see if any new update improved or decreased performance

[lyric quiver]

so far so good

the github actions works properly

I added 80% of the libraries I wanted / knew about

all that's left is to add 480p, 1080p and the comparison from weekly runs

[lyric quiver]

@torn fjord

torchcodec doesn't seem to be much faster than torchaudio

[torn fjord]

I brought this up to the torchcodec maintainer! Hoping they reach out with some thoughts soon!

[lyric quiver]

Here's something slightly more updated

should have most bugs squashed

[junior narwhal]

something appears to have beaten the "Max Theoretical" 😉

[lyric quiver]

it's pretty hacky

it does yuv420p decode then converts it to rgb24

whilst max theoretical does pure rgb24

if I do yuv420p on max theoretical on my system with no rgb24 conversion

[long knoll]

https://discuss.python.org/t/inline-script-metadata-for-zipapp-archives-and-directory-execution/90964/7

I thought there was the start of an interesting idea here. I can imagine two modes of operation:

1. Given a script with PEP 723 metadata, generate a temporary "project" folder with a corresponding pyproject.toml, and create a wheel from it. Prompt the user for missing info such as a license SPDX, etc. (There's already a project mentioned in the thread that makes a zipapp instead, bundling a script runner. One could imagine bundling a package installer and the wheel into a zipapp...)

2. Similarly, but instead generate a temporary virtual environment, then create a zipapp which includes that virtual environment along with logic to unpack it somewhere on startup. (The user needs to be made aware that the result is platform-specific.)

actually, since I'm designing PAPER to start from a self-installing zipapp (which pre-populates its cache and bootstraps from there)... I could potentially make something that does that, but also includes the script wheel and dependency wheels...

[lapis solstice]

I haven't looked into how the PR is implemented, but your highlighting this comment brought me to the open shiv PR adding the capability: https://github.com/linkedin/shiv/pull/267

[lyric quiver]

When pip installing -r requirements.txt, I wonder, is there a way for the output to have a time logger, say, [08.05.2025 | 10:23 ] - { Text }

similar in nature to logging

[robust sandal]

The question is, is the next release (the one with the first beta out now) Python π or πthon?

And I can't type that without triggering the printing dialog... (on macOS)

[mild pollen]

Printhon

[mighty flower]

pip itself doesn't have any such feature, you'd have to write a wrapper around the stdout to add timestamps at the start of each line of any command, assuming there isn't a popular too do that already

[lyric quiver]

Ty big mans

[dreamy hatch]

also check your terminal settings, although they might not show for every line. for example in iTerm2:

[lyric quiver]

In that case it's within a react tool

I capture the stdout and pipe it to a list and then display it like a built in terminal

But without the typing

I will just capture the stdout with the logging module on the python side and add date time there

And then just display the generated log.txt inside the makeawish terminal I made

[hexed briar]

@lyric quiver I think we include that in the --log file.

[dreamy hatch]

Good news, OSUOSL is funded for the next year https://fosstodon.org/@osuosl/114478619839118774

[jaunty gust]

yaeeeeeeeayyyyy

[lapis solstice]

Tab completing 𝜋 in an active Python 3.14 venv may suggest an answer here 😉

[lyric quiver]

has anyone played around with the free threaded variant of python 3.13? Is it mature enough / supported by enough libraries to be usable in production or is it still in 'alpha'

I also wonder if it will become the defaulto with 3.14/3.15

[kind moon]

It may eventually, but definitely not that early

It's still in opt-in alpha

[lyric quiver]

so you'd say maybe 2027+-

[dreamy hatch]

It may become "officially supported" instead of "experimental" in 3.14, but will still be opt in. See https://peps.python.org/pep-0779/

[lyric quiver]

Read it

Interesting findings

15-20% more memory usage is a bummer

But assuming it doesn't impact the total global memory usage but instead just only on an interpreter level

Going from 100mb to 120mb is virtually nothing

The performance degradation makes no sense to me

Single core performance loss of up to 10% is terrible 😔

[lapis solstice]

One big interpreter state lock that isn't actually checked very often (as CPU latencies go) has lower single core overhead than lots of small locks that need to be checked on every access.

So even if the default eventually flips, there may still be value in the GIL-enabled build as a way to improve single core performance and reduce memory requirements.

[lyric quiver]

🤔

that makes sense and I hate that it does make sense

[long knoll]

is there something that prevents switching between using or not using the GIL at runtime? or would that just not help?

[lapis solstice]

The free threading build can turn the GIL back on, but I don't actually know if that eliminates the performance penalty or if it's a "worst of both worlds" situation where we still pay for checking all the small locks even when they're redundant. It definitely won't eliminate the extra RAM usage.

Edit: I checked with the free-threading folks, and indeed, the runtime toggle brings the improved thread safety back, but not the improved single core performance.

[long knoll]

mm. well I figured it wouldn't help with RAM because of the stuff that gets statically compiled into the data structures to make finer-grained locking work
it doesn't seem like that will be a major overhead in practice though. (and there are probably worse things...)

[lapis solstice]

It wasn't a coincidence that the Faster CPython activities and the free threading work were approved in parallel - even if the latter gives up some of the single core performance gains from the former, the combination should still offer faster single core performance than 3.12 did (let alone 3.11 or 3.10).

[robust sandal]

In Python would it make sense if http_error_default added the url as a note? I keep hitting urllib.error.HTTPError (in pypa/build, pypa/cibuildwheel, scientific-python/repo-review, maybe others) and have no idea what URL is returning a 404 from the trace. Pytest's showlocals would help if urllib.request.Request had a nicer __repr__, but it doesn't.

[junior narwhal]

my preference would be to deprecate standard library HTTP utilities...

[jaunty marlin]

😬

I was just thinking there should be better standard library HTTP utilities... haha

[robust sandal]

There are a ton of places where quick and dirty access to http is really handy and not worth a library. build has to be minimal dependencies for bootstrapping, for example! It probably isn't needed in all of those cases, though, but it's "good enough" (though the error reporting is lacking as noted)

The one in cibuildwheel is actually setuptools using it when building something in the tests.

[junior narwhal]

I do understand your point but for that particular example, I don't view build as meeting the criteria for bootstrapping currently because it doesn't vendor and has several dependencies already

[robust sandal]

Most of its dependencies are optional, there’s just no support for default extras yet. There’s a very specific formula for boot strapping it with flit.

I followed that plan in Conda Forge, actually, it bootstraps there

(it may only partially bootstrap there, I think I use the build flit-core formula. But it doesn’t have to)

[junior narwhal]

fair enough, although I don't understand how it can bootstrap without vendoring its 2 required dependencies

[robust sandal]

I’ll run through the formula at some point to make sure it works properly and add it to the documentation, I think it’s just an issue or a message somewhere

[junior narwhal]

I mean even outside of this particular project, I don't understand how any project can bootstrap with required dependencies that aren't vendored

unless you're saying that it gets the required dependencies by virtue of requiring flit_core during the build which happens to vendor the required dependencies and you use the paths within that package to import?

[robust sandal]

I'd have to look it up. Flit-core has no dependencies and a boot-strap script. I believe build can build itself and installer without any other dependencies. Then you can use that for the other required dependencies (which also use flit-core) and you are good to go.

But I don't remember exactly.

[long knoll]

last I checked, flit-core accomplishes that by vendoring tomli (but it's simple enough to not require other dependencies)

[robust sandal]

Flit-core does vendor tomli, everything else is implemented from scratch

[long knoll]

yes, that matches my recollection.

[junior narwhal]

finally finished a blog post I had pending for a while https://ofek.dev/words/guides/2025-05-13-distributing-command-line-tools-for-macos/
related to Python since apps can be distributed as a standalone binary easily nowadays with something like #pyapp

Apple documentation is quite horrendous

[robust sandal]

No packaging summit schedule yet?

[long knoll]

Are t-strings implemented in currently available 3.14 source, or will I need to wait for a specific prerelease (or even for 3.14.0) to try them?

(I have an idea for an alternative to PEP 787 that I'd like to prototype.)

[long knoll]

Also, are there any official plans to backport templatelib (users could still explicitly create Template instances without t-string support)?

[onyx spindle]

I think 3.12 and 3.13 have it under string.templatelib

[long knoll]

I can't find anything like that. (string.Template is far older, and unrelated)

unless it will be in a new patch

[onyx spindle]

nevermind, had a wrong information

[robust sandal]

The beta has everything that will be in the final release, no new features are allowed. T strings are there.

I’ve got a copy of it using pyenv. I also have one built from source I was using for git bisect. I don’t think you can quickly get it from UV yet, I believe that is a6.

[jaunty marlin]

Yeah I'm a little behind on support for the latest 3.14 because there are build system changes 😦

[long knoll]

I've generally been happy to build Python from source

(for my personal shelling-to-subprocess needs, Plumbum looks great)

[robust sandal]

Need to see if and how t-strings make sense in plumbum. I've delayed adding 3.14 since it depends on that pywin32 and I'm assuming that will not be compiled until RC 1 (or later, last year it was later, I think)

[robust sandal]

More build system changes for beta 2, probably, since the windows builds have issues (GitHub built them with the free-threading define on)

[mighty flower]

Well my plane to Pittsburgh is super delayed 😭

[lapis solstice]

The specific minimum CPython version for t-strings: 3.14.0b1. It wasn't ready for the final alpha.

No library backport plans, since without the new syntax, the template definition UX isn't any better than what existing 3rd party templating libraries can offer.

(Also worth noting: the PEP 787 discussions so far prompted the creation of https://github.com/ncoghlan/tstrprocess and we don't actually know exactly where we're going to take that experimental API yet: https://github.com/ncoghlan/tstrprocess/issues/1)

(Why, oh why, are the GitHub Discord embeds so uselessly huge?)

[shadow zealot]

I think it’s less Discord more GitHub gives a giant banner like image in its embed

[lapis solstice]

Yeah, I'm also pretty sure it's a setting on the GH end. However, clients are the ones actually displaying the embeds, so they could absolutely enforce a maximum size on them.

[fierce horizon]

I just saw this new podcast episode on reddit and thought the blurb about tooling was glancing over the standards work that made this possible too much. https://corrode.dev/podcast/s04e03-astral/

Up until a few years ago, Python tooling was a nightmare: basic tasks like installing packages or managing Python versions was a pain. The tools were brittle and did not work well together.

Then, suddenly, we saw a renaissance of new ideas in the Python ecosystem. It started with Poetry and pipx and continued with tooling written in Rust like rye, which later got incorporated into Astral.

Astral in particular contributed a very important piece to the puzzle: uv …
The website is open source so I’d like to improve the wording a bit, but I’m not sure abiout the timeline. I also don’t want to bloat the text too much. What do you think about this?
[…] ecosystem. It started with Poetry and pipx, which set a new benchmark for usability, and inspired increased efforts for interoperability PEPs. Unshackled from implementation-defined behavior, eyes were set on speed with new tooling written in Rust, like rye.

Astral incorporated rye as a spiritual base for a particularly important piece to the puzzle: uv …

[onyx spindle]

you can wrap a link in < > and it won't show any embeds

[lapis solstice]

I typically click the "Remove all embeds" button. It's just annoying because the embeds could be useful, but websites deciding "All your vertical screen real estate is ours now" ruin it for everyone 😛

[onyx spindle]

I agree, just wanted to share a tip that could be useful for such cases

[lapis solstice]

It didn't really start directly with poetry and pipenv either, it started with PEP 440 (standardised version numbers) and PEP 508 (standardised dependency specifiers). Those were what decoupled transitive dependency resolution from the idiosyncrasies of easy_install and the pre-standardisation pip resolver (and eventually PEP 517 was able to knock out the reliance on ./setup.py install as the expected build backend interface). Something like:

Up until a few years ago, Python tooling was a nightmare: basic tasks like installing packages or managing Python versions was a pain. The tools were brittle and did not work well together, mired in a swamp of underspecified implementation defined behaviour.

Then, apparently suddenly, but in reality backed by years of ongoing work on formal interoperability specifications, we saw a renaissance of new ideas in the Python ecosystem. It started with Poetry and pipx and continued with tooling written in Rust like rye, which later got incorporated into Astral.

[fierce horizon]

That sounds great! Shall I take that and make a PR there or do you want to? https://github.com/corrode/corrode.github.io/blob/master/content/podcast/s04e03-astral/index.md

[lapis solstice]

Please go ahead

[junior narwhal]

definitely more evidence of what I was talking about in the fall about failing as a maintainer in the modern age, I didn't make much use of social media and therefore it was mostly unknown that Hatch was the first to add a Python management command which is attributed to UV, first to implement PEP 723 script metadata which is attributed to UV, etc.

I was very poor at evangelism and I feel bad about that

[fierce horizon]

PEP 723 was first implemented by a dedicated tool, but of course I get what you mean!

[long knoll]

It seems that users only care about the standards work when they have to conform to a standard. :/

(and it sucks not to have the social reach to inform people about these kinds of things)

(oh! this is a blurb describing the episode. I was expecting an excerpt from a transcript, so I was wondering why make a PR.)

[lapis solstice]

@junior narwhal You're not a VC-backed Silicon Valley startup with the time to write things from scratch in Rust, so folks get initially attracted based on speed and bootstrapping convenience, and then discover the assorted other features uv offers.

The research failures of folks that then forget the difference between "I first encountered this feature via this project" and "This project was the first to implement this feature" aren't on you 🙂

[long knoll]

uv's design is great (notwithstanding that I don't like all-in-one tools), but it personally irritates me when people focus on it being written in Rust when so much of the speed etc. is a consequence of the design

(in my view, there are some things you can only really accomplish by writing things from scratch, whatever the language)

[lapis solstice]

I hadn't looked into all the details of the speed (although I did know some of it was from the caching design, and some was from making formally-invalid-but-usually-true-in-practice assumptions to reduce the numbers of downloads needed).

[long knoll]

(well, it depends on your use case, too. Getting frustrated at complex solves is different from getting frustrated at simple things not being instantaneous)

[lapis solstice]

On a completely different tangent (in regards to notifying users of new features), I'm reminded of the time I gave a PyCon AU lightning talk because I discovered from talking to people that we'd completely forgotten to mention the addition of zipfile and directory execution support way back in Python 2.6. The addition of zipapp to the stdlib was as much about rectifying that oversight as it was about adding the module itself 🙂

[long knoll]

ah! interesting

I do have a vague idea of the history of Pex etc.

zipapp as a distribution system doesn't seem to have taken off, although the various pyinstaller type things all seem to work similarly... ?

but part of it is that a vanilla zipapp just runs the code, without unpacking anything or doing other "installation". (For PAPER, I have a self-install; the zipapp includes a pre-seeded cache and then hands over installation to one of the seeded wheels, similar to ensurepip)

[lapis solstice]

It's definitely used, mostly via things like shiv that have that unpack step. zipapp largely gave us a better place to document the feature than having it buried in the CPython command line invocation docs, rather than being something we expected to see significant direct use (as with so many things, the lack of extension module support is a deal breaker in even mildly complex use cases).

[long knoll]

Aha. Makes sense. I'm using it too, then :)

...oh, you mean that it won't load e.g. a .so directly from the archive?

(In the long run you kinda want to unpack things anyway, and maybe even set up a private venv. For a short while I had Pipx "installed" by having the zipapp on my path. It mostly works, but in particular, Pipx can't do the self-upgrade of Pip that way.)

[mighty flower]

Well, I do think the resolver needs to be written in a language that can make low level optimizations to get within an order of magnitude of uv's resolver. I'm working to make pip better, but I've been convinced that some of the strategies to optimize resolution depend on things like version comparison being super fast and being able to scale it up to millions of operations.

[junior narwhal]

I do think the resolver needs to be written in a language that can make low level optimizations to get within an order of magnitude of uv's resolver
I don't have data to support this but I disagree with that assessment, I think that can be achieved with pure Python as long as you have real threading capability i.e. the new free threading mode. I think performance can be further enhanced like you said by compiling but not via a complex extension module but rather with complete type hinting and Mypyc

[mighty flower]

I don't think parallelism or concurrency helps much here, the operations are so small and dependent on each other that it would be tricky to find a way to parallelize the problem. The issue is that representing a PEP 440 version that behaves correctly with all the specifiers is non-trivial and you end up with a massive object in memory.

And there are algorithms that depend on being able to scale up doing comparison operations on them, which also has non-trivial edge cases in PEP 440.

Maybe it is possible to make this fast in Python, but it couldn't then be based on the current packaging library

[junior narwhal]

the parallelism is important because you're doing network requests at the same time

technically today you could use threads because I/O releases the GIL, but it's not as performant because you have to do comparison operations sequentially

[long knoll]

I've been operating under the assumption that the actual resolution algorithm isn't usually the bottleneck - because I've never observed it being the bottleneck for my use cases. (which is to say: I've seen Pip spend a long time downloading multiple versions, but then it seems clear that the bottleneck is my Internet connection. Pipx kinda hides whatever Pip is doing, but.)
But some people do appear to have very different use cases.

regardless, resolution certainly becomes faster in general when you're allowed to, say, restrict options to what you already have in cache.

but this stuff is definitely not my bag, and I'm definitely going to be just using resolvelib at least to start.

(and hopefully resolvelib continues to improve going forward)

[mighty flower]

I'm talking purely about the resolution, sans IO

[junior narwhal]

then yes I agree with you, I wouldn't use packaging

[mighty flower]

Well, pip does! And we're unlikely to move off it.

[junior narwhal]

I think the solution would be to improve the performance of that library

I'm not sure if it would necessitate breaking changes or if it would expose low-level high-performance constructs

[mighty flower]

uv does a lot of clever things to fit Python packaging versions in very small memory space and with fast comparison that no Python library can do

[mighty flower]

The problem with resolvelib is it's design is to be super generic, so many optimizations aren't possible because you can't make basic assumptions about the type of dependency graph your resolving. Like that they have version numbers, or that you can prove logical impossibilities. I don't see that changing.

[junior narwhal]

yeah after my pip deep dive in the fall I totally agree with you, it would require a foundational rewrite

[mighty flower]

Big improvements could be made here by designing a resolver that can assume Python packaging standards.

[junior narwhal]

I basically couldn't implement what I wanted to because of that https://github.com/pypa/pip/issues/13111

it would require maintainers deciding what they want for the future of pip UX

[mighty flower]

Well, I would like to design a resolver that could support that, but it's a wish at the moment. And yes, pip would need to make UX choices to actually do cross platform resolution that I think would require more standardization than there is right now

[junior narwhal]

I don't think standardization is the blocker but rather maintainer time, it could just be implemented

* maintainer time and backward compatibility

[silk jungle]

Yeah. It's also a matter that we don't really have overlapping expertise on the core team (at the moment) so there is little review capacity for major projects.

The most active maintainers are Paul, Stéphane, Damian and I. Damian and I are both new maintainers and have specialized in certain areas. Damian does dependency resolution primarily, while I'm honestly a jack of all trades, but I'm pretty useless for fundamental changes to the resolver, wheel installation logic, etc.

[lyric quiver]

Off-topic as per usual, could one write a flask server around a CLI

Currently the workflow is something like

python .\main.py --input .\input\1080.mp4 --dedup --benchmark

I wonder if I could have a flask server that listens to localhost and allows parsing a .json with the same 'args' and then run main.py. The main idea behind is bypassing the obnoxious import times and stuff alike.

Ofc, stuff like Pytorch and TensorRT will still linger around in memory waiting to work and it will probably be a GC nightmare but it does seem like a good way to bypass it. Unless there's already alternatives for such

[lyric quiver]

funnily enough, I kinda did it

far from perfect, my tool is too stupid at times

but I am getting somewher

🤣

It's trying to do something

This is blasphemy

for some awkward reason, it falsely detects my system as Linux based 🤣

No shot 🤣

[onyx spindle]

WSL by any chance?

[lyric quiver]

Nope, bare metal Windows

[onyx spindle]

impressive

[long knoll]

Something that occurred to me recently: if you sys.stdin.close() inside a REPL, it doesn't quit (although it can with ctrl-D), and it can still read input for the REPL (although not for input()).
But the REPL isn't, as far as I can tell, a separate process from the interpreter. So it must be doing some kind of proxying of stdin to the code, yes?
(Scripts also don't stop simply because the standard input is closed, so I assume the REPL also has to process ctrl-D explicitly)
(s/ctrl-D/ctrl-Z/g for Windows of course)

[limpid iron]

On Linux at least, on recent versions of Python, with the fancy new REPL, it appears that the REPL interacts with file descriptors 3 and 4, which appear to be dup'd from stdio, so interacting with stdin doesn't affect it.

See Lib/_pyrepl/readline.py:

    def __post_init__(self) -> None:
        if self.f_in == -1:
            self.f_in = os.dup(0)
        if self.f_out == -1:
            self.f_out = os.dup(1)

The commit message doesn't say why they did this, it's the big "new cool REPL" PR, maybe you can dig something out of the PR itself (GH-111567)

[limpid iron]

This seems perfectly reasonable as a design in general, some notes:

1. Presumably you're going to listen on/connect to localhost, make sure other users can't access it. There are a few ways to do this. An easy one is to put a random string in a file that's readable only by the current user and use that as a password for the API (which also gives you a place to write down which port to use, since the default Flask port might already be in use). You can also do fancier things like checking which user is connecting or listening on some magical local network thing that's not TCP, which are a little more secure but more complicated (I'm more familiar with doing this on Linux than Windows, I'm sure it's doable on Windows somehow)
2. If you automatically start a background server, ideally make sure to automatically shut it down after an idle timeout or give users a way to shut it down. (e.g., Bazel does this and there's a bazel shutdown command)
3. Related to that, make sure you have a plan to tell the server to exit and relaunch if there's been a version upgrade, otherwise a user is going to upgrade the software and wonder why it upgrade didn't seem to work.

[lyric quiver]

Wow, thanks a lot. For now I plan this to be an opt in feature or rather something your turn on through a 'switch button' on the frontend side. It will only work if the end user wishes it to.

The automatic turn off is interesting, will thinker about it

for the 3rd point, the 'server' won't run 24/7, it will run locally on the end user's machine while running after effects

just for faster processing through the python backend

I've noticed, based on user feedback, that at least 20 if not 30% of the time is spent on constantly importing and reimporting libraries each time python is executed by the end user to do some processing.

2.5s total import time, +- 1.5s for torch, 0.9 for tensorrt and the rest is misc. 0.5 to fully initialize the decode, inference and encode so +- 3s for the workflow to begin. Those 3s could be pretty big assuming the user ends up running the script every minute or so for some X workflow.

[long knoll]

perhaps this aspect of it isn't new? let me see if I can find something similar

Up to Python 3.12, the interactive shell of CPython was written in C as a special mode of the parser.

well, that's likely beyond my paygrade then

rather, in principle I know C but I'm not great at trawling through large volumes of it.

[limpid iron]

@serene fulcrum https://github.com/rust-lang/rust/issues/28179

[silk root]

Were weak symbols being produced? On Linux multiple identical strong symbols should be a link-time error (but with multiple weak symbols the choice is arbitrary).

[limpid iron]

see my very last comment in that issue, this is indeed with strong symbols on Linux!

[clear wigeon]

yes, i've kept a webserver alive precisely for thngs like this

[silk root]

Wow that is surprising. Also what are the chances that we wind up chatting here and on GH for the first time within 24 hours 🙂

[limpid iron]

oh! hi lol

and about symbols in dynamic libraries lol

I am actually kind of worried about how many libraries are "working" in our new builds of python-build-standalone because they do link -lpython but they're loading that libpython.so into the existing Python process which also has a copy of libpython in it and now you have two sets of libpython symbols....

[silk root]

Yeah I'm not sure. At this point I think we are strictly "on-topic" so perhaps we should move this discussion elsewhere

But I agree, there are definitely potential concerns here. I'm going to look at our auditwheel logic later today to see why we never caught this before

[limpid iron]

Context for other folks: python-build-standalone is now statically linking libpython into the bin/python interpreter, and also shipping a libpython.so for embedders, and this is causing some amount of problems with extension modules that are linking libpython.so, which is strictly speaking incorrect but is happening a little more than we expected, https://github.com/astral-sh/python-build-standalone/issues/619

(FYI we're at PyCon sprints and I'm going to get Zanie's attention in a sec to discuss in person, if anyone else is still at PyCon and has an opinion on the topic wants to wander into room 316 feel free. Also we're ordering pizza.)

[silk root]

I'll follow up in a DM

Sent some stuff, let me know if they're coming through (I'm still getting used to Discord 😄)

[lyric quiver]

Has anyone ever fooled around with Pydantic

Seems to be a static type checker for python(?)

Through type hinting

I wonder if it has any jazz to it other than visual ricing

[mild pollen]

I'm pretty sure it was more of a dynamic type checker.

[kindred hound]

Pydantic is a data validation library. Think of it as a super powered dataclass that guarantees that the data inside your class matches your type annotations with runtime validation.

[lapis solstice]

I evaluated both Pydantic and msgspec for the wire protocol handling in lmstudio-python. While msgspec fit that specific to/from Python builtins type conversion use case slightly better (at least for my brain), I still use Pydantic indirectly via typer for the CLI implementation.

FastAPI is another neat example of what Pydantic can offer in terms of runtime data transformations.

[lyric quiver]

Oh.. interesting

thanks for the insight

[forest fulcrum]

Hi all 👋
I'm looking for some advice on how to source some code review as a solo package maintainer.

Actually asking for the code review here is on option, as is the python discord server. r/python, r/learnpython don't look like good fits based on their rules.

Does anyone have any better / other suggestions?

Thanks in advance 🙏

---

In terms of the actual request prompting this, I'm the sole maintainer of python-json-logger [1] after taking it over from the previous creator/maintainer.

I'm looking at making some reasonably substantial changes to the API and have been now been doing this long enough to know that getting a second pair of eyes is a great idea for all of the usual reasons that getting peer review of ideas and code is good.

Actual changes can be found in the open PRs of the linked repo.

[1] - https://github.com/nhairs/python-json-logger

[lyric quiver]

Implementing this on the react side was an absolute nightmare that I don't even want to bother with for now.

Great idea in practice, it works overall but holy sh*t is it a nightmare to work with async functions that work based on hopes and prayers that your Input will also have an output at any arbitrary timepoint.

[long knoll]

I think that would be okay on the Python forum (discuss.python.org) but it might not get much attention

[dreamy hatch]

please fill out this silly little survey for Thomas, he's cooking up something fun!
https://social.coop/@Yhg1s/114613499974645384

[onyx spindle]

https://tenor.com/YqCL.gif

[lyric quiver]

Aight

giving up on decoders

all of them are bad, in a way 🤣

yuv420p simply can't be properly converted to rgb24

by any decoder I've tested so far

first is the raw input, the 2nd is basswood av

the only pure way is to decode with ffmpeg rawvideo in yuv420p format and then manually converting it to rgb24 on the python side but that shoots up cpu usage to the moon.

my 13700k goes from 40% usage to 100% usage

I think a C++ implementation that isn't reliant on the 'amazing' library that opencv is - is needed

torchaudio can't handle it, pyav can't, basswood-av can't, video-reader_rs can't

all of 'em get screwed up 🤣

I think it's related to the fact that yuv420p is 16-235bit range and rgb24 is 0-255

        yPlane = self.width * self.height
        uPlane = (self.width // 2) * (self.height // 2)
        vPlane = (self.width // 2) * (self.height // 2)
        reshape = (self.height * 3 // 2, self.width)
        chunk = yPlane + uPlane + vPlane

        process = subprocess.Popen(
            self.command,
            stdout=subprocess.PIPE,
            stderr=subprocess.DEVNULL,
        )
        self.decodedFrames = 0
        self.isFinished = False
        try:
            while True:
                rawFrame = process.stdout.read(chunk)
                if not rawFrame:
                    self.decodeBuffer.put(None)
                    break

                rawFrame = np.frombuffer(rawFrame, dtype=np.uint8)
                print(rawFrame.min(), rawFrame.max())
                rawFrame = rawFrame.reshape(reshape)
                rawFrame = cv2.cvtColor(rawFrame, cv2.COLOR_YUV2RGB_I420)
                rawFrame = torch.from_numpy(rawFrame)

                decodedFrames += 1

7 255
7 255
7 255
0 255

print output

                rawFrame = np.frombuffer(rawFrame, dtype=np.uint8)
                beforeMin = rawFrame.min()
                beforeMax = rawFrame.max()
                rawFrame = rawFrame.reshape(reshape)
                rawFrame = cv2.cvtColor(rawFrame, cv2.COLOR_YUV2RGB_I420)
                afterMin = rawFrame.min()
                afterMax = rawFrame.max()

                if beforeMin != afterMin or beforeMax != afterMax:
                    print(
                        f"before ({beforeMin}, {beforeMax}), after ({afterMin}, {afterMax})"
                    )

before (2, 255), after (0, 255)
before (2, 255), after (0, 255)
before (2, 255), after (0, 255)
before (2, 255), after (0, 255)

😭

Why are there so many color formats 😭

[long knoll]

There was a recent HN post about "falsehoods programmers believe about video" that I assumed was out of date but it probably isn't

[forest fulcrum]

I asssume you mean this one:
https://haasn.dev/posts/2016-12-25-falsehoods-programmers-believe-about-[video-stuff].html

10/10 read 😂

[silk jungle]

Guess which idiot forgot their security key PIN... it's me 🖐️. I usually don't use passkey so I never use the security key PIN, there is one service I do use passkey with UA verification enabled which I now can't easily sign into (desktop is doable, mobile is not). I really ought to consolidate all of my credentials properly so I don't do this again.

[marsh kite]

Long shot but if your security key is a yubikey and you happen to remember the admin key you can reset your pin if I recall correctly

[silk jungle]

it is a Yukikey, but I never set an admin key. It's also a security key (fido only) so I don't think that's even a thing for this key.

[lyric quiver]

hardware decoding is always faster than software decoding

that's funny

who believes that?

I needed about 8h to figure that FFMPEG's sws_scale is absolutely ancient and unmaintained

had to force zscale otherwise for some reason sws_scale would internally screw with the video. I assume it's something related to forcing bt601 yuv420 conversion internally despite the video being bt709

had fun debugging that ngl

[lyric quiver]

^ was caused by sws_scale

https://trac.ffmpeg.org/ticket/979 it's been a bug for 13years now.

https://patchwork.ffmpeg.org/project/ffmpeg/patch/[e-mail]/ and only now is it getting fixed 🤣

[lapis solstice]

"Hardware is intrinsically faster than software" is a misconception that can come up in lots of places (mostly due to the fact that all else being equal, a dedicated hardware implementation will be faster than software running on a CPU of comparable complexity. It's just that all else is rarely equal, so an absurdly overpowered multi-GHz silicon hot rod CPU can easily end up running rings around a poky underpowered dedicated hardware processor with fewer transistors than even one of the CPUs logic units).

The same issue comes up in cryptography, too (hardware crypto is often more focused on being secure than it is on being fast)

[lyric quiver]

went out of my way to remind myself just how 'bad' nvenc is

=== DECODING PERFORMANCE ===
NVENC H.264 (GPU Accelerated):
  - Decoding Time: 1.0634522 seconds
  - Average FPS: N/A
  - Speed: N/Ax

x264 CPU:
  - Decoding Time: 0.3385758 seconds
  - Average FPS: N/A
  - Speed: N/Ax

Decoding Performance Ratio (GPU vs CPU):
  - Time Ratio: 3.14x
  - Speed Improvement: x

I failed to capture the fps, but who cares, 1080p

3090 nvenc vs 13700k, it's pure comedy nonetheless.

I remember someone arguing that NVENC > CPU because it's made by NVidia 🤣

[quartz yew]

sounds like you need #venvstacks 😅🤣

[onyx spindle]

nah, what they need is wheel variants and wheelnext

[lyric quiver]

I ended up writing my own little script that downloads portable python, gets pip and downloads the requirements

dependency updates are automatic during runtime through a simple hash check on the requirements.txt

as barebone as possible without any extra headache 👍

[long knoll]

(I feel that. For me the "weight" of just Numpy often seems unbearable)

(I don't really like that that's my go-to example; it feels unfair. But.)

[lyric quiver]

tell me

pytorch cuda 2.7.1, tensorrt 10.12 is about 6.5GB of that iirc

pytorch 2.8 will be hopefully 300mb smaller, that's a big W for us storage peasants 🙏

[long knoll]

Does GitHub have any kind of command-line API to get releases? like say I want "the latest tag of https://github.com/foo/barproj in .xyz format"; do I have to figure out the URL for curl/wget myself?

[onyx spindle]

I think gh (official github cli) might have something?

@long knoll gh release list -R <repo> should give you what you want. with --json flag, it should be machine-parsable (gh also has a --jq for using jq expressions)

[shadow zealot]

gh is also open source so you should be able to figure out what it does underneath and replicate
https://github.com/cli/cli

[onyx spindle]

there is also Github API, which is public and documented https://docs.github.com/en/rest/releases/releases?apiVersion=2022-11-28

[long knoll]

[kind moon]

And if gh doesn’t have the thingy you need, you can gh api to make the request manually and get auth handled for you

[lyric quiver]

I did a fun experiment

https://github.com/NevermindNilas/rife-sonnet

Gave Sonnet4.0 and Cline Agent mode free access to a VFI arch

allowing to optimize it as much as it could

it went well the couple first prompts, but it starting acting dumb

it has like 6 different benchmark scripts

none of which are working and it will under no circumstance fix them

even if repeatedly told to fix them

Programmers are safe for now 😎

Instead of updating the already existing benchmarks, it simply created new ones 🤣

[lyric quiver]

@torn fjord

Having issues with torch.compile, do you happen to have a minute?

[clear wigeon]

https://www.codeflash.ai/post/llms-struggle-to-write-performant-code
shameless plug

After optimizing 100,000+ open-source functions we found that LLMs struggle with performance. We found that 90% of AI-suggested code optimizations are either incorrect or provide no performance benefits.
the revelant part

[long knoll]

I mean, how much reasonably performant code is in the training data :V

[lyric quiver]

I was able to copy some of it

Well the code it generated

And slap it into mine

1-2% performance gains

Absolutely not worth

I won't be able to use copilot for the rest of the month

[torn fjord]

Hey sorry! Just seeing this, if you're seeing a particular issue can you post an issue to pytorch/pytorch and I can CC the correct maintainers there?

[lyric quiver]

I figured it out, seems to have been a bug for over a year now https://github.com/pytorch/pytorch/issues/138961 https://github.com/pytorch/pytorch/issues/123177

multithread + torch compile ain't buddies 😎

I was hoping to use reduce-overhead due to the built in cudagraphs

[lyric quiver]

  - **SuperUltraCompact:** 80.80fps → 92.07fps - Mode: Max-Graphs
  - **Rife 4.6:** 140.48fps → 165.51fps - Mode: Max-Graphs
  - **Small Depth Anything V2:** 108.74fps → 131.19fps - Mode: Max-Graphs
  - **GMFSS:** 6.05fps → 9.51fps - Mode: Max
  - **OG Small Depth Anything V2:** 15.72fps → 16.02fps (likely OpenCV bottlenecked) - Mode: Max-Graphs

fwiw

torch.compile is amazing

[torn fjord]

Happy to hear you're having a good experience! 😄

[mighty flower]

Oh fun, an AI library thought it would be a good idea to make give one of their main modules the same name as a builtin: https://docs.modular.com/max/api/python/driver

[clear wigeon]

by the creators of: Mojo

(literally)

[lyric quiver]

Mojo, the project no one ended up using

(literally)

Can't wait to try it out on Windows in 2035 tho'

[shadow zealot]

Did anyone actually tried Mojo? It’s not like people tried it and it didn’t work. More like it just didn’t happen at all.

I always felt like it should at least be a viable option for extension packages.

[lyric quiver]

The actual ai bit is under such a restrictive license no one in their right mind will use it

I believe it was called Max engine

[shadow zealot]

That makes sense

[clear wigeon]

i tried using it at one point but it wasn't mature enough for me to keep using it.

I was looking at it this morning and it looks like they've changed their messaging from "replacing python with mojo" to "working alongside and helping better python" (https://docs.modular.com/mojo/why-mojo)

[junior narwhal]

my understanding is that it's exclusively for folks writing software that takes advantage of hardware capabilities and nothing more, only Python-like because the target audience is familiar with it and if the target audience was more familiar with something else then it would look like that instead

[junior narwhal]

I encountered something novel (?) that I've never seen before. There is a cool analytics service that I follow but never used. I mentioned it during an interview today with a database expert and so afterward I looked at their changelog to find that their CLI now supports Windows. The installation is beautiful: https://www.tinybird.co/docs/forward/get-started/quick-start#install-and-authenticate

The server looks at headers apparently and returns the install scripts under certain conditions rather than them being hosted at particular paths like others do. Such a great idea 😄

[lyric quiver]

What's the point then if you already have the almighty pytorch at your fingertips 😎

[junior narwhal]

it's not my area of expertise but it made sense listening to Chris on a tech podcast

[lyric quiver]

https://github.com/HigherOrderCO/Bend

I am really interested in thus

[lyric quiver]

It's dead on arrival, great in theory but noone ended up using it

Tensorflow and pytorch are good enough and have permissive licenses

[junior narwhal]

For those who use AI clients, I made an MCP server for any Python CLI based on Click https://github.com/ofek/click-mcp-server

[midnight shore]

Hello,
I am a bit unsure where I should ask this question.
I have a project using file locking. (https://codeberg.org/pierrot/oups)
I could see that virtualenv is using itself filelock library. https://github.com/tox-dev/filelock
I could also spot flufl.lock library with, as far as I could see, same purpose / equivalent features. https://gitlab.com/warsaw/flufl.lock

I could not find a comparison of them however.
filelock is actually mentioning flufl.lock in their documentation page, but without highlighting the differences between the 2 lib.
I dared to ask in their github, but there hasn't been any answer
https://github.com/tox-dev/filelock/pull/167#issuecomment-2960311151

As I see that filelock is a virtualenv dependency, I dare to ask if an investigation of both libraries has been made by virtualenv team and if yes, what have been the main reasons to retain filelock?

For my own work, working with Claude, and having it to ~somehow~ compare both of them based on their documentation, it kept telling me that flufl.lock, supporting NFS, is the solution most suited. But well, it remains an AI, and quite frequently, I am not super satisfied by it.

Thanks forr you feedbacks!

[long knoll]

I would try asking in #virtualenv .

[junior narwhal]

nested group/command declaration in Typer is so confusing for seemingly no reason, I dislike this very much and wish I didn't have to support it in a project I'm working on

[lyric quiver]

Conda has the ability to install cuda-toolkit with a simple command ( https://anaconda.org/nvidia/cuda-toolkit). I wonder how hard would it be to replicate a pip wheel with the same functionality?

I assume conda doesn't actually use a .whl but instead runs the .exe automatically.

this is primarily because some libraries like cupy also require cuda-toolkit installed in order to work.

https://stackoverflow.com/questions/67912832/why-cant-i-install-cudatookkit-10-1-using-pip-but-can-using-conda

apparently pip wasn't built for this 😭

time to make a .whl which downloads cuda-toolkit and runs it in a subprocess 🤣

[torn fjord]

nvidia has been trying to do some of this with their pip packages but unfortunately no windows support (we've asked before)

PyTorch's linux packages depend on the nvidia's cuda packages as an example

[lyric quiver]

Diabolical

It's time for a caffeine-dependent freelance developer to tackle the responsibilities of a large corporation.

[lyric quiver]

https://github.com/NevermindNilas/TheAnimeScripter/issues/126

I have been experiencing this weird issue typically when pip installing requirements within a subprocess, some few end users get a deadlock while installing them and I for the love of God can't replicate it no matter how much I try.

Could it be a weird Windows issue where it just decides that this subprocess is a virus or is installing malicious stuff or what could it be?

I tried everything I could think of, maybe my code is simply put subpar

https://github.com/NevermindNilas/TheAnimeScripter/blob/main/src%2Futils%2FdependencyHandler.py

This is my code

It may be the actual line in iter for loop that's causing it

[junior narwhal]

get rid of bufsize=1

[lapis solstice]

Blocking on subprocess pipes

[lyric quiver]

@junior narwhal

I thought you might find this interesting, new contender for the best decoder

idek how is something based around ffmpeg faster than ffmpeg itself, I think subprocesses may have some innate overhead, God if I know

Windows only rn, but damn is it fast

oh, it could be the fact that I use popen and not run

[junior narwhal]

that's cool

sure it only works on Windows?

[hexed briar]

https://discuss.python.org/t/99385 is giving me complicated feelings.

On the one hand, the user is clearly using LLMs as a disability aid (this was undisclosed earlier, but has been disclosed now).

OTOH, there are multiple things in there that are LLM generated weirdnesses that is gonna be exhausting to flag/correct, meaning it'll be significantly more effort to address it in the discussions.

It's uncomfortable enough that I feel like I should disengage. It doesn't help that I don't like the proposal either. 🙈

[lyric quiver]

yup

the dev is my buddy 🤣

[junior narwhal]

1. I think it would be bad for the community and users don't actually want applications exclusively installed via Python packaging tooling
2. I won't be commenting because I've been on the precipice of burning out for longer than I'd like and don't have the bandwidth
3. It seems to me that the primary issue is the lack of English fluency rather than dyslexia
4. Even if they were a native speaker, it's completely valid to publicly express qualms about AI usage. I can only point and click the on-screen keyboard one character at a time and I don't generate output using AI except for inline code auto-completion. The only exception is Gmail suggestions when a valid response is something like "Thank you."

[frank shore]

We're gonna see more and more of that and should at least start to think about a policy. I'd also have liked to see the disclosure up front since it's way too hard to miss at the bottom of a wall of text (which TBH, TL;DR).

I won't be commenting because I've been on the precipice of burning out for longer than I'd like and don't have the bandwidth
😢 - @junior narwhal reach out if you need someone to talk to.

[junior narwhal]

I'd like to see something not quite as strict as LLVM's new policy but equally as authoritative (admittedly that's about code and wouldn't exactly be applicable to this situation)

[junior narwhal]

commented with an alternative idea https://discuss.python.org/t/pre-pep-add-ability-to-install-a-package-with-reproducible-dependencies/99497/8

[long knoll]

I think the general idea behind the proposal is pretty clear despite the obvious LLM-isms.

FWIW I'm generally supportive of the idea of having a means to "install an application" from a wheel, and broadly that's my understanding of why there's a pipx in the first place. I actually wish that this actually was within the OP's intended scope — I'd like to be able to support an intermediate class of end users who know what Python is but don't want to manage venvs manually etc.

From my own experience: in the cases where dependencies resolved in the moment from wheel metadata are good enough, the pipx install experience is quite nice. It's slow and opaque, but both those things could be fixed. It would be much nicer still if that process could use a standards-compliant lockfile. But even today in many cases I can pipx install foo (having previously done sudo apt install pipx) and it Just Works. (Or similarly with uv, using their installer. Although actually I have a test copy of uv installed with Pipx!)

On the less technical side, uv has already shown that standalone Python binaries are quite workable (and PyBI should hopefully further improve that story in the future); doing a "proper" application provisioning system doesn't require a lot more than one of those plus a venv connected to it.

[junior narwhal]

try rethinking what you would do if a project did not offer a CLI entry point in wheel metadata

[long knoll]

well, no; I think of an "application" as specifically something which does so. (For GUIs you just need a .desktop file or a whatever-w.exe launcher, etc.) If people want to distribute something that can be launched that way, then of course it's on them to define an entry point for it, since Python isn't a language with an official blessed main() function.

I can understand why people want to install code with locked dependencies that is then used from a live environment, which is part of why I don't like that pipx bails out when there isn't one. But the pipx choice to do so is arbitrary. A tool can just as well set up an entry-point-less environment in the same way, and perhaps provide a wrapper for "run the python interpreter in this environment"

[lyric quiver]

Ignoring the LLM assertions, which, honestly, will become increasingly prevalent, (I barely recall composing a lengthy email independently without LLM-assisted revisions, thus my predisposition), I posit that the fundamental concept is providing pre-built, or readily deployable, software akin to a pip wheel. If so, why re-engineer existing solutions instead of streamlining and enhancing the Python compilation process—mirroring C compilation—and then distributing it using established software delivery methods? This approach would inherently ensure compatibility with numerous platforms, ranging from Steam to contemporary packaging solutions such as winget and choco.

Rewritten with the built in LLM in my phone 😂😂😂😂

[junior narwhal]

I barely recall composing a lengthy email independently without LLM-assisted revisions, thus my predisposition
did you never have to write a long email before 2-3 years ago?

[lyric quiver]

It was more of an exaggeration for comedic purposes

The LLM took the soul out of my message

[lyric quiver]

But to be frank, not really. I only really started writing long emails once I got semi-decent at coding and a few client work came in, anything else was usually already a template that I could easily copy and paste with few modifications.

[dreamy hatch]

"I didn’t have time to write you a short email, so I wrote you a long one." -- Blaise Pascal, 1657

[lyric quiver]

That's good, I will remember that

[junior narwhal]

just when I thought we were at peak syntax, nope. what a delightful proposal! https://peps.python.org/pep-0798/

[long knoll]

hmm. there's a certain elegance to that but it really doesn't add much on reflection

[boreal bramble]

It doesn't add much but it's the sort of thing I try sometimes and then get reminded it isn't actually a thing

very natural extension of the syntax

[lyric quiver]

I for one don't like it

https://tenor.com/view/why-fail-poor-choice-yikes-gif-19205909

[long knoll]

personally I'm more annoyed that I can't do e.g. foo, bar, baz = **{'foo': 1, 'bar': 2, 'baz': 3}

[kindred hound]

Is this unpacking based on the key name or order?

[junior narwhal]

order

[kindred hound]

That seems a bit niche then, doesn't it?

[junior narwhal]

no that is how the underlying methods work

[long knoll]

I'm imagining something based on key name; I guess ofek is referring to the PEP... ?

[junior narwhal]

oh yes my bad

[kindred hound]

I was referring to Zahlman's foo, bar, baz = **{'foo': 1, 'bar': 2, 'baz': 3}. I should have been clearer 😅

[long knoll]

yeah, I imagined that it would look for keys in the dict matching the variable names

but I can imagine disagreement over what to do when they don't match 1:1, so :/

[kindred hound]

Yep

[kindred hound]

This would also mean that the proposed syntax will work only with dicts whose keys are legal variables.

[long knoll]

unless those are ignored, which is one of the ways of handling it

[mild pollen]

Presumably it would be the same semantics as when calling functions.

[long knoll]

Ah! of course

so you get an error that you can suppress by adding **_ to the lhs

now that you say it, I must have already thought of it and forgotten again

[kindred hound]

I think the best way to write this would probably be reusing match case's appraoch

{"foo": foo, "bar": bar, "baz": 3} = **{'foo': 1, 'bar': 2, 'baz': 3}
{"foo": foo, **_} = **{'foo': 1, 'bar': 2, 'baz': 3}
{"User-Agent": user_agent, **response} = **{"User-Agent": "requests/v1", "status_code": 200}
{None: value, **_} = **{None: "NoneType", str: "string type"}

[lapis solstice]

These days, if we were going to offer more complex destructuring assignments, I'd propose allowing match expressions along the lines of match pattern from value (throwing an exception if the destructuring fails) as syntactic sugar for code like:

match value:
    case pattern:
        pass
    case _:
        # The compiler would make the pattern details available to the error message
        raise ValueError(f"Value {value!r} does not match pattern {_pattern_text}")

So the above examples would be:

match {"foo": foo, "bar": bar, "baz": 3} from {'foo': 1, 'bar': 2, 'baz': 3}
match {"foo": foo} from {'foo': 1, 'bar': 2, 'baz': 3}
match {"User-Agent": user_agent, **response} from {"User-Agent": "requests/v1", "status_code": 200}
match {None: value} from {None: "NoneType", str: "string type"}

[lyric quiver]

INPUT: C:\Users\������\Desktop\���� � ��������\AE �������\TAS-PreRenders\Leonardo_DiCaprio_Pointing_Meme_JsRhB.mov
OUTPUT: C:\Users\������\Desktop\���� � ��������\AE �������\TAS-Depth\Depth_h55b.mp4

...

Error installing requirements: 'utf-8' codec can't decode byte 0xcc in position 68: invalid continuation byte
Error installing requirements: 'utf-8' codec can't decode byte 0xcc in position 68: invalid continuation byte

how do you guys typically handle non utf-8 compliant paths

Arabic & Japanese people have been killing me lately ngl 🤣

[long knoll]

you're sure the problem is with the path, not the file contents? (I guess so, if they're movie files...) Where are they sourced from? And what does the stack trace look like?

that said, maybe os.fsdecode helps: https://stackoverflow.com/questions/45724240

[lapis solstice]

If they're using conda on Windows, they're going to run into problems regardless (due to https://github.com/ContinuumIO/anaconda-issues/issues/10595)

Otherwise, using either os.fsdecode or errors="surrogateescape" may prevent the initial read codec error. Whether that's sufficient depends on how the paths are ultimately consumed (e.g. an external subprocess may still object to them)

[lyric quiver]

Portable python, no conda

[lyric quiver]

Will look into that

[lyric quiver]

User dependent, can't accurately provide a response to the source of the video and how they handle their paths

It's just that they use non utf-8 / Japanese , Chinese, Arabic or even emojis as characters in their windows username or video path and it completely breaks when parsing said characters

[long knoll]

Windows does it all in UTF-16 IIRC.

[lyric quiver]

I think I figured it may actually not be a python issue but the frontend which is typescript and react

Altho' my implementation on the backend was still poor and I improved it now.

[kindred hound]

It's not even a guarantee that windows paths are going to be valid utf-16

they can be any arbitrary sequence of non-zero 16bit values, likely valid utf-16 but not guaranteed to be so.

[dark anchor]

unix paths are arbitrary bytes

mercurial took extreme pains to always represent paths as bytes for that reason

takes, I guess

[lyric quiver]

https://github.com/Trentonom0r3/CeLux/releases/tag/v0.7.0

Latest release has linux support

finally a good Video Decoder for Python

restrictive license tho', so this may be a bummer for some.

Latest benchmarks ran on Windows, absolutely destroys the competition.

[dark anchor]

I think the pytorch team is working on https://github.com/pytorch/torchcodec

[lyric quiver]

https://github.com/NevermindNilas/python-decoders-benchmarks

check out the release

it's on the same level as torchvision in terms of speed.

running a new run with the hopefully working celux on linux.

[dark anchor]

I’ve had to touch this ecosystem a little bit to get people to ship free-threaded wheels

ffmpeg is thread-safe so that helps a lot

[lyric quiver]

ffmpeg is amazing

[lyric quiver]

https://github.com/pypa/pip/pull/13489#issuecomment-3157124611

What the actual f is going on in the pip repository

🤣 🤣

Absolutely hilarious behaviour

[torn fjord]

While I emphathize with someone being frustrated at LLM generated contributions I feel like the behavior of following someone around and harrassing them is over the top

[long knoll]

Inclined to agree in general; but did this happen more than once?

[torn fjord]

Looking at the users history it does seem like a pattern but also looking at the PR it doesn't seem like the worst PR in the world

[long knoll]

I meant, did the callout happen more than once

[torn fjord]

oh unsure

[lyric quiver]

Based on the wording I got the idea that the user was actively chasing the OP because they made repeated LLM generated contribs

But honestly, the OP is using his own money to help Open Source software improve, so I don't see anything wrong here. Be it LLM or not.

There is the possibility of acquiring tech debt but at such a small contrib that gets heavily verified before merging. I am p sure it's safe to assume the OP means no harm

[junior narwhal]

the person is hiding a fundamental change and only talking about performance of another change, I'm going to comment

[lyric quiver]

O

Keeping my eyes on the PR

[silk jungle]

y'all, can you not use pip as your avenue to battle with AI nonsense?

that PR has been derailed enough

[lyric quiver]

It's funny 🤣

[junior narwhal]

commented, not sure why they added that part to this PR

[torn fjord]

LLMs are notorious for doing too much

[junior narwhal]

I was willing to give the person the benefit of the doubt but after looking more closely at other contributions, and the current one, I would just not accept the PR outright.

[clear wigeon]

yes.

[junior narwhal]

Wild https://github.com/schellingb/ZillaLib/pull/21#issuecomment-3124536210

[torn fjord]

big yikes, then I agree on the general stance that's being taken.

[lyric quiver]

Jeez, having been told repeatedly that you are a nuissance and yet still doing it.

Couldn't be me 😎

1200 hours of prompting LLMs 🤣

so he lives in "Virgin", USA and potentially leaked his personal phone number on his website

crazy

[clear wigeon]

this guy has gotten PRs merged in multiple repos, so it's a number game at this point

[kind moon]

Looks like 3/21 so far this month?

[dark anchor]

i guess github has a whole AI assistant product that’s integrated into IDEs - they’re not exactly discouraging this behavior :/

like, if you reported that account to github, is it violating ToS? it’s clearly wasting a bunch of people’s time, but I wonder if that’s enough for them to be banned.

[silk jungle]

I sincerely doubt that their behaviour would be bannable given Microsoft/GitHub's big push on AI

[clear wigeon]

They do? Which one?

Are you talking about copilot?

[dark anchor]

yes

https://code.visualstudio.com/docs/copilot/overview

[torn fjord]

If it's done tastefully its fine, but to openly admit that you don't know what you're doing and you're going to submit code anyways is definitely a... choice

[lyric quiver]

Pytorch 2.8.0

Dropped support for like a dozen gpus

And it somehow ended up with an even larger wheel?

That makes no sense mathematically

Did NVidia decide to add 500 mb of minified code to cuda 12.9 or what

[lyric quiver]

Also my pr didn't get mentioned in the new release 😔

I feel like a fake

[long knoll]

I mean you can examine the wheel contents easily enough

[dark anchor]

come contribute to numpy, plenty of things to do there. The release notes are prepped by a dude named Chuck who isn’t employed by a megacorp

[lyric quiver]

I feel like you have something against pytorch

[dark anchor]

nothing against pytorch, just trying to communicate that numpy development has a little bit more of a personal touch

i’ve contributed stuff to pytorch

my PR getting reverted because tests internal to facebook failed wasn’t very fun

[lyric quiver]

I like numpy as much as the next guy

I use it actively through opencv actually and a few other things

But to put it simply, cuda is better

But the wheel sizes have started being outraging all things considered

Tensorrt is like 2.5GB, pytorch is like 4GB now

[lyric quiver]

I take that back

🤣

[lyric quiver]

I just re-discovered this cool toy namely CompactGUI

it does amazing jobs

didn't affect start-up time from my very very minimal test

seems to wrap the already built in command compact from Windows

maybe something worth testing as a post-optimization step for packaging

[somber badge]

byebye

[lapis solstice]

Enterprises gonna enterprise (from contextlib2's Python version download stats for the past 30 days):

[lapis solstice]

Although I guess the second highest count is the most enterprisey one (at least there's no Python 2.6 still on the list):

[dreamy hatch]

enum34 (forJuly):

┌──────────┬─────────┬───────────┐
│ category │ percent │ downloads │
├──────────┼─────────┼───────────┤
│ 2.7      │  35.69% │ 2,387,369 │
│ 3.8      │  22.54% │ 1,507,936 │
│ 3.9      │  14.04% │   938,974 │
│ 3.10     │   9.84% │   658,181 │
│ 3.11     │   5.35% │   358,106 │
│ 3.7      │   5.18% │   346,836 │
│ 3.13     │   2.79% │   186,357 │
│ 3.12     │   2.36% │   158,156 │
│ null     │   1.63% │   108,935 │
│ 3.6      │   0.47% │    31,439 │
│ 3.5      │   0.11% │     7,254 │
│ 3.14     │   0.00% │        82 │
│ 3.4      │   0.00% │        26 │
│ 3.3      │   0.00% │         4 │
│ 2.6      │   0.00% │         3 │
│ 3.1      │   0.00% │         2 │
│ 2.8      │   0.00% │         1 │
│ 3.2      │   0.00% │         1 │
│ Total    │         │ 6,689,662 │
└──────────┴─────────┴───────────┘

[fierce horizon]

Meanwhile, the scientific Python community follows SPEC 0 and bumps minimum versions aggressively (Python 3.11 is officially supported, so new versions of our stuff currently support 3.11+)

[onyx spindle]

2.8?!

[marsh kite]

probably https://github.com/naftaliharris/tauthon

[dreamy hatch]

just have a look at some of these versions ! it only takes one download of a custom Python build. the 3.2026 and 3.24 were probably me when doing feasibility tests for PEP 2026 🙂

[marsh kite]

We found a time traveler! Python 133.7 is pretty advanced!

[lapis solstice]

contextlib2 bumped its minimum version a while back in order to bring in the async components without it being painful, but the default pypinfo query for Python versions aggregates across all releases of the specified project. Showing the contextlib2 version as well gives the expected split of newer Python versions using the newer release.

[long knoll]

I guess downloads like that happen because it still works and nobody has a reason to test "wait, can't we just remove this?".

if you think that's wild, check out the downloads of six. (Oh, pypistats is still down...)

[junior narwhal]

is that service just dead now? I feel like it's been down for over a month

[marsh kite]

Sounds like the PSF will be taking it over: https://github.com/crflynn/pypistats.org/issues/82#issuecomment-3189840848

[dreamy hatch]

Here's the pip installs for six from PyPI for July 2025:

┌──────────┬─────────┬─────────────┐
│ category │ percent │   downloads │
├──────────┼─────────┼─────────────┤
│ 3.11     │  22.54% │ 120,264,931 │
│ 3.12     │  16.25% │  86,739,386 │
│ 3.9      │  15.67% │  83,653,849 │
│ 3.10     │  14.47% │  77,210,207 │
│ 3.8      │  10.63% │  56,723,400 │
│ 3.13     │   7.68% │  41,003,427 │
│ 3.7      │   5.28% │  28,160,966 │
│ null     │   5.19% │  27,674,614 │
│ 3.6      │   1.22% │   6,520,802 │
│ 2.7      │   1.02% │   5,468,060 │
│ 3.5      │   0.03% │     153,406 │
│ 3.14     │   0.02% │      91,367 │
│ 3.4      │   0.00% │      10,128 │
│ 3.15     │   0.00% │       2,144 │
│ 3.3      │   0.00% │         508 │
│ 2.6      │   0.00% │          14 │
│ 3.1      │   0.00% │           2 │
│ 2.8      │   0.00% │           1 │
│ 3.2      │   0.00% │           1 │
│ Total    │         │ 533,677,213 │
└──────────┴─────────┴─────────────┘

[lyric quiver]

Funny eol python is still being actively used

Crazy to me how people would support legacy releases

[dark anchor]

I remember in 2017 or so being at a NumFOCUS summit where an executive from JP Morgan was telling the group about their multi-million line “risk platform” that ran on Python 2.7. And JP Morgan is a relatively well-resourced organization! There’s going to be Python 2.7 code running for just as long as COBOL will be running.

[onyx spindle]

six is keeping up because of packages like python-dateutil, which still maintain compatibility with py2 (for reasons unknown)

[kind moon]

Even Azure’s Python SDK still imports it in some places.

[long knoll]

(I've met quite a few people who still resent the 3.x migration and continue to complain about it today)

[dreamy hatch]

and it's back online! https://pypistats.org https://github.com/psf/pypistats.org

[lapis solstice]

I'm still mildly surprised nobody paid Red Hat to extend their formal Python 2.7 support past 2024. Although I guess the public EOL being reached doesn't preclude private deals.

[dreamy hatch]

Still available from ActiveState... https://docs.activestate.com/activepython/2.7/

[fierce horizon]

That's wild. Still? I remember when Click whined about the Unicode support in Python ≤3.6 years ago, and thought that they were kind of the last holdouts.

[long knoll]

The support was definitely not perfect out of the gate.

But I was using it happily starting with 3.2, and by 3.4 I was already wondering why people were dragging their feet.

[onyx spindle]

until 3.6, Python 3 wasn't as fast as 2.7

[kind moon]

(By how much?)

[long knoll]

Overall, yes. But at the time, I got to be pleasantly surprised by this algorithmic improvement: https://stackoverflow.com/questions/9815252/

(for what it's worth, probably almost everything I would write today is 3.6-compatible.)

[lyric quiver]

But what is really the point in using eol python

I for one migrated my project through all the versions from 3.9 to 3.13 and had no issues whatsoever, just better performance, better startup and even more goodies I could use

[dark anchor]

the 2.7 port is nontrivial, especially for a really ancient codebase or a codebase that had to do a lot of string processing or played fast and loose with the string/byte distinction. Some organizations can’t or won’t justify the resources needed to port if the code works as-is.

The old 3.x versions are harder to justify but more of the same probably “if it ain’t broke, why fix it?”

also, it might surprise you, but there are definitely corporate Python codebases that no single human could update

[lyric quiver]

Never thought about it

I am surprised python got where it got despite being considered a scripting language by many

[long knoll]

Being able to provide interfaces to useful, well-designed C libraries caused people to perceive "scripting" as radically more useful than it was before.

and it was IMO better able to do this than e.g. Perl through whatever FFI that has, because of the better UX of actually writing a few hundred lines of Python.

[frank shore]

I'm banking on it for my retirement. I plan to charge exorbitant hourly fees to support business critical Python 1.5.2 code.

[lyric quiver]

🤣

There has to be some semi automated way to migrate to newer releases

Those ancient pythons must be filled with bugs

And ways to penetrate it

[marsh kite]

But see, these systems are critical, they cannot go down

[lyric quiver]

I read somewhere that YouTube / Google also relies heavily on python

Not sure about the version they use

[onyx spindle]

As funny as that sounds, I am sure there are some businesses that actually run Python 1.* for some mission critical apps and can't upgrade for reasons

[lapis solstice]

python-modernize and python-future both do a pretty decent job of porting from pure Python 2 code to the common subset of Python 2 & 3.

The problem some legacy code bases face is that they don't have good automated tests, and any bugs they do have are frequently known quantities that other components are already working around. It makes any large scale change a massive project, even if making the change itself can largely be automated. If a system isn't suffering from the horror show that is Py2's unicode text handling, not imposing a performance bottleneck that can't be solved by "throw more or better hardware at it", and not exposed to the kind of untrusted inputs that make CVEs a big concern, then "keep it limping along in maintenance mode until the entire system gets replaced entirely" can actually make a certain sort of sense.

If anyone is interested in learning more about the history of the transition, https://python-notes.curiousefficiency.org/en/latest/python3/questions_and_answers.html may be of interest.

[fierce horizon]

any bugs they do have are frequently known quantities that other components are already working around
Skill issue. 90% of the workarounds that I write just deactivate themselves when things get fixed upstream and the rest will loudly crash.

I don’t think any workaround code block I ever wrote has led to a subtle bug when upstream fixed things.

[lyric quiver]

For me I found it difficult to write test cases because I mostly work with video I/O

So really the only thing that can break is the output video which I have to manually visualise to see

Like a loss of clamp to 0-1 range

And sometimes even a cuda race somewhere can lead to malformations in the whole pipeline which is really hard to detect without using Nvidias inspector

I would love to write C/I tests tho'

[lyric quiver]

Interesting

Thanks for the insight

My project is only like 50-60k lines rn, never worked on a project so huge that migrating it would be a challenge

Even then, probably a divide and conquer approach would I assume work

[vast wren]

Think of it this way. Porting is a risk, never porting is also a risk. Sometimes the risk of porting is greater than the risk of not.

[lyric quiver]

I was thinking for a small little side project to make a monthly github action runner that benchmarks the import time costs for the top 50 ( initially, then maybe more ) libraries out there just for the funs.

I wonder if such a thing has been attempted before.

Would be an interesting database to reference, at least for me

[jaunty marlin]

Unfortunately GitHub Actions is very noisy

You need a bare metal machine for realistic benchmarks

[lyric quiver]

sad

maybe through a local docker setup 🤔

but it will be annoying having to run that manually every so often

was hoping for something more automated

and reproducible

[torn fjord]

Yeah github actions is typically on shared machines

[lyric quiver]

What if one would do say... 5 cold and warm runs

And average them out

[dreamy hatch]

Spotted in Athens

[dark anchor]

TIL that https://pypi.org/404 has an easter egg (scroll down)

[onyx spindle]

it goes back to the previous implementation of PyPI, which was called cheeseshop

[vast wren]

To be clear, it’s every 404 page 🙂 that’s just a convenient url for a 404 lol

Couldn’t let PyPI take itself too seriously

[frank shore]

Thank goodness! IIRC it was MvL who thought "Cheeseshop" was too unprofessional 😛

[vast wren]

I think I waffled on the name, but mostly I just wanted it to have a name 😄

[kind moon]

Hey

Warehouse is a great name

Super explicit and self-defining

[vast wren]

I talked so much about warehouse on Twitter that at one point an actual warehouse (like… the kind with forklifts and what not) followed me

[frank shore]

Warehouse for sure is a great name for the software!

[fierce horizon]

I agree, but I hate the progressive de-fun-ization of everything

Like the requests docs were so cute with their photos and the konami code and everything, and as soon as PSF took over, all that was removed (even the logo for a minute, but luckily that was put back!)

[onyx spindle]

requests has a logo?

[fierce horizon]

Yeah, Kenneth Reitz has it as a tattoo on his body. https://requests.readthedocs.io/en/latest/

[onyx spindle]

damn, that's dedication to the project lol

[mighty flower]

IMO this is a conflation of two different movements that tend to happen at the same time but doesn't need to. First is the actual progressive rewording of things, i.e. understanding whether such words do actually have problematic roots. Second is corporatization of things, which actively seeks to make everything bland, for a variety of reasons, and therefore de-funs things. In my experience the later tends to use the former as a reason or excuse to make changes that are far outside the scope of the former.

Sometimes there are actual technical reasons, rust use to give some random error messages, but it was genuinely bloating size of executables

(when I say used to I think I mean pre-1.0 days)

[fierce horizon]

Haha I was literally about to post this! https://github.com/rust-lang/rust/issues/13871

It’s a good example, because of course these 2k of text shouldn’t be in every binary. On the other hand, why not have them in debug builds? I’m all for quirky when there’s near-zero impact for users.

We’re in a forum about a programming language named after Monty Python of all things.

If we don’t do silly humor, then who?

[mighty flower]

I did just get pointed to the 404 easter egg on PyPI: https://pypi.org/404 , I'd not spotted it before because it requires you to scroll down

[raw lantern]

3.14’s venv has absolutely no silly humour whatsoever. That would be absurd!

https://youtu.be/qfwTRVnO5No

[desert verge]

Do you have to be forklift certified to contribute to pypa development?

[timber sphinx]

It definitely helps to maneuver all the pallets of gold that are sent in... 😂

[clear wigeon]

without any context this sounded like the pypa org is rich in gold

[kind moon]

Aren’t they?

[long knoll]

they're rich in Fastly's good will, I suppose.

[clear wigeon]

nope.

if it was well funded then astral would've never needed to happen

[long knoll]

the entirety of the PSF gets only a few million annually in funding, and a big chunk of that goes to making PyCon happen, basically

Mozilla and Linux are basically the only projects in the FOSS world that get major funding.

as far as I've been able to discern, at least.

[mighty flower]

Wow: https://pypi.org/project/nucliadb/6.8.1.post4961/#history

[onyx spindle]

they use github.run_number from CI as their build ID and paste that into .post{} in version

[kind moon]

Cool
but why

[onyx spindle]

feel free to ask them

[long knoll]

I guess that way they don't have to tell users a separate process to get "nightly" builds? :/

https://pypi.org/project/boto3/#history is also impressive

for that matter, and that's one of the most popular packages.

[onyx spindle]

the boto family is the nightmare case for all dependency solvers

[mighty flower]

It builds character for dependency solvers

[lapis solstice]

I'm pretty sure that was mostly me (although I do recall MAL being in favour of preferring the mundane name - I don't think MvL really weighed in one way or the other). For me, it wasn't about hating fun, it was the fact that trying to introduce the "Cheeseshop" naming up front raised a pointless additional barrier to entry on an already complex topic. I'm fine with it as an alternate "in joke" name, but as the sole (or even preferred) name it was a genuine problem when it came to improving the clarity of introductory documentation.

(the boring name also works better for fundraising purposes, which I believe was the angle MAL was mostly approaching it from)

[languid crystal]

Hi everyone,

I'm organizing a "packaging themed" Python group presentation night in Berlin, Germany in November. I'm wondering if there's anyone here with contacts in Berlin that wouldn't mind presenting on this topic. Feel free to thread here or DM me if you can help out 😃

[dreamy hatch]

Berlin

[desert verge]

I .... I don't think I've ever scrolled down. It looks so non-scrollable....

[vast wren]

Wouldn’t be much of an Easter egg if it wasn’t hidden a bit 😉

[mighty flower]

Reading all the Ruby community issues today has been wild, I have no prior knowledge of the Ruby community and organizations, so I'm mostly familiarizing myself with their structure as I'm reading about it

[desert verge]

What happened with Ruby??

[onyx spindle]

I assume they are referring to a recent turmoil around RubyGems maintainership.
Writeup by one of the maintainers: https://pup-e.com/goodbye-rubygems.pdf

[desert verge]

Sheeesh

[mighty flower]

I read this overview today: https://joel.drapper.me/p/rubygems-takeover/
And also this: https://gist.github.com/simi/349d881d16d3d86947945615a47c60ca

[desert verge]

Wow lol

When the Advanced Custom Fields plugin was stolen by WordPress, DHH said “This is totally crazy. Like if the operators of rubygems dot org just decided to expropriate the official Rails gems, hand over control to a new team, and lock the core team out of it. We’re in uncharted and dangerous territory for open source now. What a sad sight.”

[mighty flower]

There's also some (possibly related?) Ruby community drama seems to stem from "DHH", it's possible a long term issue, but recently he wrote a blog post about London which I personally find incredibly insulting in a number of ways, I really don't think the contents are worthy of discussing here even in this "off-topic" section, so I'm not going to link to it, but if you're interested it should give you some context on the personalities involved

[desert verge]

I have already been reading it

Unfortunately

[vast wren]

oof

that sucks all around

[desert verge]

I then got sidetracked into python drama when trying to see if we have any political rifts within python..

[clear wigeon]

i went thru that rabbit hole just yesterday.

it's eye opening and demotivating tbh

[desert verge]

I gave up before finding a political rift inside python

[clear wigeon]

good for you (mean it seriously, not ironically)

[desert verge]

I'm sure it exists

But there's enough of minority leadership in Python across the varying organisations that I'm not so worried about that

I'm more worried about the Astral rift and tools relying on a company

[clear wigeon]

i think someone recently in the astral discord said something like

i am both astral's biggest hater and supporter

and i gotta say i agree.

[desert verge]

That was me

[clear wigeon]

😆 😆 😆

small world

on one hand i gotta say i love the "innovation" or fire in the ass astral has caused in other projects to optimize for performance and the "innovation" it's driving within the python community, working with it to draft up / push foward PEP and so on.

on the other hand, forced pushes can cause the fuel to get used more quickly and burn out people

[desert verge]

There's not really good competition for uv, at least for my needs.

[clear wigeon]

i think we need to remember that uv is quite new.

it takes time to competitors to rise up

[desert verge]

Poetry's forcing of poetry.lock being Forever Accurate and requiring me to requires-python with an upper bound is a non-starter for anything I might publish to pypi.
PDM is incredibly slow when locking and resolving.
Hatch is pretty cool but is a little odd sometimes, and I just haven't been able to get started with it

[clear wigeon]

Hatch is pretty cool but is a little odd sometimes, and I just haven't been able to get started with it

lol that was my experience as well, it's good but just not quite there yet

though some of the times i had those experiences it was just skill issues of mine

[desert verge]

Ofek is looking for another maintainer so that's good

[mighty flower]

If anyone wants to pay me a NYC dev salary for a few years, I can make, maintain, and improve, a fast pure Python resolver library that lots of the existing Python packaging applications could plug in to 😜

[desert verge]

...would you take a raincheck?

[clear wigeon]

https://xkcd.com/927/

xkcd: Standards

Fortunately, the charging one has been solved now that we've all standardized on mini-USB. Or is it micro-USB? Shit.

[desert verge]

there is resolvelib but it seems to abstract a bit too much away from what I can tell

[mighty flower]

There aren't a lot of competing standards, in Python there's resolvelib, and poetry vendored mixology, and that's it?

[desert verge]

huh

[clear wigeon]

wait, for a few years?

[mighty flower]

Yeah, I don't want to be jobless, lol

[desert verge]

uhhh what about 3 months rural rate, take it or leave it

[clear wigeon]

make it 6 months with my help

[mighty flower]

Yeah, Poetry vendored in mixology and develops it on their repo, the published one is more PoC as I understand it, so it's not designed to be used outside Poetry, as I understand it

[mighty flower]

Abstraction and flexibility, it is as an unopinionated DFS resolver that let's the provider plug in basically anything and resolvelib doesn't try to make any requirements about behaviour the provider should guarantee, this makes it impossible to move to learning algorithms like pubgrub

[desert verge]

like tomlkit and cleo used to be :^)

pdm has a lot of similar libraries

[mighty flower]

Yeah I almost ended up using https://github.com/pdm-project/dep-logic from PDM for some resolving tricks in pip, but eventually I realized there were too many issues, and I'm trying to add minimal functionality for my needs in packaging

[desert verge]

yeah :/

I partially want to submit some fixes to pdm but I don't even know where to start

[clear wigeon]

try hammering in the solution with a screwdriver

[desert verge]

At the moment I don't even use pdm or black or flake8 but I still follow pdm and black's development

-# flake8 left off because its sooooooo slow and the maintainer is uh

same with prek and pre-commit

I never want to return to pre-commit

[clear wigeon]

pre-commit was pissing me off just yesterday, though not sure if it's pre-commit faults or ruff

prek looks neat, though very early on

[desert verge]

I already use prek in production lel

[mighty flower]

I've started evangelising for prek to anyone who will listen

[mighty flower]

Same, lol

[desert verge]

I don't even touch pre-commit anymore

[clear wigeon]

lmao

[vast wren]

they definitely exist, but afaict nothing anything remotely like rubygems has ever really happened to my knowledge

[desert verge]

for sure. But to the best of my ability I haven't seen anyone in the realm of a python dhh

[vast wren]

I think we've been pretty lucky that most people in Python are generally pretty reasonable and the folks in various places of power have generally been pretty ok

[desert verge]

ye

[clear wigeon]

inb4 you jinxed it /hj

[vast wren]

I don't honestly know how RubyGems hosting is setup, they've made a few statements that the code and the service are different.. .which is obviously true. I'm kinda curious how true it is, but that's nothing but morbid curiosity lol.

[desert verge]

There are a few people whom I think there's great distrust and dislike of, but I don't believe those same people are transphobic.

[vast wren]

like Warehouse is technically distinct from pypi.org, and you could, in theory, deploy Warehouse somewhere else

[desert verge]

Yeah, RubyGems (the code) and RubyGems (the service) are different, and.... I should just check if there's workflows in RubyGems (the code) that deploy directly to RubyGems (the service)

[vast wren]

but Warehouse is very much designed to be pypi.org and only pypi.org

so there's very little affordances made to be anything but pypi.org

[clear wigeon]

There are a few people whom I think there's great distrust and dislike of, but I don't believe those same people are transphobic.

i strongly believe that some of those people who fit the bill, simply are products of their times, nothing more than that, and it can come off as a cultural shock.

[vast wren]

I'm still catching up on it, but it's kinda wild that it appears part of this was caused by some of the rubygems / bundler maintainers making a new tool? lol

that's wild

[desert verge]

rv yeah

which is like... lol

[clear wigeon]

hmmm, maybe it's competing with rubycentral's products?

[vast wren]

Fun fact, the reason I'm a PyPI admin is because I made a competitor to PyPI, and kept sending patches back to PyPI because I kept finding bugs

[desert verge]

wow lol

[vast wren]

and Richard got tired of getting .diffs in email (because I refused to learn how to operate mercurial)

[mighty flower]

Oh, I saw rv got announced, I didn't know what was in the mix of this drama

[vast wren]

and was like, why don't I just make you an admin

[clear wigeon]

(because I refused to learn how to operate mercurial)

to be fair, perfectly sane.

[desert verge]

rv is asdf for ruby afaict

[vast wren]

https://bsky.app/profile/rmfranca.bsky.social/post/3lz7alpobhc2x

Rafael França (@rmfranca.bsky.social)

-# ↩ Ufuk Kayserilioglu (@ufuk.dev)
Yes, I agree. And some of the “admins” even announced publicly many days ago they were launching a competitor tool and were funding raising for it. I’d not trust the system to such “admin”.

Bluesky · Sep 19, 2025, 4:05 PM

[clear wigeon]

called it 🥴

[vast wren]

I guess that Rafael person is part of spotify or something

er shopify

the other sify

[desert verge]

yeah don't confuse spotify (the company that pays their creators like crap (not to be conflicted with youtube)) with shopify (the company that pays all of the youtube creators (the aforementioned ones who don't get paid enough) in exchange for sponsorships and also organises a hostile takeover of rubygems)

rv is basically just uv (which is basically just cargo) but for python but for ruby

[vast wren]

it's kinda interesting though that "make a competitor" was such a big threat, when in python land we're just like, "here we're gonna make it as easy as possible to make competitors, knock yourself out!"

[dark anchor]

shopify is also the company that is happy to be the e-commerce provider for literally anyone, including nazis

[desert verge]

My mind has gone, hopefully y'all can still follow what I'm writing

[kind moon]

[desert verge]

so like cloudflare and hetzner but for e-commerce

I think I might be on my "hate big tech" arc today

[dark anchor]

cloudflare seems to be doing a lot to support DHH’s projects too, they recently announced support for the omarchy project

[clear wigeon]

OOL- what's DHH?

[vast wren]

the creator of ruby on rails

[desert verge]

the right-wing shopify board member who also iirc had a big role in ruby on rails and <speculation> probably spearheaded this takeover

[clear wigeon]

ahh

[dark anchor]

also related: https://rustfoundation.org/media/rust-foundation-signs-joint-statement-on-open-source-infrastructure-stewardship/

[desert verge]

What I want from a language is "no drama"

But that's impossible. Like someone once said, When you put three developers in a room together and ask them to answer a yes or no question, you're going to get three answers you didn't expect.

[dark anchor]

https://openssf.org/blog/2025/09/23/open-infrastructure-is-not-free-a-joint-statement-on-sustainable-stewardship/

[desert verge]

GitHub (npm) or Microsoft (NuGet)
oh yeah right, Microsoft owns a whoooole bunch of the toolchain now

open source IDE, anyone? 😔

[kind moon]

VSC is technically open source

[desert verge]

no. Its not.

[kind moon]

Time to go back to vim?

[desert verge]

It has so much microsoft magic added to it when its built that its not

[onyx spindle]

Poetry doesn't force upper bound on Python for some time now

[desert verge]

ooh nice!!

I'll have to try poetry again

[lyric quiver]

Vscodium also exists

Altho I may be wrong but some extensions may not work due to the recent marketplace changes

Just like how they screwed over cursor

[clear wigeon]

openVSIX is still an option

[clear wigeon]

i mean, not surprising tbh

[lyric quiver]

True

[onyx spindle]

Technically, it was never "forced", just default

[fierce horizon]

haha, as soon as I learned of it I did sudo pacman -R python-pre-commit. if there was a prek.ci (free for open-source projects), I’d be very happy

[fierce horizon]

uv uses pubgrub I think? pixi uses uv for PyPI packages, and conda-lock uses Poetry, I think.

small digression re:solvers; I’m flabbergasted that not only conda still hasn’t fixed the glaring, huge, and real issue of “doesn’t resolve PyPI and conda packages together”, but mamba doesn’t either and neither does rattler. Only pixi and conda-lock do (and rip did, before pixi’s devs abandoned it for rattler). Like, does nobody care that the solver actually can be relied on to actually honor the constraints you give it? Just me? OK then.

[mighty flower]

It only really works from the perspective of conda first though, and if you're not bought in on the conda world it's pointless. Even if you are there are no standards to follow, you have to fudge the two worlds together. I'd say you have to be highly motivated to tackle that problem.

[long knoll]

maybe I should see if I can get this working more easily than resolvelib (which, yes, is quite abstract and hard to understand)

(also, let me just add myself to the list of people interested in being paid a NYC developer salary to do that sort of thing)

[fierce horizon]

conda & PyPI

[mighty flower]

FYI, someone's done something like that here: https://github.com/ddelange/pipgrip

[lapis solstice]

And later talked to me about it, and we hatched a scheme to make the whole thing your problem (albeit with the PSF providing operational support). Pretty happy with how that turned out, TBH 🙂

(imagine if you had been running some xz style long play)

[vast wren]

and here I thought we were friends

[lapis solstice]

Hmm, that was long enough ago that it was before I started making sure people really knew what they were signing up for when volunteering... so, sorry? Kind of?

(Ah, that's it: I'm genuinely sorry for not making sure you knew what you were getting into. I'm not sorry at all for how it turned out in practice)

[vast wren]

lol, I never had a problem with it anyways 🙂

as much as I'll joke about it, all the packaging stuff has been very rewarding tbh

[lapis solstice]

Now I'm tempted to trawl through my emails and find our original discussion about you being the default delegate for PyPI PEPs. I wouldn't have done that without asking first, but I have zero recollection of how I actually phrased the question (and you weren't in Santa Clara, so I couldn't have asked in person)

And yeah, seeing where things are now vs 10 years ago definitely makes all the frustrations along the way feel worth it.

[lapis solstice]

Oh, memory, you frail, frail, thing. I had completely forgotten that my original Python 3.4 packaging plans were more in line with Tarek's 3.3 plans, and you got directly in touch with me because you were (rightly) concerned that wasn't going to work (for the same reasons it hadn't worked the first time).

I subsequently added Daniel to that thread, and things progressed from there.

[long knoll]

starred it, might browse the code for reference but seems unlikely to contain anything I can use directly

[clear wigeon]

before I started making sure people really knew what they were signing up for when volunteering

reminds me of the time when i wanted to become an asyncio maintainer and then i realized what i was volunteering for

https://tenor.com/view/топ-gif-8645824773769135803

[desert verge]

I really want a "resolve oldest package" thing

Like what's the oldest package of x dependency that was built for python 3.9, starting from package<=5.6

[mighty flower]

uv has it, I'm not sure it would require any changes to resolvelib, I think if the collector worked in reserve order it should work?

[long knoll]

I'm inclined to agree with that

[desert verge]

Hmmm I'll have to give it a shot for resolvelib

[hexed briar]

I'm still a bit sad that we didn't end up with crates.io tho. Was a really good name.

[vast wren]

it still messes with my head when people talk about rust packaging

[desert verge]

@dreamy hatch I gave this a shot but the flattening of tables, especially the ruff tables, makes this ridiculously unreadable. While the sorting and pep440 normalisation is okay fine, it falls apart when every ruff definition is in full length. Same with entry points.

[onyx spindle]

Exactly why I am not using this. Too opinionated on unreadability

[desert verge]

I also have questions about its release process but I should raise an issue about that...

There's two packages developed in that repository. They both have shared rust and shared python code

The shared python code lives in a secondary repository and is published to pypi, and then used as a dependency for both of them.

In other words, the monorepo stores the shared python code in a different repository and depends on it through pypi

Which absolutely baffles me

and that common code file isn't even owned by tox on PyPI, which, ayiyiyi

and they pin that common file https://github.com/tox-dev/toml-fmt/blob/0aefe3ada580878a7c004f392f0d366c9acf954f/pyproject-fmt/pyproject.toml#L35 so you can't have two differening versions of pyproject-fmt and toml-fmt

[dreamy hatch]

Yeah, I prefer the other way, but I value being able to diff files across projects more, makes it much easier to maintain

I think https://github.com/tox-dev/toml-fmt/issues/5 is what you want? Bernát is happy for a PR to add an option

[fierce horizon]

see also https://github.com/tox-dev/toml-fmt/issues/35

[desert verge]

Yeahhhhh

I have a local copy which I was attempting to learn rust by hacking that option in

[lyric quiver]

Anyone has any idea when will it be as easy as running .half() on a model to have fp8 inference in Pytorch.

Been wanting to add fp8 support for ages now but it seems rather... finnicky?

[kind moon]

pyproject.html

What an unfortunate typo
I would cry

[clear wigeon]

i'd feel just like the project name, a birdbrain

[long knoll]

Only the first time in the message; the repo has the correct filename.

but there are a lot of things the author can learn about project structure and the import system, still.

I left my recommendations, but it seems the discussion was many hours ago, so.

(It appears that these "birds" are programmable robots used for teaching)

[long knoll]

@mild pollen the rest of the discussion presumably doesn't pertain to pip, but

anything to do with "just compile against XYZ instead" is non-trivial really. it has to provide an identical interface
I'm sure it could be done in principle, but people interested in things like zlib-ng might also be interested in swapping at runtime

more generally, I've been hoping for a smoother interface to compression formats (and I found someone over on the Python discord who seems to be attempting one). The 3.14 changes are nice, but I think wholly inadequate

[marsh kite]

What do you have in mind for a smoother interface to compression formats?

what are the pain points?

[long knoll]

oh, the someone is @kindred hound :)

lots of membership overlap.

[kindred hound]

😄

[long knoll]

the main thing for me is that working with different kinds of archives feels totally different. The interfaces don't all work the same way

[marsh kite]

the main thing for me is that working with different kinds of archives feels totally different. The interfaces don't all work the same way
Have you tried using https://docs.python.org/3/library/shutil.html#archiving-operations ?

[kindred hound]

I've been keeping a note of these!

https://github.com/Ravencentric/archivefile/blob/v2/NOTES.md

[long knoll]

shutil definitely helps, but it only gives really high-level interface (extracting an entire archive or making one from a folder).

[marsh kite]

yeah, that's fair

[long knoll]

incidentally Raven as part of PAPER I've been working on a tool for generalized tree filtering and processing operations, with some built-in support for filesystem trees. The idea is to compose some simpler pieces so that you don't have to actually write the for x, y, z in path.walk(): and manipulate generator state for filtering etc.

[kindred hound]

Sounds great. I've needed that more than a few times. You should also checkout rust's walkdir crate for some prior art if you haven't already 😄

[long knoll]

interesting

Unfortuantely it seems I can't discuss in detail at the moment. Need to make dinner while it's my turn in the kitchen, etc.

[lyric quiver]

we won

torch is now something like 2GB smaller

[onyx spindle]

How?

[lyric quiver]

I would assume it's the entire drop of Pascal and Volta support

including upgrading to cu130

can't wait for my 3090 to become paperweight in 4 years time

[lapis solstice]

On the topic of file tree walking, I've been considering dusting off https://pypi.org/project/walkdir/ for 3.14+, as the addition of caching to pathlib addresses one of its biggest performance issues (repeated syscalls)

[long knoll]

Perhaps you'll be interested in my release then.

[mighty flower]

Can I no longer download the full Anaconda installer without creating an account with Anaconda?

I was about to spend 15 mins debugging a pip issue we are getting from Anaconda users, but if I can no longer download the installer without an account I think I will just close any Anaconda specific issue and point the users to the Anaconda issues GitHub

[lyric quiver]

Well that's stupid

It already was basically a bloated repackage of python, why would I even create an account for it

[mild pollen]

Looks like it's still downloadable from here: https://repo.anaconda.com/archive/

[fierce horizon]

I’ve been doing this since forever. They repackage stuff, so they are the first address for issues. It’s the same way with stable Linux distros like Debian. Only if they determine that it’s an upstream bug you’ll have to start to care.

[dark anchor]

micromamba installer + packages from conda-forge is the way to go in 2025

or pixi

[fierce horizon]

That’s of course also true, but I assume that @mighty flower said “full Anaconda installer” because he meant to debug that distribution specifically.

[dark anchor]

that’s why I didn’t chime in earlier, but it hadn’t come up yet and I wanted to add the context that conda itself isn’t all bad, it’s mostly bad decisions made by anaconda inc

[boreal bramble]

<@&815744082823348274> the guy above spammed all of the channels in this server.

[silk jungle]

banned and message history purged ✅

[boreal bramble]

thanks!

[mighty flower]

My 4 year old PC that just got upgraded to Windows 11 last week now appears to be in a boot loop 😭

[clear wigeon]

i try to avoid upgrading as much as possible, i just delete the OSs and install the new one on top of the clean drive

[mighty flower]

Yeah, back in the day I would have done that, I prefer not to spend my days doing OS installs now though

I managed to get Windows Recovery Screen to find out that while my BIOS can detect and receive input from my wireless mouse and keyboard the Windows Recovery Screen cannot 😤

[mighty flower]

Also, FWIW is my hypothesis is it's less to do with the upgrade method and more to do with Windows 11 turning on sleep mode. I think some issues to do with sleep mode broke my PC. And it seems I threw away my last wired keyboard so it will have to wait till tomorrow to attempt to fix.

Hopefully no urgent pip release issues till then

[junior narwhal]

What happened to the encode org? Many of the projects have gone away or been archived https://github.com/encode

[desert verge]

Well. I went to try and find you the issue but found this instead
https://github.com/Kludex/starlette/issues/3042

[desert verge]

Here it is! https://github.com/encode/starlette/discussions/2997

[junior narwhal]

Oh wow it didn't click until now: the owner of encode is the same person killing (in effect) MkDocs, I didn't realize who it was

[kind moon]

It's RubyGems all over again

[kind moon]

Them too? What happened there?

[junior narwhal]

Long story short is that maintenance was lacking and the owner came back but hasn't really done anything except expressed a desire to turn it into something else like a service

So this led mkdocs-material folks to fork/rewrite in Rust which is now in private preview

Honestly that's the best outcome though, that theme is synonymous with the framework itself and they have monetary backing already

[kindred hound]

Are there any issues on any of that? I use mkdocs-matetial extensively and haven't heard anything about this lol

[junior narwhal]

The fact that they are doing a large update is quite public on social media, they haven't landed very many big features recently and in their issues they always state it's due to the current foundational changes being made

As far as the internals, that's private as is the new company website

[kindred hound]

Oh, I haven't followed any of their social media

[mighty flower]

Literally just plugging a wired keyboard into it allowed Windows to boot up, very 2000s

[long knoll]

does this one also have hordes of otherwise uninterested parties arguing about the non-FOSS-related politics of the dramatis personae?

[stable zealot]

I think the owner is focusing on httpx

As for Starlette and Uvicorn, happy to answer any questions related to it 🙏

[junior narwhal]

I had planned (it's still slowly moving up through the backlog of my old team) on switching everything at work from requests to httpx but to be quite honest I have a bad feeling about that nowadays. I think probably the wisest choice is urllib3 even though it doesn't have the same number of direct integrations.

[kindred hound]

There's an open issue about 3.14 compatibility in httpx (https://github.com/encode/httpx/issues/3644) and there seems to be a plan to split httpx into two seperate sync and async packages (https://github.com/encode/httpx/discussions/3344#discussioncomment-14170874) so I'm also a bit worried about its future

[junior narwhal]

I didn't see that, definitely concerning.

[desert verge]

i don't want httpx to be split 😔 otherwise I'd just use requests and aiohttp like the old days

[clear wigeon]

😮

+1 on httpx not being split, it's nice to have a single unified interface, however i do appreciate aiohttp's performance

[desert verge]

i actually love httpx as it is cause it allows custom transports
-# so i just use httpx with aiohttp already

but that allows cross compatibility with libraries that need an http client but don't care how

httpx is actually GREAT for that reason

https://karpetrosyan.github.io/httpx-aiohttp/

sometimes I love python. sometimes I don't

[pure plover]

Hi Friends 👋

The all-contributors bot maintainer team needs to expand to support our widely used bot. The bot allows you to easily add contributors to your project README, recognizing all types of contributions.

If you know anyone who is great with:

• JavaScript development and/or
• Wants to help with triaging issues and documentation updates

Please direct them to this issue:
https://github.com/all-contributors/all-contributors.github.io/issues/959

We're looking for people with bandwidth to consistently contribute and help revitalize this widely used project in our community!

[silk jungle]

https://github.com/pypa/packaging-problems/issues/1019#issuecomment-3478090375 This is disappointing to see from a Linux Foundation project.

I guess everyone involved with the CD Foundation are paid professionals and thus don't see how asking for essentially feedback on their promotional material is a bad look.

[long knoll]

such a blatant "form letter", too.

But I guess it's better than writing something untrue about PEP 751 and then not soliciting feedback...