pipx

[old hare]

pipx 1.2.0 has been released! https://github.com/pypa/pipx/releases/tag/1.2.0

[sweet loom]

Hello there, would pipx be a good fit for a graphical application ? Like to install/run a pygame game

I'll elaborate further, most of the time when students ask about packaging, they ask specifically about "how can I share the game with my friends on windows?" and they usually like tools like pyinstaller/cx_freeze for that purpose

[tranquil bay]

in theory yes, but if you want to distribute your game, I would look towards pyinstaller and similar solutions

[sweet loom]

I'm not a fan of making .exe out of python code and I've be searching for years for an alternative

[tranquil bay]

well, depends on the target audience

pipx still requires some understanding, where pyinstaller just gives you working binary

but I agree with you on your opinion on making exe out of python code

[sweet loom]

Well usually students ask me this question so I guess just to share it with their fellow students

I basically advise against using Python for games, using a real engine (unity/unreal) is just so much better

but meh... students 🤷

would it fit inside of a batch script? like a script that just open the Python page on the windows store, pip install pipx, pipx install <path-to-wheel>

and then a shortcut on the desktop that just runs the correct pipx command to start the game?

sorry, this question is highly specific, I've discovered pipx just last day, it's great tool to installer/isolate python "binaries"

[karmic plume]

Is there any way to get extra installation verbosity from pip?

based on installation time in CI it looks like the cache is not being respected as usual but I cannot confirm without more output

[old hare]

Currently no I think

[zinc wedge]

Does anyone have a github actions example that caches pipx things somehow?

[summer steeple]

does anyone know why pipx run wheel fails with:

'wheel' executable script not found in package 'wheel'.
Available executable scripts:

but if I install wheel into a new virtual environment there's a wheel script

[random sequoia]

I'm pretty sure it's because pipx installs wheel as part of its "shared libraries".

Which it probably shouldn't be doing anymore.

[old hare]

Fix available here: https://github.com/pypa/pipx/pull/970

[summer steeple]

pipx run wheeel works instead

[summer steeple]

ah this is because pip used to need it, but now it works without it?

[summer steeple]

probably the fix should be to drop wheel from the shared libraries

[random sequoia]

Yeah, and same for setuptools.

[summer steeple]

pipx run pip wheel doesn't work either

so that would need a fix like #970

[summer steeple]

https://github.com/pypa/pipx/issues/990

[karmic plume]

regarding running scripts https://discuss.python.org/t/pep-723-use-a-new-run-table-instead-of-project/31656/33

would maintainers here be supportive of a metadata option specifying a constraint on the Python version and erroring out if that could not be satisfied?

[vivid crescent]

Or be supportive of runtime requirements for a single-file script being specified as TOML and somehow embedded in the script?

[river hazel]

/edit: I didn’t see the channel at first, so I didn’t interpret “maintainers here” as “pipx maintainers” and blabbed on. Sorry about that!

So the problem is like the old “requires_python in setup.py is of limited use, as you kinda need to know it before running setup.py”, and the solution we had in that case is “use a static metadata file instead of code”.

We definitely need to solve that here too. A requirements list without a requires-python is just incomplete.

~~I think:

1. I don’t like marrying ourselves to markdown syntax. The Python world is too entwined with rST to make a part of our standards Md-specific.* So a Markdown code block in the module docstring is out.
2. Scripts shouldn’t hardcode the minimum Python versions into their shebangs, so #!/usr/bin/env python3.9 is also out.
3. I can’t think of a way to reliably parse either a docstring or a __pyproject__ variable with an arbitrary older Python version, so unless we can figure that out, either approach is out, too.
   So I can think of two ways:
4. Make option 3 work using something hacky like “if we can’t parse the full file, just parse an increasing number of lines until the parsed part contains the __pyproject__ variable or we hit the end of the file”
5. Make option 2 work by adding some way to express “Launch Python version x.y or later”
   *And as an aside, I like rST. Its extensible roles and directives are plain better for technical documentation than the inflexible Md, and extending that with proprietary ad-hoc syntax extensions just makes documents unreadable when renderered with a generic renderer like GitHub’s~~

[karmic plume]

have you read the 723 proposal about using a regular expression?

[river hazel]

I did not, oops! So putting requires-python into there is perfectly valid, nice!

[karmic plume]

over the weekend I'm going to change to [run] as proposed inside a comment (even though I dislike the comment approach) with initial keys dependencies and requires-python

[vivid crescent]

That's one way to try and get the [run] proposal back into pyproject.toml 😁

[tranquil bay]

the what? you got any links?

[karmic plume]

the issue I'm struggling with is how to indicate that "this is where the magic happens"

everyone including myself dislikes the markdown code fence that Rust uses

[vivid crescent]

Discussion is at https://discuss.python.org/t/projects-that-arent-meant-to-generate-a-wheel-and-pyproject-toml/29684

Strawman proposal at https://discuss.python.org/t/projects-that-arent-meant-to-generate-a-wheel-and-pyproject-toml/29684/56

[vivid crescent]

Yeah, it's a tricky one. I think as long as you have the one you like the most in there and have an open issue on other potential ideas you think are viable that's totally fine and we can ask in user studies which format people like the most

[opal slate]

We could do this https://bpa.st/HYGA 😄 I bet that won't conflict with anything or be used accidentally anywhere.

[vivid crescent]

You forgot the (╯°□°）╯︵ ┻━┻ to properly express how people apparently feel when doing anything related to packaging 😉

[tranquil bay]

people: python packaging is a mess
PyPA: ok, we will make new standards to fix this
people: (╯°□°）╯︵ ┻━┻

[random sequoia]

┬─┬ノ( º _ ºノ)

[autumn mural]

Thank you
Have some coin for helping me clean up 🤌

[summer steeple]

What's the pipx run command to install PyQt5 PyQtWebEngine orange3 and run orange3?

[latent moth]

Going to cut a release mostly for https://github.com/pypa/pipx/pull/1071 since it seems to entirely break editable installs. Since it will be 1.3.0 are there PRs anyone wants to also include?

[teal zenith]

Oh, cool, my log path fallback will get released

And there's a support for single file scripts with dependencies similar to what PEP 722 proposes 👀

This one is rather disruptive though https://github.com/pypa/pipx/commit/c3d8de92b1b6c5c6f637ff16f591598fcf98567a

To maintain compatibility with older versions, pipx will automatically detect the old paths and use them accordingly.
Nevermind

[vivid crescent]

Yeah, I'm wonder if we should hold off until PEP 722 or 723 is decided on, @latent moth ?

[latent moth]

What’s the expected timeline? If possible I’d prefer cutting a release relative soon (days)

[vivid crescent]

The decision is coming this month, probably in two weeks (on vacation next week). After that it all depends on how much time it takes to resolve any open issues on the selected PEP.

It's not that bad if pipx supports a format that isn't standardized, but it could end up supporting two formats if PEP 722 isn't chosen (or two even if PEP 722 is chosen but the format shifts somehow)

How bad would it be to do a release with the PEP 722 support pulled out?

[teal zenith]

It doesn't support 722

It supports something similar to 722, 722 has changed since

I'm actually not sure if it aimed to be equivalent to that.
While the most easy to notice is detection of a differently named header: https://github.com/pypa/pipx/blob/b489691c61622c495ae8aeece2ef1624eb45310c/src/pipx/commands/run.py#L332, it's not the only difference - it doesn't use indentation which means that empty lines for readability aren't supported, it also doesn't support comments

Welp, PEP actually mentions this:

A format similar to the one proposed here is already supported in pipx and in pip-run.

[latent moth]

Maybe it’d be best to release a 1.2.1 from branching our 1.2.0 with only the editable thing to avoid confusion. I’ll do that instead.

[latent moth]

Heads up: I’ve released 1.2.1 by cherry-picking the fix to -e (packaging 23.1) on top of 1.2.0 in a new branch. This is uncharted territory since pipx has never done this before and the release is not fully automated (unlike normal releases from main) so I’m not sure if I’ve done everything correctly. Please report if you see anything unusual.

[lament bolt]

Trying to run the nox testing, and hitting "ERROR downloading black-22.10.0==1fixedarch" - anyone quickly know why that happens? I'm on 3.11, and targetting 3.11, and only see that (weirdly named package) in the package lists for 3.7 and 3.8

Also was working a bit on adding 3.12 testing, but #1098 is needed first, I expect.

Something doesn't handle build numbers. There are new macOS wheels for that version with "1fixedarch" as the build number, but it's misparsing and putting the version number as the name, and the build number as the version. Ahh, think I found the bad regex.

[lament bolt]

Besides my fixes (which are nice to have), I've also made https://github.com/pypa/pipx/pull/1100 which adopts the (accepted a few moments ago) PEP 723! 🙂

[hardy burrow]

* provisionally accepted 😉

[random sequoia]

Quickest implementation ever?

[vivid crescent]

As Hugo said, it's provisionally accepted on the condition that the relevant table changes make it into pyproject.toml and that's still an active discussion (last I checked; haven't read discuss.python.org since Friday).

[tranquil bay]

I guess it's time to dive into WTF is [run] table

[hardy burrow]

PEP 723 support has been released today in pipx 1.3.0 https://pipx.pypa.io/stable/changelog/

[vivid crescent]

Well that makes things a bit awkward around the discussion about dependency groups and how to declare dependencies 😅

[zenith lance]

Just in case it wasn't fully intended -- the old pipx docs links don't have a redirect

(i.e. they don't work anymore -- the one my CI noticed being https://pypa.github.io/pipx/installation/ )

Don't care personally, I've updated to the new pypa.io links 🙂 but just mentioning

[old hare]

Thanks for the heads up! Just fixed and now it should work

[lament bolt]

Slightly old news, but pipx 1.4.2 supports modern PEP 723 & pipx 1.4.3 includes the name of the file in the traceback when it fails (local files only, loading via URI's still says <string>).

[zinc wedge]

Does anyone know if pipx not upgrading packages from git is on purpose if the version in git doesn't change? I currently work around that using "pipx reinstall", but that seems weird.

[old hare]

Probably related to https://github.com/pypa/pipx/issues/782?

[zinc wedge]

ah, thanks, I somehow missed that issue

[tranquil bay]

Could pipx be shiped with pip by default?

[waxen adder]

Hi buddies, just want to have your attention about the GitHub Discussion on the limitations of pipx-managed pipx since it is explicitly warned in the documentation but I use it quite smoothly recently. Any comments are welcome. Thanks in advance!

https://github.com/pypa/pipx/discussions/1341

[hardy burrow]

Good feedback for pipx and PEP 723 👏 https://fediverse.speckledmonkey.com/@aj/112595494965204114

[lament bolt]

I missed that the ./ is no longer required, nice!

(pipx run test.py vs. pipx run ./test.py)

[river hazel]

Now we just need this for ephemeral kernels in Jupyter notebooks.

[left apex]

I'm thinking about what to do about custom pipx.run entry points over in uvx, would it make sense to propose a standard entry point name instead of one that's tool specific?

[vivid crescent]

In anyone was like me and didn't know what pipx.run entry points are, https://pipx.pypa.io/stable/changelog/#01610 mentions it, but there isn't any documentation yet (https://github.com/pypa/pipx/issues/1371)

As for whether it makes sense to standardize, I don't think it would hurt, although I would expect questions around why <pkg>.__main__ is insufficient in this case.

And if anyone was like me and didn't know what uvx is, it looks to be a symlink to uv tool run (https://github.com/astral-sh/uv/blob/26d8879c3abc8e8ec064ee60847ab6866f08a418/crates/uv/src/commands/tool/run.rs#L36-L41); as for what uv tool does (let alone uv tool run), I don't know as there is no mention of it at https://github.com/astral-sh/uv 😅

[left apex]

Yeah it's experimental still 😓 I'm working on the documentation.

[left apex]

Yeah I don't fully understand it, it looks like it was added just for PyPA's build library: https://github.com/pypa/build/pull/247

We're already getting asked if we'll support it and I'm a little hesitant.

[vivid crescent]

I can sort of see it as "I want to be runnable generally, but I don't need anything installed in bin/", but that still makes me think __main__ satisfies that need via runpy. Maybe people wanted a more explicit way of calling than "import a module"?

[left apex]

I think it was a "we want to support running build but don't want to take that name in people's bin/"

[vivid crescent]

The other potential reason is when the project name doesn't match the package name and they wanted a way to map from pyproject.toml to the main function w/o worrying about that discrepancy (requiring that the entry point name matches would also take care of that post-install)

I mean if the standard was a way to say what the main object to use was and you could signal it's also a GUI or console script that should share the project name (or some other name), then it might help consolidate that concept a bit more to simplify that common case, but that only makes sense if people know how to e.g. pass arguments in as if it was called as a CLI. In that case you could get your pip API by programmatically specifying what to call and saying the signature is whatever sys.argv[1:] would have been

[lament bolt]

Some packages that use this are build, rich-cli, sp-repo-review, pyodide-build, id, pypi-command-line, dracula-cli, rich-tree, setuptools-scm, and validate-pyproject-schema-store. There are others that use it to remap - to _, which I assume might not be needed, and some that just add it in addition to having a matching console entry point, just to be explicit, I guess. But the above list are ones that use it for one of these things:

• Providing a name that the package isn't willing to add to your PATH. Something like build or id is too general to add a command for, it will potentially collide. But it's a fine PyPI name.
• Providing a simpler command on the PATH than the package; modern PyPI names are much longer than good command names. pypi-command-line is an example.
• A lot of *-cli PyPI packages don't add -cli to their command (like rich), but the package can't be called rich, because that's the library it wraps.
• A few packages are plugins that export a parent command (sp-repo-review uses repo-review, validate-pyproject-schema-store uses validate-pyproject) Both known examples are my packages, by the way. 🙂
• A few packages don't want to provide a command at all besides -m ..., but also want to support pipx (setuptools_scm I think is in this category)

The pipx.run entry point has solved those issues, but it's got "pipx" in the name (even though entrypoints are just arbitrary text), and is defined by pipx. Also, the last two use cases aren't covered by "just use the one command the package provides", and that also doesn't support a package that has even one single extra command.

By the way, this isn't perfectly covered by entry-points. Some packages provide binaries that don't need to start up Python; having to wrap this in a Python entry point would be potentially wasteful.

[left apex]

I'm not sure who the relevant maintainer to ping here is https://github.com/astral-sh/uv/issues/12303#issuecomment-2734990553

(pipx has the same behavior)

[river hazel]

Does anyone know why pipx stopped using platformdirs on MacOS and Windows? I tried to reproduce the problem that apparently led to this but was unable to: https://github.com/pypa/pipx/issues/1327#issuecomment-2792428785

[hardy burrow]

pipx & platformdirs

[mortal iron]

pipx defaults to invoking pip as-is, yes? So for example, it will default to build isolation if and only if the pip in the shared venv defaults to build isolation?
(I'm trying to diagnose something remotely on an older machine...)

(basically, someone else tried to install a wheel not from PyPI using pipx, using --system-site-packages following third-party installation instructions; it failed in a way that looks like Setuptools found an old version of Packaging (cf. https://github.com/pypa/setuptools/issues/4483). I'm trying to figure out how, since the pipx shared environment wouldn't normally have that explicitly installed.)

[lament bolt]

Build isolation is broken in pip, it can leak older versions into the build environment. For things that are likely to be on the system already (packaging), you can't require recent versions of things if you've got enough users to hit it. Fixing it would require using real virtual envirnoments, which pip won't do for performance reasons (and also it's a big change) last I asked. (Rather delayed response, sorry!)

[left apex]

(I think pip was exploring switching to real virtual environments...)

[flat nest]

The plan is use real virtual environments at some point, but it's going to take a while. Given how long my in-process build dependencies PR took (like a year), I won't be foolish enough to give you a time, but it is on the "roadmap"

[mortal iron]

I imagine the blocker there is having a better/faster approach to --python, since otherwise there wouldn't seem to be major performance issues?

[flat nest]

I don't think so? That's orthogonal to using real venvs. What really needs to happen is A) we use venv to create a proper virtual environment, and then we update our backend hooks code to use this venv, possibly...? Although TBH, I don't understand how the isolation is achieved in the first place.

I'd want to only support venvs with the in-process build dependencies which does complicate it (it necessitates a subprocess call to inspect the environment's installation scheme) but we should be cleaning up this anyway.

[warm oriole]

Couldn't we use in-process to target some temporary directory, then create a venv and configure it to to use that temporary directory as one it's site package directories?

venv's are probably my weakest are of knowledge when it comes to Python packaging so this is me earnestly and naively wondeirng.

[flat nest]

--target wouldn't work due to poor support with console scripts, otherwise I have no idea!

[mortal iron]

that was my point. if --target isn't sufficient, then aren't you stuck with --python to install into the new environment? and then that works by re-launching pip in that environment.

[vivid crescent]

https://snarky.ca/how-virtual-environments-work/ covers how virtual enviornments work, but basically you can do with Python (e.g. PYTHONPATH) will work with a virtual environment

[flat nest]

Well, I'd also like --python to not just re-invoke pip in a new subprocess. With a small script, you can inspect a Python environment and get the information you need to operate as if pip is in that environment.

It is another orthogonal concern though.

[mortal iron]

yes, I consider that core functionality for PAPER

although since these are build environments, where the build backends potentially (especially in the case of Setuptools) run version-specific Python code, there will still be a subprocess call I assume

[flat nest]

We don't control the build backends

[mortal iron]

the thing about re-invoking pip is that it also re-imports a ton of modules.

[flat nest]

Well yeah, that's why I don't pip to do that.

I mean, it's all a matter of time and resources. If someone could employ me to work on pip, then you'd see a bunch of (more) cool stuff land in pip, but that's neither here and there 😄

[mortal iron]

entirely valid

[flat nest]

All I'm trying to say is that this is treaded territory.

[lean spruce]

We've now official joined PyPA - pypa.github.io/pipx/ 🙂

[agile plaza]

As a follow up from our discussion on Twitter... https://twitter.com/jugmac00/status/1397867186866331648 ❯ which pipx /home/jugmac00/.local/bin/pipx This means probably I installed it via pip install --user pipx, right? I was young(er) and naive. So, the way to get a pipx itself on 3.9 is to pip uninstall pipx and then use pipx-in-pipx to reinstall it, right?

Jürgen Gmach (@jugmac00)

@gjbernat Back in the days I installed pipx on Python 3.6.

What is the best way to bring pipx and all managed apps to Python 3.9?

Twitter · May 27, 2021, 10:48 AM

Hm, I cannot get rid of it... pip uninstall pipx does nothing... ```❯ pip uninstall pipx
DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality.
WARNING: Skipping pipx as it is not installed.

Hm, I am seriously confused ... ```❯ ls -la .local/
total 28
drwx------ 7 jugmac00 jugmac00 4096 Apr 25 2019 .
drwxr-xr-x 80 jugmac00 jugmac00 4096 Mai 27 13:50 ..
drwxr-xr-x 3 jugmac00 jugmac00 4096 Mai 27 14:31 bin
drwx------ 8 jugmac00 jugmac00 4096 Feb 18 17:03 lib
drwxr-xr-x 5 jugmac00 jugmac00 4096 Aug 14 2019 pipx
drwxr-xr-x 41 jugmac00 jugmac00 4096 Mai 27 13:31 share
drwxr-xr-x 3 jugmac00 jugmac00 4096 Feb 11 2019 state

~
❯ ls -la .local/pipx/
total 20
drwxr-xr-x 5 jugmac00 jugmac00 4096 Aug 14 2019 .
drwx------ 7 jugmac00 jugmac00 4096 Apr 25 2019 ..
drwxr-xr-x 3 jugmac00 jugmac00 4096 Dez 4 2019 .cache
drwxr-xr-x 6 jugmac00 jugmac00 4096 Aug 14 2019 shared
drwxr-xr-x 2 jugmac00 jugmac00 4096 Mai 27 14:31 venvs

~
❯ which pip
/home/jugmac00/.local/bin/pip``` How did I install pipx back in 2019 and how do I get rid of it now?

puzzle solved... I have tons of different pip versions.. for reasons - and pipx was installed via pip3.6. ❯ ls -la ~/.local/bin/pip* -rwxr-xr-x 1 jugmac00 jugmac00 227 Feb 18 17:03 /home/jugmac00/.local/bin/pip -rwxr-xr-x 1 jugmac00 jugmac00 223 Feb 17 21:29 /home/jugmac00/.local/bin/pip2 -rwxr-xr-x 1 jugmac00 jugmac00 223 Feb 17 21:29 /home/jugmac00/.local/bin/pip2.7 -rwxr-xr-x 1 jugmac00 jugmac00 227 Feb 18 17:03 /home/jugmac00/.local/bin/pip3 -rwxr-xr-x 1 jugmac00 jugmac00 227 Feb 18 17:03 /home/jugmac00/.local/bin/pip3.5 -rwxr-xr-x 1 jugmac00 jugmac00 221 Feb 15 10:06 /home/jugmac00/.local/bin/pip3.6 -rwxr-xr-x 1 jugmac00 jugmac00 227 Feb 18 07:59 /home/jugmac00/.local/bin/pip3.8

[lean spruce]

is it user level installed?

[latent moth]

What is the content of /home/jugmac00/.local/bin/pipx? If it’s a Python entry point, it should contain a shebang that points to the Python interpreter pipx was installed against

[lean spruce]

did you try python -m pip uninstall pipx

less will also work 🙂

[latent moth]

(My guess is your pip and possibly python is pointing to something else, maybe due to version upgrade or something, but pipx is still installed against an older python that’s still on your system but not in PATH)

[agile plaza]

Sorry for the late reply, I had a meeting. As I wrote above, the puzzle is solved - pipx was installed with python3.6 - which was the first Python 3 version I had installed on my system. I was able to uninstall pipx via python3.6 -m pip uninstall pipx. Python on Ubuntu is still pretty confusing. pip3 now points to python3.5, which I needed to install to support Zope/Plone packages later on (via deadsnakes). That is why I was unable to uninstall pipx via pip3 uninstall pipx. There is this one .local and it looks like that all Python interpreters install stuff in there.

[latent moth]

Python on Ubuntu is still pretty confusing.
Linux in general really.

[lament bolt]

Is there a reason that pipx run build==0.4.0 --sdist couldn't be supported? In order to download the package, one has to know that the package is build anyway, so couldn't that be used for the entrypoint lookup? Currently one must write something like pipx run --spec build==0.4.0 pyproject-build --sdist instead. I actually thought this worked and wrote it in a few places...

[latent moth]

Because nobody asked for that before basically :p The pipx run build==0.4.0 syntax was added relatively recently (it used to be you must write --spec) so there are still assumptions the app argument must match the entry point name and no special parsing is done

I think pipx run build --spec build==0.4.0 -- --sdist would work though?

[lament bolt]

Okay, thanks, wanted to make sure it wasn't already something discussed; opened an issue. https://github.com/pypa/pipx/issues/697

[vivid crescent]

Definitely recommend always running pip via python -m pip

[agile plaza]

My Ubuntu installation predates your blog post 😂 https://snarky.ca/why-you-should-use-python-m-pip/

[lament bolt]

I keep wanting to recommend people do something like pipx run clang-format==12.0.1 but https://github.com/pypa/pipx/issues/697 . If no one bites I might have to try in a couple of weeks.

[high jewel]

Fedora is fine: https://fedoralovespython.org/

[latent moth]

I'm pretty sure Debian loves Python as well.

[lean spruce]

Loves to patch it.

[latent moth]

They all do which is exactly the problem. pip gets to experience it first-hand, 99% of reports here is because someone somewhere patches CPython https://github.com/pypa/pip/issues/10151

Red Hat's patches are not less problematic than Debian's, and they are also reused by as many down-downstream distributors (Debian's by Ubuntu etc., Red Hat's by CentOS etc. plus Cygwin and MSYS)

[lament bolt]

I wonder if RH's patches don't bite quite as often because they ship such old versions of pip they are basically unusable anyway.

$ docker run --rm -it centos:8 bash
# yum install python3-pip
# pip3 --version
pip 9.0.3 from /usr/lib/python3.6/site-packages (python 3.6)

[agile plaza]

The images in the readme file render properly on GitHub, but are broken on PyPI

[agile plaza]

Also broken on https://pypa.github.io/pipx/

[latent moth]

It's not broken on GitHub because someone submitted a fix. It's broken on PyPI and the home page because the fix has not been packaged into a stable release yet.

[lean spruce]

😬

[fiery pagoda]

modern python versions go into extra modules and one is supposed to use virtualenv&co for s2i images ^^

[waxen jungle]

Hi everyone, I am Chad, the creator of pipx. I just discovered this discord channel, but unfortunately the circumstances under which I got here are not the best. I come with no cycles left to support pipx, which has actually been the case for some time. As of right now, there are 12 unmerged pull requests, and various issues adding up.

I am extremely grateful for all the help and work put into pipx by the community, and for the adoption into the PyPA. However, the state of the pipx project is not really healthy. Matthew Clapp has been handling the lion's share of work, but he has recently been unable to devote time to it, and this is not likely going to change for the foreseeable future. Does anyone have suggestions on how to improve this situation?

[latent moth]

I can spend some more time on it for now. There are a few PRs I have reviewed but don’t have too much incentives merging and let hung there. I probably won’t be working on the code base myself (honestly I don’t see much how pipx needs to improve at this point, it does what it does), but I can review and merge others’ contributions.

I don’t know how to make releases though, is there documentation on this?

[waxen jungle]

Thanks! Yes there is documentation here - https://pypa.github.io/pipx/contributing/#releasing-new-pipx-versions

honestly I don’t see much how pipx needs to improve at this point, it does what it does
I agree, but then there are 99 open issues. People always seem to find bugs or new features to add.

For example, I recall seeing a bug in the last release where the version was not correct or something along those lines, and Bernat didn't have the bandwidth to release a fix. And features like the request for search functionality would be nice to add, and probably a good bit of work, but I guess not strictly necessary.

[waxen jungle]

Another thing I think would be a big improvement would be to create an installer of pipx that’s not pip. Like piping to bash or something. I used to have this but swapped it for pip. Would be good to have both.

[latent moth]

piping to bash
Like bringing back get-pipx.py? Personally I would prefer if pipx is standalone, bundled with a working Python but require the user to set DEFAULT_PYTHON. I created a Scoop package for this purpose that uses the Embedded Python distribution on Windows, but afaik there’s not a direct equivalent on other platforms.

[waxen jungle]

Yes exactly, like get-pipx.py. Bundling with Python sounds pretty cool, but I don’t really know how to do that. Even pip isn’t always bundled with Python. My motivation of piping to a shell is to get around the issue of polluting the system with pipx’s dependencies. If there are other ways to do that, it sounds good to me. I guess I would defer to you and the community since I don’t really have the ability to stay too involved anyway.

[marsh meadow]

How about a zipapp (with shiv or some similar tool). It does seem to hit the sweet spot between a single distributable file that does not require running an arbitrary script froma URL an is not as heavy as bundling Python all togheter....

[latent moth]

Anyone has the release permissions for this? https://github.com/pypa/pipx/issues/773

[waxen jungle]

This sounds like a good idea to me

[waxen jungle]

I can add permissions for you if you want

[lean spruce]

Happy to announce that we've decided to mark pipx stable, and version 1.0.0 is out today https://pypa.github.io/pipx/changelog/

[agile plaza]

Is it possible to install both tox and tox4 via pipx?

[latent moth]

--suffix

[summer steeple]

You can make a tox4 package on pypi that depends on the new tox pre release and sets the entry point

Like this ish https://github.com/celery/pytest-celery/blob/master/pyproject.toml#L15

[summer steeple]

you can now run pipx run tox4

[fiery pagoda]

are there any plans to enable cache-able "immutable" envs - imagine being able to have a tool like "build" reuse them on each run instead of recreating them every time

[lean spruce]

Within pipx?

[fiery pagoda]

yeah

[fiery granite]

Why would you do that in pipx instead of build? pipx's envs serve one purpose, which is to expose console scripts

[fiery pagoda]

@fiery granite pipx also has run once envs, an having one tool that manages lru caches of those instead of all tools making own caches of those seems like a "good idea" to me

[waxen jungle]

pipx caches the envs of pipx run for 14 days, so you can re-run and it works instantly until the 14 day mark is crossed

I decided not to cache them indefinitely so things didn't get too stale. And decided not to have them totally disposable because that would be very slow. So 14 days was an arbitrary middleground I chose. It could pretty easily be made configurable if you think that would be useful.

I just spent some time looking through all the pull requests on pipx. Got them all reviewed and merged, plus a few changes and cleanups of my own. I'm thinking of making a new release soon.

One other thing I've been wanting to do for a while is bring back the install script where you can run curl ... | python3. That is originally how pipx worked.

[karmic plume]

@waxen jungle hey! would you be open to a PR switching setup.py & setup.cfg to https://www.python.org/dev/peps/pep-0621/ ?

[waxen jungle]

Yes definitely

[lament bolt]

I love the 14 day pipx run caching ❤️ - I can just use pipx run anywhere without ever having to think about installing or upgrading anything ever again. 😉

[waxen jungle]

Awesome, very happy to hear that

[karmic plume]

@waxen jungle I wrapped up work a bit later than expected, but here's the PR! https://github.com/pypa/pipx/pull/817

I left a comment/question 🙂

[fiery pagoda]

Would it make sense to enable longer term store for international pinned ones (but ensuring regular eviction)

[karmic plume]

@lament bolt I pushed a final commit to https://github.com/pypa/pipx/pull/817

[waxen jungle]

What is an international pinned one?

[fiery pagoda]

Most painful typo of intentional

[waxen jungle]

Changing the global value of the cache time seems like a fine option to add. But IMO making package-specific cache durations, which would require additional pipx-managed state for each package, seems like a very specific use-case that would add more complexity than its worth.

[fiery pagoda]

the surrounding idea of mine would be to enable tools like tox/nix/pre-commit to have some kind of nice caching + cache eviction, but its a bit unclear where to put this

[fiery granite]

Do you mean nox?

[fiery pagoda]

nox is another of those tools^^

[marsh meadow]

Hello guys, I recently noticed that pipx has this entry-point API for pipx.run that is currently being used by build.

Do you know if it also works with extras?

Currently I know that I can do ```
pipx run --spec 'mypkg[extra]' mypkg --help

I was wondering if I declare a `pipx.run` entry-point I could skip the `--spec` entirely...

Forget about that, just checked here and this seems unnecessary 🙂

[lament bolt]

It should handle most anything you throw at it these days, extras, version limits or pinning, etc.

[marsh meadow]

Yeah, it is amazing, the more I use it, the more I like it

[agile elbow]

hi. I'm discovering the latest version of pip in my envs from repos that are several months old.

Ex: > pipx runpip pytest list | grep pip shows the latest release (~1 month old) even in a repo i made many months ago and haven't touched since.

Question: which pip does pipx use in its environments? pip in environments don't match the version of system pip (the pip used to install pipx).

[summer steeple]

Is it the pip from venv? Or virtualenv?

[latent moth]

pipx shares pip across environments and automatically upgrades the shared pip periodically

On my machine for example

uranusjr@tp-sb $ pipx runpip mypy show pip | grep Location
Location: /home/uranusjr/.local/pipx/shared/lib/python3.9/site-packages
uranusjr@tp-sb $ pipx runpip flake8 show pip | grep Location
Location: /home/uranusjr/.local/pipx/shared/lib/python3.9/site-packages

[agile elbow]

the system pip is python3-pip3 and venv is python3-venv

thx. that's helpful to know. i'll have to look into how pipx upgrades and see if pinning is an option.

[empty patio]

I'm new to pipx.

if pipx run --spec black==19.10b0 black --version fails, who should I expect is at fault?

The black packaging or pipx itself?

fails with ImportError: cannot import name '_unicodefun' from 'click' (~/.local/pipx/.cache/1baf6ecc4d1f193/lib/python3.10/site-packages/click/__init__.py)

So I guess the wrong version of click is being used? And that's the black packager's fault from back in the day?

[flat nest]

Yup, you can blame us for not writing a correct error handler 😦

(I'm one of the co-maintainers btw, PyPA is not at fault)

[empty patio]

so is this recoverable?

I assume the software worked when it was released

would i need to edit the requirements.txt in that release to pin click to whatever it was at the time if I want to use pipx to run this specific version of black?

If it worked when it was released, but the old version doesn't run with pipx now, this kinda feels like it must be pipx's issue, no?

How can the black authors of the past be expected to handle all future software changes in every dep that will ever take place?

Is it that https://github.com/psf/black/blob/6bedb5c58a7d8c25aa9509f8217bc24e9797e90d/setup.py#L38 is never an okay way to specify a version dep?

[flat nest]

It should be possible to change packages in a tool's env

I wonder if pipx inject black click==7.1.2 would work

[latent moth]

It should

Alternatively pipx runpip black install click==7.1.2 should do the job as well

[empty patio]

ok, so I've done:

pipx install black==19.10b0
pipx inject black click==7.1.2
pipx run --spec black==19.10b0 black --version

and that didn't work

[latent moth]

No the last one won’t work

pipx run installs and runs the package in a temporary location, and does not use the environment populated by the first two commands

[empty patio]

ah

[latent moth]

So after the first two, you should be able to just run black in your ~/.local/bin (or where you tell pipx to install commands to)

[empty patio]

nice ~/.local/bin/black --version works now

thank you both

but this is a serious issue, right?

we should expect every single project that specifies a dep version >=X to fail increasingly as time goes on

here's a feature idea: if you install from a git spec, pipx uses the git history timestamp of the release to pin the max versions of all the deps

so you never have the issue of running with a dep that didn't exist at the time the software was released

i mean assuming you have a mapping of release versions to timestamps for every package ever 😉

[latent moth]

That would be a disaster security-wise

Say Click 7.1.2 has a vulnerability and they released 7.1.3 to fix it, but pipx users won’t receive it because it’s never considered

Ultimately dependency specification is something the dependant package should specify correctly (i.e. Black) and the package manager can’t guess like this

[empty patio]

I guess specifying a dep <=X or ==X is the same security issue

[latent moth]

Generally you’d want to specify something like click >=7,<8 so you get compatible bug fixes without breaking your app

[empty patio]

I don't know, it seems like pick your poison: either security issues, or stability issues when a dep changes in an unexpected way

[latent moth]

But here Black is using private attributes from Click without informing Click, and Click though it can freely change that (because it’s private)

So ultimately it’s Black’s responsibility to be more responsible specifying and using dependencies

[empty patio]

yeah, that's def a bug

but the fact is the software worked on release, and all of the future changes that might be made to dependencies is hard to account for

i'm gonna start being a lot more careful with my dependency specs now!

[latent moth]

You’re basically describing dependency management, and it is an entire field to research. People have already come up with a ton of contracts to address it like for the past 30 years and counting. But ultimately all the solutions need to rely on social contracts, and in this instance Black broke that contract in the fist place. “Hey but we ought to be able to do better in this situation”, and yes of course we can, if we consider the incident in vacuum, but trying to do better for this particular case will make a lot of other situation worse, and ultimately will not help anything in the long run.

[empty patio]

I'm sure you're absolutely right though, if every spec was give as click >=7,<8 and everyone went back and released security fix point releases for everything, then the world is right

[latent moth]

If you’re interested in this kind of things I would definitely suggest investing time into it though; dependency management is an interesting topic and we can certainly use more people interested in it. There are a lot of things you can read, even an innocent question of how to version things is a giant time sink if you look deep into it

[empty patio]

I've read at least parts of the semantic versioning spec in the past 🙂

It is interesting, though I wouldn't class it as the most riveting thing I've ever read

[latent moth]

How semantic versioning doesn’t work is probably more interesting than how it works. And why you should probably still use it anyway, or maybe not, or maybe fix it

[empty patio]

but I do think that giving the user the optional choice (non-default) to pin the max versions of deps to what they were at the time of the software was originally released would be a beneficial feature if the user understands the security tradeoff they're making. I can't imagine any other way you could possibly make it easy for an average user to get an old piece of poorly written software to work.

[opal slate]

in practice the python community is typically pretty good about fixing things that break compatiblity somewhere along the line, either in dependent projects or the dependency

[latent moth]

Release mutability is indeed another area in dependency management that has much research done in

Let’s just say it’s all well-trotted topics, and generally those are questions that still don’t have a perfect answer available. And yes Python is actually doing pretty well comparatively in many of the areas.

[opal slate]

generally speaking if you don't want to deal with random breakages, you're best using something like pip-compile to turn a set of loose dependencies into exact pins, installing from that, and adding automation to periodically bump those pinned versions

[empty patio]

hah, makes me think back to a year or two ago when I tried to do something with ruby. holy smokes, the versioning/dep issues were insane

maybe just because I'm bad at Ruby

[opal slate]

it is also true that every ecosystem basically has it's own problems with dep management, and people familiar with that ecosystem will tend to have less problems, because they've already internalized the ways to work around them and what to do to avoid those problems 🙂

[empty patio]

maybe this is why I like Arch

Arch Linux, Never Look Back

[lament bolt]

@empty patio Maybe read https://iscinumpy.dev/post/bound-version-constraints/ - also note some recent breaking releases were Click 8.1 (note that while this was a private attribute that broke Black, it was a public attribute that broke Typer, in the same release), pytest 7.1, setuptools 61.1, etc. There are some reasons to limit deps, but most important is that if you limit to some version, you should have assurance from the developers that that version will continue to receive bugfixes after a newer one is out. In Python, this is almost never the case. For example, git just released security fix that breaks most packages using it programmatically - if you limited to setuptools_scm/versioneer less than some version, you are now (partially) broken. You have to be able to get updates if you limit.

No limits for libraries + pip-compile or some other locking system is much better in Python. (JavaScript has a very, very different system due to nested dependencies and common backport releases)

IMO, Ruby is pretty similar to Python for dependencies, good and bad. Though it has good support for a locking system via bundler (rather like PDM/Poetry in Python)

[marsh meadow]

setuptools 61.1
Is there any bug I forgot to fix?

[lament bolt]

No, it's just a change that broke libraries that incorrectly depended on the internal names of build directories. (and there's a small chance I forgot the correct number, but I remember it ended with .1)

My point being that limiting to <major doesn't mean you won't be broken, and it does mean you might miss out on critical bug fixes. I'm sure all 60+ major releases of setuptools are not still receiving bug fix releases. 😉

[marsh meadow]

I am relieved 😮‍💨 ! Thanks @lament bolt.

You are definitely right!

[summer steeple]

I use a requirements-pypa.txt created with pip-tools to bootstrap from

[waxen jungle]

pipx 1.1.0 has been released https://github.com/pypa/pipx/releases/tag/1.1.0 https://twitter.com/cs01_software/status/1530402945094651904

Chad Smith (@cs01_software)

pipx 1.1.0 is now available:

pip install pipx --upgrade

See the release notes or the tweets below for details 👇

https://t.co/Njn3xZQemc

Twitter · May 28, 2022, 4:17 AM

[waxen jungle]

I created a PR to add an installer script for pipx: https://github.com/pypa/pipx/pull/849

If anyone would like to test it out, you can try

curl https://raw.githubusercontent.com/pypa/pipx/cs01/add-get-pipx-script/get-pipx.py | python

[zenith lance]

Cool! Though wouldn't one consider users would prefer a zipapp (or shiv) instead for a tool like pipx?

[marsh meadow]

I started implementing something in my machine using shiv for pipx (as a proposal to be submitted), but I never finished... It seems to work well, but I was stuck at writing tests...

[waxen jungle]

Maybe! That’s a good suggestion

[waxen jungle]

This is actually trivial with pex:

pipx run pex . --console-script pipx --output-file /tmp/pipx

[waxen jungle]

One thing that is a little tricky is the interpreter is hardcoded, so if someone's python3 isn't recent, the pex won't work:

pipx run pex . --console-script pipx --output-file /tmp/pipx --validate-entry-point --python-shebang "/usr/bin/env python3"