general

[dire vortex]

I can't install anything with pipenv on a brand new mac(M1)

I have a repo with a Pipfile, and installing it results in a lots of An error occurred while installing

I tried to remove the lock file, it didn't help

I also don't get any meaningful error, just a blurb of a "var export":

...
...```

[lapis tundra]

Hi All,

I have written a python script which requires user input via flags eg.

docker run recaller --file /file/path/here and i want the user to run this via a webpage/service.

I have never integrated a python script with the web before (i dont really know HTML very well) but I am trying to create something very basic - literally a landing page where the user adds the filepath and that get sent to the script as a the --file parameter and then the output of the scripts gets displayed on the webpage. Any ideas how best to achieve this?

[high stone]

Ask for support on the issue trackers please.

[coarse iris]

A POC code would be as below

result = subprocess.run(["docker", "run", "recaller", "--file", "<user_input_file_path>"], capture_output=True)
return build_response_from_string(result.stdout)

P.S. this is not related to this forum.

[lapis tundra]

Using what framework to display the stdout?

[lapis tundra]

Where should I post it then? 😬

[coarse iris]

do you mean UI framework? any will do

[lapis tundra]

I guess I mean how would I get the stdout to be display on a webpage and how do I pass the parameter via the webpage?

[coarse iris]

that should be covered by the documentation of the web framework you choose. you should find out the answer of such basic questions by learning the docs

[low drift]

Hello. Thank you for participating in the Packaging User Survey. The responses are summarized in this post- https://discuss.python.org/t/packaging-vision-and-strategy-next-steps/21513. Please can you participate in the poll and indicate how you would like to participate in the strategy discussions. The poll is open to all maintainers/contributors.

[west drift]

asking here since there's no packaging channel: what are the exact semantic of PEP 508 marker comparisons?
E.g. for Requirement('numpy; "1.0" <= "1.1"').marker.evaluate() i get an error that 1.1 isn't defined

i'd have expected markers to be constraint to just <key> <op> '<value>', but PEP 508 allows the same on both sides of the operator

Also Requirement('numpy; sys_platform <= "p"').marker.evaluate() and Requirement('numpy; python_version <= "p"').marker.evaluate() being true was unexpected since <= is in <version_cmp>

[spiral urchin]

IIRC version_cmp falls back to string comp if the value is not a valid version so that’s probably expected

[distant briar]

One of the sides needs to be a key

At least with the current implementation

[spiral urchin]

The spec does allow comparing two literals so we may want to amend that. I think it’d be worthwhile to open an issue on GitHub
https://peps.python.org/pep-0508/#grammar

marker_var    = wsp* (env_var | python_str)
marker_expr   = marker_var marker_op marker_var
              | wsp* '(' marker wsp* ')'

[west drift]

are there any cases where a silent fallback from PEP 440 to string comparison is the intended behaviour?

[west drift]

in which repository?

[spiral urchin]

pypa/packaging

[spiral urchin]

All cases, IMO. The only viable alternative is to crash, which would be quite frustrating for everyone. Since the intended use case is to parse package metadata, whatever tool generating the metadata should have validated it, and if it allows the value to get in, the most reasonable thing behaviour would be to assume it’s a string (well it is)

[west drift]

what do you think about specifying that tools such as setuptools/poetry/hatch/twine must not allow to create and upload package with string metadata where PEP 440 versions would be expected and that installers need to warn whenever they encounter something like this?

[west drift]

what i meant with intended was intended by the user rather than intended by the installer. I'd have assumed that every case where someone compares python_version with something stringly it's because they accidentally typed in a non-PEP 440 compliant version

[soft glen]

I would prefer to crash (this is what Poetry currently does, because we wasn't aware off the fallback for a long time). It's surprising if suddenly "3.10" is smaller than "3.9". It can be quite time consuming to debug a problem that arises due to not PEP 440 compliant versions.

[spiral urchin]

If the version is not PEP 440 compliant the tool generating the metadata definitely should crash, but I’m not sure the installing tool crashing is the most friendly (crashing is still a reasonable result, of course, but it’s not as obvious as the only choice). If a user accidentally typed in a non-compliant marker I’d say the tool receiving the input (again, that would be the one generating metadata) should have validated it and not blindly put it into metadata for the installer.

[west drift]

i think the installing tool should print a warning

currently packaging just silently switches behaviours (i'm currently writing an issue for that)

and imho ideally we'd specify that a wheel-building or wheel-uploading tool needs to validate that

[lusty quarry]

Hatchling errors as per https://peps.python.org/pep-0621/#version

[west drift]

also when writing something like numpy; python_version >= "3.9."?

(the trailing dot is forbidden in PEP 440 for versions, so pypa/packaging falls back to string comparison)

[lusty quarry]

no special logic for that, no

[west drift]

does hatch also use packaging for marker evaluation?

[lusty quarry]

yes

[west drift]

yeah i think migrating packaging and potentially poetry (i haven't check what poetry uses) would fix the entire ecosystem

[lusty quarry]

have you tried packaging 22.0 released yesterday?

[west drift]

yes

https://github.com/pypa/packaging/issues/632

https://github.com/pypa/packaging/issues/633

opened two issues in pypa/packaging to track that

[charred swift]

what are my options for a PEP 660 install (editable/development mode) if I wasn't going to use pip?

[mossy pulsar]

#installer ?

[charred swift]

can it do that?

[mossy pulsar]

Well PEP-660 reuses the wheel spec, so there needs to be nothing more than supporting wheels 🤷

[charred swift]

installer operates only on .whl files right?

[mossy pulsar]

PEP-660 generates .whl files

there's nothing custom in it

that would have been PEP-662

[charred swift]

what tools can make a PEP-660 wheel?

can #build do that?

[mossy pulsar]

why do you need a tool for it?

what you're trying to achieve?

[charred swift]

i'm just exploring my dev workflow

and I'm am curious if it's possible to get pip out of it

[mossy pulsar]

it is 🙂

kinda 🙂

[charred swift]

so I hand craft a .whl then install it with installer into my venv?

[mossy pulsar]

you'll still want to use pip to install deps from pypi

[inland prawn]

I would start with "why?"

[charred swift]

Yeah, for a second, maybe we can just assume that the deps are magically taken care of

[mossy pulsar]

#build delegates the job to https://github.com/pypa/pyproject-hooks/blob/main/src/pyproject_hooks/_impl.py#L252, which supports editable wheels, though note #build does not exposes the editable feature via CLI for now, so you'd have to invoke Python code to build the wheel 🙂

[charred swift]

yeah, i was reading PEP-660 and it was pretty adamant that such wheels are transient

[mossy pulsar]

you could also use the subprocess frontend of https://pyproject-api.readthedocs.io/en/latest/#pyproject_api.SubprocessFrontend to invoke the build_editable msg from Python

[charred swift]

I must delete them when I'm done! 🙂

[charred swift]

https://github.com/pfmoore/editables hm

hatchling seems to use that. wonder why it doesn't use pyproject-hooks

[charred swift]

learning has occurred @mossy pulsar !

# /usr/bin/env python3

from tomli import load
from tempfile import TemporaryDirectory
from shutil import move
from pathlib import Path
from pep517 import Pep517HookCaller

dist_dir_name = "dont_dist"

Path(dist_dir_name).mkdir(exist_ok=True)

with open("pyproject.toml", "rb") as ppjt:
    data = load(ppjt)

some_hooks = Pep517HookCaller(".", data["build-system"]["build-backend"])

with TemporaryDirectory() as bldd:
    ewhl_file = some_hooks.build_editable(bldd, {})
    move(Path(bldd) / ewhl_file, Path(dist_dir_name) / ewhl_file)

thank you for your help/advise!

[mossy pulsar]

editables is meant for backend users to use to implement editable

but is not meant to be used by the frontend directly

[noble matrix]

@lusty quarry 👆

[lusty quarry]

Hatch currently can only hatch build projects that use Hatchling b/c it's simpler for arbitrary environments like Docker. maybe eventually

[coarse iris]

editables is a backend assistant and runtime dependency to help build editable wheels, different from what pyproject-hooks does

pyproject-hooks is the bridge between build frontend and backend

pdm backend also support building editables flavored wheels

[lyric hedge]

Traceback (most recent call last):
  File "generate.py", line 38, in <module>
    req = Requirement(raw_req)
  File "/home/ichard26/programming/oss/black-deps-ci/venv/lib/python3.8/site-packages/packaging/requirements.py", line 37, in __init__
    raise InvalidRequirement(str(e)) from e
packaging.requirements.InvalidRequirement: Expected end or semicolon (after URL and whitespace)
    aiohttp @ https://github.com/aio-libs/aiohttp/archive/master.zip 
              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^

heh I did not expect packaging to become stricter on terminating whitespace

broke my CI, interestingly packaging 21.3 comes pre-installed on Python 3.9 Windows

[coarse iris]

This seems incorrect according to the spec: cc @high stone

Leading to the unified rule that can specify a dependency.:

   specification = wsp* ( url_req | name_req ) wsp*

[high stone]

Is that trailing whitespace not being handled by packaging?

I'd appreciate if an issue could be filed on pypa/packaging. :)

[lyric hedge]

Yeah I had to remove it to avoid the exception

[high stone]

Yea, that's a bug. Whoops.

[lyric hedge]

I can tomorrow, it's past midnight and I'm tired :)

[high stone]

Absolutely no hurries! Have a good night. :)

[high stone]

https://github.com/pypa/packaging/pull/642/files

Slipped through due to a faulty test.

[lapis nest]

Could we mention PEP 656 in https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/ ? I think that might be missing from the page.

[noble matrix]

Yeah, it should defintely be mentioned there, not sure how it got missed. The least intensive approach would be to add a new section to the Linux section mentioning it (with tool compat, etc) and removing the outdated last sentence of that section (though ideally, the specifications from those PEPs would be migrated to be inline parts of the overall platform compatibility tag spec, that's a fairly non-trivial task).

[lyric hedge]

I'm installing Black 22.12.0 on CPython 3.9.0b1, surprisingly the tomli requirement is not installed. https://paste.pythondiscord.com/iwixisuwab

(3.9venv) ichard26@acer-ubuntu:~/programming/oss/black$ python -m pip install black && black --version
Collecting black
  Using cached black-22.12.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Collecting pathspec>=0.9.0
  Using cached pathspec-0.10.3-py3-none-any.whl (29 kB)
Collecting click>=8.0.0
  Using cached click-8.1.3-py3-none-any.whl (96 kB)
Collecting typing-extensions>=3.10.0.0
  Using cached typing_extensions-4.4.0-py3-none-any.whl (26 kB)
Collecting mypy-extensions>=0.4.3
  Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Collecting platformdirs>=2
  Using cached platformdirs-2.6.0-py3-none-any.whl (14 kB)
Installing collected packages: mypy-extensions, typing-extensions, platformdirs, pathspec, click, black
Successfully installed black-22.12.0 click-8.1.3 mypy-extensions-0.4.3 pathspec-0.10.3 platformdirs-2.6.0 typing-extensions-4.4.0
Traceback (most recent call last):
  File "/home/ichard26/programming/oss/black/3.9venv/bin/black", line 5, in <module>
    from black import patched_main
  File "src/black/__init__.py", line 45, in <module>
    from black.files import (
  File "/home/ichard26/programming/oss/black/3.9venv/lib/python3.9/site-packages/black/files.py", line 32, in <module>
    import tomli as tomllib
ModuleNotFoundError: No module named 'tomli'

I suspect our use of python_full_version is to blame, but I'm not sure:

  "tomli>=1.1.0; python_full_version < '3.11.0a7'",

[queen hornet]

https://peps.python.org/pep-0508/#environment-markers says python_full_version is equivalent to platform.python_version():

>>> platform.python_version()
'3.9.16'
>>> platform.python_version() < '3.11.0a7'
False
>>> '3.9.16' < '3.11.0a7'
False

[lyric hedge]

https://github.com/pypa/packaging/pull/523 turns out this has been fixed already actually

packaging 22.0 is behaving as expected

>>> from packaging.markers import Marker
>>> m = Marker("python_full_version < '3.11.0a7'")
>>> m.evaluate({"python_full_version": "3.8.8rc1", "python_version": "3.8"})
True
>>> m.evaluate({"python_full_version": "3.8.8", "python_version": "3.8"})
True

[queen hornet]

I'd probably just simplify it to something like tomli>=1.0.0;python_version<"3.11", no need to worry about 3.11 alphas now

[lyric hedge]

Fair point

[distant briar]

@supple palm I don't know your exact setup but this is the basic idea:

from pathlib import Path
 
from setuptools import setup
 
ROOT_FOLDER = Path(__file__).parent.absolute()
REQUIREMENTS_FOLDER = ROOT_FOLDER / "requirements"
 
 
def get_requirements(fp):
    return [
        line.strip()
        for line in fp.read().splitlines()
        if line.strip() and not line.strip().startswith("#")
    ]


extras_require = {}
for file in REQUIREMENTS_FOLDER.glob("extra-*.in"):
    with file.open(encoding="utf-8") as fp:
        extras_require[file.stem[len("extra-") :]] = get_requirements(fp)

pins = set()
for file in REQUIREMENTS_FOLDER.glob("*.txt"):
    with file.open(encoding="utf-8") as fp:
        pins.update(get_requirements(fp))


extras_require["pinned"] = list(pins)


# Metadata and options defined in pyproject.toml
setup(extras_require=extras_require)

[echo ruin]

do we have a tool that knows how to install the earliest / oldest versions of all dependencies of a package?

pip-compile doesn't know how to do so right? (in that case to generate a requirements.txt with those versions presumably)

but e.g. if your package foo has lower pins on bar and baz, do we have a tool runnable in CI which can install the earliest feasible versions someone may get

[high stone]

I don't know of such a tool.

I know that a lot of people have asked for this tho.

https://github.com/pypa/pip/issues/8085

[indigo token]

It's the way one can have a meaningful dependency array. Without it, maintainers can only helplessly gesture in the direction of “yeah maybe update your packages and it'll work then” whenever users encounter a problem.

[golden falcon]

https://github.com/FFY00/python-resolver#mindeps-cli

[supple palm]

openstack folks have some tools for building an oldest constraint file if i remember well. But if you add an option to pip-tools to allow creation of such file, I would be happy to review it.

[lusty quarry]

@supple palm Hatch makes that a lot easier https://github.com/repo-helper/hatch-requirements-txt#usage

[fallen shuttle]

In setuptools nowadays you can also have a {"file" = ...} directive to make this easier https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html#dynamic-metadata

(As long as all lines in the requirements.txt file are PEP 508 compatible)

[echo ruin]

thanks for the link

[echo ruin]

cool! thanks too

(and happy holidays to all)

[golden falcon]

Linking it again in case you missed it… this will calculate the min versions of your deps which you can then install with pip: https://github.com/FFY00/python-resolver#mindeps-cli

[distant briar]

when do you think all of tool.setuptools table stops being in beta?

[echo ruin]

amazing, this indeed looks like it's the thing :), another point for @tidal kiln -- I can't seem to run it though, it seems to want the build backend installed in the same virtual environment, but it conflicts with the versions required by hatchling so I can't do so

and.. I can't get hatch to spit out dependencies either

[tidal kiln]

it doesn't support sdists though!

it's not really very production ready

[echo ruin]

sorry I hope that didn't ping you, I saw you had DND on after

[tidal kiln]

I set DND on discord years ago, I don't really update it

no worries

feel free to ping me

[echo ruin]

ah! yeah understood on sdists (I'm not sure that'll affect me, we'll see, I think jsonschema has no non-binary deps so maybe not)

[tidal kiln]

I managed to sneak out of christmas and am now trying to fix my electron microscope

so not really that busy

[echo ruin]

But am I using that wrong? If I install it into a virtualenv I just get ERROR Backend 'hatchling.build' is not available.

[echo ruin]

(and hatchling isn't installable there, it conflicts with the packaging version dep)

oooooo wow that looks cool

[tidal kiln]

I don't think it fetches the metadata in a isolated environment

should be easy enough to fix

[echo ruin]

ah got it ok -- gonna make lunch myself then here, but good to know this exists, will play with it again later

what do you use the microscope for?

oscilloscope stuff?

[tidal kiln]

I want to use it to make semiconductors, but first I need to get the thing to actually boot up properly though

it has been a PITA

[echo ruin]

what do things like that even run

some custom mini-OS?

[tidal kiln]

windows xp

🤣

it's a bunch of electronics connected to a controller computer

mine is from ~2000, it originally ran windows 95, but was upgraded

it uses a SCSI bus to communicate

[echo ruin]

... ha. ha ha. hahahahaha.

[tidal kiln]

and serial for some stuff

[noble matrix]

Classic Filipe 🤣

Meanwhile I snuck out of Christmas to check on my GH backlog (and you guys, of course)

[rapid glacier]

This message is what makes burn-out psychotherapists see 💰-signs 😉

Just joking, I endorse sneaking out of christmas for any reason!

[noble matrix]

Just like my favorite Christmas story, "The CAM who stole (out of) Christmas"

[woven plover]

I'm wondering, anyone aware of a nice way to inject distributions and their Metadata into a python process

Im trying to figure a sane/safe way to bootstrap execnet into local/remote pythons that works without frankenstein-ing python modules into strings for exec

[lusty quarry]

fyi #hatch repo is getting spammed, I keep having to report and delete comments. 3 in the last half an hour

[lyric hedge]

that's unfortunate :(

[distant briar]

a lot of GH repos are getting spammed right now

seen it on dotnet/maui, community/community, python/typing

[rapid glacier]

What kind of spam? Advertisement? Non-english language? Random gibberish?

[noble matrix]

Reported to GitHub

[distant briar]

I only reported 2 out of 3 I saw because I got rate limited by the report system

[soft glen]

Same thing happens at the moment at the Poetry repos. 😕

[spiral urchin]

Breaking: Python packaging finally getting the attention it deserves

[rapid glacier]

Would be really interesting to see how they determine who to tag, as it seems to be people not connected to that repo, just random.

Still shit that this crap now has reached github en mass also.

[noble matrix]

I reported it directly to GitHub people

[high stone]

Oh hey, I wasn't the only one. ;)

But yea, this looks like a giant bot farm so... 🤷🏻‍♂️

[topaz pine]

Oh really? That's awful 😞

[lyric hedge]

oh hi, haven't seen you around in a while!

[topaz pine]

Oh, that's right!

[lyric hedge]

I don't think we've met actually. I'm Richard, I lurk the pip repository too much 😅

[topaz pine]

Oh, nice to meet ya! 👋

BTW, have I ever posted in this server before? 🤔

I think I hadn't

But anyway, I promise to hang around a bit more 🙂

[echo ruin]

I don't know whether this is a tox issue (unlikely), a pip issue (unlikely?), a markupsafe issue (unlikelyish), a GH actions issue (possible but I have no idea how), a Julian issue (always likely), or an other issue (likelyish seemingly), but some of my installs fail only when run in GitHub actions via cron but not when run via trigger??

This seems to happen somewhat deterministically recently. E.g.: https://github.com/python-jsonschema/jsonschema/actions/runs/3791488367/jobs/6446954364

(The same commit passed when run normally, and the same sort of behavior has happened with the scheduled builds the past few weeks, with different commits)

Where what happened there is somehow markupsafe wasn't installed, even though it's present in the requirements.txt (https://github.com/python-jsonschema/jsonschema/blob/main/docs/requirements.txt#L41 ) -- in some builds I've seen installs saying that what was installed was markupsafe==0.0.0 so some packaging metadata being screwy seems involved

But can anyone think of what might be happening there?

[mossy pulsar]

You likely want to enable temporarily running with tox -vvv so the logs are more descriptive 😊

My guess is that pip fails to install latest and falls back to older version 🤷‍♂️ so perhaps pypi/GitHub networking issue

[echo ruin]

ah and older version has metadata issues

ok good guess

wait but it's == pinned

-vvv obviously is a good suggestion, lemme try that and we shall see this evening...

[gritty roost]

👋

[lusty quarry]

done! https://pypi.org/project/hatchling/

[lusty quarry]

@merry rune you might like this https://discuss.python.org/t/pypackaging-native-content-about-the-state-of-using-native-code-python-packaging/22306

[queen hornet]

"Compromised PyTorch-nightly dependency chain between December 25th and December 30th, 2022." https://pytorch.org/blog/compromised-nightly-dependency/

[gritty roost]

Are there any dependency resolvers today that use lock files and also include build dependencies in their lock files?

[blazing lantern]

I don't know why, but I expected an electron microscope to be bigger. Thinking about it, it does somewhat make sense that something which looks at REALLY small things doesn't need to be big 😅

[blazing lantern]

Not at the moment. I think https://discuss.python.org/t/pep-665-take-2-a-file-format-to-list-python-dependencies-for-reproducibility-of-an-application/11736/ is the latest discussion on the topic that includes talking about locking build requirements (but I thought we had talked about it more recently than last January on discuss.python.org)

[tidal kiln]

the newer ones are smaller, a lot of the electronics can be sized way down from what they were 20 years ago

but you always have the column, and the vacuum pumps, which cannot really be sized down much if you want to keep the same performance

mine is the same as this one, but with an extra vacuum pump on the back, and an extra box with vacuum valves and stuff

in newer models you can get rid of the operation unit, because of the smaller electronics

but you'll always need a table anyway 🤣

ah, you also need a water chiller on older units, because they use diffusion pumps, instead of turbomolecular pumps

[topaz pine]

Hi, welcome!

[gritty roost]

I think maybe it's been discussed in the recent lock file format thread

[vast jasper]

Hey guys, tryiong to get a PyPi or Warehouse server running in a Docker Container with packages being stored in S3 anyone ever seen or done something similar?

[blazing lantern]

I believe you're after something like https://pypi.org/project/devpi/

[gritty roost]

Anyone interested in cross compiling python modules?

I've been working on new Python rules for Bazel on and off for a while. Recently I've been playing around with zig cc for cross compilation and, combined with a fork of https://github.com/benfogle/crossenv/,

I've been able to build some linux wheels on macos, complete with running through a monkeypatched auditwheel: psycopg2 dynamically linked to libpq and libssl (auditwheel copies and patches correctly), and numpy statically linked to openblas.

Can also build macos wheels on linux, but I still need to get delocate to work on linux.

[blazing lantern]

I'm definitely interested from the perspective of getting WASI .o files to statically link to a Python WASI binary/app

[gritty roost]

Haven't played with WASI too much but I do know that zig cc can target it.

[tidal kiln]

I am. I'd really like to get better cross compilation support in CPython and sysconfig, for packages.

Isuru (https://github.com/isuruf) has also been doing a lot of work in this area, might be worth to reach out

[gritty roost]

ah yeah I've seen Isuru around.

[tidal kiln]

if you have any thoughts on how we could improve support in the upstream, I am happy to hear btw 😊

I particularly dislike how everyone has been relying on _PYTHON_SYSCONFIGDATA_NAME

@blazing lantern did CPython ever support multiple arch installations?

by this I mean, like 32bit and 64bit installations in the same place

[gritty roost]

I'm relying on that as well, I believe. It's the best way I've found.

[tidal kiln]

yep, you need to, there's no alternative

[gritty roost]

If you're curious, here's the definition of psycopg2 that links against libpq and libssl. It cross compiles from macos to linux: https://github.com/jvolkman/rules_pycross/blob/dev/external/examples/external_linking/deps/psycopg2/BUILD.bazel

I added a small README in the parent directory

The biggest problem is sys.platform, as its use is way overloaded and its value has to differ based on who is asking

And so through the magic of module-level __getattr__, I ended up with this, which has worked better than it has any right to: https://github.com/jvolkman/rules_pycross/blob/dev/external/pycross/private/tools/crossenv/scripts/sys-patch.py.tmpl#L21

[tidal kiln]

humm, interesting, that's very nice to know! I would have assumed sys.platform was easy to deal with because how seemingly simple it is

okay, do you know what breaks when it is not patched?

btw, if you have instructions for me to try out your workflow, it'd also help

[gritty roost]

Problem is there are things in the standard library that look at it. Dns resolution maybe? I haven't looked for a while

[tidal kiln]

we have an issue to stop relying on sys.version in sysconfig https://github.com/python/cpython/issues/81773

we could consider doing the same for sys.platform

or at least add a way to better override it when cross compiling

[gritty roost]

I added some instructions here: https://github.com/jvolkman/rules_pycross/tree/dev/external/examples/external_linking/deps/psycopg2

let me see what breaks if I get rid of that hack

[tidal kiln]

ah right

🤦‍♂️

I'm pretty sure stuff will break in sysconfig, but it'd be great to know if it's isolated to that module or something else also breaks

[gritty roost]

Ah yeah, importing urllib.request is the first thing that breaks

If I'm targeting macos, because https://github.com/python/cpython/blob/main/Lib/urllib/request.py#L2637

That's just the first thing; i'm sure there will be others

[tidal kiln]

hummm let me look

what is your system and what are you trying to cross compile to?

[gritty roost]

That breaks if cross compiling to macos from linux

    from _scproxy import _get_proxy_settings, _get_proxies
ModuleNotFoundError: No module named '_scproxy'

[tidal kiln]

but this is with or without the sys.platform patching?

[gritty roost]

that's without

with the patching cross compiling works

[tidal kiln]

uh, how exactly does urllib.request break?

compiled module for your platform is not available?

[blazing lantern]

Not that I am specifically aware of, but I'm not an expert in this area

[tidal kiln]

that'd be my guess

[tidal kiln]

okay, thanks, is there any reason you can think off why we need to include the arch in the sysconfig data module name?

[gritty roost]

yeah, I guess _scproxy doesn't exist on linux builds of cpython.

[tidal kiln]

I am not really seeing anything right now, but I haven't looked into it that much

[blazing lantern]

For stuff that ends up in the .so file name, it's useful for installing multiple wheels for the same package over itself so you can ship multiple wheels easily without having to care what platform the user is on (e.g. it's how we ship accelerated debugpy support in VS Code)

[tidal kiln]

gotcha

but I am talking about the _sysconfigdata_* module we generate

here https://github.com/python/cpython/blob/1d4d677d1c90fcf4886ded0bf04b8f9d5b60b909/Lib/sysconfig.py#L465

[blazing lantern]

Still not an expert 😉

[untold shadow]

👋

[lusty quarry]

cross-posting here since I think this might affect us one day https://github.com/python/cpython/issues/100827

[lusty quarry]

@merry rune if you have any news to share regarding your work on native code/extension modules there is talk about that happening now 🙂 https://discuss.python.org/t/python-packaging-strategy-discussion-part-1/22420

[merry rune]

Thanks! I'll try to read through it and respond soon. Final grades due today for the class I was teaching 😓

[lusty quarry]

no problem! it seems many in the thread have doubts about the path forward for building extension modules in general 😢

I dunno, all the comments make me feel like I'm missing something fundamental but I really don't think I am

[lusty quarry]

just a heads-up since this might require changes to support the new ABI wheel tag https://peps.python.org/pep-0703/#backwards-compatibility

[inland prawn]

https://tenor.com/view/oh-my-god-its-happening-ok-stay-calm-the-office-gif-6091046

[blazing lantern]

I wouldn't get excited just yet. A PEP just means someone wrote an idea down, not that it will get accepted. There's a lot to consider as to whether dropping the GIL, subinterpreters, or neither solution is the best thing for Python (e.g. imagine all those broken extension modules 😅)

[blazing lantern]

It might, but I'm not going to worry about until we know whether the PEP will be accepted

[blazing lantern]

I think some of the folks who are doubting your approach have been burned for so long by packaging they are just inherently pessimistic by any attempts to fix it. And on top that they spend a lot of their time just dealing with troublesome modules, and I can see why they wouldn't believe it could be made easier. I think the best you could do is show them how it would work for a variety of extension modules ranging from the simple to the most complex (and I would ask them to make a list of like 3 - 5 extension modules of varying complexity to use as examples).

[high stone]

FWIW, I don't have the bandwidth for extended responses on that discuss.python.org thread, but +1 to what Brett said.

[inland prawn]

Of course, but at least there is something written down, not just discussed as pure theory. I had a chance to talk with Łukasz Langa in December and he explained some of the issues around dropping GIL

[stable laurel]

i have some code i would like to test, can some tell where to go or help me with the code

[inland prawn]

I would recommend general Python Discord: https://discord.gg/python

[stable laurel]

im new to python and cant seem to find the problem, I hope some can help me. I would like to ask someone in privet about the problem

[noble matrix]

As @inland prawn mentioned, the linked Python Discord is the place for that, not here, sorry. This Discord is for discussion of Python packaging specs, tools and related topics. You could also ask in the Python Help section of the Python Discourse https://discuss.python.org/c/users/7

[lyric hedge]

*puts on staff hat* Just a quick note that we generally discourage helping in DMs at Python Discord. It's not prohibited, but we prefer that you use our help system and open a thread. (Check out the #how-to-get-help channel for more.) You're totally welcome in Python Discord, but I do want to set expectations early.

[inland prawn]

Seems like Github finally figured out how to make dependency graphs based on PEP 621 metadata

[lyric hedge]

That's nice to hear. Are the linked repositories still totally messed up?

[inland prawn]

https://github.com/psf/black/network/dependents

[topaz pine]

Hi, welcome!

[distant briar]

I've just learnt that setuptools will not be a part of ensurepip in 3.12. Do I understand correctly that this means that projects which only have setup.py (no pyproject.toml) will no longer build due to lack of setuptools in the build environment? Or will pip be changed to use the default build-system table in such a case? Apparently that already happened in pip 22.1, should have read the relevant issue on cpython repo...

Looks like once pip 23.1 is released, I can stop telling users to run python -m pip install -U pip wheel and instead only tell them to run python -m pip install -U pip as in pip 23.1 this behaviour will be applied to all environments regardless of whether they have setuptools or not (currently if they do, setup.py install is used instead)

[lusty quarry]

meta: https://news.ycombinator.com/item?id=34390585

Kwpolska

How to improve Python packaging

Hacker News · Jan 15, 2023, 4:27 PM

[lusty quarry]

it's a bummer that to understand the arguments against PEP 582 one has to have a nontrivial understanding of Python and packaging

(of course some advocates do have a deep understanding but many seem to be newcomers)

[inland prawn]

honestly, I just prefer having virtualenvs. it gives me so clean separation of packages and control over what is installed where

[noble matrix]

A fairly big tangent but FWIW, I think that's one place where the "A" in PyPA standing for "Authority" instead of "Association" is misleading to the point of being actively harmful. Sure, it's a cute and funny in-joke, but one that, by it's inherent nature, outsiders aren't going to get. In this case, it results in a collective perception that a monolithic PyPA is somehow the bosses of the Python packaging ecosystem and willingly let things get how they are, instead of being a collecitve of most (but not quite all) Python packaging projects trying to collaborate and work toward interoperability.

Both in the blog post and in HN and the Reddit threads I saw plenty of people complaining specifically about the "Authorities". And that's how you get people saying things like "The Python Packaging Authority must be destroyed."

And people seem to think the PyPA's role is to write and mandate standards and are complaining about that, rather than just being a community of projects where 99% of the effort goes toward actual maintenance. Again, "Association" rather than "Authority" would clarify that true role

Ultimately, the PyPA is composed entirely by the maintainers of the tools. So naturally, their primary task is going to be maintaining said tools like people want, not working on standards. But conversely, it is unsurprising to have said maintainers not be able to agree to all dump their own tools and agree to promote another tool instead.

Another thing I gathered from both the blog post and the HN thread was that we really need to either update, hide or remove the older guides on the packaging.python.org site, as they are clearly doing nothing but confusing people and drawing them away from the main "How to package Python projects" tutorial, i.e. the one guide/tutorial that's actually mostly up to date, complete and important. I've really wanted to help work on that some more, but I just haven't found the time yet...

[bitter prawn]

Hi every one, im new here
How can i use Event Listener with tronpy library please

[lyric hedge]

Hi! Blockchain is wildly off-topic here. We talk about packaging, not the TRON protocol. Maybe discord.gg/python will be better?

[golden falcon]

Well, the PyPA might not be an "authority" but it is special. It has a mandate from the SC, its structure's codified in a PEP, it produces its own PEPs, its tools come bundled with Python, several of its members are core Python devs or triagers, etc. There are all these things that make people reasonably think that PyPA gets to decide the direction of Python packaging (which it very often does - see the kerfuffle around Poetry not supporting the project table cuz they don't like the newly standardised dependency format), the term "authority" aside

[hidden flame]

What would it take to turn "authority" into "association"?

[queen hornet]

a PyPA committers' vote? https://peps.python.org/pep-0609/#pypa-committer-votes

this discussion last year on renaming to "Python Packaging team" also had a suggestion for "Python Packaging Association" which had quite wide (but not unanimous) support
https://discuss.python.org/t/how-do-people-feel-about-a-python-packaging-authority-pypa-python-packaging-team-renaming/14272

[spiral urchin]

I think the main blocker is someone needs to nominate themselves to make all the necessary PRs to fix all the things, because while the people are mostly OK with the change nobody really wants to bother

[queen hornet]

I can help with that

[rapid glacier]

Will not be a real solution as the internet is filled with guides that are from different times. The way how packaging changed relativ quickly in recent times means that quite a lot is wrong and worse, wrong in different ways.

I am mostly talking about blogs, stack overflow and even some reddit posts.

[topaz pine]

I can also help! I'm a bit busy now, but if this gets done at some point, I could spend my free time if there's no problem 🙂

[noble matrix]

I can commit to helping as well; at least in terms of resources the PyPA controls, what are the other places (that are PR-able) besides the PyPA website (pypa.io), packaging.python.org, PEP 609, and maybe a few scattered references on Warehouse, the CPython docs, the SC docs, and individual PyPA projects' readmes/docs? It helps a lot that the name is pretty infrequently spelled out, so there likely aren't that many instances to update in any place that it is used.

[noble matrix]

I would refer you to the classic story of The Star Thrower: https://en.wikipedia.org/wiki/The_Star_Thrower

Yes indeed, there is absolutely a ton of outdated and just plain wrong out there in blog posts, tutorial/guide sites, content farms, forums, and (less so especially over time, due to its nature) SO, and others, and likely a majority (if not a large majority) of users find their packaging advice there rather than on packaging.python.org or other PSF/PyPA related resources.

It's of course not a magic bullet, but it would still make a difference, given it's clear that a substantial proportion of users are referencing the site, given these issues were called out repeatedly in the blog post and thread comments. And those blogs and guides such have to get their information from somewhere that's a source of official truth—if even the quasi-"official" site is outdated, unclear and confusing for the average user, how can we expect anyone else to do better?

That said, my talk is cheap; what really makes a difference is me or someone else actually stepping up and successfully getting PRs merged to do this, which has been mostly Henry and others so far—I'd burned myself out on it before trying to do too much, but it's high on my priority list of things to get back to.

[rapid glacier]

I didn't want to say, the effort is useless, or something like that.

My intention was rather twofold:

1. If you do not see an effect after fixing the official site, consider the blogs, stack overflow,... and not only the actual content of the official site.

2. Getting directed to the official site was ,at least for me some time back and even a few python-contributor from what I saw (heck I think I saw Pypa-contributor here that did not know it existed), difficult. Getting the official site more popular than all the blogs and stuff will probably help a lot.

I want to add, that those wrong solutions can also be a resource:

*Maybe add a section that takes the most popular wrong approaches (from a google search maybe) and how to convert them to the right approach.*

---

I would really like to help with the stuff, but to be honest, I am one of the people that is so confused by the current state, that I really do not know how to do my own packaging currently , correctly.

[noble matrix]

Improving Packaging.Python.org

[blazing lantern]

See https://pradyunsg.me/blog/2023/01/14/python-packaging-organisation/ for an explanation and the discuss.python.org thread that I think Pradyun started once about tweaking the name (which ultimately didn't go anywhere conclusively other than not to bother).

[blazing lantern]

I don't give much attention to those who use such phrasing, especially when they are not participating in any way in the work

[blazing lantern]

You still owe me an updated PEP on licenses first 😉

[lusty quarry]

@noble matrix regarding ^, is there anything I can do to help? I have a vested interest because I get pinged 2 or 3 times a month about Hatchling's support of your PEP because linters do not yet recognize a string for that field 😅

[inland prawn]

anyone has any materials/articles about difference between "bdist" and "wheel"?

[queen hornet]

https://docs.python.org/3/distutils/builtdist.html ?

[golden falcon]

a "bdist" is a built distribution in setuptools parlance, the wheel format is the only surviving type of built distribution

[queen hornet]

Wheel is just one bdist, there used to be loads of others in active use, bdist_rpm, bdist_msi etc

https://peps.python.org/pep-0527/ got rid of a load of old ones

[inland prawn]

Thanks

[hidden flame]

Does anyone know if a packaging summit will happen at PyCon US 2023 and who organizes it?

[blazing lantern]

I'm not aware of any explicit plans, but I also don't know if anyone has brought it up to start trying to organize it either. I can say that when I tried to organize a WebAssembly summit I was told there were no rooms available on Thursday.

[hidden flame]

I somewhat remember that @tidal kiln said they might be organizing, but maybe I'm misremembering.

[hidden flame]

Regarding WASM summit, I think I'd be interested in that as well 😬

[blazing lantern]

It's actually not happening due to me not being able to make it Wednesday (language summit), and no rooms on Thursday. I suspect we will try to do a BoF at least and try to coordinate better for next year (unless you all coordinate during the sprints, but I have to miss the sprints this year).

[hidden flame]

What if we would organize the summit outside rooms available to PyCon?

[blazing lantern]

You mean in the hall, or at an external venue?

[hidden flame]

An external venue, or if the conference venue has rooms available not booked by PyCon?

[blazing lantern]

Last people I know to try this who weren't a business was the TIP BoF and they decided not to do it as they didn't want to worry about conduct, cost, and trying to get folks off-site

[hidden flame]

right, all reasonable reasons

[coarse mason]

Preferring virtualenvs

[high stone]

There's plans, and we've (CAM, Filipe, I) have triggered the discussions with the organisers. Once we have a clearer picture, we'll announce it.

[lusty quarry]

is virtual participation a thing or do I have to be there?

[lusty quarry]

well, we now have one partial endorsement for the extension module standardization 🙂 https://github.com/PyO3/maturin/issues/1419#issuecomment-1397062760

[topaz pine]

Would be great to see virtual participation is possible. I probably won't be able to attend in-person for a while, and it seems like I'm not the only one? 🤷‍♂️

[round rapids]

Are the #pypa/#pypa-dev channels on Libera still active/valid? Does anyone know how I can get some eyes on https://github.com/pypa/wheel/pull/491? TIA

[high stone]

Virtual Participation in a PyCon Packaging Summit

[topaz pine]

Oh, hello, good to see you here!

[drifting sigil]

👋

[lusty quarry]

@drifting sigil I very much enjoyed your Twitter thread 🙂

[drifting sigil]

that makes one of us

[inland prawn]

any chance for a link?

[lusty quarry]

https://twitter.com/theavalkyrie/status/1614842051580813318

Stargirl🌠 (@theavalkyrie)

So You Want to Solve Python Packaging: A Practical Guide

Likes

465

Retweets

101

Twitter · Jan 16, 2023, 4:28 AM

[inland prawn]

interesting and true

[high stone]

Welcome! It's lovely to see you here! :-)

[mortal shore]

I know how to fix Python packaging

aws s3 delete-bucket --bucket pypi-files

https://giphy.com/gifs/moodman-modern-problems-require-solutions-9058ZMj6ooluP4UUPl

[inland prawn]

imagine someone actually did this...

[mortal shore]

technically that command would be fine to run, since we stopped using S3 but I'm too lazy to look up the invocation for google cloud 😛

[inland prawn]

I meant more generally, just the fact of the deletion

[mortal shore]

yea 😉

it would depend on exactly what they did

PyPI has a mirror of PyPI (though it may not be working fully atm...) in a dedicated DC that doesn't use cloud storage

so you could recover from that for instance

[inland prawn]

that's reassuring... I could only imagine how slow the GH Actions pipelines would be with everyone recompiling packages after such disaster

[mortal shore]

the pipelines would be very fast

since source dists would be deleted too 😄

I'll be honest though, I'm actually fairly optimistic that it's possible to "fix" Python Packaging, it's just a lot of work

[inland prawn]

I thought about platform-specific wheels (especially the ones with $LANGUAGE extensions)

[mortal shore]

yea, PyPI just doesn't differentiate in storage

it's just blobs of bits identified by a filename

[echo ruin]

Them: "Abolish PyPA" Not Them: "You said abolish PyPI right?"

[drifting sigil]

"Abolish PyPA", "okay fine but we're taking our toys (PyPI) with us"

[high stone]

Anyone here planning to stay for the sprints at PyCon US? How long will you be staying for?

[lofty aspen]

@drifting sigil nice encapsulation that doesn't even touch the technical issues of the bootstrap problem that plagues all the attempts by web devs .

[hidden flame]

There s plans and we ve CAM Filipe I

[distant briar]

TIL Pradyun isn't a CPython core developer

I should add yet since there's a vote to promote that ends in 4 hours lol

[queen hornet]

🎉 the vote passed! https://discuss.python.org/t/vote-to-promote-pradyun-gedam/23001
it's still a yet: to make it official, the next step is Steering Council's stamp of approval, shouldn't be an issue 🙂

[topaz pine]

Congrats Pradyun! 🎉

[high stone]

Thank you! ^.^

[bright parcel]

@high stone Really great blog post, thank you for that. I've been sharing it with colleagues.

[lyric hedge]

Seconded, it's a great read. Echoes some of the things I've heard users say to me.

[bright parcel]

I do wonder about the conclusion/proposal to make pip the standard workflow manager for Python. I understand the pros and cons, but I worry about the cons more (like all the legacy stuff pip has to carry around). My other question is whether it really makes sense for the full pip to be included in the stdlib. Like, do people need that full functionality or do they only need parts of that. I genuinely ask that question because for me, all I need is support for installing packages in a python -m venv.

[lusty quarry]

I do wonder about the conclusion/proposal to make pip the standard workflow manager for Python.
@high stone can address that directly to you but actually I found out while chatting with him over DM that he wasn't expecting people to interpret that as a proposal (that was my initial interpretation as well)

[lyric hedge]

I also interpreted it that way 😅

[high stone]

Yea, I wrote that more as "Don't write it off" and to state that it is still an option. At this point tho, I'm not sure it's worth editing it to clarify that.

[mortal shore]

My argument for including... whatever the "full tool" is (which atm is pip) in the stdlib is that users obviously want a full tool, there are mutliple of them available... and they're still asking for it?

so just having it available for them to install doesn't seem to be enough

[mortal shore]

Though a posy style thing if we start pushing that on python.org and all of the various docs, etc would work as well.

[elder kettle]

this posy? https://github.com/njsmith/posy

[mortal shore]

Yea

meaning, I don't think end users particularly care whether they're told to install the package manager and that brings along Python or if they are told to install Python and that brings along the package manager

but they seem to not want to have to install and manage both independently

[queen hornet]

the idea from the big Discourse topic that if the user calls pip thing, and there's a pip-thing installed, pip can just call that pip-thing, that sounds like a workable solution?

one problem with adding more stuff to pip is all the legacy code, they'd need to be added to the project, need to help out with all that

whereas a pip-thing could be in its own repo and let the new devs do their own thing. would help keep things separate

I'm probably oversimplifying things though!

[mortal shore]

I think the legacy code thing is kind of overblown, like yea it exists and it makes it harder to add code to pip than in a greenfield project

but I think the real difficult thing is just like

deciding if that's what we as an ecosystem want to do

and folks are doing good work in removing a lot of the legacy code from pip anyways 😉

[high stone]

Also, a big reason it's harder to add things into pip is also because you'll get pushed to cover more "edge" cases (like what if someone's doing this offline and whatnot) which a greenfield tool can just... ignore?

[mortal shore]

Well

yes and no

I think they can just ignore them to start out with

[high stone]

Yea, exactly.

Until they can't.

[mortal shore]

but part of why pip has to cover those edge cases is because it's "the thing" that is the "default", so it's more like you get to defer thinking about them

[high stone]

eg: Poetry's hitting this to some extent with the caret operator pushback.

Everyone uses Semver-like until they don't.

[mortal shore]

I don't honestly care whether pip gains those features or some other tool gets adopted tbh. Like I think most of the options have various trade offs and most of them are something we can shape into a better world

[inland prawn]

Well, in our case it's more that it's not part of a bigger packaging environment and it clashes with what is expected (mostly python version >=3.x vs ^3.x)

[mortal shore]

I personally suspect the... social aspects of adopting another tool as "the default" is harder than the technical aspects of evolving pip if we're sticking with the "you install Python, Python brings the installer" model, but that's just like, my opinion

[spiral urchin]

One reason I really hope pip does not become the standard workflow management tool is pip’s current feature set is decidedly not for workflow management and I can’t imagine the mess its interface would look like during the transition period when its both a workflow tool and not a workflow tool

[mortal shore]

eh

I don't actually think that is particularly hard

like cargo is workflow management right

and it has similiar commands that pip already has

[inland prawn]

Looking at how many issues Poetry has being "pip does this, do the same", I can see the problems piling up

[high stone]

Same FWIW -- I think we need a default and I don't care if the default is pip or something else (as long as we can all get on board with it).

And, agree on the social aspects.

[inland prawn]

Yeah, I agree. IMHO pip should remain as it is (maybe provide a library-like interface for using it from other tools) which is a low-level installation tool

[mortal shore]

I suspect the chances of pip gaining a library like interface are very low, all of those legacy code issues affect the ability to library-ize pip much more than any other idea for what to do with pip. I think the general idea to spin things out of pip into their own libraries, that pip then consumes is still the best and most likely path forward for that

I think "low level" is the wrong way to phrase what pip is or has ever tried to be, that's much more like what pypa/installer is trying to be IMO. I think just historically it's been primarily focused on installing packages, and left environment management and project management up to the end user.

Though the history on that is a bit funny, because if you go back far enough, virtualenv and pip were basically developed in tandem, by the same people, and were somewhat conjoined (you couldn't release one without the other if my memory serves..., the test frameworks for them both were very entertwined and there were tests for pip in the virtualenv test suite, and tests for virtualenv in the pip test suite). So in some ways environment management wasn't... not something that was part of the larger "pip" project.

I... think that it even had facilities to create virtualenv environments implicitly at one point if we go back far enough. A lot of that stuff got removed over time because it was implemented in a super janky way, and the implementation was causing tons of problems, and also the two projects ended up getting better isolated from each other so they can be developed independently just for like, the maintainers own sanity.

[spiral urchin]

I think pip is considered “low level” in the same sense gcc is considered low level (by some). It’s actually pretty high level in many senses but still is not designed as, and also should not be the command users directly invoke in their development workflow (in general, at least)

It’s all relative when people talk about high and low levels

[inland prawn]

by "low-level" I meant that pip has a lot of knobs and switches to tweak how the package gets installed. compilation, sources, binary/no-binary etc

[mortal shore]

oh pip almost certainly has too many knobs 😉

just in like general

[inland prawn]

I agree, but "workflow tools" often replicate the code that pip already has and that is some kind of abstraction over what those libraries provide. and that abstraction layer could be the "library" I had in mind

[distant briar]

@noble matrix Did GitHub finally add PR template chooser to web UI?
(re: https://github.com/python/peps/pull/2998#issuecomment-1412856422)

(just realised, should have been off-topic but this won't turn into discussion so I guess it's fine)

[noble matrix]

Not officially yet, it's still pre-alpha experimental, enabled for the peps repo only by our special request. But eventually they'll likely release it officially, hopefully with some more polish (ideally, a chooser like for issue templates, or at least support for the data block at the top to set the title, labels, description, etc.).

We basically had to merge the original PR blind and then try to figure out what worked and what didn't, since there is no docs and very limited information from them, and there was no way to test it before merging to the upstream repo.

[distant briar]

Cool, I wonder if they have any plans for forms as well or just templates

[noble matrix]

I hope so, I will mention that...it's still in early development at this point.

[hidden flame]

Yep, pip and virtualenv were not really two projects, remember that Ian worked on them in tandem and we tried to continue that tradition when we took over, which failed when we added more people to the team and had a lot more feature churn/creep

I would say, our lack of organizational skills for a project like that were not enough, and yet it was good enough to make progress compared what existed before, so 🤷

I'm fascinated by the conversations a few days ago btw, this is probably almost identical to conversations we had a decade prior 😬

it would be really beneficial to python packaging if we'd resist the temptation to work on library APIs for the sake of having an API and instead focus on what end-users (read: people who don't care about packaging if they have a choice) want. Which is a predictable, consistent user experience when dealing with complex problems. That's why I agree with the various opinions that ask for a common UI in front of spec-based libraries, like posy or Ralf's pyp (https://labs.quansight.org/blog/python-packaging-where-to-next)

[inland prawn]

Oh no, did Ruff run out of PyPI space again? 🤣

[wide stump]

Hahah no I'm just here to hang out! 🙈

[inland prawn]

On that note, thank you for Ruff, I love it (and I think a lot of people here agree that it's great)

[lyric hedge]

... 👀

[wide stump]

Wow thank you for saying that! It's really nice to hear especially after a long day of grinding through issues 🤣

[lyric hedge]

I wish that I was even 50% efficient as you. I could never work through issues that fast.

[wide stump]

I'm sure you're 500% more efficient than me on projects that aren't Ruff 🙂

[lusty quarry]

oh hey!

[mossy pulsar]

This is how it always starts, first you just hang out, then you write that tool that becomes mega popular, a year down the line now. You're trying to fix python packaging forever

[lyric hedge]

Just replace packaging with anything and that's the life of a maintainer in a nutshell 😅

[wide stump]

A little naivete can go a long way 🤣

[mortal shore]

you can still run away

it's not too late

https://giphy.com/gifs/outlander-season-5-starz-05x07-VEzEYnPPio0f24lh3L

[bright parcel]

"Open source is great until you have users" someone once said

[inland prawn]

I am getting that on my wall

[high stone]

Welcome @wide stump!

[blazing lantern]

I work in open source so I know dreams don't come true

[rapid glacier]

Petition to rename discord server to Anonymous open saucers sourcers

[wide stump]

Thank you! Really nice to be here. It's amazing how much good conversation is freely accessible online if you just go looking for it 😂

Actually that's not true, I didn't go looking for it, I was invited by @elder kettle so thank you

[wide stump]

"I'm suffering from a serious case of 'having actual users'" is a sentence I said out loud once

[elder kettle]

no prob 🙂 love it here

[queen hornet]

This FOSDEM talk is starting in half an hour, there's streaming video too https://fosdem.org/2023/schedule/event/python_install_malware/

[bronze vector]

I have got a local pypi index that I use for uploading/installing pre-release packages
Now I would like to setup a mirror of my local pypi index
Once I have mirror index, I will add that in extra-index-urls in pip.conf
In this way, I will have a failover pypi index

The problem that I face is that, both python setup.py bdist_wheel --universal upload -r local and twine upload -r local dist/* only uploads to a single pypi index. I will have to run this again to upload to a different pypi index.

Do we have any tool to mirror local pypi index ? or publishing tool that let's you upload to multiple indices ?

[queen hornet]

have a look at bandersnatch https://pypi.org/project/bandersnatch/
they have a channel at #bandersnatch

[bronze vector]

Does this support mirroring of local pypi indices ?

[queen hornet]

ah, maybe not: that page says "bandersnatch's main goal is to support the main global index to local syncing only"

I've never used a PyPI mirror so hopefully someone else can give better info!

[woven plover]

How would i best go about renaming setuptools_scm, as its going to be more independent of setuptools in general

Im thinking of renaming it all together, and providing a shim package for old projects that just pins the new package with the setuptools extra + provides a shim

[inland prawn]

Scm_versioning?

[lusty quarry]

please use "vcs" instead of scm 🙂

[inland prawn]

Yeah, I agree actually

[indigo token]

You could join forces with dunamai maybe (not saying anything about its name, just about the fact that both packages overlap a lot)

[woven plover]

the name vcs_versioning is registred now

@lusty quarry i intend to have it superseed hatch-vcs as well

[lusty quarry]

you mean like config from your section should take precedence?

[woven plover]

@lusty quarry well, the alternative is to ensure hatch vcs doesnt take own settings in its section

a key problem with hatch-vcs is, that tis using a sifferent toml section thant setuptools_scm (which means the cli cant work on projects)

i want to provide commands for tagging new versions and similar , if a build backend uses own sections for the config, it has to reintegrate in unfortunate ways

well, atm all i have is the project name

[lusty quarry]

oh I see what you're saying I think, this new project would provide the necessary Hatchling plugin already right?

[woven plover]

yes

well, for now its just a name registred

i stumbled uppon https://github.com/hynek/build-and-inspect-python-package today - i wonder if it should be included in default recommendations

[queen hornet]

it looks very good. https://github.com/hynek/build-and-inspect-python-package/blob/main/action.yml is the file that does the work

here's an example run, scroll down and see how it makes artifacts of the metadata, packages and README, plus a nice summary of the contents
https://github.com/hynek/environ-config/actions/runs/4133100186

[woven plover]

bascially bruno rolled this into pytest and i was awestruck and wanted to share ^^

[inland prawn]

why does it sound a bit like "lets switch the plugin from setuptools to hatch as default"? (I know that it's not the goal, but couldn't help myself to notice this)

[woven plover]

@inland prawn well there is a oblivious connection, part of it is making setuptools no longer be the default an dependency, part of it is affinity with hatch

[inland prawn]

ok, let me ask a different question. If I wanted to use vcs_versioning with Poetry to allow for vcs version integration, would it be difficult? or would it be just a library with an API?

[woven plover]

depending on the apis of poetry, idealy it would provide both the apis and the poetry plugin

[elder kettle]

#4 was nice to see in this list

From the survey responses, it was clear that while Packaging tools were improving with time, most users found the landscape too complex to navigate. Based on the survey responses, the strategy discussion was condensed to these topics-

1. Development of a single Packaging tool or a more unified experience
2. Better support for Packaging users
3. Phasing out legacy systems
4. Supporting each other in the community
5. Encouraging long-term contribution

https://pyfound.blogspot.com/2023/02/python-packaging-strategy-discussion.html

[indigo token]

OK, so I read the packaging survey, specifically the “one true tool” thing and I’ve been thinking:

There’s quite some that the JS and Rust ecosystems (and to some extent Poetry) do for developer convenience that pure PEP 621 lacks. So if we want to extend something to be the “one true tool”, we need some way to do that. I’m thinking especially:

1. Lock files: Useful for application development, and when using dependabot, also for lib development (by making sure one is always compatible with the newest dep versions).
2. Automatic env management, especially temporarily pinning some dependency’s branch version with some fix/feature (yarn add $pkgname@https://github.com/$org/$pkgname#head=$some_branch)

Cargo allows extra tools via plugins, and I think we’re sorely lacking functionality to test the soundness of our dependencies like the following:

1. Testing with all extra combinations (https://github.com/taiki-e/cargo-hack#--feature-powerset)
2. Testing with minimum specified versions to ensure that the specified minimum versions are actually working (https://github.com/taiki-e/cargo-minimal-versions#usage)

Note that both of these ensure that tests are run in completely different dependency environments. So being able to very quickly whip up an environment would be nice. I think for that we lack

1. A static package metadata index: pip still needs to download packages to resolve dependencies, which I think is quite medieval. It should sync a package metadata index and then statically resolve dependencies. Insights gained from defining a lockfile format can be used for this index.
2. See above: automatic env management. I think our one true tool should (once it has synced the package index and has all necessary packages in its cache) be able to sync a ephemeral venv within a split second and then run things in it.

[lusty quarry]

https://github.blog/changelog/2023-02-13-dependency-graph-supports-the-python-pep-621-standard/

[wide stump]

I think this means I can get rid of Ruff's setup.py... but I'm scared to try.

[lusty quarry]

yup we all can now!

[inland prawn]

I have tested this already, it works fine. all projects in Pallets (Flask etc) dropped it and it works fine

[coarse mason]

It has worked fine for a while on the couple of my projects I've tried it on, but I was waiting for ⤴️ before permanently making the switch.

[merry rune]

I've generally dropped setup.py most places already, so it's nice to suddenly have the dep graph back!

[civic compass]

hi there everyone, I published recently a question on stackoverflow regarding using virtual environments with gdb. I would particularly appreciate your insight on it. https://stackoverflow.com/questions/75448549/unable-to-debug-using-gdb-with-python-inside-a-virtual-environment-venv-on-win

[bright parcel]

What about GitLab maintained projects?

[lusty quarry]

I don't know much about GitLab except that there used to be a feature request open for years for CI to trigger on forks/contributors, and I think it's still open

(we use it for CI/CD extensively internally but I don't know much about its ecosystem as a whole)

[high stone]

Oh, I should've posted a link here! A few of us were on a podcast recording for Talk Python to Me until a few minutes ago: https://www.youtube.com/live/z50B6AmQwLw?feature=share

It'll be available as a podcast episode in a few (2?) weeks too!

[topaz pine]

I found a new podcast to listen now 😁

[indigo token]

Awesome! Weird feeling to see what’s effectively a Zoom screen without any large branding or so to remind you that you’re not actually in a call lol.

Also someone has a talky birb!

[indigo token]

Hearing what @fickle hound said about “cargo and npm kinda have venvs built in”, I want to to reiterate what I said here: #general message

They can do that because they have things like

yarn add $pkgname@https://github.com/$org/$pkgname#head=$some_branch

and

cargo add --path ../mydep

We’ll always be forced to support somwhat-manually managed venvs as a workflow until we support something like that for dev workflows. We could go a somewhat different route than those tools and add a resolutions table/file which would make tools use some git branch or commit version of a dependency while developing a feature – without violating the purity of our abstract dependencies lol.

@lusty quarry being the visionary here, everyone nods: yeah, downloading the workflow tool (aka rustup) and having it download Python for the user would be the dream. Toolchain management like this (and tools integrated with it like cargo +nighty run) is so nice.

@fickle hound mandating that tool wouldn’t be how it is: Rust is also distributed in individual versions in parallel to Rustup. A user of Rust software (who still needs to compile that software, e.g. a Gentoo user) would just install the currently stable versions of rustc+cargo, while a developer uses the toolchain manager (rustup)

[high stone]

Not that Ofek isn't a smart person, but he's certainly not the first person to propose that FWIW. Posy and the original discussion for back when PEP 582 was proposed included the same suggestion and points.

[woven plover]

With his track record, I'm hopeful it will actually land and deliver

[mortal shore]

I think having the workflow tool install Python is a nice UX, but you can't have only that

you need something that can target arbitrary environments in the ecosystem at least, IMO

It sort of works fine in Rust because statically compiled language

[manic belfry]

That resolution table is actually a pip constraints file where you put VCS urls. It works quite nicely. In pip-deepfreeze I automate that workflow a little bit.

[indigo token]

Sounds nice! Now we just need to standardize all parts that can play together here (that functionality, a lockfile that supports the same stuff, …) 😄

just.

[spiral urchin]

I made a poc a while ago (pythonup) and still uses it on Windows. The problems are much more complex on posix since the Python build on those platforms are not relocatable. If anyone wants to do it, there are several problems already pinpointed you don’t need to rediscover. The idea has been around for a long time.

[inland prawn]

How is that different from pyenv?

[spiral urchin]

pyenv compiles everything, an ideal workflow tool shouldn’t need to do that. It also brings too many bells and whistles that confuse relatively inexperienced users but that’s easy to avoid for a new tool.

The main hurdle for rustup for Python is there are currently only two supported ways to install Python on a POSIX machine: 1. Compile from source (pyenv), and 2. install to a fixed prefix (the official Mac installers, which installs to /Library IIRC)

[mortal shore]

https://peps.python.org/pep-0708/

[indigo token]

Do you or does anyone else know where to find something listing those? E.g. I think Python doesn't need to be relocatable, we'd just need to host pythonup specific binaries for each platform, right?

[spiral urchin]

How would the pythonup specific binaries work if Python is not relocatable? Do you always drop those binaries into a fixed location on the target machine? I don’t think it’s possible for everyone to agree on that location though?

[lusty quarry]

this is relevant for Hatch because I plan to implement Python management in the coming months

I haven't put too much thought into it yet but I was planning on using https://gregoryszorc.com/docs/python-build-standalone/main/

I believe Bazel uses those by default as well now

[spiral urchin]

Last time I looked into this I think this is the best choice
https://github.com/Infinidat/relocatable-python3
python-build-standalone has some quirks that’s likely to frustrate people (the backspace key issue would annoy me a lot),

[lusty quarry]

the backspace key issue
what's the issue?

edit: oh I see https://gregoryszorc.com/docs/python-build-standalone/main/quirks.html#backspace-key-doesn-t-work-in-python-repl

that wouldn't be a problem actually since entering a shell would always be done via Hatch and that environment variable could be set

the majority of developers needing to enter a REPL would be on macOS and Windows which don't have that problem

[mortal shore]

Doesn't python-build-standalone require static linking compiled modules

[indigo token]

Yeah, as long as pythonup only does system wide installs and not user wide ones, it could standardize one. Like homebrew’s /usr/local/Cellar/...

Is there big resistance to making CPython relocatable? It could read its prefix from a distribution defined file instead of compiling it in.

[mortal shore]

C libraries in general are often not relocatable without some effort, OpenSSL is one where you'd have to decide between bundling it or what

because it's not relocatable IIRC

[spiral urchin]

Homebrew installation is actually relocatable and some people just want (need?) it

[mortal shore]

well

it's relocatable, but putting it in a different path means you often won't reuse premade binaries

IIRC

[spiral urchin]

And I don’t think system-wide is a viable option in a lot of setups

[indigo token]

So they just ship a fork or patchset that makes it relocatable. That could he our solution as well then.

[mortal shore]

IIRC Homebrew bottles can only be used when on the default path

or some of them anyways

[gritty roost]

Nope

Bazel has been defaulting to those builds for months now (maybe a year?) and they work great

I've also never had a problem with the backspace key in repls (on linux)

[blazing lantern]

I actually have buy-in for there to be official, standalone builds of CPython available from python.org if someone puts in the effort to make it happen. It would require the make targets and such to make them run from anywhere, defining how you handle e.g. someone wanting to use a different version of sqlite3 than one that you shipped in the static binary, but there is support for something official happening if the whole build can be an automated step as part of a release. And PyPy ships OpenSSL compiled in, so that shouldn't be a problem (especially since OpenSSL ships with the Windows install, so any major OpenSSL security issue requires a new release anyway). And conda already makes all of this work, so this should be doable.

[spiral urchin]

SQLite is another interesting problem (also readline but people probably care less about having the latest version of it), I wonder if it makes sense to make it a part of the version manager so you upgrade the manager to stay the latest SQLite that’s used by all installed Python versions.

[blazing lantern]

Since there's no "version manager" to speak of, I think it's getting ahead of ourselves to worry about such a thing. 😉

[west drift]

i'd love to have official cpython builds on python.org! would they need to be significantly different from what is already built in https://github.com/indygreg/python-build-standalone?

[blazing lantern]

I believe Greg has said previously that his standalone builds do a lot more than would probably be necessary for most people. And I personally don't think a self-contained binary is necessary as much as a self-contained tarball

[tidal kiln]

https://github.com/python/cpython/issues/62509

this is the related issue btw

it is interesting, I may sprint on it on PyCon maybe

[west drift]

maybe i'm just being naive, but what i meant was what's stopping us to put python-build-standalone on python.org? This would of course needs a lot of testing and a bugfix here and there (like https://github.com/matplotlib/matplotlib/issues/23074), but to me this seems easier than reinventing this infrastructure inside of cpython.

(i'm asking as someone who needs downloadable python builds and currently uses python-build-standalone for that)

[obtuse holly]

hmmm, is there any way to provide a Home-page field using the project table in pyproject.toml? I wasn't sure which channel I should ask this in due to the nature of it 😄

[woven plover]

I believe the project.urls table is intended for that in pep-621,

[obtuse holly]

yeah, but the wheel doesn't have Home-page set, as project.urls ends up providing everything as a table in the built metadata 🙁

[woven plover]

I'm on mobile, so i can't crosscheck, i believe a specific key name is needed

[obtuse holly]

hmm, I wonder if it's backend specific then

[woven plover]

What backend do you use?

[obtuse holly]

currently setuptools, but have been looking at switching to pdm-backend or hatchling

[distant briar]

Why do you need Home-page to be set?

Pyproject.toml metadata doesn't support setting that as it was considered redundant with project urls

[obtuse holly]

pypi treats it differently, afaict, and there's some references to it in some peps

[distant briar]

Pypi considers it the same as a url named "Homepage"

[obtuse holly]

ah okai

[distant briar]

As for peps, I believe you're right that Home-page was never deprecated in core metadata but I don't know if any up-to-date tool treats Home-page specially in comparison to how a Homepage project url would be treated

[obtuse holly]

it's mostly tools that allow viewing wheel or package metadata

it's slightly annoying to see Home-page but if it's not something that can be changed that's okay, I suppose

[lusty quarry]

before I deep dive, does anyone here know immediately what the modern equivalent of this line would be? https://github.com/Supervisor/supervisor/blob/f8bb4b1a2e233c69f6e4ef004fd6a465f3871eb0/supervisor/options.py#L393

[woven plover]

A most hackish way to make a fake entrypoint and loading it

@lusty quarry someone just added the modern version to setuptools_scm

[lusty quarry]

oh wait, are you saying that code doesn't actually need entry points but rather it's just trying to load an object using that spec?

[woven plover]

Yeah

[lusty quarry]

aha that makes a lot more sense thank you

[woven plover]

def _get_from_object_reference_str(path: str) -> Any | None: 
     try: 
         return EntryPoint(path, path, None).load() 
     except (AttributeError, ModuleNotFoundError): 
         return None

Wish there was a prettier way

[blazing lantern]

As I said, Greg I believe has told us that the build isn't necessarily a good fit for something we provide on python.org. But otherwise no one has proposed it and explained exactly how it's built, discussions on design decisions and the ramifications of them, etc. Because this may get used by literally millions of people we can't take anything blindly and need justification as to why something is done the way it is, especially when it doesn't originate from a core dev who has (hopefully) internalized general concerns and issues we run across with users

[hollow trout]

IMO, EntryPoint should provide an equivalent of load that doesn't require an object. It's useful general-purpose functionality.

[woven plover]

There is resolve of importlib

[hollow trout]

What do you mean?

[woven plover]

Apologies, seems like misremembered a pep

[fallen shuttle]

How about combining importlib.import_module and operator.attrgetter?

[woven plover]

Good point, let's see what i can do to ensure behaviour parity

[hidden flame]

Hey all, a colleague just asked me, do we have details yet about the packaging summit at PyCon US 2023?

[tidal kiln]

no, I think we just need to pick a schedule and then publish the details

I've pinged the other organizers

[hidden flame]

gotcha, thank you @tidal kiln

[tidal kiln]

@heavy relic when you have some time, could you have a look at https://github.com/python/peps/pull/3036 ? thanks!

also, is there any planned schedule regarding the PEP?

just to keep in mind

[heavy relic]

@tidal kiln I am waiting for the SC to come back about the generic idea, and then we can choose either of this. I guess it will be rejected as I can not say in the PEP that pip will support/implement this for sure.

[queen hornet]

here's the SC issue https://github.com/python/steering-council/issues/165

[tidal kiln]

@heavy relic not sure it will help much, but FYI we already do much more dangerous path calculations for normal virtual environments

we look for file in the same directory, and one directory above (or was it up to two?) of the interpreter and if found we completely change the path initialization

[heavy relic]

Yup.

[tidal kiln]

@lusty quarry did the extension modules build backend library ever go anywhere?

nvm, found it 🙃

I spent like 5min looking for it and found it right after asking 😅

https://github.com/ofek/extensionlib

[blazing lantern]

Same and one up for pyvenv.cfg (I'm working on a blog post about how virtual environments work)

[tidal kiln]

https://snarky.ca/how-virtual-environments-work/

[tidal kiln]

oh, that's interesting. I hadn't realized that the venv setup is done by the site module! that is surprising.

also FYI, I am planning to create a sysconfig backport

the main target will be the new API that I am going to propose, but I'd probably like to support backporting the venv scheme on the current API too for eg.

[noble matrix]

@high stone , @tidal kiln and I would like to officially announce the opening of formal signups and topic proposals for the Python Packaging Summit at PyCon 2023! It will be held in two sessions, Friday, April 21 from 10:30 AM - 12:30 PM, and Sunday, April 23 from 1:00 - 3:00 PM, both in room 251AB at the Salt Palace Convention center. Signups close April 1 and topic proposals April 10, so get them in now. Hoping to see you all there! More info: https://discuss.python.org/t/signups-finally-open-for-the-python-packaging-summit-2023/24835

[lusty quarry]

I'd like to start packaging apps like Hatch but for macOS nowadays one needs to sign with a verified Apple developer ID. Do we have an existing one that I can use to create a new certificate or should I purchase a developer account for PyPA and create a 1Password for us?

[blazing lantern]

If we have to buy one then we should have the PSF pay for one for the PyPA

[queen hornet]

1Password offer free accounts for open source projects https://github.com/1Password/1password-teams-open-source

[lusty quarry]

yes that's what I was planning on but it's the Apple developer account that costs $100 per year

[tidal kiln]

does Apple really requires you to sign Python apps?

I thought it was just binaries, but maybe I am mistaken

[lusty quarry]

Yes I'm going to begin distributing as a binary

and in addition to that for macOS the signing process requires specific entitlements in order to run third-party code like extension modules

I'm just going to copy what entitlements the official Python distribution uses

[tidal kiln]

@narrow hawk I remember we talked about how LLVM handles this (macOS signing), but I don't recall the details, is there anything relevant for this use-case?

[narrow hawk]

I think anything that runs of a program that require e.g. special syscalls, like ptrace. LLDB need to be signed in order to use those kernel syscalls. The apps are packed into their weird zipped app format. Everything inside it, that requires access to those extensions, need to be signed, otherwise access is denied

[tidal kiln]

gotcha, then how do you handle plugins? do they need to be signed too?

I vaguely recall a workaround for this, but maybe I am misremembering

[lusty quarry]

e.g. https://developer.apple.com/documentation/bundleresources/entitlements/com_apple_security_cs_disable-library-validation

[tidal kiln]

hummm

[narrow hawk]

Things like just-in-time compilation can also suffer from the same problem, due to process code injection. I'm not aware of how python virtual machine works, but I assume some might be affected?

[tidal kiln]

no, we don't have a JIT currently, so pure Python modules should not suffer of that issue

but native modules do, which is the use-case we were discussing

[narrow hawk]

Yeah, as mentioned by this link, everything that uses the similar dlopen interfaces are affected

Apple ¯_(ツ)_/¯

[merry rune]

We are starting to put together a proposal for a standard protocol to write plugins for dynamic metadata for build backends. If anyone wants to provide opinions early, it's at https://github.com/scikit-build/scikit-build-core/issues/230. I'll be checking with some backend & plugin authors soonish, and starting a discuss thread, but early feedback would be appreciated!

[lusty quarry]

what do others here think about shipping tests in wheels? https://github.com/pyOpenSci/python-package-guide/pull/55#discussion_r1142626510

[fallen shuttle]

My impression is that tests are very welcomed in the sdist but I usually don't ship them on the wheels (to make them more lightweight + I am not sure if testing is in the scope of wheels)...

[woven plover]

not even pytest ships its tests in the wheels, however there are a number of valid cases where tests in the wheel are the right thing to do

imho tests in thr wheel should imply tests are inside the installed package

[mortal shore]

Tests go In wheels if you install your tests

Whether or not you install your tests is a philosophical debate

[woven plover]

borgbackup is a fun example, it has a large normal testsuite with pytest, and ships a small unittest based selftest inside the package

[heavy relic]

Just like manylinux wheels, is there anyway to build similar wheels for Mac? where I can put all the library dependencies into the wheel file.

[heavy relic]

maybe https://pypi.org/project/delocate/

[lusty quarry]

yes that's the one

[woven plover]

question wrt packaging - can we get a more nuanced regex for versions - (in particual ones that match versions without dev/local part

[blazing lantern]

The regex used by packaging is https://github.com/pypa/packaging/blob/5b34465682d40ddd42dd6cba88e158e1674de7b9/src/packaging/version.py#L113-L142 , so you could copy that and strip out the parts you don't want. Otherwise I'm not quite sure what you're after since dev and local are not required, so leaving them out of a version will still match

[woven plover]

for example for setuptools-scm/vcs-versioning i want a insertable regex that doesnt prefine local/dev matching, instead i want to forbi local and forbu dev tags that have a number

[blazing lantern]

Can you just copy the regex out of packaging then and modify it as necessary? the PEP 440 version formatting seems rather stable, so I don't think copy-and-pasting being a big issue here

[woven plover]

im already depending on it and i dont exactly fancy maintaining a mutilated copy of the regex, i'll propose something

[high stone]

You can use that via packaging.version.VERSION_PATTERN as well, FWIW.

[west drift]

When packaging an extension module that has submodules, is there any way to allow importing directly from the submodule?

Let's say i write a native bio package with dna and rna submodules, all inside bio.abi3.so, is there a way to enable:

from bio.dna import transcribe

We're already generating python wrapper code to allow type stubs (from .bio import * plus setting __all__ and __doc__), but i haven't found a way to "redirect" to a native module.

[fallen shuttle]

If you are already generating wrappers, have you considered generating wrappers for the sub-modules as well?

Then you could use https://peps.python.org/pep-0562/ or augmenting vars(sys.modules[__name__]).update(...)

[west drift]

vars(sys.modules[__name__]).update(native_module.__dict__) was what i was looking for, thank you!

you do not by chance also know how to allow pycharm to see the actual symbols? currently it's convinced that the module has no symbols

(i'd like to avoid importing at build time to generate pyi files since imports can run arbitrary code)

[merry rune]

Isn't PyCharm reading the static types? You probably need to generate .pyi files.

[west drift]

pycharm seems to be able to inspect native modules, at least in the right circumstances

e.g. i got autocompletion with installed files but it ignores a bio.abi3.so in a project directory

[fallen shuttle]

Sorry @west drift, I have no idea 😅 .
I am actually impressed with native module inspection (maybe they are using something like objdump?)

There is some static analysis tool out there that can understand __all__ = otherpackage.__all__. Not sure if PyCharm is one of them, and if that would work...

Ok, pyright is the one that understand some __all__ idioms, and this is the list: https://github.com/microsoft/pyright/blob/main/docs/typed-libraries.md#library-interface

[west drift]

__all__ i'm already setting, but that didn't work somehow

[fallen shuttle]

Ah, ok, maybe it does not work for PyCharm...
In the case of pyright I think they don't evaluate the Python operations themselves but somehow do some symbolic interpretation...

(that is why the list of supported idioms is limited)

[blazing lantern]

Correct; pyright does not execute any Python code and instead parses it and analyses it.

[merry rune]

Made discuss.python post for the dynamic metadata proposal: https://discuss.python.org/t/pep-for-dynamic-metdata-plugins/25237

[lusty quarry]

https://discuss.python.org/t/pep-582-python-local-packages-directory/963/430

[strange knot]

Sharing this a little more widely: we're in the process of rolling out credential-less "trusted publishers" to PyPI: https://docs.pypi.org/trusted-publishers/. If you're interested in joining the beta, sign up here: https://forms.gle/XUsRT8KTKy66TuUp7, I'll be inviting another round of beta users later this week.

[obtuse holly]

The short-lived API token behaves exactly like a normal project-scoped API token, except that it's only valid for 15 minutes from time of creation (enough time for the CI to use it to upload packages).
I know some ci that takes way longer than 15 minutes to generate their wheels

[queen hornet]

the token is created just before upload, so after you have already generated your wheels. it only needs to be valid for the upload bit

[echo ruin]

So, I'm back to thinking about minor improvements to my "standard" packaging/testing setup this week. Specifically at the minute I'm trying to decide how best to solve the problem of (for a library, not application) "I want to install specific versions of dependencies for my normal test run" (great, pip-compile + a requirements file) but now also whilst not accidentally installing additional dependencies accidentally simply because they appear in a requirements file.

There are some "obvious" ways to partially handle this (though the one that's most obvious to me doesn't seem easily doable with pip-compile). What do y'all do for this if anything?

[hollow trout]

How could the additional dependencies appear in the requirements file?

[mortal shore]

I would just use pip-compile

not sure why you'd get additional deps in the file

[echo ruin]

if you change your package dependencies and forget to regenerate the requirements file essentially

@mortal shore yeah I think the issue though is if I do the "obvious" thing to me, it's have a test env which re-runs pip-compile and complains if you forget ^^ -- but pip-compile doesn't seem to have an option to not generate versions in the requirements file or whatever, meaning you'll get spurious failures if you diff whenever versions do change

not to mention it still doesn't handle multiple interpreter versions (though I have to re-look at pip-compile-multi I guess)

Is the scenario I'm trying to prevent sensical at least or did I not explain it well?

[hollow trout]

Doesn't pip-compile keep the versions as-is if the requirements file already exists?

[echo ruin]

Oh good point, I have completely forgotten you can run pip-compile without -U... so yes maybe 🙂 let's see

OK right that indeed does remove requirements if they disappear, great... now maybe that reduces the problem to just the cross-environment problem

(needing to manage multiple different requirements files for different versions/environments/markers), but that's probably something I need to look at again anyhow

[blazing lantern]

If you're using GitHub Actions, I have an action specifically to help with detecting when you forget to update or add a file: https://github.com/marketplace/actions/check-for-changed-files

[echo ruin]

@blazing lantern thanks, good to know about -- though I don't think it's precisely what I want for this specific use case (I'm not trying to ensure every PR updates the requirements file)

I'm trying to catch the case where your library will install A B and C when installed without a requirements file, but your requirements file accidentally includes D as well, so running your tests actually ran with the presence of an additional dep

(and at the end there my "real case" at the minute includes cases where B being a dep depends on an environment marker)

But that looks like a nice action to be aware of anyhow, thanks for linking it.

[blazing lantern]

That's why you can specify a dependency glob via the prereq-pattern; so you could set prereq-pattern to requirements.in and then file-pattern to requirements.txt, and that would cause a PR failure only if you edited requirements.in but not requirements.txt. We use this to remind folks to update tests if they update code (and opt out via a skip tests label to make it pass).

[echo ruin]

@blazing lantern I saw that :), but that still doesn't catch this kind of scenario

This kind of thing can happen with no files changing, e.g. if C was accidentally present in the requirements file but accidentally not in the requirements.in / pypackage.toml because it was depended on by B, and then B stops depending on it

Now no one modified the requirements.in, the requirement.txt still doesn't match what will be installed by a normal install, but the package may be broken

[hollow trout]

Wouldn't requirements.txt have a pinned version of B in this case?

[blazing lantern]

Then yeah, can't help with human error depending on things that it shouldn't be depending on without an update to your requirements.in.

[echo ruin]

Yes, so your tests pass, but pip install fails.

[blazing lantern]

Unless you want to write a linter to check all of your imports correspond to only things specified in your requirements.in file.

[echo ruin]

I think the "outline" of a thing "really" catching these kinds of scenarios is "is the list of packages installed by the test requirements exactly the same as the one installed by pip installing without a requirements file"

Yeah such a linter sounds like sort of that kind of thing

Somehow it feels "better" to me to not do it statically and compare the list of installed packages in two environments

[hollow trout]

If you want to test what happens when you install without a requirements file, then why even have a requirements file?

[echo ruin]

The requirements file is so that your test runs are deterministic.

(Otherwise a change in versions may fail your tests and make it harder to tell that was why they failed)

the other kind of thing I was playing with to make this work is instead turning the test requirements into a constraint file but I was running into old bugs there so I gave up for a bit, but maybe I should try to make that work

[hollow trout]

Then maybe that's exactly what you should test? Install with pip, then compare the list of installed packages to the requirements file.

[echo ruin]

yeah probably I should give that a shot directly too, thanks, good idea

[obtuse holly]

is there an api for https://discuss.python.org/ ?

oh its just a discourse deployment

[placid swan]

I'm wondering would it be possible to create a chat dedicated to pip-tools here under the "Other projects" category?

[tidal kiln]

done

[shut quest]

why don't venv's come with wheel?

[spiral urchin]

The long term goal is to nudge everything to PEP 517. In the next pip release (due this month) an environment without wheel will use PEP 517 to build wheels by default.

[shut quest]

increasing adoption of pep 517 makes sense, but why not have venv come with wheel? pep 517 installations are a much worse experience compared to wheel installations.

[hollow trout]

If PEP 517 installations are a worse experience, then why do you say increasing adoption makes sense? 😉

[shut quest]

pep 517 is a better experience than setup.py install

to be clear, i'm asking the question because i'm worried about the user experience when pip pulls the trigger on this

we've spent a long time getting users to use venv and getting library authors to build wheels...

...so it's weird to me that the default venv experience will not use wheels

[hollow trout]

But it will. Installation via PEP 517 builds a wheel and then installs it.

[spiral urchin]

I’m pre-emptively answering a common misconception so forgive me if I’m wrong: you don’t need wheel to install a wheel

[peak ocean]

If you want to build and cache a wheel, when PyPI only has an sdist, then wheel is useful to have installed locally. But once pip is using PEP-517 for that, it won't need wheel installed.

[shut quest]

okay, thanks! you're right, for various me-specific reasons, i confused myself

sorry about that! 🙂

[high stone]

IIRC, because ensurepip was implemented before wheels were fully figured, and no one went back and added wheel to the set of core venv dependencies when wheel adoption started becoming a thing. By the time someone took a closer look, we already had PEP 517 and pushing users towards isolated builds was the general concensus overall.

You did initially need wheel to install wheels with pip, for example, and I think it was a case of thing-that-slipped-under-the-radar. Besides, there wasn't really a strong incentive structure to move off of setup.py install at the time, and by the time we got there, we're already a step further and it's also gonna end up being a good way to push adoption of isolated builds a step further.

[vagrant cosmos]

<@&815744082823348274> uhhhh do you guys know that inspector is down? 502 bad gateway?

https://media.discordapp.net/attachments/1086881712434319501/1093731777559920670/image.png

[lusty quarry]

I think that service is provided as-is, no guarantees or SLAs or anything like that

[lyric hedge]

Hi. FYI the vast majority of (or all?) admins here don't maintain PyPI. This would be better as a GH issue on the inspector repository (especially as this service is not deemed critical).

[queen hornet]

reported https://github.com/pypi/inspector/issues/90

[queen hornet]

back up now

[obtuse holly]

does inspector still not use the fastly cdn?

[distant briar]

I wouldn't think that it's used enough to warrant it

[lusty quarry]

I don't think it should, way too much data needlessly cached

[obtuse holly]

sad

I wanted to create a discord integration for it

[lusty quarry]

for example, say somebody ran a job that required scraping everything and it's backed by a CDN

there would be a cached page for every file inside every artifact for every release

it doesn't make sense to pay for that, and if we don't pay it's bad etiquette in that case, almost abuse

[placid wyvern]

There also just isn't a compelling reason to use inspector to any degree except for investigating a specific instance of something or reporting something to Security. Package download endpoints are exposed through the JSON API, and those can be hit to pull down the tarballs or wheels for specific packages to handle locally.

[tidal kiln]

scanning a large amount of artifacts without downloading them all?

[placid wyvern]

Erm... I'm not sure I understand-- wouldn't that just shift the load on the currently hosted implementation of Inspector?

[hollow trout]

The layout on pypa.io seems to be bugged:

[high stone]

pypa.io layout

[civic compass]

How can i get discord token using py?

[torpid veldt]

This discord is primarily about pypa related projects and not for general help with all python projects. You probably want to ask that question in the discord-bots channel of the python discord https://discord.gg/python

[lyric hedge]

@rare hearth did it seriously take this long for you to join? >.<

[rare hearth]

Perhaps :3

But hey, I'm here now

[rare hearth]

Hallo!

[placid wyvern]

Ask him how many times we linked the PyPA Discord before he finally went "Hey can I have the Discord link?" 😛

[tidal kiln]

it's that time of the year again, I created #pycon-us-2023 for people attending PyCon to coordinate

[lyric hedge]

Classic Robin 🐦

[heavy relic]

I have a virtual environment (with updated setuptools/wheel package) (on linux distributions), and I have a list of dependencies (package names, sometimes versions). What is the best way to get the virtual environment to be in the same state of those dependencies? I can create & execute bash scripts/python scripts to do things as required.

[hollow trout]

What's wrong with pip install -r?

[heavy relic]

I should double check that the downgrade works, somehow I thought it does not happen.

And I am wrong, it just works.

[torpid veldt]

pip-sync from pip-tools?

[hollow trout]

Doesn't pip-sync need versions for everything?

[torpid veldt]

Yeah so you'd pip-compile to fill in the missing versions and pip-sync to uninstall extra packages

[heavy relic]

So, here are the current steps: create venv, install asaman tool, then install the given build dependencies for the project we will be building (from a specific index), and then try to download the project we are building from pypy.org, and then finally build.

And the build dependencies will be handwritten (for now) in a different file all together, my tool will find these information from that separate file.

[high stone]

Wipe an environment with pip uninstall --all and run pip install --no-deps <paths>?

Oh, maybe we never landed --all?

[spiral urchin]

I don’t think so

[heavy relic]

I should start writing down my whole scenario for the folks in a blog post.

[west drift]

since python platform tiers (PEP 11) came up at the packaging summit: through which wheel tags are specified in manylinux and related standards and through allowed platform tags on pypi, there's effectively a second tier system, which afaik is currently not documented anywhere

imho it would make sense to have a page in somewhere like the pypi docs or packaging.python.org that lists the supported tags

[elder kettle]

What could be a good discussion on tooling: https://github.com/mitsuhiko/rye/discussions/6

[heavy relic]

Is there any good way to extract the list of build dependencies from a package's source tarball?

[gritty roost]

pretty sure you're meant to invoke the get_requires_for_build_wheel PEP-517 hook

[heavy relic]

goes to search and read

I wonder if any tool exposes output from that.

maybe from build itself.

[gritty roost]

There's https://github.com/pypa/pyproject-hooks which I think build uses under the hood, but it's a library - doesn't look like it ships a cli tool

i'm not sure that build has a mode to just output the build deps but i could be wrong.

[heavy relic]

I guess I will have to use the API https://pypa-build.readthedocs.io/en/latest/api.html

goes to write some code to test out the idea

Okay, this will do.

@gritty roost thank you for the tips, build can give me all the details via API.

[high stone]

Oh hello there folks! What did we do to attract so many new comers over the last few days?

[placid wyvern]

I recognize a few names filtering in. Some of these are people I'm talking about packaging security with in other venues. It's a hot topic. 😛 Some very good ideas and very smart people floating around out there, hope they get involved here and can help out! ^^

[wide saddle]

Hello 🙂

[inland prawn]

👋

[lusty quarry]

oh hello there!

[lusty quarry]

a person just opened an issue on Hatch which turned out to be a user error, but they mentioned this https://peps.python.org/pep-0423/#top-level-namespace-relates-to-code-ownership

is it possible for us to modify that text to indicate that periods have no special significance?

[high stone]

423 is deferred?

It's not in action. You're basically hitting the problems of communicating with PEPs. It doesn't help that some tools are implementing draft PEPs. 🙈

[lusty quarry]

true, I didn't notice the deferred status

I wonder how that person came to that document

it would be nice if we could somehow de-rank the SEO for PEPs that have not been accepted lol

draft PEPs

speaking of... https://discuss.python.org/t/pep-639-round-2-improving-license-clarity-with-better-package-metadata/12622/91

heroes, thank you 🙇‍♂️

[high stone]

https://github.com/python/peps/issues/2719 is the fix across all PEPs.

[elder tinsel]

I'm one of those, and it's why I filtered in, just haven't had enough time to formulate everything I want to get a discussion going about. I'd like to make sure I cover obvious detractions from the suggestions, whether on limiting negative impact, or "Why not x instead", and going through a lot of what's already been proposed or less formally discussed in various other places.

The TL;DR: I'd like to make the process of the average user getting their dependencies less prone to security risk wile minimizing negative impact on end users and maintainers. I think the goal is worth it even if it only achieves small gains as long as the cost is low, and I think it's possible to make security idiomatic with minimal friction, but covering everything which would actually be required is probably a multi-stage plan, and I know I'm somewhat retreading space that's either been explored before or is currently to many people viewed as "Good enough" with active monitoring of pypi and security concious users use of internal indexes and --verify-hashes

the actual plan for that is taking time to put into a good draft that covers the tradeoffs, impacts (both positive and negative), and minimizing friction.

[elder tinsel]

<meta name="robots" content="noindex"> on peps that aren't accepted? Not sure unindexing is the best option. Maybe just a banner that shows up on not accepted peps that draws better attention to what kind of not accepted pep you are reading.

There's also SEO "hacks" that aren't guaranteed to continue working and only work on observed behavior, like using 301 redirects to intentionally lower the page ranking. (https://peps.python.org/pep-0505/ -> https://peps.python.org/deferred-peps/pep-0505/)
This might help with people noticing the state of the PEP as well.

[autumn forge]

Hello, I've been reading PyPI's Acceptable Use Policy and noticed something odd.

https://pypi.org/policy/acceptable-use-policy/

Specifically this line:

Note that this includes dual-use content, including content that is used for research into vulnerabilities, malware, or exploits, including bug bounties. We consider PyPI to be a platform used primarily for installation and run-time use of code, and not for research.

Is that rule actually enforced? Because there are many popular InfoSec tools on PyPI that fall under that definition.

[lyric hedge]

I wonder if this is rather talking about that PyPI is not a platform for conducting vuln research. I'd guess that distributing InfoSec tools is usually okay but uploading spam or malicious packages to PyPI to get data for your vuln research (like trying to perform a supply-chain attack for research) is NOT allowed.

(disclaimer: I am not affiliated with PyPI in anyway. I'm just a layperson here)

cc @strange knot

[strange knot]

Yep, that's accurate

[placid wyvern]

I am curious as to how that delineation is drawn. We've seen a few C2 frameworks with instances of compiled malware binaries, known malicious dependencies, etc.
https://pypi.org/project/pyhtools/ was the more recent one (that actually started this discussion).
At face value, it's not a malicious package, but it does contain overtly malicious payloads that are prepared or able to be staged. (Moreso, it advertises this fact: PHT also includes malwares which are undetectable by the antiviruses.)

Standing policy organizationally is to defer that level of decision-making upward, and report anything that may fall within the 'middle ground' of acceptable use, but that middle ground can be a bit of a chasm.

[uneven karma]

oh sheet its the feds

[elder tinsel]

I feel like with this in the project description:

NOTE: Do not upload/send/report malwares to anti virus services such as VirusTotal. This will make program detectable
It should be pretty clear this person isn't interested in actually improving security and is just trying to skirt guidelines.

The only point of legitimate offensive tools for security research is to improve the defense.

[subtle galleon]

T

[autumn forge]

Ok thanks that makes sense.

[autumn forge]

I don't know about that package but I will say that is a disclaimer that I've seen with legitimate red team packages.

If a project is specifically about researching antivirus evasion, I've seen people put that disclaimer due to VirusTotal sharing binaries with AVs.

[placid wyvern]

Wouldn't that fall under the research clause?

[autumn forge]

Because obviously if you're making a AV evasion framework you don't want to immediately be detected by AVs everytime someone uploads your binary even though you clearly state it's a C2 for example.

[autumn forge]

No.

Reread what they said.

If you're using the PyPI platform itself to gain data for research, that's not allowed.

Like here's an example: a supply-chain attack company wants to conduct research, so they decide to do a supply-chain attack against PyPI and upload malicious packages.

[placid wyvern]

I generally speaking don't believe that zero days or evasive compiled binaries could represent a legitimate use case for distribution on PyPI. Metasploit/Pwncat/etc., are all plenty effective, despite flagging AV pretty rigorously.

[autumn forge]

That would violate the rules.

[autumn forge]

I don't think that violates the rules.

If I'm understanding correctly.

[placid wyvern]

🤷‍♂️

[autumn forge]

If you have a package which is a CLI to build reverse shells that are supposed to be more undetectable/obfuscated to evade AVs for red teaming purposes, then I think that's ok.

Someone can correct me if I'm wrong.

[placid wyvern]

https://inspector.pypi.io/project/pyhtools/2.2.2/packages/4f/2f/8ff2d198577b1acf4e0e17979c34aaf7c02e5cb21727270ee27dbde6d16c/pyhtools-2.2.2-py3-none-any.whl/pyhtools/evil_files/ransomwares/dmsec/HowToUse.md
I don't know, give me a good use case for ransomware distribution for vuln research. Because outside of internal lab setup, there's not a lot of use cases.

[autumn forge]

Well, GitHub surprisingly allows malware samples.

There are repos like theZoo which contain dozens of ransomware samples for research purposes.

[placid wyvern]

Sure and we have one ourselves. But this isn't for research, they're specifically imploring that their ransomware remain undetectable. >.>

[autumn forge]

Anyway some use cases:

Malware analysis, dynamic/static analysis, AV testing/benchmarking, etc.

[placid wyvern]

And... uploading these to PyPI for the sake of AV benchmarking... wouldn't indicate research into supply chain security levying PyPI?

I think there's a reasonable level of delineation that can be created on red teaming tools that are qualified and legitimate versus those that are explicitly designed to evade detection and generate payloads for novice users to use.

[autumn forge]

Well, it all depends on the intentions of the creator.

[placid wyvern]

Which you cannot possibly glean; but I'd argue that Metasploit does not market itself as "DO NOT UPLOAD OUR BINARIES, HERES HOW TO RANSOMWARE". Neither does Pwncat. Or even CobaltStrike.

[autumn forge]

It sounds like dumb classification but it literally comes down to the creator.

There is a popular open source RAT on GitHub, which is different from a script kiddie making a RAT to use on people and upload it to PyPI.

[placid wyvern]

Realistically it's the caveat of open source; what is 'acceptable use' and what isn't when it comes to redteaming versus blackhat. But I still have an iffy feeling about how that package targets its audience versus one of the many examples I've given so far.

It's definitely a very multi-faceted discussion.

[autumn forge]

Ok look at his GitHub and tell me what you think of him based on what you notice: https://github.com/dmdhrumilmistry

[placid wyvern]

I don't think that's wildly relevant? You cannot possibly imply that we need to OSINT every developer that pushes something malicious to PyPI.

[autumn forge]

It is relevant like I said.

Here is what I see subconsciously with a glance:

1. Uses his real name.
2. Has a Paypal, Patreon, and social media.
3. Offers freelancing through Fiverr.
4. Shows his CCNA badges.
5. Recently opened an issue alerting a repo that their code is vulnerable to XSS.

Conclusion: very unlikely to be a threat actor.

[placid wyvern]

You still maintain a burden of responsibility in how your tools will be used.
Me slapping "Don't use these for badguy things," and dumping my entire malware repo recklessly to the internet is irresponsible.

[autumn forge]

I'm sorry but I have to completely disagree.

Otherwise malware sample repos couldn't exist and it would cause a huge barrier for amateur malware research. That is one example.

[placid wyvern]

The same repo's I had to... qualify myself to access?

[autumn forge]

Those are different.

Lookup "malware samples GitHub repo" on Google.

[placid wyvern]

I mean I understand what you're saying, I'm just not sure the rules make a clear delineation on whether or not that's acceptable use or not is kind of my point. Whether or not I personally agree with it, the fact that you're creating an evasive payload and broadcasting it on a platform that specifically forbids content that does not have a clear dual use scenario (and I would argue novice crafting of ransomware falls well within that category) doesn't allow for that concession.

[autumn forge]

Checkout theZoo for example which I mentioned earlier.

https://github.com/ytisf/theZoo

[placid wyvern]

yes I am aware of this repo. GitHub allowing the content isn't relevant to PyPI allowing the content.

[autumn forge]

I'm not talking about PyPI right now, I'm talking about the general idea of allowing repos like that.

[placid wyvern]

This is the PyPA Discord m'brother. If you wanna' talk about GitHub ethics, we have better places to talk about that.

[autumn forge]

Well let's see what the admins think.

[placid wyvern]

Ultimately its the security team's decision. Like I said, we defer this decision point upward by allowing that decision to be made at the appropriate levels.

[autumn forge]

Are packages like we talked above against the PyPI rules?

[autumn forge]

No, we're asking for a clarification on the specific rule.

[autumn forge]

I only talked about the ethics because you brought it up.

[placid wyvern]

If of the ~250 malicious packages we've reported so far also encompass several within the chasm of 'grey area' red-teaming toolkits, I'm willing to accept deferring the decision point above us. I'm not sure we need direct clarification, because I can't fathom a situation where that can be encompassed in a rule set that isn't overtly restrictive.

[autumn forge]

Saying that it's reckless, which is a valid opinion about the idea of free and open access to malicious tools.

[autumn forge]

Well I'd like clarification for myself.

[placid wyvern]

Fair.

[elder tinsel]

There's a difference between someone actively being a threat actor and them fundamentally missing the point of researching offensive techniques. There's nothing novel in not being detected in this repo, this isn't AV evasion research, it's a working exploit not currently detected by some.

This falls under at minimum "totally missing the point" of security research imo. It's also at minimum callous and reckless to express wanting working exploits to remain working

Them putting their own credentials up next to it doesn't change that.

[lyric hedge]

<puts on mod hat> Please note that this is bordering on or is flat out off-topic. If you choose to continue the conversation, please use #off-topic or somewhere else. Discussion in relation to PyPI is on-topic for this channel. Thanks!

[obtuse holly]

have you ever looked at https://github.com/robots.txt

at the bottom of the page: Disallow: /Explodingstuff/

simply put

the other interesting headers are

Disallow: /ekansa/Open-Context-Data
Disallow: /ekansa/opencontext-*

[autumn forge]

I haven't, that's interesting.

Why did they single out that user in particular?

[obtuse holly]

i'm not sure why ekansa was singled out, but explodingstuff feels self-explanatory

[autumn forge]

How so?

There are malware repos which contain WannaCry.

Gotta be a reason why they picked that user.

[obtuse holly]

fair ¯_(ツ)_/¯

[elder tinsel]

In an attempt to focus on things more on topic, is there a channel I should discuss potential changes to packaging prior to drafting a PEP? The project specific channels would all likely want to be aware of the proposal I'm working on, but the only one that would require any changes (And thus a higher barrier to buy in, even though I'd hope to get buy in from the whole packaging ecosystem on this) would be #pypi , would this channel be fine for discussing some of that with others to get feedback?

Context for this is here: #general message

[queen hornet]

https://discuss.python.org/c/packaging/14 is a good place, and drop a link to the discussion here. There's lots of people who don't use Discord.

[coarse mason]

Hey all! Wanted to let everyone here know about a free Python distribution/packaging roundtable that OpenTeams is hosting next week, at 11am EDT on Tue 9 May. It’s part of our initial community-events effort for the open source architect network we’re trying to build.

Some may recall this Twitter thread started by Travis Oliphant (https://twitter.com/teoliphant/status/1628454465866022912), downstream of the recent Discourse packaging discussions and the Talk Python To Me packaging panel (https://talkpython.fm/episodes/show/406/reimagining-pythons-packaging-workflows). Essentially, while the PyPA umbrella is big, there is a lot of Python distribution and packaging that happens outside of it, and we wanted to get representatives from that broader community together and discuss use cases, features, challenges, etc. from that perspective. We’ve got folks lined up from Conan, Conda, CPython, EasyBuild, Homebrew, meson-python, Nix, scikit-build, and Spack.

If you’re interested, the registration page has more info: https://hopin.com/events/python-distribution-packaging-roundtable-20230509?utm_campaign=ot-event-20230509&utm_medium=social&utm_source=discord-pypa&utm_content=20230509-py-dist-pack-rt

Please ask me any questions you might have.

(I’ll note that we do recognize that this panel does not represent the entire extended universe of Python distro/packaging, and that this event could be the first of several. Recommendations welcomed for other projects we could loop in for future events.)

(Oh, and—this is the first, last, and only announcement I’ll make here; won’t bother you with it again.)

Thanks!

[hidden flame]

glad you joined, how's it going with rye so far?

[blazing lantern]

It wasn't a life goal, but getting quoted in the Register and being portrayed in both a positive and negative light in the same article was not something I expected to happen to me https://www.theregister.com/2023/05/04/a_python_tool_called_rye/