#off-topic

1 messages · Page 2 of 1

silk jungle
#

I don't have any middle names...

#

Y'all have like at least four middle names

muted whale
#

Only two officially, heh

kind moon
# silk jungle I don't have any middle names...

Simple skill issue
All it takes is a quick trip to the records office, and you can have as many names as you want.
Put them all in the middle and you don't even have to change your paperwork!

muted whale
#

No skill required

onyx spindle
#

Most of my team at $WORK use GUI for git, guess who is a go-to man when they break stuff...

silk jungle
#

How many hours do you think you've spent fixing other people's git messes?

#

It reminds me of this

// 
// Dear maintainer:
// 
// Once you are done trying to [teach someone git],
// and have realized what a terrible mistake that was,
// please increment the following counter as a warning
// to the next guy:
// 
// total_hours_wasted_[because_of_git] = 42
//
#

42 might be a bit low ...

muted whale
onyx spindle
muted whale
#

Fun fact: The A stands for A-a-ron, which is not-uncommon internationally but relatively rare in the US, especially around the time I was born, whereas the girl's name Erin (pronoucend the same) was much more common. I was originally supposed to be named "Aaron Christopher" but when I was born, my then-6 year old cousin told my parents "You can't call him Erin, that's a GIIIIIIRLLLL's name!!!" so they swapped it a few days after I was born. My mom also added the "Manning" a few days before the 1-year-old deadline for having to go through a whole process to do so before a judge (it was her last name, but my middle name).

#

And as they say, the rest is history.

shadow zealot
muted whale
shadow zealot
#

Hah I guess he did. He’s the younger brother of Nick Carter, a member of the Backstreet Boys. Think Justin Bieber but 2000.

muted whale
shadow zealot
junior narwhal
#

I miss that show. they ended while it was still great like Seinfeld and Breaking Bad did

muted whale
muted whale
hexed briar
silk jungle
junior narwhal
#

I find it quite rare to have to document an elaborate explanation of some code, but that is a great example of such a case

hexed briar
#

Oooo. It reminds me of how 8% of 25 is tricky but 25% of 8 is not; and they're both the same.

#

In any case, that comment is out of date; coz Sphinx 4.4 fixed things.

onyx spindle
#

@silk jungle pipe as Union is also available with from __future__ import annotations, so that isn't that bad

silk jungle
#

Yea I stay away from that future mostly because of the future of the future is not certain (i'm sorry)

#

What is even the situation between that PEP and the alternative? I haven't looked into it in a while.

onyx spindle
#

what PEP? 🤔

onyx spindle
silk jungle
muted whale
onyx spindle
muted whale
muted whale
#

AFAIK It is not official yet, but it looks like PEP 649 will be accepted IIRC in favor of 563

onyx spindle
muted whale
#

It's not clear right now how the whole __future__ situation will be handled, but it seems likely the old __future__ import will stick around at least until the new annotations are availible on all supported Python releases. For now I'm just continuing to use from __future__ import annotations.

amber shuttle
#

(BTW, sorry for coming late to this conversation, but I've been really busy outside of the PyPA org)

silk jungle
#

We aren't some corporate entity trying to mandate "team-building" activities :)

amber shuttle
#

That's why I always try to come here, you are a nice community 🙂

muted whale
shadow zealot
#

I wouldn’t object if someone wants to host some PyPA team building activities, they are fun to attend

nocturne swallow
#

Perhaps that pycon, but aren't summits basically a team building activity where we try to convince people outside of the packaging group that we know what we are doing 😅

muted whale
onyx spindle
muted whale
amber shuttle
#

I understand the "Authority" thing in PyPA was intended to be a joke, right?

#

'cause we don't look too authoritative

amber shuttle
muted whale
#

I'm not the authority on that, but AFAIK yeah it was supposed to be an in joke because it has no hard authority

amber shuttle
#

Yup

muted whale
#

But unsurprisingly, it can result in rather misleading expectations for anyone not "in" on the joke

#

IMO, it would be worth changing to "Association", but predictably no one could agree on it

amber shuttle
#

Yeah... a while ago, if I remember correctly, there was a discussion about that

muted whale
#

Yeah, on Discourse, and I'm pretty sure there were others

amber shuttle
#

Oh yes, I think I even saved that Discourse thread

muted whale
#

You know what they say about the two hardest problems in programming...

#

Just replace "cache invalidation" with "packaging"

#

At least these days

onyx spindle
onyx spindle
muted whale
#

Er, I mean aren't our three chief weapons problems fear, surprise, ruthless efficiency and an almost fanatical devotion to the Pope?

shadow zealot
muted whale
dreamy hatch
#

which joke name came first? PyPA, PyCQA or PyCA?

shadow zealot
#

IIRC PyPA was first

shadow zealot
lean bear
#

Seems like auditwheel, delocate, delvewheel all do the same thing for different platforms. It's a pain when cross compiling. Thinking about making one repairwheel that just mixes all three and works cross-platform.

#

"any wheel, anywhere"

shadow zealot
#

I think the main problem is administrative, the three really aren’t that interested in other platforms. A wrapper project combining them and helps redirect issues to relevant parties (important) would be useful, sure

lean bear
#

More than a wrapper, unfortunately. auditwheel for instance fails explicitly if sys.platform != 'linux', which makes sense given its scope. Its cross-platform gap is pretty small, though, and I will probably work around it with some monkey patching to start. And I think delvewheel is already platform-agnostic.

quartz yew
#

I have considered doing that but I lack time

#

after I get meson-python in a good state, I can see if work thinks it's a beneficial enough issue to work on

lean bear
#

biggest gap I think is codesign on for delocate. But I think maybe ldid can replace it for delocate's purposes. It only ever does ad-hoc signing. https://github.com/tpoechtrager/ldid

#

Everything else in auditwheel + delocate can be handled by LIEF, I think, which ships wheels for multiple platforms

quartz yew
#

dependencies for this tool is't a big concern as it is intended to be ran on dev environments

lean bear
#

true but no environment other than macos can have codesign

atomic meteor
#

Hey guys having some issues with my private pypi server (devpi) on the server everything works great with the global index set to http://localhost:3141/root/zeuspi/+simple/ but on other machines I can't get pip to work. I have tried http://root:****@172.31.94.115:3141/root/zeuspi/+simple/ and http://172.31.94.115:3141/root/zeuspi/+simple/ but both fail with the error

ERROR: No matching distribution found for six```
The package install works fine on the server. For authentication it is default username of root with no password since the network is isolated. Not sure if this is something wrong I have on my devpi server or the client machine
vapid kite
#

Are you sure you've configured devpi to listen on some other interface than just localhost?

#

Check with e.g. lsof, or look at whatever the devpi default is

#

Also, check what pip install -vvv says

#

Also also, just because something's on a private network, doesn't mean it's safe to not have authentication

marsh fog
onyx spindle
muted whale
# onyx spindle Well, if someone doesn't know all core devs by name, it's an easy mistake to mak...

And Skip's technically a retired core dev, though FWIW his name is listed at the top of his post (which is why IMO signatures are rather redundant in this context to begin with). I read it as intended, but that's because I've interacted with Skip a fair amount so I was used to his signature, and also knew he wasn't someone who'd respond like that. Ironically, Skip was recently on the other end of such an unnecessarily dismissive exchange with another core developer on a PR of his, and (alongside myself and others involved) expressed disappointment that it was handled in that way.

frank shore
onyx spindle
#

the OGs from before I was born...

#

the pictures give such a 90s vibe

dreamy hatch
#

fully compressed gifs 🙂

frank shore
#

What I wouldn't give to have all that hair back. Ponytail FTW!

junior narwhal
#

aww you look so happy in that picture

muted whale
#

On one hand, it's weird to think that all happened a few months before I was born. On the other hand, it's weird to think in "just" the time I've been alive, Python (and you all) grew up from where it was then to the respected status of today.

silk jungle
#

I love the spam fill-in photo.

muted whale
#

And the backstory too

ionic tulip
#

Im wondering, is there any library that has as a target building semi-cacheable from a given set of requirements (like tox and hatch and pre-commit)

I need something similar for execnet remote bootstrapping and absolutely hate the idea of reinventing yet again

frank shore
muted whale
silk jungle
#

offshoot of @muted whale's post on the A in PyPA being a problem. Man it does not feel good to read the linked HN thread.

#

I'm not even one of the core PyPA members and yet seeing all of that frustration at once was not pleasant. I get it, there's a lot of things we need to improve, I guess it just feels like a hose.

#

Not to say they're being unreasonable, but yeah...

#

Pretty random I know, but I'm probably not the only one feeling this way from time to time.

muted whale
#

The blog post itself had, surprisingly, some of the most extreme language, e.g. "The PyPA should be destroyed."

silk jungle
#

I haven't read that, and I probably shouldn't (right now at least).

west basin
#

I was gonna say HN is a nest of self-aggrandising negativity (which it usually is), but the replies honestly aren't that bad - it's mostly people coming out in favour of more centralisation or complaining about things that are mostly outside PyPA's control (e.g. venvs)

west basin
vast wren
#

what HN post is this

#

also it'd be great if we got to a point that what the A stood for is the most pressing problem in packaging 😛

onyx spindle
#

Aren't we at that point now?

silk jungle
#

Strong disagree. The bad advice (even from official-ish sources!) is still a major problem.

vast wren
#

oh I just read the post

#

meh

#

rehashing the same arguments again

amber shuttle
#

Sorry for my misunderstanding, but what is HN, and what's the post we're talking about?

junior narwhal
amber shuttle
#

Ohh, I see

#

My mother always says: "when they criticize you, be respectful, take what's useful to improve and drop everything else"

onyx spindle
#

@analog oyster TOML allows for non-data information, like comments and whitespace between data tables. most libraries, like tomli (called tomllib in stdlib) only read data and discard comments and whitespace. "full" library like tomlkit, while slower, do read whole file and preserve information about comments and whitespace

analog oyster
#

gotcha. interesting. makes sense. is tomllib the library that recently was absorbed by the standard lib? i feel like i remember reading about a pep about toml

onyx spindle
#

yeah, tomllib is in stdlib since Python 3.11

analog oyster
#

nice. ok its clicking

onyx spindle
#

but it only reads data

analog oyster
#

not comments/whitespace

onyx spindle
#

it doesn't allow for data dump

analog oyster
#

btw thanks for explaining

onyx spindle
#

tomllib was known (and still is) as tomli package. there is also "write" version, tomli_w

analog oyster
#

gotcha

boreal bramble
#

this is interesting

image.png
onyx spindle
#

oh, interesting

#

I wonder what has to be broken in the data for it to be invalid

boreal bramble
#

I do wonder if this is just about types

onyx spindle
#

maybe

boreal bramble
#

i.e. keys in toml are strings so having ints as keys in a dict would be invalid input

#

not saying that this specific example would be problematic

#

but I just wonder if it's just about these sort of things

onyx spindle
#

well, that would be easy to overcome, you can just "1" iirc (or just sanitize and raise exception before dumping). My guess would be some custom classes without __str__ or a way to dump to dict easily

boreal bramble
# 
>>> import tomli_w
>>> tomli_w.dumps({-123.0: "value"})
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/tmp/tomli_w_test/.venv/lib/python3.10/site-packages/tomli_w/_writer.py", line 39, in dumps
    return "".join(gen_table_chunks(__obj, ctx, name=""))
  File "/tmp/tomli_w_test/.venv/lib/python3.10/site-packages/tomli_w/_writer.py", line 73, in gen_table_chunks
    yield f"{format_key_part(k)} = {format_literal(v, ctx)}\n"
  File "/tmp/tomli_w_test/.venv/lib/python3.10/site-packages/tomli_w/_writer.py", line 152, in format_key_part
    if part and BARE_KEY_CHARS.issuperset(part):
TypeError: 'float' object is not iterable

welp, that fails to serialize but still, maybe it's something else with types

ionic tulip
#

floats are not suported as keys, but the error could be nicer

boreal bramble
junior narwhal
junior narwhal
#

FYI I won't be available for questions tomorrow I'm taking that open source day off, I got the new covid booster today and it is wrecking me

hexed briar
silk jungle
#

I've been sick for the past week, not pleasant.

amber shuttle
meager jay
#

@nocturne swallow if you are around I can maybe avoid to open an issue in the tox issue tracker. 🙂 I have this in the config of a tox environment:

install_command =
    python -m pip install --no-deps {opts} {packages}

In tox3 this leads to the behavior, that the package under test was installed, but neither its direct nor its transient dependencies. Since tox4 the direct dependencies are installed as well (but not its transient deps). According to the docs (https://tox.wiki/en/latest/config.html#install_command) the tox4 behavior is expected.

I wonder if the behavior in tox3 was never intended? Is there a way in tox4 to only install the package under test but not its dependencies?

onyx spindle
silk jungle
#

the isort pre-commit hook is blowing up, ugh

#

I'm still mildly annoyed that pre-commit doesn't let Python hooks use wheels from PyPI. I understand the reasoning, but it would save me a lot of trouble.

junior narwhal
#

the reasoning
what is the reason?

silk jungle
#

pre-commit will only ever operate with git repos

#

pre-commit is supposed to be language agnostic and I'd imagine the maintainers wouldn't approve of adding features that are very specific to a single language

ionic tulip
#

Im pretty sure Anthony would love to have a good one, but that currently does not exist

boreal bramble
#

how about doing it in a way that is language agnostic then

#

just allow .pre-commit-hooks.yaml (not .pre-commit-config.yaml) to specify cwd so that whatever command it runs to install the repo runs from that directory

ionic tulip
#

The only tool that can roughly do that is nix, and that's not exactly something most want to use or can nicely ude

boreal bramble
#

rather than the root of the repo

#

then you can have a subdirectory within the base repo where you create whatever you would previously create in a mirror repo

ionic tulip
#

Well, it's easy to make a insecure exploitable mess that does what you describe, it's hard to do it secure

boreal bramble
#

just check that it's still part of the same directory tree?

#

after resolving path

ionic tulip
#

As for subfolders and other stuff, the tradeoff between having a maintainable tool and having bells and whistles is real

#

One gotta respect that Anthony errs on not having to maintain debts others would like him to pay for, features spike the rent

boreal bramble
#

you won't know if he's open to it until you ask is all I'm saying

ionic tulip
#

He's not open to it unless it's a implementation that won't shit his future self in the face, i tracked that stuff a while, its far from easy to get to the point where it would be ok, and i got better things to do than getting a halfway thing rejected

hexed briar
silk jungle
#

isort already fixed it fwiw

#

I'm currently trying to finish preparations for Black 23.1.0 which is due later today sooo I'm a little busy 😅

onyx spindle
hexed briar
#

Ah, fair!

#

It's gonna be "fun" when pip starts rejecting invalid versions in 3-6 months; depending on how the maintainers' time ends up...

onyx spindle
#

well, not gonna lie, I will be happy to see PEP 517 builds default in pip, it will take sooo many issues from Poetry's repo.

silk jungle
#

Just to confirm, packages that don't have pyproject.toml will no longer build?

boreal bramble
#

I thought they will but will default to isolated build

onyx spindle
#

the ones that don't have [build-system] info (if I understand the whole picture correctly)

#

@hexed briar could you verify our assumptions?

silk jungle
#

I've lost track of all of the moving parts. I'm just asking because a transitive dep of requests doesn't have a pyproject.toml file yet !!!

hexed briar
silk jungle
#

Right okay, so pip will still offer a fallback pyproject.toml.

hexed briar
#

Effectively, yes.

boreal bramble
#

not providing a fallback seems like it would have been way too big a breakage

#
  • pep 517 highly suggests having the fallback
#

I wanted to say it requires it but it doesn't say MUST anywhere as far as I can tell

hexed briar
#

From https://peps.python.org/pep-0517/#source-trees:

If the pyproject.toml file is absent, or the build-backend key is missing, the source tree is not using this specification, and tools should revert to the legacy behaviour of running setup.py (either directly, or by implicitly invoking the setuptools.build_meta:legacy backend).

onyx spindle
#

I was hoping for "break things, make people take real action", but I understand why it has to work

hexed briar
#

It was always intended to be backwards compatible.

silk jungle
#

I was going to ask more questions, but I'm just making myself more confused :)

boreal bramble
#

on the bright side, this should mean that adding pyproject.toml without build-system no longer changes how pip behaves

#

looks at flake8

onyx spindle
silk jungle
#

"only 4 years late" -- every black user from a few years ago

boreal bramble
#

not that he uses flake8 anymore now that ruff is a thing

junior narwhal
boreal bramble
#

yep

onyx spindle
boreal bramble
#

lol

hexed briar
#

Happy to answer questions about pip's plans for doing pyproject.toml-based builds by default, over on #pip. 😉

onyx spindle
boreal bramble
#

which plugins

hexed briar
#

Yea, in my limited experience, if you mention something to Charlie, he'll have implemented it 4 hours later. 😅

silk jungle
boreal bramble
onyx spindle
# boreal bramble which plugins

flake8-use-fstrings - I have to verify if existing ruff rules are enough to drop this
flake8-typing-imports - ruff doesn't cover the rule 1 fully (already reported in ruff issue tracker)

onyx spindle
silk jungle
#

Are they releasing daily?

boreal bramble
#

yes

junior narwhal
onyx spindle
vast wren
#

I dunno if the space limitation on pypi are really providing much value anymore tbh

onyx spindle
#

well, I guess it's a good blocker for people not to flood PyPI with spam packages

vast wren
#

in theory the idea was to keep the storage requirements from getting out of hand so that something like bandersnatch could work out of the box without like a large raid array behind it

hexed briar
onyx spindle
#

if there would be an action to be taken, I think going with "verified"/unverified" projects, where "unverified" are limited in space and "verified" are unlimited, would be a safer option

vast wren
#

and when I implemented uploading on Warehouse I just copied that behavior

hexed briar
#

LOL

vast wren
#

that's the per file size limit

#

mostly as a sanity check to make sure people weren't uloading like, pirated movies or something

hexed briar
#

Well, this might be a topic for discussion amongst the PyPI admins. 😉

vast wren
#

then a bunch of ML packages started taking up like

#

all the space

#

and making bandersnatch mirrors run out of space

#

so we added the project limits to try and stop that

#

but I dunno, it feels like it's inherently in opposition to how the wheel spec works

#

maybe default limits make sense, but just have a boolean flag like @onyx spindle said that's just "hey we made sure this project is doing semi reasonable things" that opens it up

#

dunno

#

random thoughts (tm)

dreamy hatch
#

there is that $10k / month bill for storage that Google Cloud donates...

vast wren
#

that's mostly bandwidth IIRC

hexed briar
dreamy hatch
#

how many TB is Ruff? 😉

boreal bramble
#

(top 100 of the largest packages, not top 100 of most downloaded)

onyx spindle
vast wren
boreal bramble
#

what's the current tensorflow's project limit lol

onyx spindle
vast wren
#

you can just sum the file sizes in the json api

#

there's not a UI for it IIRC

hexed briar
vast wren
#

oh is there?

hexed briar
#

There is!

vast wren
#

neat

#

well you can also sum it from the json api if you want to do it the harder way 😛

ionic tulip
#

i wonder if it makes sense to go back to index semantics for large files (like put al lthe large binary wheels into github release artifacts or something similar

boreal bramble
vast wren
hexed briar
vast wren
#

it doesnt' really solve the bandersnatch problem

#

which idk might be unsolvable without making bandersnatch just smarter by default to not mirror the world

ionic tulip
#

links take less space than full files

hexed briar
#

And pip install's flakiness is now a sum total of pypi.org flakiness + whatever host flakiness

vast wren
#

it's something we'd have to sit down and really think about and nobody's really had the time

ionic tulip
#

true , its a can of cans of worms

boreal bramble
#

there's also that discussion about release immutability

vast wren
#

just for funsies, 100TB of GCP storage using their standard storage class is like 2700/mo

#

which is like 10x our current size

#

well 9x

#

whatever

#

many x's

hexed briar
#

7.2 but whatever

vast wren
#

math is hard ok

dreamy hatch
vast wren
#

it's not like I'm sitting at a computer that can do it for me

ionic tulip
#

but part of me is like "yo guys, machine learning models are not package data, don't use pypi as your backwater cdn

vast wren
#

most of it isn't models

#

it's because they have to bundle CUDA

boreal bramble
vast wren
#

and CUDA is massive

hexed briar
#

Yea, it's GPU-specific binaries and whatnot.

vast wren
#

they have t obundle CUDA for... I think Nvidia reasons

#

or they might be real reasons

#
  • the wheel spec
#

which doesn't allow linking to external things anyways even if the nvidia/possibly real reasons were fixed

boreal bramble
#

so much duplication 😅

onyx spindle
ionic tulip
#

hmm, so those coudl possibly safe space by better dedup/split but stuf is a mess

vast wren
#

I think a better wheel spec that's optimized for compression saves a signifcant amount of space too

#

but that's a whole extra can of worms

hexed briar
#

^ +++

onyx spindle
#

let's wait another decade before we go and change the spec, ok? 🙏

boreal bramble
hexed briar
vast wren
# 
requests:

    current: 57k
    tar+xz: 44k
    tar+zstd: 45k

pip:

    current: 1.4M
    tar+xz: 914k
    tar:zstd: 960k

tensorflow:

    current: 402M
    tar+xz: 126M
    tar+zstd: 141M
#

from like 2 or 3 years ago

ionic tulip
#

calls in for earthworm jim with the can opener gun

vast wren
#

oh that's not really tar though

#

sorry that's misleading

# 
current: The existing wheel format
current+bzip2: The existing wheel format, using ZIP_BZIP2 instead of ZIP_DEFLATED
current+lzma: The existing wheel format, using ZIP_LZMA instead of ZIP_DEFLATED
tar+gz: A new format that keeps using the existing format for .dist-info directories, but puts all the “data” into a tarball and compresses with gzip
tar+bz2: The above new format, but using bz2
tar+lzma: The above new format, but using lzma
tar+zstd: The above new format, but using zstd
tar+brotli: The above new format, but using brotli
hexed briar
#

... why do you have that handy but not a calculator!?

vast wren
#

my brain is good at remembering nonsense I've typed into the void at some point in time, not good at doing math okay

boreal bramble
hexed briar
boreal bramble
#

quite ridiculous

ionic tulip
#

if we ever do a new install format, can we please make sure dist info folders drop the version metadata from the dist info folder name

vast wren
#

so I dunno, 30% file sizes for our largest packages seems like a good reason to do a new spec

#

at least to me

hexed briar
#

Introducing a new file format, in this economy!?

#

:P

vast wren
#

the real trick is just wrapping all of the data up until an inner zip file, that doesn't use compression, and storing that inside the wheel as data.compression.zip

onyx spindle
vast wren
#

that's where most of your savings come from

hexed briar
boreal bramble
vast wren
# 

    output/current: 402M
    output/current+bzip2: 314M
    output/current+lzma: 136M
    output/tar+brotli: 148M
    output/tar+bz2: 309M
    output/tar+gz: 396M
    output/tar+xz: 126M
    output/tar+zstd: 122M
#

tensorflow

hexed briar
vast wren
#

interesting that not changing the format, but using lzma does so much better

ionic tulip
#

hmm, im totally ofr lzma wheels

vast wren
#

I think that's mostly because tensorflow's fiel size is a single massive .so

onyx spindle
vast wren
#

so the fact zip compression is per file doesn't really matter

#

well doesn't matter much

boreal bramble
#

ah yeah, it is

#

damn

hexed briar
vast wren
#

it is

#

since 3.3

hexed briar
#

Oh, then we have an option...

vast wren
#

it's "optional", so if your python wasn't compiled with support for it then you won't have it

#

but the same is true for zlib

#

just zlib is more universal

#

the problem with lzma is mostly that it's kind of slow and can be memory intensive

hexed briar
#

Isn't zlib accepted under manylinux?

onyx spindle
vast wren
#

so installing from wheels currently relies on the optional zlib stdlib support being enabled

#

if we did lzma wheels it would instead rely on the optional lzma stdlib support being enabled

#

the only real difference is it's basically illegal to have a computer without at least 17 copies of zlib

hexed briar
#

Or... useless. :P

vast wren
#

lzma has some downsides

onyx spindle
#

abstract question: what would it take to make this "mandatory"? I mean, I can smell issues every time I hear that something is going to be based on something optional

vast wren
#

I think it's slower, and more memory intensive

#

zstd is much better in that regards

#

but isn't in the stdlib

vast wren
#

the same way zlib is defacto mandatory now, it's just technically optional

#

https://discuss.python.org/t/making-the-wheel-format-more-flexible-for-better-compression-speed/3810/1

Discussions on Python.org
Making the wheel format more flexible (for better compression/speed)

While dreaming about plans to standardize sdists, I have been thinking about how sdists are strongly tied to gzipped tarballs and wheels to zip files (although obviously more wheels than sdists). I view this as unfortunate as it means neither build artifacts could take advantage of e.g. zstd for compression and speed benefits. I noticed that co...

#

the whole discussion 🙂

boreal bramble
#

though that one might not be Python up to its spec, Idk

#

but there's probably someone out there!

#

[insert relevant xkcd]

ionic tulip
#

since lzma is basicially more than 20 years old, it seems like its ok to require it

vast wren
#

IME you only end up with a python without zlib by mistake because you're custom compiling and forgot to install zlib headers

onyx spindle
ionic tulip
#

hmm, maybe a pep is needed for mandatory zlib/lzma

vast wren
#

I don't think we really need lzma to be mandatory for Python to make it mandatory for wheels

#

like there are use cases for Python that don't require installing things from wheels

#

and there's not a lot of reason to put undue burden on them

#

there would maybe be a bit of a transitionary period where common distributors have to make sure their Python has lzma support by default

#

but I think that's fine

#

it would be cool to try and get zstd into the stdlib, but I have like no motivation to actually do that, but I think if we did most of the arguments for lzma go away and zstd is the clear winner

ionic tulip
#

how long until 3.12 would be the minimal version?

onyx spindle
#

~5 years

#

3.11 will be dead on 24th October 2027

vast wren
#

well you also don't need it in every version of python you support

#

you can just say that new style wheels are only supported on python X

#

3.12 or whatever

#

presumably old style wheels would be supported near forever anyways

#

it's nice of course if you can just support every version right away 🙂

#

which is a definite point in lzma's favor

#

I suspect most packages wouldn't want to switch to a new style wheel right away anyways, until support in the packaging tools was "old enough"

onyx spindle
#

Or, hear me out, lets double the amount of wheels a project has to make! F*ck storage ||/jk||

ionic tulip
#

is lzma supportable offthe bat ? ifit works for the intensive ones, then lets give it a go

vast wren
#

um

#

I'm not sure

#

I think regardless it might require tooling changes to support LZMA wheels

#

certainly if we want to do the inner compressed data.zip.xz it would

#

If we want to just allow wheels to use LZMA compression, which would help for wheels like tensorflow... I'm not actually sure if tools would support that natively right now or not

#

I think pip etc might if they're using the built in zipfile module and Python has lzma support

#

obviously the build tools that build the wheels would need to grow support for it, but that's a shorter thing than consumers

onyx spindle
#

If it's just changing the compression lib, it shouldn't be a big problem

vast wren
#

this is pip FWIW

# 

    output/current: 1.4M
    output/current+bzip2: 1.3M
    output/current+deflate: 1.4M
    output/current+lzma: 1.3M
    output/tar+brotli: 910K
    output/tar+bz2: 1.0M
    output/tar+gz: 1.2M
    output/tar+xz: 914K
    output/tar+zstd: 959K
#

so just swapping compression to LZMA doesn't really help much, but doesn't hurt either

onyx spindle
#

I guess the change might be more significant in bigger packages

vast wren
#

yea this is tensorflow

# 

    output/current: 402M
    output/current+bzip2: 314M
    output/current+lzma: 136M
    output/tar+brotli: 148M
    output/tar+bz2: 309M
    output/tar+gz: 396M
    output/tar+xz: 126M
    output/tar+zstd: 122M
#

that mostly gets big savings because the bulk of tensorflow's size comes from a single large file

onyx spindle
vast wren
#

I haven't done extensive research to know if the big packages are big because of a single file

#

or because of lots of files

#

that would be an interesting thing to figure out though

dreamy hatch
silk jungle
nocturne swallow
#

The link to alpha is not working

silk jungle
#

I should run a linkchecker on the post 😅 Gimme a second.

#

what the hell was I thinking?

image.png
silk jungle
#

(this is partially why I make my first announcements for a new release here, it's low pressure compared to everywhere else :P)

nocturne swallow
#

Cool

dreamy hatch
west basin
#

but what about us poor Poetry users 😛

onyx spindle
#

Poetry is actively working on enabling pep 621 metadata spec

nocturne swallow
#

@silk jungle seems somehow the newer black does not recognizes trailing commas in literals 🤔

#

when there's a single element

#

nm 🙂

nocturne swallow
silk jungle
silk jungle
nocturne swallow
#

But still a regression no?

#

I mean we use this for generating easy to diff literals that start with one element but will grow

#

I guess black doesn't know if is tuple or Literal so can't handle it differently no?

#

Because I agree for tuples makes sense but not sure if it does for Literals 😅

silk jungle
#

Well it's probably possible but that's kinda ugly

#

metaphorically / from a purity standpoint, not a style standpoint 😅

nocturne swallow
#

🤔 well let me know how strong you feel

#

Because for now we pinned as it breaks one of our projects, but if this is how it's going to be going ahead, we'll just have to live with it 😅

shadow zealot
#

You can say it’s technically a regression, but Black changes its formatting rules quite often so it makes sense it’s not a bug

nocturne swallow
#

Well I meant more like I don't think this should happen for Literals while I agree makes sense kinda for tuples 😅

shadow zealot
#

Not that I agree with the change, just that it makes sense from Black’s perspective

silk jungle
#

We're trying to do better. We have a stability policy which means we don't make changes to the stable (default) style more often than yearly.

junior narwhal
#

Charlie is in #general so you can influence Ruff's incoming autoformatting style 😛

nocturne swallow
#

Called me old school, I like my formatter black 😅

#

I like it's zero configuration style, while ruff has a lot of knobs for my taste for now 😅 so didn't adopted it anywhere yet...

boreal bramble
#

Tbf, nobody would probably use linters as comprehensive as ruff without a way to disable rules 😄

#

But it could use with more rules in the out-of-the-box experience which is planned through categorization and stuff I think

onyx spindle
#

Similar to how isort has profiles would be nice

shadow zealot
#

Personally I would prefer less configuration, profile feels like a step backward after we have black

boreal bramble
#

Linters are all about all kinds of conventions. But the auto-formatting stuff in ruff is opinionated towards Black

shadow zealot
#

People also said formatters are about all kinds of conventions (oldies probably still remember yapf?), until Black took over.

boreal bramble
#

It would force Ruff to either limit scope of its rules significantly to be of use to anyone or be a dead tool that nobody wants to use because it's way too strict with all of the rules it keeps on adding

#

sane defaults imo are enough for a linter; it allows them to be widely used by people who either don't want to bother with configuration or are less experience with Python and are looking for a tool that nudges them in the right direction while appeasing a very big audience of projects with different requirements

shadow zealot
#

It’s the tool’s choice I guess. Black is Black because Lukasz doesn’t care if people want to use it and effectively it’s his way or the highway, and people (surprisingly?) bought it

#

But I understand not every tool wants to be like that

boreal bramble
#

I treat formatters as different kind than linters (though the latter can nudge you about what the former automatically fixes)

ionic tulip
#

a certain level of opinionated is important else a tool adds fog instead of value, a key feature of black is that you cant do whatever you want and that creates synergy for everyone, every time one starts to give everyone all the bells and whistles one gets excess diversity - which when it comes to adding cognitive load on how to format is a problem - i prefer people figure more important things than where exactly to put a comma ^^

vernal hornet
#

FWIW I consider myself close to an autoformatter purist 😅

#

I think it's natural for people to someone start on a new autoformatter, and see it as an opportunity to do something very different from Black (e.g. something highly configurable), but I don't really want to go in that direction

#

I do plan to make indentation and quote style configurable which is perhaps controversial

ionic tulip
#

its against the zen of python ^^ there should be preferably one way to do things

#

so configruable formatters 🫰

vernal hornet
#

We need to go back and remove tabs from the grammar... it's the only way

boreal bramble
#

or maybe... remove spaces from the grammar

junior narwhal
#

quote style would be very nice

boreal bramble
#

how about... just remove spaces from supported indentation, not all spaces

vernal hornet
boreal bramble
#

😄

onyx spindle
junior narwhal
onyx spindle
shadow zealot
#

Instead of arguing tab vs space people should all use U+3000 Ideographic space for indentation. Fixes all the problems.

vernal hornet
#

We'll also be enforcing CR line endings sadly

boreal bramble
#

blasphemy

junior narwhal
# 
from __future__ import braces
shadow zealot
#

CRLF is obviously the only correct way, you need to go down one line (line feed) and return to the beginning of the line (carriage return)

vernal hornet
#

So that your code only runs on Commodore and very old macOS versions

#

Na we're doing CR! Not even CRLF

#

😄

ionic tulip
#

ideal case would be programming languages go away from text based syntax and use a serialized ast, that only allows syntactically correct programs to be saved, then rendering them however way user wants to see them

boreal bramble
#

where did I hear that before

#

🤔

ionic tulip
#

but until then, hell no to bells and wistles, everyone gets the same format so that one doesn't have to switch format style familiarity between projects

junior narwhal
#

with AI now English is the best programming language

boreal bramble
ionic tulip
boreal bramble
#

because that's not what AST typically has while preserving some of the empty lines is crucial part of auto-formatter

ionic tulip
#

one wold certainly need a syntax concept for ligical groups

boreal bramble
#

yes, I don't mean empty lines specifically

ionic tulip
#

but thats not empty lines, its a contextual group

boreal bramble
#

but info that would allow you to differ

ionic tulip
#

those would need ast nodes

shadow zealot
#

syntax concept for ligical groups
As in… braces? shudder

boreal bramble
#

doesn't have to be braces, can be dedicated keywords for start and end of section

#

like in pascal, bash, or whatever

ionic tulip
#

@boreal bramble first error, a grup is a collection, nto a start/end marker

#

we are talking about a ast here, not tokens

boreal bramble
#

I meant from how you would type this

#

not from how it would be stored in the actual file

ionic tulip
#

one would not type this

#

one would select a number of statements, then tell the editor to put them in a group

boreal bramble
shadow zealot
#

The point being most languages do have an ast node for contextual groups (some of them e.g. C merge the concept with variable scope but the two are conceptually different, JavaScript aimed to separate those but it implemented the separation too awfully for it to be accepted)

ionic tulip
#

no progamming language has a concept for a "functional group" of statements as in a text file one would just use white-space, and it wouldn't even appear in the ast

boreal bramble
#

yeah, you're right, I'm confusing different concepts here a little bit

#

sorry

shadow zealot
#

I’d argue it’s the other way around, most programming language designers never bother to introduce statement grouping to ast because people like to just use whitespace

ionic tulip
boreal bramble
#

it does seem to imply certain strictness on what you actually type so that it ends up represented in a file properly so that other's people editors show it properly too

ionic tulip
#

yeah, no more wsgiref in the stdlib having randomly 20 lines of space becasue the implementor used a strange editor and nobody fixed it

#

alltho i never checked if they fixed it after the initial addition

shadow zealot
#

This discussion makes me think though, the entire premise is people want to have a syntax-free ast and be free to choose whatever frontend to use. But if history can offer a hint (you know, this is a packaging Discord), if you actually try to do that, people would beg whatever Authority to officially bless one of the frontends and we’re back to where we are in reality

ionic tulip
#

i recommend the cobol lookalike as official frontend for the python in ast language

boreal bramble
#

another problem, how do you represent strings in AST to account for the fact that you can't fully automatically wrap long arbitrary text and still have it look good no matter what text it actually is. How do you do this abstractly to not include opinionated formatting into the format while still representing some additional information about what that text is so that it can be displayed appropriately by the editor

ionic tulip
#

tpe annotations as prose from an mba wil lave a never to forget taste

#

long docstrigns can just be embedded restructuredtext files (potentially also speudo ast)

#

as for long text, give some examples

shadow zealot
#

embedded restructuredtext files
Not afraid to make enemies I see!

quartz yew
#

I'll be at FOSDEM, if anyone is also there and wants to meet just lmk

ionic tulip
hexed briar
#

I want it on record that I appreciate @vast wren's GIFs!

hexed briar
#

Also, today's the first time I've actively gone ahead and told discuss.python.org to not show me a person's posts!

Screenshot_2023-02-03_at_20.28.20.png
onyx spindle
#

Sigma move

hexed briar
#

(you can too, if you go a users' profile -- it's near the top right corner)

junior narwhal
#

yeah it's certainly a balancing act, I think this year I'm also going to associate less with people that are not at least moderately optimistic/encouraging

hexed briar
#

For me, I think this is a case of saying "this person will never say something useful for me" -- or, at least, that the benefit of even reading things they'd write is less than 0 overall.

#

Besides, there's a marker that I can click on to show it still; in the right spot.

Screenshot_2023-02-03_at_20.50.41.png
junior narwhal
#

something I learned over the past few years that makes me genuinely bummed out is that, contrary to my pretty hard-core stance on open discourse/dialogue, some people really have nothing useful to contribute or can in fact negatively impact a discussion with relative ease

hexed briar
#

nods.

vernal hornet
#

The corollary that I've tried to remember, but am maybe not very good at yet, is that not every comment demands a reply

silk jungle
junior narwhal
#

yeah I don't think it's worth it, as far as I know the standard practice is to still use virtual environments in containers

onyx spindle
# junior narwhal yeah I don't think it's worth it, as far as I know the standard practice is to s...

To add to that

apt-listchanges: News
---------------------

python-pip (23.0+dfsg-1) unstable; urgency=medium

  This version of pip introduces PEP 668 support. Debian's python3.11
  interpreter will soon (>= 3.11.1-3) declare the installation to be
  EXTERNALLY-MANAGED, instructing pip to disallow package installation outside
  virtualenvs.

  See: https://peps.python.org/pep-0668/

  Practically, this means that you can't use pip to install packages outside a
  virtualenv, on a Debian system, any more.

  See /usr/share/doc/python3.11/README.venv for more details.
  If that isn't available yet, check:
  https://salsa.debian.org/cpython-team/python3/-/blob/master/debian/README.venv

 -- Stefano Rivera <stefanor@debian.org>  Wed, 01 Feb 2023 19:14:08 -0400
junior narwhal
#

oh, nice!

vast wren
fierce horizon
hexed briar
frank path
#

FOSDEM

fierce horizon
#

@onyx spindle re:German I think there’s a difference between the “German sounds aggressive” meme (which really is just about an abundance of hard consonants in the language) and what @frank path and me were talking about. And I think you’re completely correct about it @frank path: I think people growing up in the US get taught to add more “politeness phrases” to everything, and paragraphs missing them are interpreted as blunt.

The American “Hihowareyou” isn’t a statement in German, it’s a question to which you expect an answer: From strangers usually a “Great, Good, OK, or ‘could be better’” without expanding on it, from friends anything from one word to a therapy session.

And there’s non-verbal components too: Russians think you want something when you smile at them, Germans rarely smile in everyday interactions (but when they do you know they’re just happy), and Americans smile when they feel any emotion other than miserable.

#

(I’m talking about Americans because I have most experience with them, but to a degree, I think most cultures are more like the American than the German one)

junior narwhal
#

Americans smile when they feel any emotion other than miserable
ouch, true lol

fierce horizon
#

Don’t even get me started on Japanese smiles 😆

frank path
fierce horizon
#

It’s 100% cultural, but that doesn’t mean the way it expresses itself isn’t also 100% through grammar. I said “politeness phrases” but it could just as well be exactly what you said and be all through sentence structure or so.

arctic copper
#

An alternative to harsh german is also viennese german 😉 , which can have the same chanson cadence as french when a particular heavy version of it is used.

The „how are you“ can also be quite aggravating for austrians or viennese, as we know they dont really care for an answer and that is insincere for us.

We would only ask how are you, when we actually want to know and this mostly is done to people that are close. So the question can insinuate a closeness that we dont see.

This can even be a problem between austrians and germans (divided by a common language). Germans often use „Hallo“ as greeting to everyone, in austria it is only used if you also are close enough to use „Du“ or if you are a child, as in close enough to not use courtesy variants. Entering a shop and getting greated with „hallo“ can be quite off putting for an austrian. (Not always and less so in younger generations).

arctic copper
#

PS: That whole thing is why I am a fan of pre-defined "tags" in stuff like code review. It takes the cultural part out of the conversation and is from what I heard also a boon for people with ASD (but no real experience).

what I mean is something like <opinion> I would do this via a list and not a custom object </opinion> (example)
where opinion is predefined as:

This is how I would do it, but this is not meant as critic or requirement.
Does not mean, the current code is bad or so.

Then people can decide to use the tags to have the intent explicitly defined.

I know it gets close to handling conversations like it is a programming language, but I just have seen to many conflicts from misunderstandings where one party assumes the intent behind a comment and becomes defensive and vice versa.

onyx spindle
ionic tulip
#

@junior narwhal that blog post on cli in rust is quite lovely, but i do wonder if there could be an equivalent of typedsettings for it

edit link for the interested

Ofek Lev
Writing a CLI in Rust

In this post I’ll walk through the setup of an example project to show how to build a modern CLI in Rust.

shadow zealot
#

I miss those macros pulling values from Cargo.toml into the CLI, I hope Python could have something similar. Those things you can only get when there’s a compilation step.

ionic tulip
onyx spindle
shadow zealot
vast wren
#

we have a compilation step if you're packaging up your software

#

it's just typically the compilation step is "copy these files"

ionic tulip
shadow zealot
#

There’s no reason you can’t technically, but Python has a user problem. Once you rely on the packaging step someone’s going to complain the code doesn’t work when they clone and run directly.

vast wren
#

it just working for basically anything in cargo.toml without needing to figure out much is really nice though 🙂

shadow zealot
#

I blame the interpreter being too easy to run, everyone and their pets use Cargo not just because Cargo is nice, but also compiling Rust code without Cargo is horrible.

vast wren
#

so we need to make Python worse

ionic tulip
#

pythonhas all the metadata avaliable, so i sdone see a issue

vast wren
#

I suspect if we made the default tool expect something like a cargo run, we'd end up in a state where most people expect to do that too tbh

shadow zealot
#

Yes

junior narwhal
onyx spindle
fierce horizon
#

my dream would be that editable installs recompute metadata automatically

shadow zealot
#

Pulling metadata from code imposes restrictions to the code base. Existing tools either performs static analysis to the code base (restricts the syntax) or actually imports code (restricts module layout and more). It’s possible to work around but IMO objectively worse. With pulling metadata into code not being possible, I’d personally prefer to rely on tools like pre-commit to automatically keep the different version declarations in sync.

#

But I understand people prefer workarounds than being theoretically superior (which is a good thing, please don’t be like me). Editable install is another one of those things I personally think don’t need to exist to begin with, since you still miss auto metadata computation, but if you implement that, you can rebuild the entire project and can use regular installs anyway…?

hexed briar
#

FWIW, you can do that today.

quartz yew
#

it's not sys.executable, that'd make things more complicated

#

it's sys.ececutable from the PEP 517 hooks

#

also, editable installs only in the pre-release

#

they work but debugging via code editors is broken, because we do not actually use the path they try to set breakpoints on 🤣

shadow zealot
#

In find it intriguing an Indian native living in the UK is using a baseball reference! https://discuss.python.org/t/23442/6

Discussions on Python.org
Python Packaging Strategy Discussion - Part 2

@pf_moore’s latest comment is the 3rd strike[1] for me writing down what I think the whole point of these strategy discussions is. Wrote that in a separate topic; in case folks wanna opine on that. That’s a baseball reference. ↩︎

junior narwhal
silk jungle
#

Remote-only, huh

kind moon
#

Nice

junior narwhal
arctic copper
#

AI and hands (and teeth), name a better nightmare fuel couple!

ionic tulip
#

hmm, still better artist than me ^^

junior narwhal
vast wren
#

We enabled it on warehouse today

#

It seems to work pretty nicely

#

Don’t have a ton of experience with it yet though

dreamy hatch
vast wren
#

Im on my phone but as far as in know it’s a different event then pull requests

#

So you could run buildbots on the merge queue safely

#

And get pre-merge protections from the buildbot fleet

junior narwhal
#

and now an intermittent bug I've been trying to investigate is fixed... 😅

onyx spindle
#

TIL Heisenbug

arctic copper
#

So, it points to task group as a fix, bug from the way I see it, tasks without a reference are often used for fire and forget tasks. With those the TaskGroup is not really a solution unless you wrap your whole programm in a TaskGroup.

I feel there is a need for a thing that wnables fire and forget. Of the top of my head just a container that keeps referencen and on adding new tasks, it checks all stored for completion and just removes those, but this would be a problem because it will stay constant if you never add one again.

Maybe just check if completed from time to time,… and we are again at fire and forget tasks 🙂

(The implementation example is somewhat of a straw man, I know and the usage of those tasks could be a huge discussion about code design and anti patterns)

shadow zealot
#

The create_task interface never really makes sense to me, I’ve always wanted something like asyncio.queue(coro) that I can just send something to the loop to run

vast wren
#

I dislike the asyncio interface entirely

#

Trio is much better imo

ionic tulip
#

asyncio has the major blunder of starting out without a async syntax, now it carries the callback hell plus the low hanging good ansyc api fruits

#

trio did so much right there

vast wren
#

asyncio had yield from which I think is enough to invent trio, albeit not quite as nice version of trio without async with statements and the such.

#

It’s just before trio callbacks and unstructured async was state of the art

ionic tulip
#

@vast wren async iteration and async contextmanager are a must have for composable bits,

vast wren
#

eh not really, like nothing in trio requires those

#

it just makes the patterns easier

ionic tulip
#

@vast wren without the patterns that enable easy and fun apis, one descends into the madness part too easy

vast wren
#

like context managers are largely just sugar over:

yield from ctx.__enter__()
try:
    ...
finally:
    yield from ctx.__exit__()
#

that's still better than callbacks

ionic tulip
#

having the syntax that allows makes it usable,

vast wren
#

like it's not like callbacks have a better answer to that problem

ionic tulip
#

@vast wren but that goes from knowing the new api towards how the abomination would look if the new style as not possible

nobody in their right mind would write a networkign famework where the users of the lib would have to do manual yield from context managers (fearfully looks into the direction of dabez)

vast wren
#

I mean, that's what the callback API required

ionic tulip
#

Well, the callback api was so terrible it made me like twisted

#

Twisted deferred and protocol concepts Brough a number of things well together

fierce horizon
silk jungle
#

@onyx spindle you really nerd sniped me on GHstats 😅. I didn't like how the front-end was made by manually copying and pasting HTML and JS. Added jinja2 templating so it can be handled automatically too.

image.png
onyx spindle
#

LOL

#

Nice

#

Glad to be of service here 🤣

silk jungle
#

it gets even fancier lol

image.png
onyx spindle
#

Nice

#

So, it asks the questions and generates website or is it just for data download?

silk jungle
#

so setup + add-repository handle the GHstats configuration, and then generate-html generates the web directory which will be used during deployment

#

the issue data was already handled by pre-existing scripts and CI

#

add-repository will invoke the generate-html command after it's done running

onyx spindle
#

Nice

silk jungle
#

Let me know if anything goes wrong :)

#

FYI the setup command doesn't handle any optional configuration, only the bare minimum. The configuration I'm using contains some optional features. In particular, the author object configures the content in the footer.

silk jungle
#

Hmm, I keep forgetting that DMs are a thing. I basically never use them. Probably should change that though 😅

dreamy hatch
#

looks good!

I tried in on python-pillow/Pillow, then started a local web server with python -m http.server --directory web , but at http://[::]:8000/Pillow/ the charts don't load and the console says:
Uncaught TypeError: Failed to resolve module specifier "luxon". Relative references must start with either "/", "./", or "../".

onyx spindle
ionic tulip
#

Open source Was never designed to ensure work is paid fairly, freedom of the users, not sustenance of the developers

#

It was always about the freedom to use, never about the freedom to get good money for good work

#

If opensource enabled sustenance in any meaningful way I'd probably be working as full-time pytest maintainer and trainer while trying to upstream key bits to cpython

#

But I wouldn't be able to contribute to the sustenance of my family then

hexed briar
#

"Maybe we made the world wrong"

junior narwhal
#

Not drop-proof.
Shocking

ionic tulip
#

the vibranium cups where not well received, unfathomable cost and breaking any flooring if you drop them ^^

silk jungle
#

It's built using Vite. The deployment + the issue data is handled by a GHA workflow which deploys to a production branch. It's a stupid design I'll admit but I've spent way too much time into it already to be bothered to clean it up anymore...

dreamy hatch
#

fair enough! looks good though 🙂

silk jungle
#

It would probably make more sense to generate the web assets at build/deployment time and hide it all away from the end user (ie. handle it the same way the issue data is). That makes tweaking on the front-end harder though as vite devwouldn't be usable any longer.

junior narwhal
lean bear
#

wow

ionic tulip
fervent copper
#

I don't see how a library can make any money vs something like docker or elasticsearch which can be offered as a PaaS

shadow zealot
#

According to the article they are indeed offering PaaS. No idea how that would work.

dreamy hatch
#

“We’re building cloud services, and we’ll have a generous free tier and usage-based pricing after that,” Colvin continued.

hexed briar
#

I'm not really surprised to see a maintainer of a popular library to take that popularity and convince a VC that it's somehow something that they can build into a platform. ⛄

onyx spindle
#

this is actually really nice situation compared to what was happening with core-js drama

#

and I guess Samuel had a good pitch for investors to give him the money. I doubt anyone would invest in something that wasn't going to be a success

#

at least potentially

hexed briar
#

I don't really care if they make a usable PaaS, I'm 90% sure that Sequia and friends won't care if this is one of the dud startups that just breaks even or even makes a loss forever, and if it means Samuel gets to work on making pydantic better in the open (like we have with Textual + Rich), all is good. As long as these folks don't turn evil or have like something bad happen to them circa VCs wanting an ROI.

onyx spindle
#

well, I guess that last part is for lawyers to figure out in contracts

hexed briar
#

Yea, exactly. And, I'd hope these folks are smart enough to run this by lawyers and reduce any personal liability if things go poorly.

analog oyster
humble phoenix
shadow zealot
#

I’m not sure if I should be happy that computers can read error messages intended for humans, or sad by the fact that aspiring programmers read errors worse than a computer.

amber shuttle
#

Maybe both? 🙃

ionic tulip
#

there always have been people that read errors worse than computers, now if all of those sad people ask chatbots instead f wasting maintainer time, its a net win

hexed briar
#

Oh wow, that's hilarious.

junior narwhal
#

@ionic tulip I saw https://twitter.com/ossronny/status/1630491958962450438

Ronny (@ossronny)

Im starting to consider #python #ruff an antipattern , whenever I see it added somewhere there is loads of custom settings and noise

The key win of the other tools was converge in style, now we get a speedrun to divergence, disheartening

Twitter ·
#

soon Hatch will take care of this

#

in the coming releases I will add commands that other package managers like Cargo have, in this case a top-level command for formatting which will manage an isolated virtual environment populated with Ruff and Black and will have predefined "good" config that will be used in lieu of user config

#

when that happens Hatch will rely on that forever and put nothing in pyproject.toml except for maybe an override or 2

#

@vernal hornet does Ruff merge configuration from multiple sources?

vernal hornet
vernal hornet
#

I've been thinking a lot about this... I was hoping to better understand the critique. Most Ruff migrations tend to be 1:1 ports from [Flake8 + plugins + isort + some pre-commit hooks] to Ruff, so the end state is often “the same”. Increased divergence or customization isn’t an intended goal, so if that's being felt, I'd love to hear more. But OP is under no obligation to engage, all good regardless.

ionic tulip
west basin
#

I don’t agree with the criticism but I think it’d be nice to have recommended presets a la eslint. I think that’s already been suggested

ionic tulip
#

having recommended presents that build on top of each other and promote convergence would be absolutely fantastic - from what i understand right now that would completely elevate the "issues" i take right now

quartz yew
#

just foud the libseccomp maintainer is also called Paul Moore 🤣

vernal hornet
vernal hornet
quartz yew
#

I had to check

#

I was like, it's not the same person as pfmoore, right?

arctic copper
#

Maybe a little input from a place that may be underepresented here. Single maintainer/programmer (I mean, on my projects I 99% work alone on everything)

Customization is really nice and one of the big reasons, I do not use black.

Generally, why should I use something that makes things harder for me to understand my own code after some time (formatter) or that makes me ignore warnings/errors in general (linter). Normally there is the huge benefit of helping to work together with others, but my projects and probably most python projects are single maintainer with no real possibility of having more maintainer in the future.

I would argue that the amount of stuff done with python, that have more than a single person working or looking at it, is probably less than single digit percent.

(Sorry for run on sentences, it was rather hard for me to formulate it differently)

ionic tulip
# arctic copper Maybe a little input from a place that may be underepresented here. Single maint...

key reason is use black is that it streamlines contribution - it completely anhillates and discussion about code style - for example for projects at the size of pytest tis a huge win

another key win of the lack of configuration is that every single project using black will have the same style, so no more getting used to the styles
for me those points are massive wins in terms of cohesion and project mobility, they enable and support broad collaboration

#

i add them to all of my projects, even the small ones, as its excellent for setting up contributor expectations

arctic copper
#

I am all for a default config, but please do not take away options for people like me.

A lot of pip (see packaging discussion) and libraries already seems to cater to the minority that write big projects with many people, this would just increase that.

arctic copper
dreamy hatch
#

I agree with Ronny's points, but if you don't like Black/autoformatters, that's fine, there's absolutely no requirement to add it to your projects

dreamy hatch
arctic copper
dreamy hatch
#

ah right

I guess Ruff will be either be configurable to disable autoformatting, or it will be disabled and you need to configure to enable

vast wren
#

auto formatters are great, no configuration options are great

#

one of the main things I even like about go is that go fmt is just more or less mandatory for everyone

onyx spindle
#

Consistent format across community makes it easier for people to get into the project

west basin
#

I do hope nobody here has had the misfortune of having to deal with historical tz offsets (local mean times)

# 
>>> datetime(1921, 9, 1, tzinfo=ZoneInfo('Asia/Nicosia')).isoformat()
'1921-09-01T00:00:00+02:13:28'
#

that's right, 2h 13m and 28s

#

thanks, I guess

quartz yew
#

yeah, paul gansle would have something to say about that probably 🤣

junior narwhal
#

documentation has never been more important! this is awesome

kind moon
#

Woah

#

Nice!

amber shuttle
#

I've also been playing around Bing's AI, it's cool until it starts stating inaccuracies 🙄

amber shuttle
#

But it's accurate most of the time 🙂

hexed briar
desert verge
#

tfw discourse breaks the REST spec

shadow zealot
#

REST is a style. There is no spec.

desert verge
#

oops

#

meant HTTP spec

shadow zealot
#

How does the above breaks any http spec?

buoyant verge
#

you can have a body in an http get request

desert verge
#

but you aren't supposed to

buoyant verge
#

it is annoying for some badly implemented caches but those implementations are wrong

shadow zealot
#

Isn’t the above the response’s body though? Ah I misunderstood, it is the request body

buoyant verge
#

are you facing any real problem regarding this?

desert verge
#

in this case I'm getting a 413 when I try to provide the body above

#

which is payload too large

#

payload is simply {"post_ids[]": 430} and I've also tried it without the []

buoyant verge
#

can you provide the full request/response raw text?

desert verge
shadow zealot
#

Actually I think the API doc is simply wrong?

#

This works for me

desert verge
#

wow

shadow zealot
#

I “reverse engineered” it https://meta.discourse.org/t/reverse-engineer-the-discourse-api/20576
(They try to make it sound cool but it’s basically opening the dev tool and go through the network tab…)

Discourse Meta
Reverse engineer the Discourse API

Discourse is backed by a complete JSON api. Anything you can do on the site you can also do using the JSON api. Many of the endpoints are properly documented in the discourse_api gem, however some endpoints lack documentation. To determine how to do something with the JSON API here are some steps you can follow. Example: recategorize a topic...

desert verge
#

thanks for the help, but in the end it turns out that endpoint isn't even what I'm looking for, lol

shadow zealot
#

FWIW I’m not even sure DIscourse itself uses that endpoint. From what I can tell its frontend only uses /t/{id}.json

desert verge
#

yeah :(

#

which is on topic 963 as the 40th comment/reply/post/something

#

oh.

#

its the stream field

hexed briar
#

I actually quite like how basically all of discourse is available as HTTP requests to get JSON blurbs.

desert verge
desert verge
verbal plover
#

Oops.....

image.png
desert verge
#

it's an out of date ruff schema on SchemaStore

silent hornet
#

Hello all, I've been working on Customs-Inspector, a poetry plugin that hooks into poetry update process and calculates the diff between package updates and asks you to audit the changes before continuing with the update. The goal is to eventually build something like https://mozilla.github.io/cargo-vet/ for federated auditing of packages. Here's the discussion I created: https://github.com/python-poetry/poetry/discussions/7715 . I would love to hear your thoughts as this is probably going to be my Bachelor's Thesis.

Introduction - Cargo Vet
GitHub
An easy way to manually audit changes in package updates · python-p...

Hello all, Recently, I've been working on Customs-Inspector, a proof of concept plugin that hooks into poetry update, generates a diff for package updates and asks you to audit before updating....

onyx spindle
#

Customs inspector

vernal hornet
# verbal plover Oops.....

This is a confusing thing where we want people to use C9 and C4 instead of C, because C9 and C4 map to different linters within Ruff ("comprehensions" and "complexity"), and each parent category is meant to map to one linter.

We special-case C in the code such that if you provide it, we expand it to C4 and C9 internally. It's probably worth marking them as acceptable in the JSON schema since this is clearly confusing.

silk jungle
#

@upper hill

1❯ ssh -T git@github.com
Hi ichard26! You've successfully authenticated, but GitHub does not provide shell access.

I'm using SSH now!

#

I'm waiting for my SSH public key to be deployed on the GCC Compile Farm machines so I can ssh into them. In meanwhile, I thought I'd try GitHub.

desert verge
silk jungle
#

rsa

desert verge
#

:(

silk jungle
# 
❯ ssh ichard26@gcc13.fsffrance.org
ichard26@gcc13.fsffrance.org: Permission denied (publickey).

seems like this may take a few business days (unless I uploaded the wrong key)

desert verge
#

rsa has been considered insecure for a while iirc

silk jungle
#

just for SSH keys or even GPG keys?

vast wren
#

RSA isn’t insecure

#

Ecdsa / ed2559 is faster and let’s you get an equivalent amount of security with a smaller key

upper hill
#

But yeah, use ed2559 these days in future 🙂

#

As Donald says

silk jungle
# upper hill Elite now 😮 
❯ ssh ichard26@gcc104.fsffrance.org
Last login: Sun Apr  9 13:57:19 2023 from <ip>
  __  __                        _       _   __  __ _ 
 |  \/  | __ _  ___   _ __ ___ (_)_ __ (_) |  \/  / |
 | |\/| |/ _` |/ __| | '_ ` _ \| | '_ \| | | |\/| | |
 | |  | | (_| | (__  | | | | | | | | | | | | |  | | |
 |_|  |_|\__,_|\___| |_| |_| |_|_|_| |_|_| |_|  |_|_|
                                ___  ____ 
    _     _ __ ___   __ _  ___ / _ \/ ___|  Welcome to Darwin on
  _| |_  | '_ ` _ \ / _` |/ __| | | \___ \  Apple Silicon (16GB)
 |_   _| | | | | | | (_| | (__| |_| |___) |
   |_|   |_| |_| |_|\__,_|\___|\___/|____/      2020 M1 Mac Mini

 Hardware donated, hosted,                   Report issues here:
 and managed by Adélie Linux.                      zv.io/contact

(1) respect resources; (2) usage monitored; (3) backup your data

***** For a temporary Homebrew environment, run 'homebrew' *****

gcc104:~ ichard26$

got access to the (most of the) farm now

upper hill
#

Cool

#

lol @ <ip>

#

It better be IPv6 😄

silk jungle
#

I don't think my network supports IPv6 or my laptop...

#

actually it does

silk jungle
#

Good to know for later!

upper hill
#

Yeah, wonder when they plan to change the default

lean bear
#

with pure-python approximations of codesign, otool, install_name_tool, patchelf

junior narwhal
lean bear
#

basically rewrite wheel extensions and include shared library dependencies, but cross-platform

#

really only makes sense if you're cross-compiling wheels

verbal plover
#

I debugged Chinese translation build for the whole night until I tracked that down...

dreamy hatch
junior narwhal
shell oracle
#

And a colossal thank you to Ruff, for setting off all our malware scanning services by leaving flake8-bandit in the package 😛

silk jungle
#

Importlib.metadata is confusing to use. The stdlib docs are surprisingly sparse. The external API reference on RTD is nice, but it doesn't list EntryPoint (or EntryPoints for that matter).

#

Not to detract from all of the hard work I'm sure goes into importlib.* (anything away from pkg_resources is great) but its docs could be easier to grok.

fierce horizon
left mauve
#

Ok

fierce horizon
#

I have a few more actual arguments supporting this if anyone cares, but right now I just found it funny that Ruff sets it off. It’s a linter. How in the world could it possibly trigger any heuristics?

shell oracle
#

I can’t see this conversation occurring in a way that brings us to a healthy conclusion. Flake8-bandit uses strings commonly used by threat actors in systems enumeration and data exfiltration. The presence of flake8-bandit, subsequently, was correctly identified by our solution based on the information present within.

fierce horizon
#

How is that “correct”? Is Ruff malware? No, so it was a false positive. I can believe that it’s hard to actually do something smart here, and that erring on the side of caution is a nice thing, but it shows one more fundamental flaw with the approach.

What if Ruff now gets allowlisted (which it should!), but then something gets compromised and a new Ruff version contains actual malware? It’s now on an allowlist and therefore has a license to contain those “evil” signatures! If the heuristic would actually do its job, it wouldn’t have detected it in the first place.

shell oracle
#

It’s not on an allowlist.

#

We do not whitelist packages for any reason.

#

We’ve correctly identified and reported over 200 malicious packages, and our data ingest is something like ~10,000 packages per week. It’s incredibly difficult to mitigate false positives, but it is an iterative process.

fierce horizon
#

Not saying identifying malicious packages isn’t a great service, but false positives can do actual harm to users.

shell oracle
#

Erm, I’m not sure if you understand my background.

I run an assertive package scanning service; I do not maintain an antivirus service.

We scan new and updated packages for potentially malicious indicators, and manually evaluate the contents contained before reporting them.

#

If my post mislead you to believing that the package was legitimately malicious, I apologize.

fierce horizon
#

No, but I did misinterpret your process. If false positives don’t actually reach users, I have no complaints 😃

#

My problem is with fully automated processes (like antivirus, that’s why I mentioned AV in my initial comment). Because that approach has a lot of problems that can in many ways reduce security instead of improving it.

shell oracle
#

I’ll take the feedback into account moving forward. Our underlying tools utilize many existing processes that have effective and proven track records.

Our entire process is currently closed source at the moment, and aside from manually discussing things like the inclusion of certain packages flagging rules unintentionally, there is no external transparency.

Unfortunately, with a team of 10 individuals, and out of pocketing all costs in pursuit of open source security, we can only move at the speed with which we’re able to develop and iterate on existent processes.

fierce horizon
#

Well, my main gripes are with AV run on user PCs, which has a bunch of issues that scanning doesn’t have. (E.g. Increased attack surface through elevated permissions)

If scan results are not automatically published without review to wherever library users will likely see them, I believe it’s a good deed to do what you do. But I do think that if automated publishing of results would end up scaring users off libraries like ruff would be a counter productive outcome. Linters like ruff help with security!

timber sphinx
#

Need a new package tool to manage upgrades, call it "molt"

arctic copper
#

I knew learning a language that incidentally shares a name with a snake species, as a ophidiophobe, was a mistake!

shadow zealot
#

That’s code name of my Rust-based workflow tool (eventually decided I prefer to write specs more and leave the actual implementation to more capable people)
https://github.com/uranusjr/molt

junior narwhal
#

Why aren't you honored that your product was good enough for Google to absorb and build off of? I'd be super proud.
I feel like I'm in a fever dream reading these comments from engineers lol

#

is my perception off and this practice is actually pretty much fine?

vast wren
#

all of those licenses require attribution

#

at least in the sense of maintaining copyright statements

junior narwhal
#

oh interesting, I didn't know that. they definitely didn't do that then

vast wren
#

the code in question is MIT/Apache v2 Licensed AIUI, MIT license states:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

Apache v2 States:

You must give any other recipients of the Work or Derivative Works a copy of this License; and
You must cause any modified files to carry prominent notices stating that You changed the files; and
You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and
If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License.

You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License.
#

since it appears to be dual licensed MIT/Apache v2, if we assume google used the less onerous of those licenses to use the code under, the code would need to maintain the Copyright (c) 2020 Ofek Lev

#

as well as the rest of the MIT license stuff

#

They've also licensed their project under Apache v2, so if they're using the original work under the MIT license, they'll have to include that license as well, as the overall work will be licensed under the restrictions of both MIT and Apache at the same time (this is different than your dual license, which allows downstream to choose either license)

#

unless they choose to incorporate your code using the Apache v2 license, which would simplify the licensing for their project, but still requires them to retain the copyright statements

junior narwhal
#

it's so odd to me to neglect attribution

#

like, clicking fork is in fact easier then copying the code to a new repo

#

beyond that, I couldn't even imagine not attributing work to others even after a significant rewrite (which this is not)

#

it would just feel terrible and wrong

#

as an example I maintain bindings to a cryptographic library and I rewrote everything except part of the build system and I acknowledged the original project's contribution to the build system

vast wren
#

yea I mean, the license requires it, and personally even if it didn't I would try to show some kind of attribution

kind moon
#

I agree.
I actually started a project last week and I was concerned I didn't add enough attribution.
Just "doesn't feel right" to not acknowledge other's efforts.

Speaking of, did I add enough attribution? The project is actually from here: https://github.com/letsbuilda/pypi-inspector.letsbuilda.dev
I cloned https://inspector.pypi.io/ because I wanted a starting point for testing some parsing of package contents.
But all I added was a single line to the README linking to the original license.
Is that enough?

junior narwhal
#

that seems sufficient, and you are already one line ahead of Google 😜

stoic mural
desert verge
desert verge
fierce horizon
#

Others made a case about “if you don’t want your code taken and incorporated into a commercial product, use AGPL”. That’s a valid point if you feel like they shouldn’t just have taken the code (ignoring the attribution)

vast wren
#

I don't disagree that there is an inherent difference between "I legally grant permission to do x" and "I think it's better manners to do X"

fierce horizon
#

I wanted to categorize the responses from HN first. Category 3/3 concerns attribution.

#

And as a scientist, I fully agree: It is a dick move

#

But I expect absolutely nothing from corporations.

#

They will do all the dick moves that could potentially save them money.

#

If you want to protect your rights, use a license that protects it. Scares them off sometimes. If you really want to protect your rights, do that and also be prepared to go to court.

shell oracle
#

I'm not sure anyone's out there to... objectively be sinister. Sometimes the simplest solution is someone was confused at some point. 🤷‍♂️

vast wren
#

yea I doubt it was anything sinister

fierce horizon
#

I was speaking generally! I think not knowing how to navigate licenses as a newish dev is common and excusable.

But my point stands: If a company can get away with something, and leadership thinks it might make money, assume they will do it.

In this case, of course, the fork is also open source, so there’s no incentive (therefore I also assume innocence). But if something is internal code that leadership thinks might give the company an edge? You better bet they steal code and slap it in there without asking.

vast wren
#

most of the companies I've worked at, particularly at Google's size, takes license stuff pretty seriously, and generally ban the use of code that isn't permissively licensed, even if they could possible use it legally without a concern.

#

of course random devs within those companies will do whatever

fierce horizon
#

Yeah. They think they can’t get away with it. Others do.

shadow zealot
#

Eh no you’re not supposed to “fill in” that part. The appendix is for teaching people reading the document how to use the same license for their work. It should be included verbatim as the rest of the document; they are doing it the right way.

boreal bramble
#

appendix can even be omitted

#

though it does seem useful

fierce horizon
#

I see, seems like I should not infer things from licenses that I've never fully read myself lol

vapid kite
#

Is anyone interested in joining up / forming a small (GH) org for maintaining some direct bindings to useful rust packages (via maturin)? In the past few months I've written initial bindings for 3 packages I needed (im-rc, rpds, and regress), all fairly mechanically, but seems like might be nice to collaborate.

kind moon
#

I don't have very much experience with Rust, so I wouldn't be able to contribute much, but I do have experience with managing GitHub orgs and workflows and publishing, so I'd be happy to help with the admin side.

vapid kite
#

I don't have much experience myself 😄 -- I did find it quite easy to get into! But yeah that kind of help is super super valuable so certainly interested in help there.

junior narwhal
#

I'm pretty good with Rust but I've never done Python bindings with it before, I wouldn't mind learning

vapid kite
#

OK I created a crate-py organization on GitHub (the best name I could think of in 13 seconds) @junior narwhal adding you to it -- @kind moon if you send me your GH username I can add you as well. I'll transfer the 3 bindings I made to the org.

#

If anyone else is interested feel free to ping me.

kind moon
analog oyster
vapid kite
#

Yup. I haven't added any org readme explaining it or anything, just created it and done the transfer

#

I couldn't think of a decent name that gave homage to jazzband (unfortunately brassband is taken on GH)

analog oyster
#

is it kind of like a nursery for python interop?

vapid kite
#

Yeah think that's the idea (or to complete the jazzband reference -- a jazzband specifically for rust->python bindings)

#

of course with 1 millionth of the effort and thought Jannis has put into this :P, I just am selfishly hoping to distribute the load on maintaining these 3 packages

analog oyster
#

Yea that sounds really cool. I'd offer to help, but my plate is pretty full atm. I'll keep my eye on it. Sounds awesome.

vapid kite
#

Very fair!

shell oracle
#

Entertaining exchange with a PyPI malware author. Asked him to knock it off and... shockingly, he did.

image.png
#

Which, in hindsight, further reinforces my intuition that a non-small part of malware authors on PyPI are probably just... kids interested in Cybersecurity that don't really understand the implications of what they're doing.

frank path
kind moon
#

We could be a windband?

junior narwhal
#

Windows will be the death of me. WSL randomly stopped working which broke Docker and my SSH, required a reinstall of everything and a Windows update

still better than macOS though

shell oracle
#

Frankly that's been a common issue lately, I wonder if... something goofy is going on. There are more than a few people in PyDis that have been experiencing issues specifically with Docker and WSL.

analog oyster
#

idk ive enjoyed macos much more than windows so far

junior narwhal
#

is anyone online right now and can test something real quick on macOS?

hollow lion
#

sure

junior narwhal
#

can you try the installer?

#

.dmg

#

just wondering if it works

#

I think I did everything correctly

hollow lion
junior narwhal
hollow lion
# 
ddqa not found
➜  ~ ```
#

I'm not sure it's installing anything

junior narwhal
#

interesting, I'm not sure why that would be

#

I'm a Windows user so I'm unfamiliar but I thought it should move that directory to your Applications folder

hollow lion
# 
➜  MacOS pwd
/Applications/Datadog QA.app/Contents/MacOS
➜  MacOS ./ddqa 
ERROR: Could not find a version that satisfies the requirement ddqa==0.0.1.dev1 (from versions: 0.0.1, 0.1.0)
ERROR: No matching distribution found for ddqa==0.0.1.dev1```
junior narwhal
#

that's expected!!!!!!!! nice it worked

hollow lion
#

I had to copy it manually to the applications folder

junior narwhal
#

now I have to figure out why the installer isn't working

#

thank you for testing 🙂

hollow lion
#

yeah np

#

With mac installers, that picture I pasted above, there's the app on the left and the applications folder on the right, you just drag and drop in that window to install.

junior narwhal
#

oh okay, and that is not working with this right?

hollow lion
#

or double click and brings up a window that has an installer kinda of like windows, where you pick the location and click through to the end

hollow lion
junior narwhal
#

if so then I'll probably just adopt his script instead

hollow lion
#

A little backwards but this is what you normally see

Screenshot_2023-05-13_at_9.26.28_PM.png
#

It copies, but doesn't add to path

junior narwhal
#

I'm wondering if spaces are disallowed in the name of the app

junior narwhal
#

oh wait I think I messed up the invocation, I think rather than targeting the app folder you have to target the parent directory of that

shell oracle
#

Never mind, figured out why he removed it. lol.

junior narwhal
shell oracle
#

There's a tidbit in the readme about 'OSX 10.5 or later'. But I believe it may be... some interesting wording by the author indicating that they don't support EARLIER versions of Mac OSX

#

Requirements

Nothing except a standard installation of macOS/OS X is required.

We think this works in OS X 10.6 Snow Leopard and later.

We'd like to keep it working in as many versions as possible, but unfortunately, we just don't have test boxes running old versions of OS X adequate to make this happen. Development and testing mostly happens in the last 3-5 years' worth of macOS releases; as of 2020, this means macOS 10.12 and later.

But if you find a bug in an older version, go ahead and report it! We'll try to work with you to get it fixed.

If you're running OS X 10.5 or later, you're SOL. That's just too hard to deal with in 2020. ;)

#

Mistook that to mean it was no longer supporting additional releases of Mac OSX, apologies for the confusion. Was following this issue passively and doing my own research.

junior narwhal
#

Oh I see, yes that is poor wording lol

#

after I fix this invocation everything is pretty much set up for Hatch to copy this config and start distributing for every platform

#

I'm so happy

shell oracle
#

I've been following the project closely, cheering for ya'!

junior narwhal
#

(deleted some messages, just a user error on the part of a few co-workers)

junior narwhal
#

I was talking with a Windows expert at work just now about build stuff and he casually said something and I'm like what and yes ... apparently on Windows there is not just "the heap" every module has their own and cannot touch others'

I feel like my brain expanded by 10x today

shell oracle
#

Referring to segment heap?

junior narwhal
#

the conversation was in reference to memory from one library being freed in another

junior narwhal
muted whale
#

Hey @robust sandal just curious, in regards to https://discuss.python.org/t/change-in-pypi-upload-behavior-intentional-accidental-pebkac/27707/ it looks like scikit-build-core normalizes wheel filenames following the wheel specs, but does not normalize sdist filenames at all per PEP 625 (same normalization as the wheel spec). Is there a plan to implement that, like all other backends do (except for Setuptools, but it appears they plan to soon)? I couldn't find an issue for that.

robust sandal
#

I can change it, I was matching setuptools originally, since scikit-build was based on setuptools. I could add a config option if there are any issues (I've run into issues before with the sdist names and setuptools vs. other backends, since some things care about the SDist name, hopefully those issues are gone now though)

muted whale
#

Anyone going to SciPy? I was organizing a scientific Python packaging BoF and was wondering if anyone else was also planning on doing so, or at least was able to make it?

silk jungle
shell oracle
#

Congratulations Seth! Also very excited to see what happens with PyPI in the coming year! Wishing you all the best!

cobalt valley
#

Congrats!!!!
I'm sure that you'll do an amazing job and I'm looking foward to hear out more about you! ❤️

robust sandal
#

I'll probably be there but exactly when is still a bit in the air. I'm teaching (a packaging tutorial) at INTERSECT here at Princeton on Monday of SciPy And I've got confernces (at Princeton) the week after, too. But I'll be at SciPy at least for sprints. Maybe all three confernces days, maybe less.

valid rover
#

Thanks @silk jungle @shell oracle and @cobalt valley 🙏

junior narwhal
#

actually maybe Linux should be changed:

❯ docker run --rm python linux32 python -c "import platform;print(platform.machine().lower())"
i686
#

I wonder if that is correct still if running normally on 32-bit rather than emulation with linux32

silk jungle
junior narwhal
#

oh cool thanks!

#

I am unable to articulate why I think this but I'm almost positive on Windows it will return i386 so I am keeping that until I am presented with proof from someone

shell oracle
#

Well I'm trying to give you a hand here but uh

image.png
#

Me and Windows are not agreeing with each other.

junior narwhal
robust sandal
silk jungle
#

it seems like a lot of work for little gain, but I don't use Windows anymore so I don't really have a stake here :)

buoyant flame
silk jungle
# 
~/programming/oss/mypy$ lexa temp | wc -l 
73

@muted whale IIRC you were the one who suggested a temp folder that's gitignored globally. A long while later, it has accumulated a lot of experiments :P

onyx spindle
#

that's an interesting approach

silk jungle
#

there's a lot of stuff here :)

image.png
vast wren
#

what I do is just create random .py files in whatever directory I happen to be in at the time

#

this is not at all confusing

kind moon
#

Nice
I eventually end up with a dedicated repo for assorted crap in all my orgs

muted whale
hexed briar
boreal bramble
#

nice idea

#

I use .git/info/exclude if it's something more permanent, if it's not then I'm just going to delete it at some point or move it outside if it annoys me to have an untracked file. I do have global gitignore as well but nothing clever like that there, may try it

junior narwhal
#

I'm so happy I could cry

junior narwhal
onyx spindle
#

a default one I mean

junior narwhal
#

you mean like a feature built-in to GitHub?

onyx spindle
#

yeah, like what gitlab has

timber sphinx
#

Slack busted for anyone else? Their status page is a little empty

junior narwhal
#

yes they have an ongoing issue

hexed briar
junior narwhal
#

as far as contact, it's a great platform for messaging actually! it's how I've messaged Hynek, Michael (Talk Python), Charlie (Ruff), Samuel (Pydantic), Sebastián (Fast API), etc.

dreamy hatch
#

Come to Mastodon! Most of those people are there, and lots of other Python people

junior narwhal
#

I did consider that but I prefer just using one app for everything and if I want to keep up on news/what's next in the zeitgeist you basically have to be on Twitter still

ionic tulip
#

I stopped using Twitter

vapid kite
#

Same, it's probably about time I not only stop using it but also plain delete the account, I no longer have it installed and don't go to the site so if anyone tries to reach me there they'll probably get no indication I'm not ignoring them.

junior narwhal
#

Twitter is still fine if you don't really care about the meta conversation surrounding leadership

#

(I don't)

silk jungle
#

twitter being sold was my excuse to get off one more social media I suppose :)

junior narwhal
#

also I like the fact that they are not stagnant anymore, even if every feature or improvement may not be noteworthy

#

I don't want to enumerate everything that has changed that I like lol but I will say one of the best things is their all-in approach to birdwatch/community notes

#

I've seen so much misinformation called out or context added to stuff, I've even done it myself and voted on community notes since I was a member early

#

from random individuals clipping a video to mislead to government officials stating actually wrong stuff, it's amazing

shadow zealot
#

Twitter is still fine
ICYMI there is no Twitter anymore so I guess that’s not exactly fine? 🤪

dreamy hatch
#

It's fine other than adding 5s delays to links to certain sites like the NYT and competitors, removing the block feature, unbanning and paying extremists, sacking 80% of the company, a rise in hate speech, suing anti-hate speech researchers

crude radish
#

But other than that, Mrs. Lincoln, how was the play?

shadow zealot
#

removing the block feature
Seems to work for me?

dreamy hatch
fierce horizon
# dreamy hatch It's fine other than adding 5s delays to links to certain sites like the NYT and...

Yeah that. It’s very much not fine when the owner promotes an interview between Andrew Tate (proud misogynist and rapist) x Tucker Carlson (white supremacist and not-so-crypto fascist).
… and then goes on to reshape Twitter to further enable and accommodate people like this.

It doesn’t matter which individual changes are maybe not 100% as bad as they are painted. Musk makes it clear who the platform is for now, harassment statistics consequently go through the roof, and as a result it’s not a platform I feel even remotely comfortable on anymore.

I’m actively building and participating in communities that make sure that they side with the victims, not the bullies. X is not a place like that.

junior narwhal
#

as far as the hate speech data, I think it's prudent to see more comprehensive studies over time because like it or not much of the current talk/studies about that are politically motivated. for example, (people outside the US might not know this) there was talk for years about a right wing pipeline that the YouTube algorithm promotes and that is actually totally false and in fact the opposite of detectable political bias https://academic.oup.com/pnasnexus/article/2/8/pgad264/7242446 (that is the newest one but most research indicates the same thing)

#

and then, which is really unfortunate, you have just poorly done studies which are referenced by large media institutions like this one that is most cited about hate speech data in relation to the acquisition https://ojs.aaai.org/index.php/ICWSM/article/view/22222/22001

I just re-read it and even the first two figures are startling. the first one shows a spike before any policy changes happened nor layoffs and the second shows an already upward trend and an example of the previous year with the same levels

#

(to be clear the authors were ethical and have a very good Limitations and Future Directions section)

west basin
#

well, it's not just internal changes that might've contributed to an increase in hate speech. For instance, hateful users might've been emboldened by Musk's acquisition. Either way, this isn't a causal study, so unless the statistical analysis is flawed, I don't see why it might've been "poorly done"

junior narwhal
#

I view it as poorly done for not taking into account confounding variables like the hate speech words chosen, other current events that are happening, etc. for example the spike in the previous year they attributed to political protests in Canada but then didn't talk about events that were happening in the spike they wanted to shine a light on

#

I definitely agree that showing a causal relationship isn't a requirement for studies!

hexed kindle
#

build backends used by the top 2500 pypi packages:

setuptools.build_meta: 452
hatchling.build: 110
poetry.core.masonry.api: 101
flit_core.buildapi: 79
maturin: 12
poetry.masonry.api: 11
pdm.backend: 6
mesonpy: 5
setuptools.build_meta:__legacy__: 4
jupyter_packaging.build_api: 3
backend: 3
poetry_dynamic_versioning.backend: 2
pdm.pep517.api: 2
flit_scm:buildapi: 1
pbr.build: 1
scikit_build_core.build: 1
hatchling.ouroboros: 1
sipbuild.api: 1
setup: 1
flit.buildapi: 1
sphinx_theme_builder: 1

source: https://old.reddit.com/r/Python/comments/166a9br/pyprojecttoml_buildbackend_statistics/jyn02k8/

Reddit
Popular-Database-229's comment on "pyproject.toml build-backend sta...

Explore this conversation and more from the Python community

onyx spindle
#

Nice. Now I got to find those 11 projects using old Poetry build API and migrate them to build backend lol

hexed kindle
# onyx spindle Nice. Now I got to find those 11 projects using old Poetry build API and migrate... 
down/codeowners-0.6.0.tar.gz: codeowners-0.6.0/pyproject.toml: build-backend = "poetry.masonry.api"
down/aioboto3-11.3.0.tar.gz: aioboto3-11.3.0/pyproject.toml: build-backend = "poetry.masonry.api"
down/colorclass-2.2.2.tar.gz: colorclass-2.2.2/pyproject.toml: build-backend = "poetry.masonry.api"
down/databricks_api-0.9.0.tar.gz: databricks_api-0.9.0/pyproject.toml: build-backend = "poetry.masonry.api"
down/fastapi-utils-0.2.1.tar.gz: fastapi-utils-0.2.1/pyproject.toml: build-backend = "poetry.masonry.api"
down/graphlib_backport-1.0.3.tar.gz: graphlib_backport-1.0.3/pyproject.toml: build-backend = "poetry.masonry.api"
down/quinn-0.10.0.tar.gz: quinn-0.10.0/pyproject.toml: build-backend = "poetry.masonry.api"
down/pytzdata-2020.1.tar.gz: pytzdata-2020.1/pyproject.toml: build-backend = "poetry.masonry.api"
down/strawberry_graphql-0.205.0.tar.gz: strawberry_graphql-0.205.0/pyproject.toml: build-backend = "poetry.masonry.api"
down/terminaltables-3.1.10.tar.gz: terminaltables-3.1.10/pyproject.toml: build-backend = "poetry.masonry.api"
down/url-normalize-1.4.3.tar.gz: url-normalize-1.4.3/pyproject.toml: build-backend = "poetry.masonry.api"
onyx spindle
dreamy hatch
#

from the top 4k (downloaded on 2023-07-04)

setuptools.build_meta: 871
poetry.core.masonry.api: 212
hatchling.build: 156
flit_core.buildapi: 100
poetry.masonry.api: 35
maturin: 14
setuptools.build_meta:__legacy__: 13
pdm.backend: 8
pdm.pep517.api: 7
mesonpy: 6
backend: 4
jupyter_packaging.build_api: 3
sipbuild.api: 3
flit.buildapi: 3
sphinx_theme_builder: 3
poetry_dynamic_versioning.backend: 2
flit_scm:buildapi: 2
whey: 1
pbr.build: 1
hatchling.ouroboros: 1
scikit_build_core.build: 1
dreamy hatch
#

and from the top 8k (downloaded just now)

setuptools.build_meta: 1263
poetry.core.masonry.api: 389
hatchling.build: 243
flit_core.buildapi: 152
poetry.masonry.api: 72
maturin: 27
setuptools.build_meta:__legacy__: 18
pdm.backend: 12
pdm.pep517.api: 10
whey: 8
poetry_dynamic_versioning.backend: 7
flit.buildapi: 7
mesonpy: 7
jupyter_packaging.build_api: 5
sphinx_theme_builder: 4
backend: 4
flit_scm:buildapi: 3
scikit_build_core.build: 3
sipbuild.api: 3
cython_backend: 1
pep517_backend.hooks: 1
ext: 1
pbr.build: 1
setup: 1
hatchling.ouroboros: 1
pdm.backend.intree: 1
junior narwhal
#

I switched away from xonsh to nushell today, it's so cool! like powershell but actually easy to use and intuitive

junior narwhal
#

everything is structured and pretty and can be manipulated by other things

image.png
kind moon
#

This is the Rust one, right?

junior narwhal
#

yeah

fierce horizon
#

Things I did with my nushell setup:

  1. activate the carapace completer (described in comments in the default config) to get a lot of tools autocompleted
  2. added a command-not-found hook for Arch Linux
  3. added a bunch of aliases like def 'pipx list' [] { ^pipx list --json | from json }

But it’s very usable without much fiddling. One just needs to learn its idioms (as with anything that’s new to you) and deal with the breaking changes (as with anything prerelease)

silk jungle
#

for RTD, is there any way to use the latest Python version automatically?

#

I don't think there is a way. Updating the Python versions every year or two isn't too that bad...

dreamy hatch
timber sphinx
junior narwhal
#

this weekend I think I'm going to make a small docker image for folks. last night I got the logic right and I have a way to turn a file with a bunch of commands into a terminal session gif. I was showing the project to someone and they messaged back that it was hard to tell the high level view of what it does without an example to show and I was like, hmm I'll have to learn how, so now I know 😃

record.gif
junior narwhal
#

should one set TERM to xterm-256color or xterm-24bit nowadays?

boreal bramble
#

ideally you would set it to something that represents the terminal you use exactly (TERM points to a terminfo file with details about your terminal which libraries such as ncurses can use to know what to do in the terminal it's running in) but that might not exist for your terminal (or if it does exist, it may not be provided with the system-provided terminfo database) or it may cause problems with apps/scripts that simply check for xterm- prefix to determine if something is not a dumb terminal

junior narwhal
#

oh okay interesting, thanks

boreal bramble
#

if selecting just from the two you mentioned, choose the one that matches the colors supported by your app, most of the scripts that check for xterm-specific TERM, should only check for xterm- prefix. I imagine there could be some that check specifically for xterm-256color but if they do, they probably just limit you to 8 colors rather than no colors at all

junior narwhal
#

I was thinking in the context of a pseudo-terminal as I'm using to produce the GIF file for terminal sessions. I wonder if I could set the latter and then the output would be truecolor

boreal bramble
#

there's actually another env var

#

COLORTERM

#

that can be set to truecolor for true color support

onyx spindle
#

when in doubt about that kind of stuff, I always look into rich code 😄

boreal bramble
#

since TERM is tied to terminfo files

#

actually, case in point, my system does not have a terminfo file named xterm-24bit and I'm running 22.04 which is not old at all

image.png
#

if a specific terminfo file is missing on the system you're using it, a lot of simple stuff will work badly

#

stuff like cursor movement/looking up history with arrows keys

#

and TERM gets propagated when you SSH into any server

#

so if the server doesn't have it, you get the weird behavior

#

based on your question, I assumed xterm-24bit is probably something widely accepted as a thing but I guess not 😄

junior narwhal
boreal bramble
#

I mostly know about those things because kitty terminal wants to do things more properly and actually chooses to define its own terminfo which has been a bit of an annoyance at times because it's generally not part of distro's base packages

#

xterm-kitty

#

arguably using xterm- prefix when kitty is not xterm is not ideal but it does mean that all the things that check for xterm in TERM work properly with it

junior narwhal
# 
awk 'BEGIN{
  s="/\\/\\/\\/\\/\\"; s=s s s s s s s s;
  for (colnum = 0; colnum<77; colnum++) {
    r = 255-(colnum*255/76);
    g = (colnum*510/76);
    b = (colnum*255/76);
    if (g>255) g = 510-g;
    printf "\033[48;2;%d;%d;%dm", r,g,b;
    printf "\033[38;2;%d;%d;%dm", 255-r,255-g,255-b;
    printf "%s\033[0m", substr(s,colnum+1,1);
  }
  printf "\n";
}'
#

looks like it is working with replay!

image.png
#

@boreal bramble thanks again for that environment variable tip!

boreal bramble
#

np

junior narwhal
#

can someone on a macOS please try something for a moment?

silk jungle
#

I'm on a SSH session to a macOS machine so if this relies on a well featured terminal, I can't help unfortunately FYI.