black-formatter

[dense jungle]

April of 2018?

[bright glacier]

yep, that clearly didn't happen 😅

[austere lava]

I clearly meant 2022

[bright glacier]

no 2030

[mint barn]

print('rick goes brrr')

[split merlin]

By the time that releases do we still use years?

[bright glacier]

I mean, I might be flooded once the stable released happens

[silent apex]

I thought I would hate the idea of not being able to configure stuff in Black (my prettier configs are a mess), but since Black follows PEP 8 well, it seems nice

[bright glacier]

it's a constant struggle of making sure we maintain the "unconfiguraility" goal since obviously black's style isn't for everyone yet it appears everyone wants to use it

[frozen badge]

🎶
I see some Python
And i want it painted black
All other code-styles suck
I want them to turn black.

[split merlin]

Just using it once ages back convinced me to use it constantly.

[late dew]

@mint barn :white_check_mark: Your eval job has completed with return code 0.

Utah bread

[bright glacier]

Łukasz or @solid adder

They probably have too many friend requests ...

No, I'm not famous 😅

[scarlet swift]

a vs code extension would be nice though 🙂 would save the work of installing it manually in each env (like the python extension auto installs flake8)

[cloud quarry]

👍 noice

[scarlet swift]

yeah same xD might just use this oppertunity to try making one 🙂

[bright glacier]

Yeah, that would make sense. There has also been some desire to improve packaging for tools to reduce duplication (I happen to be a PyPA member too).

Python Packaging Authority (not actually supposed to be an authority, it was a joke but now that's not so clear lol)

[foggy crow]

Hmm so what is black about?

[rapid yoke]

Check the pins

[foggy crow]

No pinned messages

[scarlet swift]

he means the channel discription

[foggy crow]

Oh

[bright glacier]

whelp, there's ya pins

[rapid yoke]

Wait partners can pin messages?

[bright glacier]

Well, I'm assuming everyone with the "partner: black code formatter" role can manage the #black-formatter channel 😉

[rapid yoke]

Lmao

Would be a safe assumption

[ionic nymph]

That's a correct assumption :)

[plain atlas]

sends you one

[plain atlas]

It can also auto install black too lol

& you can provide the ext a specific path to a specific black to use one outside of the venv

[plain atlas]

Is it possible to color the black role black 🥺

[frozen badge]

from a accessibility POV: probably not

[bright glacier]

It would be a readability nightmare I'd think

[delicate marsh]

It's an addition to the existing partner role, so I think it makes sense as purple, as well as the a11y concerns mentioned by laund and ichard

[plain atlas]

True. The purple is already a little hard to read.

a11y?

[delicate marsh]

accessibility

[frozen badge]

i love that a11y also can be read as ally

anyways, i came here for one specific question: is there any plans for black supporting partial formatting, like only part of the code?

[bright glacier]

Probably not since one of our goals is consistency. # fmt: off/on and our rather new addition # fmt: skip (quite buggy still unfortunately) will probably be the only ways of using black in part of a file.

[frozen badge]

hm, sad that i will have to rely on external tools to do partial formatting for projects that refuse to use black project-wide

[rocky wharf]

Are all the black maintainers here?

[bright glacier]

Well all of the active ones.

[rocky wharf]

ah ok

[glad pilot]

I have more than once tricked Black into formatting a chunk of Cython code that happens to also be valid Python code by adding a bunch of:

if 1:
    if 1:
        if 1:

above it to get Black to leave the indent alone, writing it to a different file, running Black on it, copying it back, and removing the ifs

[frozen badge]

all cython can be valid python thanks to cython support of typehints for typing (in my experience)

[plain atlas]

I agree. I don't want to format parts I'm not editing but I want to ensure my code isn't formatted with '

[frozen badge]

i have been using darker for that, it works based on git diff and runs black just for that

[plain atlas]

Yeah due to the style guide of the first project I commited to I use ' instead of "

[glad pilot]

possibly, but that's not the way that any of my Cython code is, and I'm not rewriting it. I realize most things could be done that way, though I think there are still limitations.

[plain atlas]

I actually need to do something about my vsc, it's totally buggy.

[frozen badge]

ping me in #editors-ides if you have some Qs

[solid adder]

Look at our little new channel blowing up like it's Black Friday

[rough spoke]

hahaha.

[glad pilot]

There was a PR to teach Black to format Cython a while back - why did that wind up getting dropped?

[plain atlas]

Good one. Lol

[bright glacier]

haha, totally not overwhelming or anything

[glad pilot]

too much work/too different from the normal architecture?

[haughty ocean]

First time hearing if this. And frankly, I still don't completely under Black...
But will just lurk and do some research on it

[plain atlas]

If you've ever formatted your code, that's what black is formatters 😛

[frozen badge]

's what happens when you're suddenly visible to 200k+ people

[plain atlas]

204,131 as of this message

[glad pilot]

yeah, the ping may have been excessive 😄

[haughty ocean]

I got that. But because of how it is done, how is it better?
Sure it looks like it helps with readability, but what are its long term effects

[rocky wharf]

lmao

[glad pilot]

there are no long term effects - or there aren't supposed to be.

[frozen badge]

well it was announcements only, not @ everyone

[glad pilot]

it makes your code more readable, that's all.

if you run it regularly, it keeps your code more readable.

[dense jungle]

The long term effect is that you never have to worry about formatting again

[rocky wharf]

I can write everything as a pep8 defying inline lambda and have black fix it

[haughty ocean]

Ahh I see

Well, I will give it a try in my spare time

[split merlin]

It's worth it imo

[mint barn]

What exactly is project black anyways

[frozen badge]

black is a (very good) python auto formatter

[mint barn]

Oh

[solid adder]

The channel description has a link to the project page which shows you what it does.

[neon loom]

wonders who owns black.vercel.app - Would love to get IPv6 on it

[restive sapphire]

This looks really cool so far! Looks leagues better than some other auto formatters I've used.

[graceful vault]

is there any sort of beta version of black for python 3.10 atm?

[solid adder]

There isn't yet

[graceful vault]

any way i can help contribute to that effort?

[neon loom]

I'd suggest installing latest deadsnakes release and getting our unittests to pass ...

Does 3.10 have new syntax?

(I haven't looked)

[graceful vault]

a bit yeah, has some match case thingy

[bright glacier]

"some" lol

[rocky wharf]

yep

[solid adder]

Hang on, this will take a sec to respond to

[rocky wharf]

pattern matching was added

[solid adder]

Unions for types are supported by Black already, this is using existing syntax. But the match statement is another story

[rocky wharf]

hm

match foo:
  case bar:
    do_code()
``` or something iirc

Its like how Rust does it

yep

!pep 622

[late dew]

**PEP 622 - Structural Pattern Matching**

Link

Status

Superseded

Python-Version

3.10

Created

23-Jun-2020

Type

Standards Track

[hushed flint]

lmao

[rocky wharf]

Rust pattern matching is nice :D

[frozen badge]

pep 634 actually

[rocky wharf]

welp

[hushed flint]

rust in general is nice but this isnt really the topic here

[rocky wharf]

yep

!pep 634

[late dew]

**PEP 634 - Structural Pattern Matching: Specification**

Link

Status

Accepted

Python-Version

3.10

Created

12-Sep-2020

Type

Standards Track

[rocky wharf]

oh this is the spec

[solid adder]

At the time I was working for Facebook on their internal use of Python. There were over 20 million lines of code maintained and too much time during code review was wasted fighting over formatting. Plus different projects ended up having muuuch different coding styles, including some ex-Googlers forcing use of 2-spaced indents in their favorite projects. It was a mess.

At first I tried adopting an existing code formatter, YAPF. I even contributed fixes to it and a new style that was more in line with what we wanted at Facebook at the time. However, we couldn't make it work for our 20 million lines of code. It was very configurable but also very inconsistent because of it. I can go into details but in the end it turned out the tool just can't be forced to do what we want because of its design.

So I started working on my own. "How hard can it be?" Well, it took me 6 weeks to get to the first alpha release. When I put it out on March 14th 2018 (Pi Day!), it got 500 GitHub stars in one day, Kenneth Reitz started using it right away and tweeted about it, and soon after we got pretty big adoption.

[rocky wharf]

whoah

[frozen badge]

this would be a great pin

[rocky wharf]

yes pin

but yeah, cool :D

[hushed flint]

summoneth the mods

[hushed flint]

[rocky wharf]

lmao

[hushed flint]

ok this goes to my collection

[neon loom]

I sat opposite a very sleep deprived @solid adder for those weeks 😄

[rocky wharf]

lmao

[neon loom]

remembers every second morning - Come to a white board and we'd debate formatting decisions ...

Do we do it like foo or do we do it like bar ...

[hushed flint]

what formatting style does pycharm use tho?

[solid adder]

Internally they have their own but it's made to be (sorta) black-compatible now.

[mint barn]

hi

salam

[frozen badge]

what is/was the most controversial/most discussed formatting descision?

[bright glacier]

let me guess (I don't actually know): string quotes

[hushed flint]

i mean i hope it does

[neon loom]

https://lukasz.langa.pl/1d1a43c4-9c8a-4c5f-a366-7f22ce6a49fc/

[hushed flint]

double ftw

[solid adder]

Yes, that's another candidate. Right on, @neon loom.

[split merlin]

To have people like kr announce to use it publicly the first day of public release.

[frozen badge]

thanks, that seems like a interesting read

[graceful vault]

always loved the multiline function defs, definitely more readable

[solid adder]

@graceful vault good, you can stay on the channel.

[jolly cosmos]

I like the sad face 🙂

[rocky wharf]

I like how black gives multiple functions arguments their own line

[solid adder]

all good

yes

[neon loom]

Wu Karsh ain't nothing to ....

(Refer to Wu Tang clan for full quote)

This is how my Aussie head remembers how to say his name

[solid adder]

March 14th 2018

was 18.3a0

yes

[neon loom]

ROFL that you remember that

[solid adder]

I started it 6 weeks before that, to the day

yes

[heavy summit]

hi @solid adder, is it OK if I DM you an off-topic message?

[solid adder]

sure

but I'm not into crypto, just saying

[split merlin]

If people start dming crypto Messages let us know

[heavy summit]

especially if it's a mod

or an admin

or an owner

[solid adder]

The third most discussed thing was probably the least interesting: the default line length. I just did 80 + 10% which was 88. This was in fact decided with Science (tm). We knew that 79 in PEP 8 is too little for practical coding, Raymond Hettinger said it very well in his talk "Beyond PEP 8" a few years back. So I started looking up: 85, 86, 87, 88, 89, 90, 91, 92... and as a test reformatted all code at Facebook (and the standard library of Python and a few other projects) with those line lengths. At 79 many lines are split into multiple because they don't fit the low character limit. I wanted to see how many of those lines will fit if you use 80, 81, 82, and so on. Turns out that around 91 I stopped seeing any advantage and we had a new issue: many developers couldn't fit two files side by side which is important for code review and coding. And in particular the 13" MacBook I was using at the time (a pretty standard model at FB at the time) could only fit two files side-by-side at the default GitHub font size with line numbers if the line length was 88 or 89. So: 80 + 10% it was.

[graceful vault]

science 😎

[frozen badge]

Beyond PEP 8
that is such a great talk

[prime gull]

all ray hettinger talks are tbh

I must admit i find it pretty interesting and funny that reformatting all code at facebook was part of the decision strategy, thats golden

[solid adder]

Well, if it's 20+ mil LOC then you have all sorts of code there. It's a good test for whether something works or not. At least I didn't have a better one at the time 😉

[prime gull]

yeah, definitely seems like a useful resource to have access to when making something like black

[dense jungle]

Currently the launch process also involves me formatting a couple million lines at Quora to see what happens (and a few other people doing the same at FB on even more code)

[naive iris]

How long did the Black process(es?) take to reformat all the 20+mil lines?

[solid adder]

I don't remember the exact number but you need to understand one thing: at Facebook people don't work on their laptops per-se. They use laptops, sure, but to log in to pretty powerful developer servers. Mine at the time had 256GB RAM and 64 cores, top shelf stuff in devservers at the time. They were called "bigdev" and you had to be lucky to get one. I was 🙂 This is why Black from very early versions supported parallel reformatting with asyncio. When you do black . it will split the workload across all your available cores.

So I did that, and it was actually not too bad. I could reformat everything many times over the course of a single day. This also let me discover and fix some performance bottlenecks very quickly.

[prime gull]

When you do black . it will split the workload across all your available cores.
thats really cool! and also explains the jet noises when i ran black on cpython

[hushed flint]

lol

[mint barn]

black formatter? lol now we will format a string 1000 times faster and more modern with black formatting, Great!

[frozen badge]

did you mean yeethon?

[frozen badge]

its not about string.format, its a code formatter

[prime gull]

well yes but they probably dont know yeethon lol

[solid adder]

yeethon.org LOL, I'm sure my son's gonna love this

[prime gull]

I've made some horrific technical monstrosities in the last year but yeethon is way up there at the top

[raw nebula]

i LOVE black

use it on every project

[dense jungle]

did you literally hack the grammar to make yeet an alias for del? that's... strange

[prime gull]

maybe

[mossy inlet]

yeethon? WOWOWOWOWOW

[neon loom]

Help this old man know what yeethon is?

or yeet

[dense jungle]

I'm impressed

[neon loom]

googlers

[solid adder]

to yeet is to throw something vigorously; yeethon replaces del with yeet

[split merlin]

No http redirect on yeethon sadly

[neon loom]

#impact

[solid adder]

I have some feedback for you from the release manager of Python 3.11

[dense jungle]

Also you need to make del not be a keyword any more. The Zen of Python says there should be only one way to do it

[heavy summit]

You can use cyrillic letters, aka

!charinfo уееt

[late dew]

Character Info

\u0443 : CYRILLIC SMALL LETTER U - у
\u0435 : CYRILLIC SMALL LETTER IE - е
\u0435 : CYRILLIC SMALL LETTER IE - е
\u0074 : LATIN SMALL LETTER T - t

Full Raw Text

\u0443\u0435\u0435\u0074

[heavy summit]

This will still break compatibility, but it's an unlikely breakage.

[solid adder]

\u0074 is literally "t" tho

[bright glacier]

Renaming the error messages that contain delete to use yeet would be a nice touch

[heavy summit]

Is there a fork of black that works with yeethon?

[bright glacier]

what a great idea /s

[heavy summit]

that would be a good option for april fools

[solid adder]

Well, the Yeethon organization on GitHub clearly needs a black fork. @prime gull modified CPython no problem so Black should be trivial in comparison.

[heavy summit]

sadly domains with yip seem to be occupied

[prime gull]

i was definitely not searching them earlier lol

[dense jungle]

so you'd write yip yeet instead of pip uninstall

[heavy summit]

wait, it's pypi.org. yepi.biz is available

alright, that's getting a bit off-topic, we can move to #ot0-fear-of-python

[solid adder]

Alec's fork of Black could be called yeellow.

[bright glacier]

the file manager for your system should be thelastyeet, all it can do is yeet your storage clean, something someone may want to do after enduring too much yeeting

[heavy summit]

rename /dev/null to /dev/yeet as well

[prime gull]

so there's backstory behind this, at first i replaced it entirely, but then i needed to modify the whole standard library to use yeet too. No problem, i thought naiively, I have sed I can just replace them like that. Haha, nope. That broke a lot of things as you can probably imagine (and i got it a bit wrong so we ended up with from datetime import timeyeetta). Next thing i tried was taking the ast module, unparsing and reparsing to use yeet rather than del, but for some reason it was leaving a ton of files totally untouched (it was at this point i ran black on the whole thing because astunparse made it necessary and got the jet noises). So after all of that i just settled for backwards compatibility with regular python by just aliasing it

[solid adder]

timeyeetta 😆

[split merlin]

disallow http

[prime gull]

perfect

[bright glacier]

oh no no

[solid adder]

@prime gull haha awesome 😄 not joking, look into hacking yeellow to replace "del" keywords with "yeet". This should be relatively straightforward and will show you how a concrete syntax tree is useful in those cases. You won't miss any cases and won't modify ones that you shouldn't be touching.

That's actually a great exercise to learn how the tool is built.

[dense jungle]

Yes, this would be a good exercise for contributors who want to learn how Black works internally too

[prime gull]

yeah i actually learned a decent amount about cpython's parser stuff while making yeethon, so it has actually been somewhat useful

[rocky wharf]

lmao

[heavy summit]

damn, we'd need #yeellow-formatter as well

[solid adder]

I'm LOLing IRL

[prime gull]

ah this is actually a good point, hmmm

[plain atlas]

Lmao

[heavy summit]

I made my own mini-fork as well a few months ago https://github.com/decorator-factory/cpython, it allowed extra things in names, like `spaces when incide backticks` or allowing_question_marks?

[mossy inlet]

i have a question: is yeethon simply python but del replaced with yeet, or are there more changes?

[plain atlas]

@heavy summit your account is called decorator factory? Nice.

[heavy summit]

also __yeet__ I suppose

[prime gull]

pretty much yep, I took the Python grammar file and added yeet as an alias to del

but more fundamental changes are planned

[mossy inlet]

LESSSGOOOO

[bright glacier]

the garbage collector needs to be renamed

Collector is too boring of a name

[mossy inlet]

`spaces when inside backticks` is very cool

[plain atlas]

Garbage code disposal

Runs sudo rm -rf / when the memory is almost full

[bright glacier]

or when you make a typo and use del by accident

[plain atlas]

So what does Del do anyhow?

[bright glacier]

idk, it would make a good new keyword

[split merlin]

Irrelevant object destroyer

[heavy summit]

not sure why del even exists tbh, outside of del collection[key]

[prime gull]

!e

mydict = {"a": "b"}
del mydict["a"]
print(mydict)

[late dew]

@prime gull :white_check_mark: Your eval job has completed with return code 0.

{}

[plain atlas]

Force clear object

Oh that's cool

!e

hm = "perhaps"
del hm = "this"
# is possible?
print(hm)

[late dew]

@plain atlas :x: Your eval job has completed with return code 1.

001 |   File "<string>", line 2
002 |     del hm = "this"
003 |            ^
004 | SyntaxError: invalid syntax

[plain atlas]

Oh good

[heavy summit]

!e
On its own it just makes a variable undefined. It does not "delete" the object in any way

a = [1, 2, 3]
b = a
del a
print(b)
print(a)

[prime gull]

you can del slices of lists too which is awesome

[late dew]

@heavy summit :x: Your eval job has completed with return code 1.

001 | [1, 2, 3]
002 | Traceback (most recent call last):
003 |   File "<string>", line 5, in <module>
004 | NameError: name 'a' is not defined

[plain atlas]

Yep don't mind us this is very clearly #black-formatter

[heavy summit]

uh yes let's move to #ot0-fear-of-python

[prime gull]

sorry for hijacking your channel, maintainers

[plain atlas]

No #ot1-this-regex-is-impossible

[bright glacier]

Honesty it's entertaining as heck so I don't mind

[heavy summit]

What's the proper way to make black autoformat files on save? I can't seem to make it work with VSCode. I tried inotify, but it's pretty slow.

Is blackd what I'm looking for?

[bright glacier]

no, blackd is a server version of black, it's to avoid paying the cost of starting up a process and importing

There must be some vscode option but I don't remember

[solid adder]

@heavy summit VScode can be configured to do it and then it does it semi-asynchronously.

[dense jungle]

yes, there's a config option for it in vscode. works fine for me

I think you have to select one option to make it use black (as opposed to autopep8) and another to make it format on save

[solid adder]

It would be actually cool if VScode used blackd since that would save us from paying the Python process startup cost many many times over.

[bright glacier]

happy to see my accepted label actually being accepted by the core team lol

[plain atlas]

Yay lol

[marsh geyser]

this is cool 🎉 welcome #black-formatter to the server

[plain atlas]

lmao yep

wait for the third channel to open later today

😏

Idk of any third channel its just the theme of the day

[dense jungle]

We make releases from time to time. We haven't been very good at it for a while, but recently we've been more consistent in making releases

[solid adder]

Those are our releases this year:

before that it was... August 2020, and before that... October 2019. Yeah, we've been quiet for a while O_O

[marsh geyser]

Lots of neat Black stuff this week. c00ps (black contributor) is python dev of the week on python vs mouse. https://www.blog.pythonlibrary.org/2021/05/24/pydev-of-the-week-cooper-lees/

[bright glacier]

you can thank @neon loom for being (in a good way) pushy about it 🙂

now we have a BRMFN duty, it stands for Black. Release. Manager. For. Now.

[neon loom]

Who dat guy?

[short forge]

Is https://black.readthedocs.io/en/stable/ up to date?
I can see a couple spelling mistakes but cant find them in the repo

Also, for small corrections like spelling, is there some quicker way to submit changes or do we have to go through the PR process

[solid adder]

PRs for everything please.

[neon loom]

Is 2501 an Australian post code or your AS number? If the former, I grew up in 2518 😄

[marsh geyser]

Homage to a ghost in the shell AI

The ASN is Tokyo University

[neon loom]

Yeah, I'd already looked that up.

[bright glacier]

It should be, we just reorganized the documentation to make more sense, but me and @neon loom tend to make grammar / spelling mistakes so yeah

fixes would be welcomed, just use the PR process

[dense jungle]

Also in case you're not aware, making a single-file PR can be really easy: use the pencil button at the top right of the page in github (e.g. https://github.com/psf/black/blob/main/docs/integrations/source_version_control.md) to edit and then submit a PR

[bright glacier]

Although stable does point to the newest published version of the repo. Latest contains whatever is in the main branch

[plain atlas]

Black fixes all of the other mistakes :P

[heavy summit]

Why isn't it possible to make a multi-file commit on github? That always bugged me.

[bright glacier]

Not a clue, I just make all of my commits locally (even typo fixes) because the web workflow is either impossible or incredibly annoying

[delicate marsh]

If you are tied to web then codespaces is worth looking into, but otherwise yeah local commits are much simpler

[dense jungle]

You can make multiple commits through the web UI by going to different files and then sending a PR for that branch. Requires a lot of clicking around though

[plain atlas]

I always thought git was heavily nerfed for what people said until I learned about git.

(I thought github was git until I learned of actual Git)

[spice rampart]

Does this mean the server is partnered with black?👀

[marsh geyser]

Seems so based on tags

[rocky wharf]

Does this channel have .topic topics

.topic

Guess it doesn't

[jovial sonnet]

It's not exactly a discussion channel, more like a dev one a la #dev-contrib

[rocky wharf]

Hm, yeah

[mint barn]

why does black prefer double quotes

[silent apex]

The official docs have some info on that: https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html#strings

[mint barn]

single quotes feel much easier

[chrome vortex]

Well the doc tells you to do what's easier and trust black to fix it 🙂

[heavy summit]

IMO double quotes are better, because in English ' appears in sentences quite often. So you'll end up using "..." anyway from time to time.

[mint barn]

what about nonenglish languages

[heavy summit]

well, if you're making some kind of open-source tool or a public API, it's going to have lots of English strings.

If it's something internal, then yeah, it might not have 's. That is, if your logging is not in English.

[scarlet swift]

also if you also work in any lower level language there's a chance ' are used for char and " for string o- o so its also just more convenient to have " in python too

[torpid stump]

The single apostrophe ' is used in a wide variety of ways in non-English languages, particularly when rendered into a reduced character set, see e.g,. https://en.wikipedia.org/wiki/Apostrophe#Non-English_use

[sour aurora]

" " is sometimes used to emphasize a word in a string, therefore ' are better, what do you use if you have to use both of them in a sentence though? there's no triple quotes? or is there?

[scarlet swift]

you can do """ (multi line string) actually and then use both ' and " in the string

[sour aurora]

ah okay.

[scarlet swift]

or you could just escape the " in the rare places where you have to use them by doing \"

[shell cloak]

Shouod I switch from autopep8 to black?

[gusty grail]

you should definitely give it a try :)

[mint barn]

in vscode can I disable black from adding spaces around the ** operator? so x**y instead of x ** y

I don't mind spaces around other ops

[heavy summit]

I think " is much more rare than '. ' occurs in abbreviations (like it's and don't) and when specifying ownership (Bob's request). But maybe you can take some project with lots of strings and get some statistics across string literals.

[naive dragon]

What is the recommendation for use black with teams and Visual Studio Code as editor?

[carmine parcel]

lol

[heavy summit]

[torpid lava]

Morning o/

[solid adder]

We have an emoji now, awesome

[raw nebula]

silly discrete, you arent allowed an opinion. black has opinions for you 😛

[bright glacier]

although there's an open PR for no space for doublestar (seemed to have stalled tho sadly): https://github.com/psf/black/pull/2095

[obsidian gust]

oh new channel.

[bright glacier]

welcome to the true home of dark mode 😉

we got spares of black paint brushes, don't worry about it, you can borrow a few

[tired shard]

Recently I have been going through the contributing guide and the docs for black, are there any labels on issues which tell that it is beginner friendly, good first issue or it is accepted by the core devs and I can get myself assigned to it?

[bright glacier]

sigh, this is the one area we have been lacking significantly, our contributing docs and other aides for contributors aren't the best. I've been personally working on it but my time is limited.

I don't have time to search for issues (we don't use the good first issue label well as of right now) but perhaps the other maintainers might know some good issues to work on

[tired shard]

Hahaha they are not that bad .

Alrighty!

[bright glacier]

for a project as big as black, I think we could do better 😉

[little parrot]

I think an 3.10 issue with passing the loop param it's technically not supported but works fine for most code if you remove it

[tired shard]

https://github.com/psf/black/issues?q=is%3Aopen+is%3Aissue+label%3A"good+first+issue" there are two issues here 🙃

[obsidian gust]

I used black for formatting all of my python files lol. I did black "D:\"

[bright glacier]

IIRC the PyPy turned out to be harder than expected and the other one has already been taken

[obsidian gust]

Anyway I can make black run automatically whenever someone makes pr? to auto lint the pr'ed code.

[little parrot]

Yes

[bright glacier]

for your project? Try pre-commit

[little parrot]

There's an example for a workflow as well

[bright glacier]

And then run pre-commit in CI

[obsidian gust]

hmmm

[bright glacier]

there's also (currently being rewritten) GH action for black

[little parrot]

That's what I use

Works nicely

[tired shard]

Yeah most my projects use black in pre commit hools

[solid adder]

Have you seen #846514617261621292 and #828316147657080835? Somebody threw a gountlet there for us it seems 😉

[errant barn]

Wow this channel exploded :D Hi everyone!

[bright glacier]

hi @errant barn nice to see you here 😃

[errant barn]

Likewise 😉

[muted haven]

Just to chime in, i've recently embedded typed ast support to pypy3.8 (just like how it is done on 3.8+ cpython>=), so I assume that problem will be gone when it gets released!

[plain atlas]

you got an emoji

[neon loom]

blackd running in pypy would be interesting for speed

[muted haven]

Interestingly there is no lib2to3 benchmark in speed.pypy.org, unlike speed.python.org so no estimates about how much faster the lib2to3's parser will be 👀

[neon loom]

That would be a cool benchmark

lol @ myst-parser using emojis in commit messages. We need to lift our game there Black community!

damn, seems 0.14.0 still does not support sphinx >= 4.0.0

@bright glacier ^^

bandersnatch CI agrees. They didn't sneak it in.

[bright glacier]

idk how I feel about emojis in our commit msgs and changelog lol

[neon loom]

Emoji grinch

[errant barn]

How 'bout ~~monochromatic ~~ gray emojis only :D

[mint barn]

can anyone simplify the black logo? it seems outdated

[bright glacier]

it's a play on the Ford logo and our name is a reference to Ford's quote about black being the only colour

[mint barn]

i get that but these i made probably look good

these look classy and smart

[solid adder]

Thanks, we're not looking for a new logo

[mint barn]

but i still submitted ¯_(ツ)_/¯

[solid adder]

Thanks for sharing.

[mint barn]

🙂 np

[solid adder]

I made black Black on black t-shirts for PyCon US 2018

I should probably do a new batch

[bright glacier]

of course 😉

too bad our docs theme has no dark mode

[errant barn]

Actually didn't know that, I just thought it was some random clever line on the readme 😅 sweet shirts!

[bright glacier]

Sorry, I meant black mode

dark is a copycat of black mode

[prime gull]

you know you can read #announcements or #changelog before just posting a random message saying "new channel?", right?

[frozen badge]

i recommend "i see some python and i want it painted black - The Rolling Stones" on the back

[frozen badge]

the only acceptable colour theme is black (#000000), as can be seen in my vscode

[mint barn]

{"test":"no_spaces"}

[civic bane]

I just want to say thank you--black's been a bit of inspiration for me with my own projects, so seeing you here is a special kind of amazing

[bright glacier]

And thank you for popping by!

[austere lava]

maybe I can get my hands on some then! 😛

[tribal thistle]

I'm reading up on blackd for the first time. What's the use case for it? The docs say it's to avoid spinning up a black process, so is this a tool meant for CI or something?

[dense jungle]

Could be CI, could also be useful in an editor

So you don't have to start a new process every time you want to format on save

[tribal thistle]

Gotcha

Thanks

[dense jungle]

For what it's worth we never bothered to set up blackd for CI and we check millions of lines against Black. The startup cost doesn't matter that much there

[austere lava]

^ it's extremely useful for latency sensitive situations, usually involving a human at the end of a screen/keyboard

where getting back the formatted results in 50ms compared to 350ms is a HUGE difference

[neon loom]

Wonder if you can make a store that we can add the funds to black bug fixing / parser work?

[tribal thistle]

Not to change the topic, but the prettier hook made changes to a clean clone

Is that intentional?

(Added newlines at the end of authors and changelog)

[errant barn]

are you on windows?

[tribal thistle]

Yup

[errant barn]

Happened to me too the other day. Did you also run it manually instead of in precommit?

[tribal thistle]

pre-commit run -a

[neon loom]

Does fb / vscode use it anywhere @austere lava ?

[tribal thistle]

The file certainly doesn't have a new line, so it should in theory be changed

[errant barn]

Yup, but they are symlinks so I'm not sure how they work (meaning does the newline bother the link). The problem doesn't appear when precommit runs through git so it doesn't bother me.

[austere lava]

yeah we have a wrapper on top of it

[neon loom]

Of course we do 😄

You should uvloop it for further lower latency

[austere lava]

[dense jungle]

Are you running it on Cinder?

[neon loom]

Cinder might work for it, but it's not GA available across FB. but we could prob get access and try it.

Insta team don't want the maintenance pain outside their workload. I don't blame them.

[neon loom]

I'll add it as a separate import in a try/except - if in env, use it, if not, don't (so your windows stays working)

And make an extra install

We could even do it for all of black really

https://github.com/psf/black/issues/2257 - I'll test it and get a PR up one day soon. Could be interested to see if it help perf

[austere lava]

not sure it's worth your time, do you have an idea about potential speedup?

[neon loom]

Yeah - I've seen huge latency wins on long running servers like blackd internally

Instagram seen huge latency wins from uvloop too

@austere lava It's like 6 lines of code in init.py + a few more in setup.py

I recently did it for bandersnatch: https://github.com/psf/black/issues/2257

[austere lava]

alright 🙂

[neon loom]

So at FB, if you deploy a par, basically if uvloop is included, and you'll get the latency wins for free

included == added as a dep so it's in the PAR env

[neon loom]

Pasting is killing me today: https://github.com/pypa/bandersnatch/pull/891/files

Damn, I didn't know I'd be so big here. Sorry for that.

[bright glacier]

I wonder if uvloop would help with the non-linear performance degradation when formatting many many files

https://github.com/psf/black/issues/1951

[neon loom]

Good chance to try it once I add it on that workload

I'd just imagine the exponential rate to be less, but not fixed.

[bright glacier]

I don't remember whether if the loop was the issue or something about black was the issue tho,

[neon loom]

But I've been wrong a lot in my life 😄

[dense jungle]

I don't think we ever figured out the root cause of that issue. Black is pretty fast for me when formatting thousands of files

[austere lava]

on an unrelated note, how do we feel about binary dependencies/

I remember there was some churn with the regex package needing build if you don't have a binary wheel

[dense jungle]

Better to avoid them if at all possible

[austere lava]

I'm asking because maybe we'll have a PEG parser written in rust sometime later

[dense jungle]

Though that sounds like the kind of thing where we can't get around it 🙂

[austere lava]

we can always implement it in pure python 🙂

[austere lava]

but I'm curious why do you say this

[dense jungle]

To avoid making it harder for users to install Black

Like with regex I feel like we caused a lot of pain for users for a pretty marginal bug fix

[austere lava]

right but I guess what I didn't understand even back then is that if you're running a not-so-common architecture (i.e. no bdist wheels available on pypi), isn't it a basic expectation to have a compiler toolchain?

[dense jungle]

That sounds pretty reasonable 🙂 I'm not familiar with the environments those people were using, just that we got a lot of complaints

[neon loom]

Yeah, it’s created a lot of churn for the one fix. There is an issue to try get rid of the regex use all together and drop this dep.

[dense jungle]

Hopefully the new parser will actually make it so we don't need regex any more

[frozen yacht]

With third party libraries, some are slow to update CI with newer Python versions to publish the wheels, even though the Python code itself is already compatible with newer versions. It sucks to have to depends on that.

The Apple M1 thing is a recent example of a common architecture that has had some issues with lack of wheels, but this kind of thing is a one-off. It's not often that a large userbase is introduced to a new architecture.

[austere lava]

fair enough!

[austere lava]

hmm a quick run of

python -m cProfile -s cumulative -m black my_1000_files/*

shows that for 1000 input files, there are 474 calls to schedule_formatting. This confuses the hell out of me

[dense jungle]

caching?

[austere lava]

no, there should only be one call

[dense jungle]

Does every new await get counted as a new call? I see that happen with generators, not sure if it's the same with coroutines

[austere lava]

hmmm, meaning every time the task is suspended and resumed counts as a new call? That would make sense

[frozen badge]

might be worth to use a async-aware profiler like https://github.com/gaogaotiantian/viztracer

[austere lava]

that would be sensible but I like pain apparently

[frozen badge]

specifically, i think you would want to log function entry https://viztracer.readthedocs.io/en/stable/extra_log.html#log-function-entry

[frozen badge]

oh, i should note: fair warning, if you dont filter out some things or limit it to certain functions the reports may take a long time to generate and view

[austere lava]

I ended up using yappi because I couldn't figure out how to get viztracer to output data in a nice tabular format I could sort

https://github.com/psf/black/issues/1951#issuecomment-848245423

[neon loom]

😮

Why does that await help?

[austere lava]

because then we're gonna be calling wait less frequently, which has a considerable overhead

[neon loom]

Right, that makes sense. In the tight loop you speak of. Nice find. Good learning for me.

You just scaled blackd! haha

[austere lava]

If anyone wants to send a PR, feel free to. I don't think I'll have more time these days

[neon loom]

Can discord do polls?

[hushed flint]

with reactions, kinda

ie vote 1, 2 or 3

[neon loom]

Interesting - We want to get community vote on https://github.com/psf/black/issues/2171 - And wish way to normalize binary/raw/f-strings prefixes

I wanted simple sort() and call it a day to avoid caring, but, as usual with formatting, people cared 🙂

[hushed flint]

imo rf and rb make more sense

but thats just my opinion ¯_(ツ)_/¯

[neon loom]

https://github.com/apps/polls

[halcyon latch]

We have a poll command. I believe it's locked to mods+? I'll poll the admins and see what they think about allowing it for y'all here

[bright glacier]

"[...] poll the admins [...]" 😆 (i'm sorry, i had to point that out)

but in all seriousness, thank you for asking!

[frozen badge]

i bet 5 mistakes before they figure the command out, seeing how long it took you

[dense jungle]

while we're waiting, let's do a simple reaction based one. React 🇫🇷 for fr"x" and 🇷🇺 for rf"x" see https://github.com/psf/black/issues/2171 for context

[prime gull]

fun fact: in here you can say something like psf/black#2171 and @toxic storm will resolve it for you :P

[toxic storm]

GitHub

Normalize raw f-strings prefix

[little parrot]

If you're doing that would you do the same with calls to open?

[dense jungle]

You mean the mode string? We won't change that

[split merlin]

What is the mode?

[dense jungle]

like open(file, "rb") for read in binary mode

[split merlin]

import black

code = "y = 4+  3"
code = black.format_file_contents(code, fast=True, mode=black.mode.Mode())
print(code) 

[dense jungle]

Oh, that affects how Black formats the code. It corresponds to the command-line options, like line length and --skip-string-normalization

[split merlin]

Ah, alright

[little parrot]

Why's skip string normalisation an option?

[neon loom]

Buggy

https://github.com/psf/black/issues/2188

^^ Would love help working out the last few primer found bugs so we can go default

[bright glacier]

because "pushing double quotes only" wasn't a hill we were willing to die on, IIRC the related thread(s) are quite long and heated, so -S disables the quote normalization

[prime gull]

is it possible to have black format your code with double quotes only? i need this

[bright glacier]

just don't pass -S or --skip-string-normalization

[dense jungle]

You mean even for strings that contain " inside of them?

Otherwise it's the default

[bright glacier]

--experimental-string-processing is completely unrelated to --skip-string-normalization btw

[prime gull]

this channel has taught me so much about black in the short time it's been here, wow

[neon loom]

That's default

[little parrot]

Also what do you guys think of the blue formatter that monkey patches black to do single quotes?

[split merlin]

meh

[hushed flint]

not sure if im right here for that sort of question, but do any of you have any advice on how to best use black with pycharm?

[tribal thistle]

The docs have a guide for that

I’ll link it in a sec

[dense jungle]

https://black.readthedocs.io/en/stable/integrations/editors.html

[hushed flint]

thx 👍

[little parrot]

Discord.py is trying to switch to a formatting style currently and it's hard to change minds on double quotes so I think it's a decent first step

[dense jungle]

You can start with Black with --skip-string-normalization

[bright glacier]

which is actually more like --no-touch-string-quotes

[glad pilot]

There's a fork of black that uses single quotes instead. It's a hill someone was willing to die on...

[bright glacier]

Second time mentioned in this channel.

[tribal thistle]

Got a feeling that number will only go up lol

[hushed flint]

kinda reminds my of the oh my zsh lol plugin

[bright glacier]

I'm the process of rewriting the official GH action and honestly I don't want to write tests for it, it's annoying and I don't even know if it'll bring much benefit

Cooper originally suggested writing unit-tests but I don't think I'm doing that ...

[hushed flint]

but, but you're supposed to follow standards

[bright glacier]

well the thing is that each test is quite slow (to run and to write) so yeah

[hushed flint]

then why bother?

doing certain things just for the sake of doing certain things is wasted time

[bright glacier]

that was supposed to be a response to the argument of "you must follow the standards" lol

[hushed flint]

it was more of a joke

albeit a bad one

[bright glacier]

good enough.

[hushed flint]

lmao, thanks for the pity

[bright glacier]

I've had too many close calls doing Ctrl-C while running pre-commit when it's stashing something (and if my luck runs out, losing my unadded work) 😅

[neon loom]

We can write post you pushing. We can create an issue - "good first issue" for people who know how to unit test

Should be able to mock stuff to make it fast

[bright glacier]

Testing the virtual environment management is slow though

although I have ideas to make it faster (on the testing side)

(for context, the GH action manages its own virtual environment to install black for multi-OS compatibility, a docker based action is complicated in its way, so a composite based action was better since it gives better compatibility too)

oh great, the original bug I was trying to avoid by using Python as the driver still exists

I know to fix it properly now, but I'm done working on the action today 🙂

[dense jungle]

would be nice if we could get an integration in this channel that tells us when. there's a commit to main

[bright glacier]

perhaps workflows for PRs too? (that may be too noisy though)

[halcyon latch]

I think we can help get that setup for you (cc: @delicate marsh)

[prime gull]

we have a cloudflare worker in front of update webhooks so we can filter out some noise that matches certain patterns

[tribal thistle]

The worker mainly filters bots and a few other things

A single event is better handled through the github UI

[delicate marsh]

yeah, it probably won't take a load out of black, but you can filter things at GitHub's end yeah

we'll figure something out internally and I'll reach out shortly

[bright glacier]

Does GitHub webhooks support only POSTing pushes to main? it appears it doesn't

[delicate marsh]

It doesn't, no

[bright glacier]

womp womp 😦

[prime gull]

could filter that with a worker :P

[delicate marsh]

we can filter that out, we have a Cloudflare Worker that sits between GitHub and Discord that we can strip events on

yeah

what events do you want to show up here? just pushes to main?

(would look like this)

[dense jungle]

Nice, thanks! New issues and PRs could also be nice but it might end up too noisy.

[tribal thistle]

That embed is Seb's action, no?

[delicate marsh]

sure, will get main filtering set up at our end and then reach out to set it up at the repo end, is there any one person who is best to go to for that sort of stuff?

[delicate marsh]

nope

[frozen badge]

#black-formatter-log channel when

[bright glacier]

that feels much IMO

[prime gull]

maybe oss-projects-updates

which can cover future channels too

[bright glacier]

that would be more platable 🙂

[frozen badge]

i mean, what if there are like, 3 large projects in the future? would be REALLY noisy

[delicate marsh]

let's start with main pushes, issues and PRs here, and the black team can disable issues and PRs if that gets too noisy?

[tribal thistle]

It would drown out smaller repos with bigger ones, but it's an option

[prime gull]

so would 6 channels rather than 4

[dense jungle]

@delicate marsh sounds good, thanks! I don't think it will be too bad, we don't get that many issues

[delicate marsh]

👍

[bright glacier]

So wait, does issues and PR mean the creation events? Or like creation / comment events?

[frozen badge]

by default creation and comment

tho im sure that can be filtered

[delicate marsh]

they are separate events at GitHub, so you can toggle them

issue open & issue comment are separate

[bright glacier]

I know, just was wondering what you had in mind when saying that. I don't have any issues really.

[frozen badge]

i was just going off what we have in staff

[dense jungle]

A single channel for multiple projects sounds like it'd be too noisy

[delicate marsh]

👍, will just push an update to our worker then and DM one of you the credentials required

[bright glacier]

The webhook URL? cause most of us maintainers don't have admin rights to repo

[neon loom]

https://github.com/pypa/bandersnatch/issues/925 - As usual I'd planned to use bandersnatch as my testing ground for this

[tribal thistle]

From our projects (#dev-log), I know having both opens and comments can be useful to keep you updated

Though I'm not sure what sort of traffic you usually get

[delicate marsh]

ahh, alright, I can still shoot the stuff to you if you can get in touch with folks with admin rights

[bright glacier]

It's just @solid adder who has admin permissions IIRC.

[delicate marsh]

gotcha

[solid adder]

Yeah. Not because I'm a jerk (that's unrelated!) but because the project is part of the PSF infra

So not even I am a full owner, I have a subset of admin privs on GH

But yeah @delicate marsh, DM me what you need and I'll make it happen

[delicate marsh]

perfect, just writing up the changes at our end to ignore non-main pushes

[solid adder]

Probably not like now right now because it's 1am and I need some sleep after fighting pixels all evening

Great event btw

[delicate marsh]

Hahaha, that's all good, glad to hear you're enjoying it 🙂. Will DM you stuff tomorrow.

[bright glacier]

Not enough black pixels tho (in all seriousness, it's super cool)

[hushed flint]

how come?

[solid adder]

PSF admins own the org on GH

I put the project under the PSF umbrella so it's not centered around me

And even if I disappear for any reason, it should have no consequences

[hushed flint]

fair enough

[solid adder]

I managed to gather a wonderful team of people to maintain the project

It's safe, not going away

Maybe some people wish it would 😂

[hushed flint]

why didnt you make your own org though? or is there an advantage to it being under psf

[solid adder]

"Your own org" means nothing in practice. The Python Software Foundation is an actual governing body with funding and staff

Like the Apache Software Foundation or the Linux Foundation

[hushed flint]

i get that, i just didnt think about that from that perspective

[bright glacier]

You did an awesome job of making a damn cool project so I think you can credit yourself for that.

[hushed flint]

[solid adder]

Thanks, @bright glacier.

[frozen badge]

an actual organisation backing it means that someone will at least maintain the project at minimum functionality even if the maintainers leave. Plus, i would guess that it being under a official umbrella also helps adoption as people trust that org

[hushed flint]

yeah got that

[bright glacier]

I mean, sometimes we get comments about us being a bad influence on the official python code style (which doesn't exist btw) because we're under the PSF umbrella

[hushed flint]

inb4 black becomes part of the reference python implementation

[solid adder]

Unlikely to happen in Guido's or Barry's lifetimes as they unsurprisingly have strong opinions on whitespace and formed them 30 years back

[hushed flint]

lmao

[tribal thistle]

Debugged and left a fix on psf/black#2104

[toxic storm]

GitHub

Safety checks are disabled when passing in code using -c / --code

[tribal thistle]

I'm around right now and could PR it

On a related note, is it possible to actually pass --check when passing in code? Given something like print("hello world") the formatted string would be print("hello world")\n but I can't figure out how to enter the \n without causing a parsing exception

[bright glacier]

I'm a little confused, could you explain a bit more?

Also thank you for debugging the issue!

[tribal thistle]

So if I pass in something like black --check -c "print(\"Hello world\")", that would exit with 1, because the properly formatted string would have a line break

If I try black --check -c "print(\"Hello world\")\n", that throws the following parsing exception: https://paste.pythondiscord.com/citukikave.sql

That exception was much longer than I thought lol

[bright glacier]

You'd have to do a multi-line string in your shell

[tribal thistle]

Right, that makes sense

Thanks!

[bright glacier]

~ via v3.8.5 took 263ms 
2❯ black -c 'print("hello world")
   '
print("hello world")


~ via v3.8.5 took 303ms 
❯ 

[neon loom]

Is he refactored it I’ll agree 😂

*now he

[worn zenith]

On my windows 10 machine running regular cmd prompt, I get weird encoding errors:

[golden nymph]

It's trying to display star emoji or whatever

[torpid stump]

that's because regular cmd prompt has garbage text encoding

[worn zenith]

cygwin does it too:

Hazardous to pipes if I'm automating this somehow.

[worn zenith]

Is there a way to squelch just the non-ascii art in this project?

Just say All done! and nothing else on that line?

[modest creek]

It’s 2021 it should be up to your terminal to support emoji

[worn zenith]

Should but isnt.

[torpid stump]

Is this causing a tangible problem, or is it that you don't like the ugliness?

[worn zenith]

So far just the latter but I'll be looking into tooling later. Unsure what it'll do to the logs and their repsective parsing and ELK management.

[torpid stump]

It shouldn't be any more of a problem there.

Cf. https://github.com/psf/black/pull/1270

[worn zenith]

Harumpf.

[neon loom]

I don't use windows, but for CI to pass with UTF8 on windows I've generally set the pythonioencoding: utf-8 envar

And that's helped

[worn zenith]

🤔 Perhaps. I'll look into that.

[neon loom]

https://github.com/facebookincubator/ptr/blob/master/.github/workflows/ci.yml#L40

[late dew]

.github/workflows/ci.yml line 40

pythonioencoding: utf-8```

[neon loom]

Is an example on a GitHub action

[frozen badge]

it might well be that the terminal itself is in ascii only mode, or doesnt have a font that supports the emojis.
@worn zenith i recommend using "Windows Terminal" or at least PowerShell, since cmd is deprecated in all but name (windows backwards compatability yay)

[modest creek]

yeah why use cmd/cygwin when there's windows terminal and WSL

[haughty ocean]

I got a ford advert and thought it was black trying to get my attention 😬

[scarlet swift]

wt is indeed nice since you can have powershell , cmd, WSL all in the same place but have all of them differently customized with different color schemes for each and different fonts for each (like if you wanna have your wsl to use a nerd font for a theme or something)

[flat kraken]

cooperlees

[psf/black] Pull request review submitted: #2260 Use latest Python in uploading binaries

I feel no pinning at all is needed here. I also feel this minor version pinning won’t do much as there is no idea when a 4.0 will happen …

Let’s see what other maintainers say here.

cooperlees

[psf/black] Issue closed: #2257 Add optional extra uvloop install

cooperlees

[psf/black] Pull request closed: #2258 Add optional uvloop import

cooperlees

[black:main] 1 new commit

754eecf Add optional uvloop import (#2258) - cooperlees

[hushed flint]

ok can we potentially move this in a separate channel?

[bright glacier]

eh it's not too bad (although 3/4 notifications per merge might be excessive, but it's only been like a few hours so let's give a it some more time )

[rancid abyss]

On pytest i put it in a different channel, github is excessive chatty :(

[tired shard]

@delicate marsh

[delicate marsh]

the proposal I believe was only to have pushes to main

we can also probably filter out the issue/pr close events

[neon loom]

Yeah I think me commenting is a little nosisy.

I feel PR creation, merges to main + releases

That said - @bright glacier @solid adder @dense jungle (we need a maintain tag if that possible) I'm thinking a 6/1 7+ fixes and doc fixes 😄

*6/1 release

Or maybe sooner

[dense jungle]

Let's do it, there's a couple more PRs that could go in

I can spend some time on experimental string processing this weekend

[bright glacier]

I do want get in an GH action change

[bright glacier]

Our black code formatter role doesn't work?

Or it not pingable?

[silent apex]

It's not pingable for me

The only pingable roles are mod and admin

[bright glacier]

Can confirm

[neon loom]

That was my plan too - Try get that ready - But might leave the make it default for a dedicated release

It can be a 21.6b1

[bright glacier]

Don't we start at 0 for the release number?

[neon loom]

Yes, that will be the before experimental strings release

[bright glacier]

ohhhhh I can't read

[neon loom]

😄 I can't type

@bright glacier If you can remember - do we have a section explaining the new string splitting behavior in the docs?

If not, we should before making it default

[bright glacier]

Yes! Thanks to

Nvm I did a typo

[gusty grail]

I think the role can be made pingable by an Admin- but that'd also mean that any Bob and Alice can ping the role.

[tired shard]

and annoy you guys for no reasons

[neon loom]

People can already do with slightly more effort

[bright glacier]

@errant barn is the right person :)

[bright glacier]

We'll still have to mark it as enabled by default but the docs are good for the most part.

[neon loom]

I just want somewhere to explain it for people when we get the "Why black changing my strings" comments in here post release 😄

• pin etc.

[errant barn]

That would be the "strings" section in the style docs: https://black.readthedocs.io/en/latest/the_black_code_style/current_style.html#strings

[neon loom]

Also, I really dislike when a big change comes out and it's not explained anywhere from other projects, so I try to not let ones I help with do so

[errant barn]

Also it probably should be changed from instructing to enable the option to disable it, or what happens to the option?

[neon loom]

Sweet - we can have the commit that makes it default change the wording there

@errant barn Thanks for all your docs help!

[errant barn]

My pleasure mate!

[neon loom]

And non docs help ...

Option gets deleted

[errant barn]

Ah, I see - that's probably for the best

[neon loom]

black's goal is to be as unconfigurable (is that a word) as possible

e.g. Like gofmt

All go looks as ugly everywhere! 😄

p.s. I'm not a go hater, just love to stir the pot

[tribal thistle]

This is a python server, saying otherwise would be a crime

[neon loom]

I do hate typing if err == nil tho

*!=

[errant barn]

Oh btw @tribal thistle did the symlink file formatting issue still keep happening for you yesterday?

[tribal thistle]

Yeah

[neon loom]

I use to run a go bittorent tracker at Facebook - python would have struggled with it's request rate in 1 binary ... Go's goroutines eats that shit up.

Now a proccess pool of asyncio workers with uvloop would have come close but that did not exist in 2014

[errant barn]

Damn, that's too bad

Anybody here know whether newlines in the symlink files actually break them or have no effect?

[tribal thistle]

Ultimately it's not a big deal, I just have to make sure they are not edited when I go to commit, or they break the pre-commit

I could test that now

[errant barn]

Yep, that was my conclusion too

[neon loom]

Newlines in symlink files - Does this mean a if black find a .py file via a symlink we're not adding a \n at end of file?

[tribal thistle]

There are symlinked files in the docs directory in the repo

They were added as a fix for something

They currently don't have newlines at the end

When you run lint on the repo, it modifies them

https://github.com/psf/black/commit/97c24664c5bc251f7b67756b8b184962ca44b6d9

[errant barn]

which prettier then changes when run manually

[neon loom]

I the black repo? Yeah, they should have \n imo - Put up the PR i'll merge and fix anything that breaks

*in

[errant barn]

So the files should be okay to have newlines? Sweet!

[neon loom]

I can't think of any reason they shouldn't. In the most part, most text files should

[tribal thistle]

According to the commit message, these are a temporary fix

So it might've just slipped through?

[bright glacier]

To clarify they exist to reduce duplication in the docs system (they are here to stay)

[tribal thistle]

Huh

Alright

[bright glacier]

Some symlinks were temporary but I think the two you are talking about are staying.

[tribal thistle]

In that case, I'll add the \n and make sure the docs don't break

[errant barn]

Yeah I guess it's better to have the files available on the top level too. Easier to find on GitHub when someone wants to read them

[bright glacier]

yep, authors and changelog are symlimked for that reason, so they stay top level but also exist in the sphinx docs

there's other hacks we could do to avoid those symlinks (like symlinking on documentation built) but it probably doesn't matter much

[tribal thistle]

While we're on the topic of docs

I couldn't get them to generate on windows at all

[flat kraken]

cooperlees

[psf/black] Pull request review submitted: #2259 Code Flag Options

Wow, great tests here! Thanks for your contribution.

[tribal thistle]

They'd throw a bunch of warnings

[bright glacier]

hmmmm

[tribal thistle]

Dropping the warnings flag fixes it

but...

[errant barn]

Did they refuse to generate? For me it was fine just ignoring the warnings 😅

[tribal thistle]

I should probably make a dump of the actual warnings, one sec

Yeah the contributing docs have -W which will fail it

[errant barn]

Oh I see, didn't use the full command then

[bright glacier]

There should be no warnings (our documentation build workflow rejects builds that have warnings)

[tribal thistle]

Yeah, there are no warnings on my linux machine

[bright glacier]

But I haven't tested Windows 😅

[tribal thistle]

Ultimately, it does work even with the warnings

[bright glacier]

What's the warnings?

[tribal thistle]

The first one is an encoding error

https://paste.pythondiscord.com/idigipogox.less

There are more, let me try to get the rest

[uncut patio]

windows .encode and .decode doesn't default to utf8 IIRC

[tribal thistle]

65 warnings in total

Symlinks also cause a different warning

C:\GitHub\black\docs\index.rst:102: WARNING: toctree contains reference to document 'authors' that doesn't have a title: no link will be generated

[bright glacier]

oh for goodness sake

I do have a windows machine so I can look into it later on my own

[tribal thistle]

It does seem like a bit of a nightmare haha

[bright glacier]

Black primer has am emoji in its help output which we dynamically capture and put into the docs

Setting an encoding on the subprocess should work but I need to look if the sphinx extension we're using supports configuring that

It appears there is no option O.o

I wonder if the "no title" warning is because windows doesn't support symlinks by default

[tribal thistle]

Yeah

[bright glacier]

So it's just reading the symlink target text

[tribal thistle]

Yuuuup

Removing the symlinks and copying in the files manually drops it to one warning

[errant barn]

I wonder if referencing the original files from the toctree or using a rst include would be enough to fix it

[bright glacier]

I'll have to try

[tribal thistle]

Not really sure what can be done about the emoji

I'm not even sure what codec is supposed to decode it

err, encode it

[bright glacier]

So ok it's not too bad although they sound impossible to workaround

I wonder if Cooper would find it acceptable to remove the emoji

[tribal thistle]

I haven't worked with sphinx before, what's the entrypoint to the generation?

Could we copy in the files and delete them after the process is done?

[errant barn]

the makefiles would be the place I think

[bright glacier]

I'm not a fan of using the makefiles

also windows has its own format called batch files and I have no experienced with those (almost same for makefiles)

[tribal thistle]

I was actually thinking of conf.py

[bright glacier]

There's no post build step tho (creating a custom sphinx extension may work tho)

[tribal thistle]

Hmm

[errant barn]

I think conf is only for setting up. I've installed packages using it, but in terms of tearing down.. I haven't heard about it

[tribal thistle]

We don't have to clean, we could ignore the copied file

Though that might not be the right move

[errant barn]

oh that would work

[tribal thistle]

AUTHORS = CURRENT_DIR.parent / "AUTHORS.md"
CHANGELOG = CURRENT_DIR.parent / "CHANGES.md"

Path(CURRENT_DIR / "authors.md").write_text(AUTHORS.read_text("utf8"), "utf8")

[bright glacier]

I've found that to be annoying but we might have to go that route for Windows

[tribal thistle]

This does work

It also got rid of the encoding error ????

That can't be right

[bright glacier]

Wat

[tribal thistle]

Let me delete my old build lol

[bright glacier]

Or use -E

[tribal thistle]

There it is

I found https://stackoverflow.com/questions/10199233/can-sphinx-link-to-documents-that-are-not-located-in-directories-below-the-root

Which seems to be dealing with the same issue

Haven't tested it yet

There are some limitations

[bright glacier]

I just tested this and it works!

[errant barn]

Perfect!

[bright glacier]

The document that has the include must be in MyST though since mixing two languages breaks hard

So okay just the subprocess encoding mess left

would assuming UTF-8 work fine here?

Not like it's possible with the sphinx extension but curious

OH WAIT, looking at the traceback again, click is crashing???

[neon loom]

Windows should support UTF-8 with python >= 3.6 easily. People just need to set their PYTHONENCODING=utf8 I believe (But I have 0 windows skillz)

[tribal thistle]

Does it actually encode with utf8?

codecs.charmap_encode("\U0001f3f4", None, "utf8")
This complains too

On windows and linux, so I'm assuming I'm just not understanding

[bright glacier]

I honestly have no real encoding skills

[neon loom]

@solid adder understands encoding the best out of anyone I've ever met - I summon thy 😄

Or, prob better to get a draft PR up for him to comment on

And show CI failing

Might help me understand too

https://www.youtube.com/watch?v=7m5JA3XaZ4k

[bright glacier]

I was watching that last night😅

[neon loom]

That's my little polish mate in his element.

[solid adder]

Use Python 3.8+ and then Windows encoding is assumed UTF-8 without intervention. See PEP 538 and PEP 540.

[neon loom]

O shit look out

[tribal thistle]

I'm on 3.9

[solid adder]

Yeah, but you're trying to use the charmap codec. Why?

[tribal thistle]

Click seems to be calling it

The full TB is

[neon loom]

Ahhh click and unicode 😐

[tribal thistle]

here

[solid adder]

charmap is a subset of Unicode: https://docs.python.org/3/library/codecs.html#encodings-and-unicode

that traceback is pretty unreadable lol

[tribal thistle]

Yeah it's a pain

I'll try to generate a better one

[bright glacier]

yeah I agree I'll see if I can configure our ci to try windows for docs too

[neon loom]

Seems sane since we have Windows contributors now 😄 (well more :P)

[bright glacier]

I used to be a windows contributor but then I left that world

[neon loom]

Least @bright glacier saw the light

[solid adder]

that's the same traceback but formatted for humans: https://paste.pythondiscord.com/revavopavo.sql

[neon loom]

I don't why click is us cpxxxx lameness

[solid adder]

OK, so we see that echo is using CP-1252, maybe there's an actual FSENCODING= or LANG= or LC_ALL= declared in the environment?

[neon loom]

I don't think we run primer in WIndows anywhere - I guess we could add that too

[tribal thistle]

I could get you a dump of my env, but I don't have anything special set

[neon loom]

We could also hard set black's coding if it's not everywhere: e.g. https://github.com/facebookincubator/ptr/blob/master/ptr.py#L6

[late dew]

ptr.py line 6

# coding=utf8```

[neon loom]

I've done this to make less errors with Windows

[bright glacier]

for context this crashes when sphinxcontrib-programoutput tries to run black-primer --help

[neon loom]

LoL - I haz it: https://github.com/psf/black/blob/main/src/black_primer/cli.py#L1

[late dew]

src/black_primer/cli.py line 1

# coding=utf8```

[neon loom]

So something is overriding it in your ENV I feel

Or sphinx is being "helpful" on Windows

sphinx / click

[tribal thistle]

I looked for these three flags, but none are set

running primer help on it's own is fine, but I think that was expected

[bright glacier]

It's the interaction between the sphinx extension and click it seems

[neon loom]

Yeah my coding should enforce that - I guess sphinx is being "helpful" (unhelpful)

[solid adder]

@tribal thistle what does sys.getfilesystemencoding() say on your end?

[tribal thistle]

Nada

[neon loom]

^^ I bet this differs when manually running primer then sphinx running it

[tribal thistle]

Is it supposed to output something? It just printed a new line

[errant barn]

utf-8 for me, I'm on 3.9

[neon loom]

Are you on Windows too?

[errant barn]

Yup

[neon loom]

If so, does docs running repro for you too?

I can book a VM later to try if need be

shivers at the thought

[errant barn]

Yeah it had a bunch of warnings, let me see about the encoding one though

[tribal thistle]

Most of them are the symlinks

It's just 1 encoding warning

[errant barn]

Yep I saw

[tribal thistle]

I reran this in the conf file, and the output is "utf-8"

[neon loom]

I have to go do $dayJob - But I am all for adding more windows CI once we fix all this so we don't break all you fine people and your dev experiences moving forward

[tribal thistle]

Windows CI can be a bit of a pain, but that would be appreciated haha

Catchya later

[errant barn]

Hmm, actually it does produce the unicode error with click failing. And the virtualenv does produce the same output for file encoding.

[neon loom]

If I see the PRs happy to review ... Actions make it pretty easy these days. I've had good success with other projects. We've had good success here for main black CI.

More success that MacOS X reliability

needs to poke friend @ Mac Stadium about that

[bright glacier]

I'm working on the docs on windows ci config rn just messed up the tank syntax lol

[neon loom]

(FWIW GitHub outsources the Mac fleet)

[bright glacier]

YAML***

[heavy heron]

I once saw a picture of their Mac "servers", it is really weird stuff

[neon loom]

@bright glacier - FWIW I've had to do this to get utf8 happy in GitHub Actions Windows land: https://github.com/facebookincubator/ptr/blob/master/.github/workflows/ci.yml#L40

[late dew]

.github/workflows/ci.yml line 40

pythonioencoding: utf-8```

[neon loom]

Lets try not to, but if we have to ...

[errant barn]

But.. Linux is slowly getting pretty much all the software I care about, even the audio workstations and games so maybe I ought to consider a change once more :D

[neon loom]

I also vote doc builds should only run in latest Python (e.g. 3.9) since it's easy to get latest stable Python on Windows these days too

[bright glacier]

I made that change too already

[neon loom]

Thanks to the legendary M$ Core Contributors

[bright glacier]

You've complained if I didn't 😉

[tribal thistle]

os.putenv("pythonioencoding", "utf-8") resolved the issue

[neon loom]

Damn right

[neon loom]

Ok, so something is setting it tho. I put money on sphinx

[bright glacier]

In the conf.py?

[tribal thistle]

Yup

[bright glacier]

Huh

[neon loom]

That's prob a safe hack

[bright glacier]

here's the windows workflow fyi

https://github.com/psf/black/runs/2676894366?check_suite_focus=true

same crash

somehow the symlinks didn't break tho

[tribal thistle]

Actually got all the warnings resolved with https://paste.pythondiscord.com/ekosehupah.py

Symlinks and encoding

[neon loom]

But maybe worth an issue on sphinx/click if we root cause the problem to say it's 2021, lets default to utf8 per PEPS (see numbers above)

[tribal thistle]

But it seems someone found a better fix already (for symlinks)

[bright glacier]

I was thinking that, but the simple docutils include work fine 🙂

also that would have to be its own external extension so incremental builds still work

[tribal thistle]

That's a much better solution, espically for ro systems lol

Could someone rerun this test?
https://github.com/psf/black/pull/2259/checks?check_run_id=2676253811

It seems to have bugged out

I could actually run this on my own fork lol

[bright glacier]

I'll rerun it

[tribal thistle]

Thanks

Hmm

[bright glacier]

did it crash again

[tribal thistle]

A different suite with the same error

[bright glacier]

OK, it's probably time to remove the parallelization then, our test suite is pretty fast on one core still 😄

[tribal thistle]

I forgot to add the docs change to the changes

I'll do that now lol

[bright glacier]

does anyone know what https://github.com/notifications/beta/archive is all about

while going through my notifications, sometimes I just randomly get directed to that page

[dense jungle]

404 for me 😄 I guess you're on the other side of an AB test

[bright glacier]

seems like it

[tribal thistle]

I'm investigating psf/black#2250

[toxic storm]

GitHub

Comments being removed in .pyi files

[tribal thistle]

Something worth noting is that the behavior is limited to ... on pyi files. I could not reproduce it in .py files, or when replacing the ellipses with any other code. The formatted output places the ellipses on the same line as the colon, which seems like a hint for what's happening.

This is not documented in the black style page, but I'm wondering if the code style enforces ellipses on the same line. That would be consistent with the behavior you'd experience if you remove the comment manually.

Is that standard for stub files?

[bright glacier]

Black uses the typeshed formatting convention for Pyi files

I'm sure @dense jungle can explain

[dense jungle]

Yes, that's right. Typeshed's style guideline is to put ... on the same line as the :, and Black enforces that guideline

But I guess in the process we eat the comment

Probably we shouldn't put ... on the same line as : if there's an intervening comment

[bright glacier]

https://github.com/python/typeshed/blob/master/CONTRIBUTING.md#conventions

https://black.readthedocs.io/en/latest/the_black_code_style/current_style.html?highlight=pyi#typing-stub-files

[tribal thistle]

Would comments in general be in violation of

Stub files should only contain information necessary for the type checker, and leave out unnecessary detail:

lmao was that section always there

[bright glacier]

yep even in the stable docs

https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html#typing-stub-files

[dense jungle]

@tribal thistle No, comments are allowed to explain why stubs are the way they are. We may also move toward allowing docstrings in stubs in the future

(for context I'm also a typeshed maintainer)

[tribal thistle]

Black seems to eat up any body content when it sees ..., so I think I may have a fix

[bright glacier]

I'm surprised, my guess would be that the prefix handling code is misbehaving for ...

Admittedly I have not much skill with AST wrangling code

[tribal thistle]

Actually you may still be right

It only eats up the body when the body is only comments

[flat kraken]

ichard26

[psf/black] Pull request review submitted: #2259 Code Flag Options

Thanks for working on this fix! I got some feedback but nothing truly blocking (except for possibly a Windows bug).

[flat kraken]

ichard26

[psf/black] Pull request opened: #2262 Fix and test docs on Windows

There's some weird interaction between Click and
sphinxcontrib-programoutput on Windows that leads to an encoding error
during the printing of black-primer's help text.

Also symlinks aren't well supported on Windows so let's just use
includes which actually work because we now use MyST :D

[tribal thistle]

Ooh nice

!remind 1H review docs PR

[late dew]

Absolutely!

Your reminder will arrive in 1 hour!

ID: 2255, Due · May 26, 2021, 11:36 PM

[late dew]

@tribal thistle

It has arrived!

Here's your reminder: review docs PR.
[Jump back to when you created the reminder](#black-formatter message)

[tribal thistle]

Just in case I hadn't raised enough windows problems for y'all to look at, pre-commit does not run the black hook using the virtualenv

It tries to use the system env

And if you don't have it in the system env, which is my case, it flat out fails

We ran into the same problem with python-discord projects in the past

I'll try to find the fix

Right, so there are two fixes:

1. Update the entrypoint to explicitly use pipenv (pipenv run black)
2. Have the end user manually edit their installed pre-commit file to activate the virtualenv. Downside here being this can't be done automatically, and must be configured by the end user

[frozen yacht]

What leads you to believe this is blacks fault

[flat kraken]

HassanAbouelela

[psf/black] Pull request review submitted: #2262 Fix and test docs on Windows

Tested locally, and everything works as expected.

Adding windows to the CI is an appreciated touch.

[tribal thistle]

The pre-commit is meant to run the dev copy of black, otherwise it may as well use a remote one

[frozen yacht]

So this is about the hook used for its own development, not the one it "exports" for other users?

[tribal thistle]

Yeah

https://github.com/psf/black/blob/main/.pre-commit-config.yaml#L5-L13

[late dew]

.pre-commit-config.yaml lines 5 to 13

- repo: local
  hooks:
    - id: black
      name: black
      language: system
      entry: black
      minimum_pre_commit_version: 2.9.2
      require_serial: true
      types_or: [python, pyi]```

[tribal thistle]

this one

[flat kraken]

cooperlees

[psf/black] Pull request review submitted: #2262 Fix and test docs on Windows

LGTM - Thanks doc guru.

cooperlees

[psf/black] Pull request closed: #2262 Fix and test docs on Windows

cooperlees

[black:main] 1 new commit

6613e76 Fix and test docs on Windows (#2262) - ichard26

[neon loom]

Wow, so this must install in all our main python environments and no venv? Is there an option here for use a venv

?

[bright glacier]

I thought pre-commit was supposed to create its own environments??

But I guess that step is skipped for local hooks...

I suppose windows means even an activated dev virtual environment won't make pre-commit use the env's black?

[bright glacier]

FYI this is the type of feedback we truly need, I've been trying to improve the contributing experience and we would have never knew of rhis

Cross linking psf/black#2238

[toxic storm]

GitHub

Improving the contributing experience

[bright glacier]

So thank you even if it's annoying and makes me dislike Windows even more 😅

But that truly confuses me because if pre-commit doesn't manage its own envs for local hooks, why would it try using the system python when you're in an activated virtual environment?!

[tribal thistle]

Pre-commit does use a virtual env, but it gets a bit more complicated when you try to run a module

The full explanation goes a bit too far into the weeds of how it works

But the basic rundown is

[bright glacier]

Especially since you should be calling an install of pre-commit that is installed in said dev environment

Perhaps I should do a full run through of a dev workflow on windows

might catch some of these annoyances

I suppose python discord has hit a lot of these issues ay?

It sounds like this is the 20th you've hit this :p

[tribal thistle]

HA most of them are me

[bright glacier]

that works too lol

[tribal thistle]

Alright, basically ['.virtualenvs\\black-pgyv376z\\scripts\\python.exe', '-mpre_commit', 'hook-impl', '--config=.pre-commit-config.yaml', '--hook-type=pre-commit', '--hook-dir', 'C:\\GitHub\\black\\.git\\hooks', '--']

This is what it runs

very pretty no doubt

it doesn't actually activate that virtualenv though

Why it doesn't? I can't answer that

[bright glacier]

so it completely isolates it from the parent process which probably has been activated

[tribal thistle]

But do you see a situation where someone may need to run the dev hook, without having pipenv available?