#black-formatter

1 messages Β· Page 12 of 1

tribal thistle
#

it calls it through SystemExit(subprocess.call(CMD))

#

with CMD being that list up there

#

It does something slightly different on other OSes, this is just windows

bright glacier
#

no, we do support lightweight testing / documentation environments but they're more for CI so probably not?

tribal thistle
#

If you change the hook entrypoint to invoke the manager directly like pipenv run black, that'll guarantee it runs the right version

#

The other option would be calling activate before that CMD, but that requires manual modification of the git hook file

bright glacier
#

And sounds painful

tribal thistle
#

Pre-commit is a miracle of hacks and workarounds in the name of cross-compatibility

bright glacier
#

many TILs today haha πŸ˜…

tribal thistle
#

But it falls short in places

bright glacier
#

Yeah, although if it was up to Cooper, he would probably straight up get rid of it

#

I still like it for making it hard to not mess up the basics

#

well not that hard (a lot my commits use no verify but I digress)

dense jungle
#

I don't use it actually. I get impatient when git commit takes too long

bright glacier
#

I find the speed to not a big deal personally most of the time but rapid iteration (i.e. when I make too many mistakes) makes me use --no-verify a lot

tribal thistle
#

Hmm that does make me think

bright glacier
#

but then again it wouldn't be the worst thing to remove it entirely from dev environments by default since CI would still catch lint issues

tribal thistle
#

I touched on it earlier, but you could use a remote version of black, like you do for the other hooks

#

It would fall behind, but it avoids this whole mess of envs and versions

bright glacier
#

That would lead to some very confusing (probably rare cases) where pre-commit fails but the tests pass (our tests do a pre-commit like run of black too)

#

Hmm, I'll think about it (and I guess see what's other maintainers' opinions on this) for a little bit

tribal thistle
#

For now I’m perfectly contempt modifying the file on my own, since it’s a simple one time fix

#

But it isn’t the most user friendly

bright glacier
#

Well wait we do call pre-commit without pipenv provided env for lint CI but that's on Linux so I guess that avoids the issue?

tribal thistle
#

Uhh

#

Perhaps lol

#

I’ve never seen anyone run into this on anything other than windows

bright glacier
#

I don't know, the cloned black should be the only copy of black on the machine anyway

tribal thistle
#

The command itself should be pretty similar on linux and mac

#

the only difference being the env setup, and the invocation, which is: os.execvp(CMD[0], CMD)

#

same CMD as above

bright glacier
#

I haven't ran into issues like this on Linux like you said too. Ugh, I wish windows would disappear for the time being lol

tribal thistle
#

Not many projects give too much thought into getting things working on windows

bright glacier
#

I'm not saying Windows sucks, it just isn't ideal for development

tribal thistle
#

Which is, fair enough I guess

#

Yeah

#

I appreciate the effort

bright glacier
#

I mean, making the lint job use the pipenv env might not be as slow I would expect (especially with the excellent caching you can do with locked requirements)

#

So pipenv run black at first glance seems to be the most appealing

tribal thistle
#

It’s what we settled on for our projects

bright glacier
#

I have never used pipenv run btw

#

Pipenv is too slow

tribal thistle
#

But admittedly we do force people to use pipenv

frozen yacht
#

Why is it even set to system rather than python?

bright glacier
#

I literally created an alias for the activation script of the pipenv managed env

#

Pipenv shell imports a lot stuff

tribal thistle
#

Pycharm has pretty great integration with pipenv

#

It does basically what you described

bright glacier
#

Vscode for me πŸ˜‰

tribal thistle
#

Just uses pipenv as a package manager

bright glacier
#

It does the same thing but I still likr working in a basic uncomplicated terminal

frozen yacht
#

It doesn't activate a virtual environment because system tells it not to use one. If you use python instead, then it will install black into its own venv. However, I'm not sure if it supports local/editable installs.

#

I don't see a reason for it not to support editable installs though.

#

The only benefit I have encountered for system so far is to avoid maintaining duplicate lists of dependencies in the pre-commit config and in a requirements.txt or whatever.

#

However, in this case it's just black without anything else. I don't see why it's set to system.

bright glacier
#

I haven't taken a good look at our pre-commit configuration like ever honestly

#

Definitely worth a look over

rocky wharf
#

Hey, are you guys planning on switching your parser? I remember seeing something about that in pygen

dense jungle
#

Yes, we'll have to eventually because the current Black parser isn't compatible with CPython's new PEG parser

rocky wharf
#

Ah OK

#

Have anything in mind?

#

Or are you rolling one from scratch

dense jungle
#

libcst and parso are candidates. we haven't really done much so far except realizing it's a problem πŸ™‚

rocky wharf
#

Alright

bright glacier
#

It's basically a rewrite of the project so we have been working on other stuff in the time being

#

We plan to do planning and do a POC but it's all very much in the air still

rocky wharf
#

Are you considering writing one from scratch? Or just searching around for a generator

dense jungle
#

We'd rather use something that already exists

rocky wharf
#

Yeah, a parser might be hard to get right from scratch

#

Isn't libcst only up to 3.8?

dense jungle
#

Yes, that's a problem

#

Though I read it's based on Parso and Parso does support 3.9.

#

Haven't tried it myself yet though

bright glacier
#

Still says nothing about the shiny match statement though :(

dense jungle
#

3.10 is going to have a ton of new syntax too so that'll be interesting

rocky wharf
#

Yeah 3.10 is exciting

#

Do you guys have any plans for adding 3.10 support when it's out?

dense jungle
#

Haven't really talked about it. I'm guessing it's pretty hard because match and case are soft keywords

#

So our existing parser won't be easy to adjust

rocky wharf
#

Hm, yeah

#

Parso seems like a solid option

potent bone
#

when will you add ai so the formatting better?

modest creek
#

how does AI make formatting better

potent bone
#

Well, I can format myself much better than this, so I figure ai is what's missing, right?

flat krakenBOT
#
GaneshSai
[psf/black] Issue opened: #2263 error: cannot format test\_bert\.py: Cannot parse: 26:0:

error: cannot format test_bert.py: Cannot parse: 26:0: def most_similar(doc_id, similarity_matrix, matrix):

def most_similar(doc_id, similarity_matrix, matrix):
# Finding similarity if Cosine similarity is chosen:
if matrix == "Cosine Similarity":
similar_ix = np.argsort(similarity_matrix[doc_id])[::-1]
for ix in similar_ix:
if ix == doc_id:
continue
return({similarity_matrix[doc_id][ix]})

def bert(text, url):
save_not_...

modest creek
#

if I've learned anything about AI in recent years, adding it will only make black racist, somehow

potent bone
#

lol

flat krakenBOT
#
jacobian91
[psf/black] Issue opened: #2264 Multiline Context Manager 'with/as' parsing error

Describe the bug
Black reports a parsing error when using multiline context managers using the 'with/as' pattern.

To Reproduce
Run the below code for the parsing issue. Then remove as x as y as z, this will run without issue.

with (
    open("xxxxxxxxxxxxxxxxxxxx", "w") as x,
    open("yyyyyyyyyyyyyyyyyyyyyy", "w") as y,
    open("zzzzzzzzzzzzzzzzzzz", "w") as z,
):
    x.write("x")
    y.write("y")
    z.write("z")

Expected behavior
No e...

muted haven
#

have you guys considered directly using pegen with a forking the current grammar and rewriting the actions to construct a CST instead of an AST? since the rules are there, the only extra is going to be the token captures and new nodes to represent those (or even a generic form like Node(node_type1, [token1, token2, node1, token3]) instead of creating and handling each node type separately)

neon loom
#

No. Nothing has been considered. The refactor out of 1 big file was preferred and getting a lot of bugs out and soldiering on with blib2to3

#

But we’ll take all POCs and suggestions

#

It will basically come down to which was Łukasz wants to go. He’s a parser nerd with opinions πŸ™‚

#

He will listen to people tho - Im not saying he will dictate.

austere lava
#

Unfortunately 3.9 unofficially snuck in the "parenthesis in context managers" feature, so technically there are valid 3.9 programs that parso can't deal with

austere lava
muted haven
austere lava
#

Thanks for the offer! Right now I'm struggling to convince my employer to fund this work (it will never get done if I do it in my spare time)

nimble urchin
#

Is Black formatter supported in PyCharm?

austere lava
nimble urchin
#

Oh, I have only come across with an issue on YouTrack that it still wasn't supported

#

All right, thanks for the info πŸ‘ I'll look it up

errant barn
#

I'm not really sure what this whole discussion resulted in. But for me pre-commit runs just fine with nothing installed on the system level when running manually (and it does reformat python files too), but fails in the git hook πŸ€” could that be my PyCharm being too smart with pipenv then..

final swan
neon grove
frozen badge
flat krakenBOT
#
vijayant296
[psf/black] Issue opened: #2265 INTERNAL ERROR: Black produced different code on the second pass of the formatter\. Please report a bug on https://github\.com/psf/black/issues\.

Can you please suggest on below error while running black:
error: cannot format /home/vijayant/Desktop/phoenix2/phoenix/src/device/deviceconfigserver/deviceconfigsdk_py3/device_sdk.py: INTERNAL ERROR: Black produced different code on the second pass of the formatter. Please report a bug on https://github.com/psf/black/issues. This diff might be helpful: /tmp/blk_c0tsf65p.log
Oh no! πŸ’₯ πŸ’” πŸ’₯
1 file failed to reformat.

flat krakenBOT
heavy heron
#

Well

flat krakenBOT
heavy heron
#

Hey! Is it the right channel if I have some questions about the future development of Black? It is about configuration

dense jungle
#

sure!

heavy heron
#

Alright, are they any plans to add feature switches to Black? For example (this is just an example, I don’t remember if Black actually did that, it may have been another tool) if Black wants me to formate my slices as [a : b] instead of [a:b], will I be able at some point to ask it to not reformat that?

dense jungle
#

We're generally very hesitant to add flags that affect formatting. The goal is that all Black-formatted code looks the same

torpid stump
# 
__main__.py, version 21.5b1```
Hmmm. Figure out how to fix Click's program name introspection for Windows?
#

Or tweak Black to simply use @click.version_option(version=__version__, prog_name="black") instead of relying on Click to figure it out?

heavy heron
#

Mhm, but don’t you think that it would be better to sometimes allow configuration and have Black fit nicely in an existing code base? For example I can imagine that if you already have a 100k lines code base or even more and want to start using Black, you’d better be off making Black use your current styling guidelines than generate thousands lines diff

raw nebula
#

just squish all former commits and force push it, bam no diffs

torpid stump
#

If you want configuration, there are other tools like autopep8, yapf

dense jungle
#

@heavy heron there are ways to make git blame ignore the reformatting commit, it's in our documentation.

heavy heron
#

That’s interesting

#

it will still generate tons of conflicts though, is there a way around it?

dense jungle
#

You mean for pending PRs? If people reformat their branches with black too there should be few conflicts

mint barn
#

what is this? someone mind explaining to me

torpid stump
frozen badge
#

in that doc page, are you going to change the IRC link? or is that still relevant?

bright glacier
#

We (read: me) just haven't gotten around to it :p

prime gull
bright glacier
#

oh my πŸ˜†

prime gull
#

that was both harder and easier than i expected lol

#

simple but obscure change

dense jungle
#

The double quotes make a difference?

prime gull
#

yep

#

double quotes are soft keyword single quotes are hard keywords

dense jungle
#

interesting design decision

prime gull
#

if ambv hadnt told me that I'm pretty sure I'd never have found that

#

its such a subtle difference

torpid stump
heavy heron
#

Just as a side note, I don't know how familiar you are with discord but you can create an invite that will drop users directly in this channel by clicking the little user icon next to the channel name, that could be useful for the doc page

austere lava
#

ye just make sure the invite never expires πŸ˜‰

prime gull
#

not yet yay, turns out the soft keyword version works but the keyword itself doesnt

bright glacier
#

I'm not 100% sure that's right tho.

delicate marsh
bright glacier
#

hooray πŸŽ‰ thanks for checking, still very much a discord noob :p

flat krakenBOT
bright glacier
#

hmm, the help output from click is pretty unreadable on main

#

stable doesn't have the same issue, will look into it

#

It might actually be a Click 8 thing. Click 7.1.2 (which is was the newest at the time of stable's build) doesn't have the same behaviour either.

dense jungle
#

Sorry which part do you think is wrong? The fact that it scrolls to the right?

bright glacier
#

The lack of spacing between options.

dense jungle
#

Oh yes, with spacing it's nicer

bright glacier
#

Reading the changelog, this was actually a revert, and there's no easy config to get back the spacing. And the formatter class's relevant method isn't friendly to override as of right now unless we want to copy and paste a bunch of code :/

tribal thistle
#

Can you link the changelog/reason for revert?

#

If it’s not resolved soon, I could look into it sometime

bright glacier
tribal thistle
#

Interesting

#

Thanks

bright glacier
#

And thank you for being willing to help out!

plain atlas
#

Before Black

file_handler = (logging.handlers.RotatingFileHandler(log_file,maxBytes=5 * (2 ** 12),backupCount=5,encoding="utf-8",).setFormatter(logging.Formatter(fmt=fmt,datefmt=datefmt,style="{",)).setLevel(logging.TRACE))
#

After Black```py

file handler

file_handler = (
logging.handlers.RotatingFileHandler(
log_file,
maxBytes=5 * (2 ** 12),
backupCount=5,
encoding="utf-8",
)
.setFormatter(
logging.Formatter(
fmt=fmt,
datefmt=datefmt,
style="{",
)
)
.setLevel(logging.TRACE)
)

neon loom
#

I know what's much more readable πŸ˜„

plain atlas
#

the first one ofc

#

it makes the program much shorter in line count

flat krakenBOT
red sedge
#

The b stands for black-formatter

flat krakenBOT
#
tmotyl
[psf/black] Issue opened: #2267 Black is wrapping result of an if/else oneliner in tuple \(changing variable type\)

Describe the bug A clear and concise description of what the bug is.

In some cases black is changing variable type by wrapping if/else oneliner expression in a tuple, which changes the variable type.

To Reproduce Steps to reproduce the behavior:
See example code:
https://black.vercel.app/?version=stable&state=_Td6WFoAAATm1rRGAgAhARYAAAB0L-Wj4ACqAHxdAD2IimZxl1N_Wmlq4KNAXQ0SPvPbuEKTxVQ0VY8PC6zOM3kOy_hazUIwFPqfT_KXfZ566uiV3GFO2KICIF0rToCYmmHFekAa5Voo3cqCmEerIqUxWsd_agoWXOoD5xyc...

heavy heron
#

You bug template makes me think of something

#

You can use anchor tags to place arbitrary comments in the file and they won't be shown in the rendered version

#

That's what we do at pydis to keep issues less wordy

#

Maybe that could be nice to have for Black ?

flat krakenBOT
flat krakenBOT
#
tmotyl
[psf/black] Issue opened: #2268 Black is not removing unnecessary \+ signs in concatenated strings

Describe the bug A clear and concise description of what the bug is.

second_string = ("def"
    + "ddd" + "ddd")

is formated to

second_string = "def" + "ddd" + "ddd"

where I would expect:

second_string = "defdddddd"

To Reproduce Steps to reproduce the behavior:

https://black.vercel.app/?version=main&state=_Td6WFoAAATm1rRGAgAhARYAAAB0L-Wj4ADPAHpdAD2IimZxl1N_Wl9XM4Hukmv0lHvH8ZV8H_6vvOURMPls_WzMNcku5-VbAQaYsBSET4vbhPFAzFY5DcRsFuGPJMGD-DLKFHB...

bright glacier
heavy heron
#

Oh, probably haha

#

I can submit a PR if you wish

bright glacier
#

That would be appreciated ☺️

flat krakenBOT
flat krakenBOT
#
Akarys42
[psf/black] Pull request opened: #2269 Issue templates: use HTML comments

This commit makes use of HTML comments inside GitHub issue templates
to make sure that even if they aren't removed by the issue author they won't be shown
in the rendered output.

The goal is to simply make the issues less noisy by removing template messages.

You can try them out on my fork. I haven’t added a news entry as this doesn’t seem to fit, let me know if it needs one! 😁

heavy heron
#

Tadam!

#

lol

solid adder
#

@visual grotto ohai

visual grotto
#

πŸ‘‹

bright glacier
#

πŸ‘‹

solid adder
#

OK, I see you're listed as a Python Discord partner already. Good. This is our new spot. How do you like it?

#

It's got 100% less Facebook.

visual grotto
#

It's nice! I like discord as a tool for this kind of thing more than, say slack (or fb πŸ™ˆ )

#

Tried to make the case a while ago to the Functional Programming slack but no one bought in

#

Some communities I know use discord for the whole livestream rooms thing which let them also do, like, meetups and stuff in them

solid adder
#

I mildly prefer Slack functionally because it's got threads and I tend to use those for work rather aggressively.

#

But there is no Python Slack like the Python Discord.

visual grotto
#

I have a slight irrational grudge against slack threads because they more or less ruined the irc-terminal-client-to-slack plugin i used to use to avoid running the actual slack client πŸ˜…

#

I have made my peace now, though

solid adder
#

I have a tolerate/hate relationship with Slack due to its irrational handling of text input. Most of the weirdness can be turned off but not all.

#

Plus, it's got this comedic memory usage that is literally the butt of jokes for years now. And they don't seem to do much about it.

#

OK, heads up: we're officially moved to Discord for good, I kicked everybody out of the Messenger group chat.

short forge
#

you had a messenger group chat?

bright glacier
#

RIP messenger chat, its constant deadness will be (not) missed

heavy heron
#

Messenger group chat for OSS developments sounds really yikes

#

Imagine posting code in messenger lemon_pensive

plain atlas
#

wut @solid adder got their emoji πŸ‘€

short forge
#

i thought you guys came from IRC, messenger is kinda... haha

heavy heron
split merlin
#

Seemed like a joke

bright glacier
#

It was just for the core team (which was like originally 100% Facebook really back then, and we didn't switch until now)

tribal thistle
#

Does Facebook usually use messenger for internal comms? Haha

solid adder
#

When I started Black it was just me, and soon enough a few other people at Facebook. So we used their tools. We pasted code in Messenger sometimes, usually formatting results. It's got the triple backticks formatting although without syntax highlighting.

heavy heron
#

Oh does it really have triple backtick markdown

solid adder
#

Yes.

#

Now we don't have a good spot for pasting those results since this is a public channel and some of that code was internal, with the understanding that it's only to look at formatting mishaps or whatever.

heavy heron
#

Nice, I didn’t know that

solid adder
#

I guess we can make a group chat on Discord for that purpose if we need it πŸ˜„

#

And Discord devs are gonna be like πŸ‘€

bright glacier
#

Was thinking the same thing (although I half expected us to continue use the messenger chat for strictly internal comms)

heavy heron
#

Yeah, sadly there isn’t really a way to make a private channel in this server because of how the permission system works

solid adder
#

Don't worry about that, if we really do need it, the group chat option is not half bad. And it's better to have both here instead of separate apps.

#

I hate having 17 messaging apps on the phone.

carmine parcel
halcyon latch
solid adder
plain atlas
mint barn
#

true

prime gull
#

yeah slack style threads are set to release on discord in the nearish future

halcyon latch
# solid adder What about them drives you bonkers?

Because no one uses them correctly, in slack you can't expand it, so any long convos in there are pure hell, and they don't really in my experience contain conversations but just split them awkwardly

solid adder
#

So, don't get me wrong, Slack and Discord are for ephemeral messaging. This won't ever be a knowledge base and making it totally orderly like a forum or wiki is impossible. Then again, I'm in multiple Slacks right now where threads are being used sensibly. So maybe your particular community doesn't find it a good fit, but it's not universal.

#

In particular, in the EdgeDB internal Slack it's mandated to only post long intro posts straight in the channel, everything else needs to be in a thread. And once you get used to it, it works pretty well. #random is an exception, as it is everywhere I'm at.

#

The Polyend Slack is less rigid but there also things that start off conversations that would be considered "off-topic" are very often threaded and it works well.

#

A few of my friends and I also have a "PyCon in absentia" social Slack where threading is used for "I liked this detail of what you said here and want to talk more about it without disrupting the flow".

#

On Discord it's great but look at my wall of text here and suddenly...

solid adder
#

and now at this point it's not clear whether the conversation should be continuing on what was said before the reply... or whether the reply should continue. On this particular Discord server I see people usually continue the "main" topic and the reply dies out. Unless somebody replies to a reply. This is an ad hoc thread.

heavy heron
#

I find replies to replies a decent enough way to have threaded conversation. At least we can have two or three conversations at the same time in one channel in off-topic without tooooo much issues

flat krakenBOT
tribal thistle
#

I'm working on making the --code option use reformat_one to fix it's behavior

#

Right now I'm stuck trying to figure out how to handle the stdin_to_stdout func

#

The part I'm struggling with is getting the encoding & newline, since I just have the content as a str, an empty buffer I can't read from, and a closed buffer

#

For encoding, I'm thinking I can just ignore that and force use utf-8

#

But not sure what to do for line endings besides checking the system os

late dewBOT
#

src/black/__init__.py line 760

src, encoding, newline = decode_bytes(sys.stdin.buffer.read())```
tribal thistle
solid adder
#

Taking a step back: what's wrong with the current way --code is handled?

tribal thistle
dense jungle
#

It ignores most options and safety checks. I think one of the reason I added it is for quick testing of changes in black itself, and then it's bad that it doesn't do the AST safety check

tribal thistle
#

In an attempt to make it more like the other formatters, it's become quite bloated and repetitive

#

And pretty similar to other formatters, so I'm trying to get it to just use the other formatters

#

The only problem I'm running into is I can't get the newlines from the code, it's passed in as just a string

solid adder
#

So good news is that newlines are sniffed for files just the same, just not sure whether the API allows for bytes input and I'm on the phone right now

#

But it shouldn't be a big stretch to replicate the same behavior for a string.

#

In terms of encoding, a string is already pre-decoded while bytes are raw. I'd have to check which type you get from --code. If it's string, don't worry about it.

plain atlas
bright glacier
late dewBOT
#

src/black/__init__.py line 349

code: Optional[str],```
solid adder
#

Good, because if it's bytes, there's an issue: the file content might specify a pragma with encoding (# -* coding: utf8 -*- and friends). But what's given on the command line might not be in that encoding.

plain atlas
tribal thistle
#

Seems the textIOWrapper can handle itself just fine without newlines anyways

plain atlas
#

why are there 35 different tests?!

tribal thistle
#

So forcing encoding to utf-8, and forcing newlines to None should cover everything I think

tribal thistle
plain atlas
#

omg

#

because it tests and builds on every plat

#

for version 3.6 3.7 3.8 3.9 wow

bright glacier
#

8 primer jobs, 12 tests jobs, 1 lint job, 4 fuzz jobs (or 3 IDR), one doc job, one changelog check

plain atlas
#

what's a fuzz job?

#

coveralls?

bright glacier
#

oh yeah, unrelated but we also get 5 more statuses from coveralls

#

but fuzzing is basically throwing a bunch of different input to some piece of code and seeing if it fails

plain atlas
#

πŸ™‚

#

And the primers?

bright glacier
#

for us, we count a failure as an either parse failure or a format failure (instability or invalid code generated)

plain atlas
#

hm, do you have tests that intentionally pass invalid code?

bright glacier
plain atlas
#

ah

#

but its a check

bright glacier
#

GitHub provides an API for status checks

#

So as GitHub Actions Cloud runs the Test workflow jobs, one of the steps is uploading the collected coverage data to Coveralls. Once Coveralls gets that data and processes it, it (probably) uses the API to create a status check for the commit / PR. So yeah.

#

Primer (which is actually calling black-primer) is one of our dev tools. Primer allows us to run Black over a huge array of Python projects and their code. We use it to determine differences in formatting between one revision and another one.

bright glacier
plain atlas
austere lava
tribal thistle
#

It's not too late haha

bright glacier
#

I actually haven't seen an usage of --code out in the wild before

neon loom
#

is a BOFH at heartz

bright glacier
#

I wonder whether replacing the IRC link in the left sidebar with a link to a "Contact" page would be better.

#

If we hide the contact page from any TOC (i.e. make it an orphan page) and only link to the version of latest, that would mostly avoid the issue of outdated contact info in older releases of the docs

#

That may be overly complicated tho

bright glacier
#

writing this gh action is painful

#

I know what to do, but y'know, obstacle after obstacle πŸ™ƒ

dense jungle
#

minimal(ish) repro of the sqlalchemy crash with --experimental-string-processing: ```
def _legacy_listen_examples():
text += (
" "listen for the '%(event_name)s' event"\n"
"\n # ... (event logic) ...\n"
% {
"since": since,
}
)

#

going to spend some time trying to fix it now

flat krakenBOT
bright glacier
#

oh look, an issue missed my giant close wave πŸ˜„

#

(for context, I closed a ton of issues when Łukasz finally resolved the magic trailing comma instability)

flat krakenBOT
#
JelleZijlstra
[psf/black] Issue opened: #2271 \`\-\-experimental\-string\-processing\` crash on string \+ % \{ 
% cat legacy.py                                   

def _legacy_listen_examples():
    text += (
        "    \"listen for the '%(event_name)s' event\"\n"
        "\n    # ... (event logic logic logic) ...\n"
        % {
            "since": since,
        }
    )
% black --experimental-string-processing legacy.py
error: cannot format legacy.py: Apparently, old parentheses do NOT exist?!
Oh no! πŸ’₯ πŸ’” πŸ’₯
1 file failed to reformat.

Minified from a file in sqlalchemy that we...

dense jungle
#

Felt like I had to close at least one issue before opening a new one πŸ˜„

flat krakenBOT
bright glacier
#

Haha, you will be free from the guilt of making this line go up then :p

dense jungle
#

I made it go down now!

bright glacier
#

Can confirm that the line is now down by one, also just looked into the data and on that day I closed around ~65 issues 🀯

flat krakenBOT
bright glacier
#

OK this is getting out of hand, Jelle is pulling ahead (in a not serious contest I made up) in their count of issue closures πŸ˜‘

flat krakenBOT
bright glacier
#

...

dense jungle
#

Now this is a challenge

bright glacier
#

You're now trying to steal the title of "most active issue triager" (which btw, I don't actually mind, cause it's dull πŸ˜… ) huh?

#

And honestly I think I already lost that title a while back

flat krakenBOT
bright glacier
#

In all seriousness though, this is great @dense jungle , better done than never (if was up to me, I would never respond to them)

tribal thistle
#

Damn that’s some serious triaging

#

poke @vivid pasture we should take notes, close everything πŸ‘€

flat krakenBOT
bright glacier
#

10/10 highly approved

#

(I'm disclaiming all liability of injury in case a serious inter-team fight breaks out)

vivid pasture
#

Currently a little burned out of programming, but def will still take notes :D

bright glacier
#

The only notes you'll need is "does it have a red magical close bottom? yes? press it hehe" /s /j

flat krakenBOT
bright glacier
#

I'm impressed.

flat krakenBOT
dense jungle
#

@bright glacier do you know if we any docs that say that after reformatting with Black, you may have to adjust magic comments in some cases (e.g., # noqa, # type comments, etc.)?

bright glacier
#

Nothing comes to mind quickly so I'm assuming no 😦

dense jungle
#

Let me add it to the FAQ

bright glacier
#

πŸ₯°

#

Now Jelle only needs one more closure to beat Zsolt πŸ˜„

jelle-is-killing-it.png
flat krakenBOT
bright glacier
#

in announcer voice: whelp, zsolt has fallen to jelle's triaging might

flat krakenBOT
bright glacier
#

OK, y'all were all right that this wave of notifications of issues being closed is satisfying as heck

flat krakenBOT
bright glacier
#

I didn't believe it when I was responsible of such wave, but now on the receiving end, I must concede the annoying bits are highly outweighed

flat krakenBOT
tribal thistle
#

Damn at the rate these bugs are being closed, there might be time to actually work on new features /s

bright glacier
#

I'm trying to join in the fun

flat krakenBOT
bright glacier
#

my goal is to close at least one issue tonight

flat krakenBOT
dense jungle
#

To be fair I'm closing a bunch of them based on your comments πŸ™‚

flat krakenBOT
bright glacier
#

YAY

tribal thistle
#

Ooh I found 1801

#

Review my PR, get two birds with one stone

#

Not advertising my own PR or anything

bright glacier
#

what contest did I start...

flat krakenBOT
dense jungle
bright glacier
#

Probably and I'm taking of the advantage to gain one more closure so sure

dense jungle
#

it's yours πŸ™‚

flat krakenBOT
tribal thistle
#

psf/black#898

toxic stormBOT
tribal thistle
#

Seems resolved

flat krakenBOT
tribal thistle
#

Oh I reached the end of the list

dense jungle
#

I'm always a bit scared of vim plugin issues

#

But I guess you're right

flat krakenBOT
dense jungle
#

There's also a lot of deduplication to be done on the "Design" issues. People often ask for the same changes

bright glacier
#

Yep, I've avoided doing that task because I am scared of missing a tiny detail here and there πŸ˜…

flat krakenBOT
bright glacier
#

We did well! @dense jungle

Screenshot_from_2021-05-29_00-10-44.png
dense jungle
#

now down to 100 open issues

flat krakenBOT
dense jungle
#

I mean we should go down to 100 open issues. There's still like 340

flat krakenBOT
tribal thistle
#

Still on that streak haha

#

Actually pretty cool

flat krakenBOT
bright glacier
#

Ok, I gotta go now, that was fun @dense jungle πŸ˜„

dense jungle
#

I'm going to close a few more hopefully πŸ˜„

flat krakenBOT
dense jungle
#

^ this one has the best repro case.

flat krakenBOT
tired shard
#

πŸ‘€ gosh that;s lot of triage

flat krakenBOT
dense jungle
#

That's the last one for today. Only 318 issues left

bright glacier
#

(had to check before truly disappearing for the day): good work

#

I'm going to wake up to a very full GH inbox (I ignored it while you did your thing)

neon loom
#

Wtf you all do to my inbox? Haha

#

Nice work.

errant barn
#

Whew! What a sight to wake up to πŸ˜„ nice!

flat krakenBOT
flat krakenBOT
#
DrGFreeman
[psf/black] Issue opened: #2274 Docstring new line reformatting can violate line length contraint

Describe the bug

Removal of new line in docstrings can violate the line length constraint if the last line is with three characters of the max line length value.

To Reproduce Steps to reproduce the behavior:

Run black on the following code with the default line-length constraint of 88 characters.

def f():
    """89 123456789 123456789 123456789 123456789 123456789 123456789 123456789 12345678
    """
    pass

Expected behavior

No reformatting i...

flat krakenBOT
neon loom
#

@dense jungle Do you think it’s only that a bug with SQLAlchemy for new string processing?

#

Anything I can help with (besides going over the PR soon)

flat krakenBOT
dense jungle
#

I'm not aware of any others. I pretty much gave up on actually fixing the crash for now. Seems pretty tricky

#

I think the person who originally contributed the string processing has been pretty quick in submitting fixes in the past, maybe we should try to pull them back in

neon loom
#

Want to email and ask to come here to discuss or see if they understand the bug?

#

Maybe explain on the issues actually so majority of the conversation goes there. Maybe even a tag there will do?

#

*issue

dense jungle
#

yeah let me do that once I review this other PR πŸ™‚

neon loom
#

I love these Lemon emojis. Haha

#

I’ll merge your docs PR. Anything else we all want in before a release?

flat krakenBOT
dense jungle
#

my other docs PR? πŸ˜„

neon loom
#

The code flag options?

dense jungle
#

that'd be nice too

neon loom
#

Let me see how happy people are with it

#

If it’s just typing want to merge and fix the typing?

#

Or we could a push fix to the PR

dense jungle
#

Yeah I can apply my suggested change

neon loom
#

FWIW - I agree

#

We have an anti-pattern in FB where people do β€˜import typing as t’

#

It makes my eyes bleed

dense jungle
# 
a + b
reformatted <string>
All done! ✨ 🍰 ✨
#

feel like this would be better without the "All done" and "reformatted" lines. maybe we should make -c quiet by default

neon loom
neon loom
dense jungle
#

Not quite because it does go to stderr

#

Just distracting

neon loom
#

O, then people can add -q if they want

#

But I agree with -q by default there.

#

I don’t know why, but could someone disable -q if we went -q default?

dense jungle
#

I guess -v would override it

neon loom
#

But then be more output?

dense jungle
# 
a + b
(black) jelle@mbpt-root black % black -v -c "a+b"
a + b
reformatted <string>
All done! ✨ 🍰 ✨
#

Not that much more output, it just makes it also print something if the input was unchanged. But -c always formats only one "file" so it's not going to be that much

heavy heron
#

Does Black accept -- as an input to say use stdin?

dense jungle
#

- but yes

neon loom
dense jungle
#

Huh there's still a difference: black - picks up a pyproject.toml file but black -c doesn't

neon loom
#

Yeah that should be consistent

heavy heron
#

Isn't -- the standard over -?

bright glacier
#

oh for goodness sake

#

why was this even more complicated

neon loom
#

Is there a β€œam I Unix compliant” CLI tool tester? That would be handy too.

#

(Sorry windows people - I’m super bias)

#

I doubt it since it’s such a loose standard

dense jungle
heavy heron
#

Oh

#

Right

bright glacier
#

well most of it, the mode is still in main but like that shouldn't matter

dense jungle
#

I might need to look into it more. What I saw is that black -v - prints a line about pyproject.toml being found and black -c doesn't

bright glacier
#

Also, good mornin'

#

I was going to say I want to get in that action pr

neon loom
bright glacier
#

But like it's been so painful that waiting for me is probably a bad idea

neon loom
#

I’m happy to wait for this one to get into the release. It’s handy and overdue on our side.

#

So don’t rush.

bright glacier
#

I gave up on the tests, just need to add even more hacks for Windows

neon loom
#

We can make em posix is only …

#

*OS

#

@jagged wasp() decorator

bright glacier
#

It's fine, it's just I need to rewrite my entry point script that is supposed to handle the windows

neon loom
#

cpython itself does this πŸ˜„

dense jungle
#

confirmed the pyproject discrepancy:```(black) jelle@mbpt-root tmp % black -c "a = 'b'"
a = "b"
(black) jelle@mbpt-root tmp % echo "a = 'b'" | black -
a = 'b'
All done! ✨ 🍰 ✨
1 file left unchanged.
(black) jelle@mbpt-root tmp % cat pyproject.toml
[tool.black]
skip-string-normalization = true

neon loom
#

(Where it makes sense - e.g. pathlib)

bright glacier
dense jungle
#

I think it's because we pass the value of the src param to find_pyproject_toml and I guess - is enough for it to find the file

neon loom
#

AFK for 30 or so.

bright glacier
#

I would have expected the search path to default to Path.cwd()

dense jungle
#

Nope it defaults to /

bright glacier
#

I don't know tho, I need to get on my laptop first before claiming I know anythingπŸ˜†

dense jungle
#

So I guess you could put a pyproject.toml at the root of your filesystem and affect -c

bright glacier
#

ouch

dense jungle
#

Now, should we use pyproject.toml when using -c/stdin? I'd say yes because that lets you test how code would be formatted in your project

bright glacier
#

I suppose the user config is also ignored by -c?

#

(I almost forgot we added that feature)

dense jungle
#

I think it would still use that

#

As long as /pyproject.toml doesn't exist

flat krakenBOT
bright glacier
#

Alright, cool now time to go through my GitHub Notifications and see what replies we got after that closing wave

#

I also need to regenerate those graphs again πŸ˜„

dense jungle
#

Not that many actually

bright glacier
#

I saw one talking about a CoC violation which certainly did not happen, so I'll mark it as spam

#

Or at least respond to it in some manner

dense jungle
#

I can see how that issue could be seen as a CoC violation. Don't really want to engage with it though

bright glacier
#

Yeah, I suppose, still makes me uneasy

flat krakenBOT
bright glacier
#

The noy issue is an interesting read haha

bright glacier
#

IIRC we do and I'm currently looking for it

#

Spoke way too fast, already found it lol

solid adder
#

Mike Bayer just wrote an INCREDIBLE unprompted testimonial for Black

#

Look: https://twitter.com/zzzeek/status/1398661499288272902

mike bayer (@zzzeek)

@chriswithers13 @chrismcdonough @hynek I dont think I could overstate how much productivity Black has enabled us to have over here. I can now do refactorings in about 1% of the keystrokes that it would have taken me previously when we had no way for code to format itself.

Twitter Β·
#

https://twitter.com/zzzeek/status/1398663464709656580?s=20

mike bayer (@zzzeek)

@chriswithers13 @chrismcdonough @hynek with commits like that, it's not just the hand strain of manually formatting all that, it's the eye strain of visually looking at all the spacing, the mental strain of deciding how to format every line, I've wanted that automated for years. why woud someone not want that?

Twitter Β·
bright glacier
#

The testimonials we have in the README looking a bit weak now πŸ˜†

tribal thistle
#

Damn that’s pretty amazing lol

#

Those other comments make me think we may have world peace if python only had one type of quote 😬

dense jungle
#

Nah the tabs people would still declare war

bright glacier
#

Nah, the tabs vs spaces debate was intense

tribal thistle
#

Hahaha

bright glacier
#

Also the sadface was pretty inflammatory

solid adder
#

the entire tool is controversial but that's part of the appeal.

bright glacier
#

Peace is too much to ask for here

solid adder
#

it's a holistic solution. some would rather say "unholy" but that's subjective πŸ˜‰

flat krakenBOT
neon loom
neon loom
#

Thus, I like black.

solid adder
#

which issue are we talking about?

neon loom
#

Bah asottle asking for emojis to be changes cause their remind him of someone …

dense jungle
neon loom
#

It was a silly reason for the request imo

flat krakenBOT
solid adder
#

✨ 🍰 ✨ is now more recognizable as a Black thing rather than a KR thing.

#

Thanks for closing it, and it's a wise decision not to get further involved.

neon loom
solid adder
#

There is a big problem with it

#

Originally Black wasn't looking at pre-existing formatting at all, applying consistent styling altogether. Then I caved and changed behavior to leave some vertical whitespace intact, and to this day we have a problem of the fact that this is some and not all vertical whitespace (see: imports).

#

Then I caved and we did the magic trailing comma, rest is history, it was a big problem for a long while with both stability and usability. And to this day it's surprising some users.

#

The same will happen when we cave to leave some parentheses intact but not others. It will surprise people, others will want previous behavior when stuff is formatted automatically more radically. We might introduce another magic trailing comma disaster if the same issue happens:

  • Black leaves existing parentheses;
  • but sometimes it puts parentheses by itself;
  • those same parentheses are now treated as pre-existing on another run of the tool.
neon loom
#

I agree, but in an ideal world we should just change this to be default imo ...

#

It's way more readable

#

We're still beta ... Is it worth it to make this consistent?

#

I won't push hard, but I do like this and do write it this way to have black change it a lot.

#

I just live with it ...

#

But if there is a chance not to

solid adder
#

In the ideal world, knowing what I know now, we would never put or remove parentheses automatically.

#

Is it too late to remove this now? Hard to say.

solid adder
neon loom
#

You were hesitant then if I remember correctly too

#

On another side not, I started using usort with a project yesterday - John and Tim did a good job: https://github.com/facebookexperimental/usort (Non configurable isort that is designed to work with black)

GitHub
facebookexperimental/usort

A safe code formatter that sorts python imports. Contribute to facebookexperimental/usort development by creating an account on GitHub.

#

*note

#

I love it's simplicity (externally - haven't read the code at all)

bright glacier
solid adder
#

well, we already agreed it's time to declare stable

#

the remaining things to be done are mapped out

flat krakenBOT
solid adder
#

I'd actually turn on experimental string processing right now as there are no other formatting changes save for not inserting spaces into empty docstrings

#

so it is really the only consequential formatting change

#

other changes are gitignore this, gitignore that, details

neon loom
#

We have to one bug to fix. I vote to enable it after a release today or tomorrow ...

#
  • In it's own release, as I expect the "Why my strings changing" issues coming for this one
solid adder
#

releases are now cheap to make but they are not cheap to adopt

#

people sometimes don't re-pin for long periods of time

rancid abyss
#

pre-commit autoupdate automatedis a big help,

neon loom
#

That's out of our control and peoples (imo poor) decision to pin πŸ˜›

rancid abyss
#

there may be a need for a lint helper tho

bright glacier
#

\me wonders how the version usage graph is looking now, it's been a while

rancid abyss
#

pinning is a good decission, not having regular updates however turns it rotten

solid adder
#

you know this graph too well. we only overtook 20.8 with the latest release last week

Screenshot_2021-05-29_at_18.46.07.png
neon loom
#

I see both sides. A lot of people don't have the automation to say "hey there is a new verson" - If you have that, I like pinning.

#

So I include that in my hating of pinning more than pinning itself

rancid abyss
#

these days the general suggestion should be to jsut use black with pre-commit and have a automated regular autoupdate (for example via pre-commit.ci)

dense jungle
#

For Black specifically pinning makes sense so you don't have to suddenly change a bunch of formatting when you want to make an unrelated change

rancid abyss
#

at work we have a job with warnings that goes yellow every time a pre-commit update is need , and recently added a gitlab bot to do the rest

neon loom
rancid abyss
#

the key and major issue that makes pinning a pain is staleness - so pinning abslutely requires to set up automation or it will cause pain

neon loom
#

^^ This is basically what I say, just better phrased.

rancid abyss
#

a key reason why i wholeheartedly support/appreciate anthonys pre-commit.ci - it created pre commit pin upgrades for all of the pytest projects on a regular shedule that fast paced enough

neon loom
#

O, maybe I need this CI for bandersnatch and I'll dislike pre-commit less

#

My hate is i have requrements.txt automated and when I last looked long ago updating pre-commit via CI didn't exist and I hadn't looked at doing it yet

bright glacier
#

You can also get rid of the lint job IIRC

neon loom
rancid abyss
#

pre-commit has a autoupdate command that typicall works out, so if you have a pr creating updater surrounding things, then you can just use that typically

neon loom
#

Yeah, I know how to do that, but was just annoyed I needed a sepereate ting when I already have requirements.txt updating - but i get the advantages / disadvantages

bright glacier
#

Hopefully Ronny can correct me if I'm being an idiot but pre-commit.ci pretty much does takes over the job of the lint job

flat krakenBOT
tribal thistle
#

Could someone rerun tests on/force merge that PR ^

flat krakenBOT
tribal thistle
#

sweet

flat krakenBOT
tribal thistle
#

@dense jungle I'm looking into the comment about pyproject being loaded incorrectly

#

That seems to be something to do with click or the way the options are defined

#

I ran the same project with both options, and at the very beginning of the main function, config would be None for --code, and the pyproject for anything else

#

This is before any thing done by the function

dense jungle
#

Yes, it's done by the click callback. I think my comment fully explains what's going on

tribal thistle
#

Do you think this should be fixed in this PR, or a follow up?

neon loom
#

Should just need a rebase now right?

dense jungle
#

@tribal thistle Might as well fix it now; it's part of the same idea that -c and - should work the same

#

@neon loom no this is a diffferent issue

neon loom
#

kk

#

My bad

bright glacier
#

Aftermath of yesterday's closing frenzy πŸ™‚

blackissues2.png
dense jungle
#

We can do better than that

bright glacier
#

haha πŸ˜…

neon loom
#

Yeah, I might have a dig tomorrow - Off to a BBQ soon. But yeah would love to get us to < 100

bright glacier
#

You got some work to do πŸ˜‰, Jelle and I are unofficially fighting for most issues closed (i'm only half kidding)

cooper-has-some-work-to-do.png
#

Look at your pink line so far away from us

#

Enjoy your BBQ though!

heavy heron
#

A different message for every new contributor? That's so sweet! I'll make sure to comment on the issue once I'll have an actual code contribution as I wouldn't have much to say right now :P

#

Also bon appΓ©tit !

bright glacier
#

I shamelessly stole the idea from a blog post I read a while back πŸ˜›

heavy heron
#

Haha, that isn't a bad idea

#

Maybe you should also pin the issue?

bright glacier
#

Maybe ... didn't really think of that

heavy heron
#

You could also pin it on discord honestly :)

bright glacier
#

Also, I have a base message for my thank you: Thank you so much for your contribution! This project is only possible by contributions like these πŸ–€. You're awesome, @

#

But I make sure to add a personal touch specific to the contributor and/or contribution

#

Makes it way easier to be consistently welcoming tho, so it's IMO worth it

tribal thistle
#

Finished writing tests for the path resolving bug

#

I'll push it now, it's a pretty small fix

#

If y'all still want to push this in the next release, should be possible

violet echo
#

is there any plans to support "incremental" file formatting? often i want to use black while editing code to reformat some snippet that i'm working on, without reformatting the entire file. this naturally fails on anything indented. i could write a wrapper script to un-indent code and then pipe it to black -q, but it'd be handy if this were a feature in black itself

#

is this a "pr accepted" situation, or a "no thanks, we don't want to support it" situation, or something else?

dense jungle
#

There are issues about it and I think some third-party tools

#

Feel like this would have to be mostly implemented in third-party tools like editors

#

If your snippet is valid Python code by itself, you can probably just do deindent + black -c + reindent

austere lava
#

woohoo I have a peg parser that can parse

@hello
def f(): ...
#

including all the whitespace

muted haven
austere lava
#

yep

#

it also doesn't allocate any new strings after reading the source text, all CST nodes refer to slices of the original string

solid adder
#

this sounds very promising

#

final version when eta

#

(Don't look at me like that, I learned to talk like this on this Discord)

bright glacier
#

@neon loom finally got the action pr ready

flat krakenBOT
bright glacier
#

"bashslashes" lol

#

#typo

plain atlas
#

ah yes

plain atlas
#

i love those

#

I once merged a minor changes pr to my projects with 285k edits

#

and that's why the github language breakdown says the project is entirely css

bright glacier
plain atlas
#

oh god

#

why is that pr

#

every single whitespace known to main

plain atlas
bright glacier
#

I don't know, it is from like 10 years ago tho

plain atlas
#

my browser is so slow from that

austere lava
bright glacier
#

so 3 years huh?

austere lava
violet echo
flat krakenBOT
flat krakenBOT
bright glacier
#

@dense jungle closing more again?

#

or just one off closure?

dense jungle
#

Probably will be some more. I am trying to put at least one label on all issues

#

And then look at some interesting labels for more cleanup

bright glacier
#

cool, I might be able to help out πŸ™‚

flat krakenBOT
bright glacier
#

I was about to check that one @dense jungle πŸ˜‘

#

and unzip the zip lol

dense jungle
#

too much work

bright glacier
#

I suppose you went the route of "unlabeled"

#

I'll leave those allow then

#

alone*

dense jungle
#

plus what if it's a social engineering attack to put a virus on a maintainer's computer

flat krakenBOT
bright glacier
#

aren't they inspectable?

#

e.g. zipinfo file.zip

#

although there are probably ways to get around that

dense jungle
#

no unlabeled comments left πŸ™‚

flat krakenBOT
bright glacier
#

I'm leaning towards no because it feels too niche to be worth providing in the core tool

#

I'm asking because there's a similar issue (although it appears they want line granularity too): psf/black#1516

toxic stormBOT
dense jungle
#

I'm not too enthusiastic about it, but I guess some people need it

flat krakenBOT
bright glacier
#

is it just me or is copying text on gh gotten worse?

#

when I double click, I copy a bunch of HTML instead of text :/

flat krakenBOT
dense jungle
#

yeah I get that occasionally too

flat krakenBOT
bright glacier
#

"one day Black will have an API"

#

I love it so much

#

nice new labels, they are helpful additions πŸ™‚

dense jungle
#

current project is looking through "T: enhancement" for issues with no other labels

bright glacier
#

O, I didn't notice until now, but we went sub 300 πŸŽ‰

dense jungle
#

πŸŽ‰

bright glacier
#

My project is still "just scan the issue tracker for good candidates for closing", it works shockingly well, although I think most if not all of the easy to spot issues are closed already so I should probably adopt a more focused strategy πŸ˜„

flat krakenBOT
bright glacier
#

Currently looking at some of the probably heavily duplicated design issues

flat krakenBOT
dense jungle
#

I was trying to gather courage to do that, I feel like there's a bunch of things that keep getting requested

#

Might be worth adding some labels for specific design requests

bright glacier
#

What makes this annoying is that not all issues are lablled correctly, a lot of code style changes are labelled enhancement when in reality they should labelled with design and vice versa

dense jungle
#

I fixed a few of those πŸ™‚

flat krakenBOT
dense jungle
#

What do you think of adding a label for cases where Black doesn't obey the line length limit?

bright glacier
#

seen enough cases to be worth it I suppose

dense jungle
#

added F: linetoolong

flat krakenBOT
bright glacier
#

got one duplicated design issue at least πŸ™‚

flat krakenBOT
dense jungle
toxic stormBOT
flat krakenBOT
dense jungle
#

also I'm going to add F: linebreak for all the complaints about how we split lines

bright glacier
#

good call

bright glacier
#

it basically comes to down to favouring the RHS or LHS although the cases where this debate happens are limitless

flat krakenBOT
bright glacier
#

@dense jungle running my issue / pr data scripts again

#

in this time period, we made modifications to 219 issues 🀯

flat krakenBOT
bright glacier
#

and closed ~43 today

dense jungle
#

nice, we undid most of 2020

bright glacier
#

Between the two of us, I think it's safe to say we probably checked each issue in the tracker at least once πŸ™‚

#

today

#

insane

bright glacier
#

It's getting quite a bit harder to find issues to close now.

#

Most of issues left to close are probably design / enhancement issues.

#

I think we checked the bug issues pretty well by now.

dense jungle
#

yes. maybe there are some hiding under labels I didn't check

#

I tried to put an F label of some sort on every design issue

bright glacier
#

Thanks!

#

I gotta go again like yesterday. Pleased as heck though.

flat krakenBOT
#
PeterJCLaw
[psf/black] Issue opened: #2279 Inconsistent wrapping for long call chain

Describe the style change

Method chaining is inconsistent when the overall expression is a call (rather than, say, an assignment).
It would be great if black were consistent in its wrapping such that it either chained horizontally or vertically, but not both within the same expression (at least within the same level in the call-nesting of an expression).

Examples in the current Black style

def func():
    thing("argument-1").thing("argument-2").thing("argum...
flat krakenBOT
#
nat-n
[psf/black] Issue opened: #2280 Valid toml syntax causes black to crash

Describe the bug

I have a pyproject.toml containing a list containing both strings and inline tables, similar to the following example from the v1.0.0 of the toml spec.

contributors = [
  "Foo Bar ",
  { name = "Baz Qux", email = "bazqux@example.com", url = "https://example.com/bazqux" }
]

When I run black I get a cryptic traceback from an error inside the toml library. I'm not using the pyproject.toml for configuring black.

To Reproduce

mkdir ...
flat krakenBOT
abstract owl
#

what is GitHub used for? for codes for projects?

bright glacier
#

At its core GitHub serves as the primary git repository for many F/OSS projects.

abstract owl
#

okay, getting the idea a bit

bright glacier
#

Although it has many many many other services that aide in development too, like issues trackers, pull requests, account and organization management

#

CI/CD platform

abstract owl
#

oh, cool

flat krakenBOT
#
enzet
[psf/black] Pull request opened: #2281 Possible fix for \#2254

Not sure the fix is right. Here is what I found: issue is connected with line

first.prefix = prefix[comment.consumed :]

in comments.py. first.prefix is a prefix of the line, that ends with # fmt: skip, but comment.consumed is the length of the " # fmt: skip" string. If prefix length is greater than 14, first.prefix will grow every time we apply formatting.

bright glacier
#

AFAIK @dense jungle there's actually another issue with --experimental-string-process: psf/black#1469

toxic stormBOT
dense jungle
#

Sounds like that fails even without --esp?

bright glacier
#

we don't touch f prefixes for strings in normal string processing

#

so --esp only bug

dense jungle
#

Oh right

#

Yeah just confirmed locally it crashes with esp only

#

You're kind of asking for it if you use nested f-strings though

bright glacier
#

yeah, pretty unreasonable

flat krakenBOT
#
bbugyi200
[psf/black] Issue opened: #2284 The \-\-experimental\-string\-processing flag does not split long strings that start with \`==\`

What is this feature supposed to do exactly? I tried it and it transformed the following string, which seems unideal to me, since it breaks the max line length of 88 by a large margin.

I'm using black version 21.5b0.

# input code:
assert str(suffix_arr) == (
    "['$', 'angaroo$', 'angrykangaroo$', 'aroo$', 'garoo$', "
    "'grykangaroo$', 'kangaroo$', 'ngaroo$', 'ngrykangaroo$', "
    "'o$', 'oo$', 'roo$', 'rykangaroo$', 'ykangaroo$']"
)

# black's output code:
ass...
bright glacier
#

Looking into that regression Jelle FYI

dense jungle
#

Thanks!

bright glacier
#

was literally a one line fix, now to write the changelog entry and regression test πŸ˜„

flat krakenBOT
bright glacier
#

Good catch Jelle, I'm surprised no tests failed

tribal thistle
#

Probably a good sign tests should be updated haha

bright glacier
#

Yeah definitely

#

the only issue is that I don't actually know how to test this part of the codebase

dense jungle
#

Yeah I guess it might lead to crashes with an uppercase F-string with escaped braces in it?

bright glacier
#

probably

dense jungle
#

There's also one on line 544 with r. Might make it merge an uppercase R-string with a normal one

bright glacier
#

yeah, I'll fix them all (using Ctrl+F) but I need to figure what to do for testing

flat krakenBOT
bright glacier
#

Ugh it appears some of the regressions my code would've introduced are actually avoided because the prefixes are already lowercased once the string transformer runs

#

I would imagine changing the order of transformers would make those regressions actually start crashing things

#

Although I got a feeling the order is important

#

If feasible, I'll add cases for all 5 instances in trans.py but they'll probably pretty useless πŸ™‚

#

Although, not useless enough to be worth not writing haha, if we ever rewrite the project using a new parser, we will want all of the regression tests possible

#

Actually, everything I just said might be wrong

#

The prefix lowering that happens before the string transformer run is in the AST visitor IIRC so there's practically no chance of ever actually hitting a case where the F prefix would stay capitalized

#

The only safe conclusion here is that this string code is new to me and I have a lot to learn πŸ™‚

flat krakenBOT
solid adder
#

@bright glacier do we lower-case raw-string prefixes?

bright glacier
#

We currently do for docstrings which is what I'm trying to fix

solid adder
#

yeah, there was GH-1244 about this

bright glacier
#

it's a regression 😦

solid adder
#

we shouldn't touch upper-case R""

bright glacier
#

get_string_prefix unconditionally lowercases the extracted prefix before returning, unfortunately some of codebase depends on this behaviour while others break on it :/

solid adder
#

ah, apart from --experimental-string-processing we only did it for docstrings which is inconsequential

#

this is how we've done this all along so that's not a regression on its own

flat krakenBOT
dense jungle
#

^ trying to understand the regex in that PR. What does (?R) mean? I can't find it in the re documentation

#

Oh it's a regex thing

#

A recursive match

bright glacier
flat krakenBOT
dense jungle
#

Thanks, I'd never seen it before

flat krakenBOT
bright glacier
#

Me neither actually

neon loom
#

Damn - So I guess I could do the PR to try and run primer on sqlalchemy again with --experimental-string-processing

#

Great work. I was out paddle boarding and I come back to just win after win.

bright glacier
#

I suppose excluding tests/test_runner_apps/tagged/tests_syntax_error.py for django is necessary in primer.json

#

if we do that AND sqlalchemy passes, then we can close psf/black#1446 πŸ˜„

toxic stormBOT
bright glacier
#

I'm still eager to close more issues haha

#

our biggest chunk of open issues is "design" atm

#

doesn't surprise me at all

#

unfortunately they also happen to be issues that are annoying to deduplicate

neon loom
#

If they are hard to repro - ask for an easier repro or it won't be considered imo ...

#

If people really want it they will reopen etc.

bright glacier
#

design issues are about changing the style so I'm confused by asking for a repro

neon loom
#

Sorry - I am tired and read deduplicate completely wrong

bright glacier
#

we did deduplication for bug type issues pretty welll

#

we only have 70-80 of that type

dense jungle
#

Yes, the crash issues are pretty well organized now. There's only a handful

#

Most of the "bugs" are also formatting complaints, not clear bugs

flat krakenBOT
bright glacier
#

oh wow you marked a ton of issues with linebreak

#

it felt like there was a lot of those sort of issues but there really is a ton of them

dense jungle
#

yes, probably a lot are duplicates

bright glacier
#

@neon loom django timed out LoL

#

did not expect that to be the failure mode

sturdy dirge
bright glacier
#

Hi @sturdy dirge , congrats on your first project! Please note that this channel is specifically for the Black formatter project, so showing off your project (while may be sweet) is not acceptable in this channel. I'd suggest posting somewhere else. And if you're looking for feedback, a help channel may work wonders #β“ο½œhow-to-get-help. Thank you for understanding πŸ™‚

tired shard
#

You can share your projects and get feedback on them in #python-discussion, we don't have a "project showcase", etc. channel here.

bright glacier
#

Yeah, that's why I avoided suggesting any specific channels, wasn't sure if python general was an acceptable choice, good to know!

#

I went with the IMO safest option of "you can get feedback in a help channel" πŸ™‚

flat krakenBOT
#
fpdotmonkey
[psf/black] Issue opened: #2289 long blocks of unstructured text should be limited to 72 characters

Describe the style change

Per PEP8,

For flowing long blocks of text with fewer structural restrictions (docstrings or comments), the line length should be limited to 72 characters.

...

Some teams strongly prefer a longer line length. For code maintained exclusively or primarily by a team that can reach agreement on this issue, it is okay to increase the line length limit up to 99 characters, provided...

pure wren
#

πŸ‡§ πŸ‡± πŸ‡² channel

flat krakenBOT
bright glacier
#

I guess I should sneak in my action for the ESP by default release haha

#

I'll deal with the existing review comments in the free time I have right now once I get done with this other RL tas

neon loom
#

So, does this mean wait for relase?

#

*release

#

What's ESP?

bright glacier
#

no, it's fine, I can wait for the next release (which is soon for the experimental string processing right?)

neon loom
#

Ya, I would say in a week or so we turn that on by default and release

#

Or what ever you're all comfortable with

flat krakenBOT
bright glacier
#

I suppose give me like 30 minutes and if we can get the action PR done annd landed then great, if not, continue with the release

#

I don't want to block it that hard since it's just an action

flat krakenBOT
bright glacier
#

oh that works lol

#

it's such a fast workflow to make releases now

#

it's awesome

neon loom
#

pypi done. Docker and self contain binaries still building

bright glacier
#

Admittedly our releases also now cause less disruption since there isn't llike months between them lol

#

Did we test the updated binaries workflow?

#

I hope the PR author did. Although it wouldn't be the end of the world if it crashes.

neon loom
#

Surprised me how Linux is the fattest binary.

#

*fattest

bright glacier
#

whew

errant barn
#

whew :D

neon loom
#

3.9 sexy binaries too πŸ˜›

bright glacier
#

wonder if it's the regex binary

#

IDK, I have never looked into the binaries that deeply, just made sure they actually worked πŸ™‚

neon loom
#

Poor Docker builds are slow. Cause I have it do a qemu arm64 build

bright glacier
#

ah emulation, how fun

neon loom
#

I guess we could open a task to see if anyone has cycles to help us debug

errant barn
#

But in all seriousness, is there a way to test CI runs like that without actually releasing Black?

neon loom
#

Yeah, fork and test there

errant barn
#

Oh you're totally right, that didn't even occur to me πŸ˜…

bright glacier
#

For certain basic workflows, you might be able to get away with running them locally with act

neon loom
#

And there is some attempted container thing I've seen @bright glacier use right?

bright glacier
#

although unless you want to pull down some massive images, they aren't compatible with many GHA tools and workflows

#

also there is just some plain limitations with the fact GHA uses VMs while act only uses docker containers

#

It's been okay, I find the ergonomics not to be that great, but it is faster than pushing a change and watching GHA cloud do its thing

neon loom
#

Yeah, fork and test is the way to go imo.

#

But cool to know this exists

errant barn
#

Actually now that I had a look, the actions have been running all along. I was under the impression that forks were somehow at a disadvantage with that and started looking for some configuration :D nevermind and thanks!

bright glacier
#

Well, IIRC github disables GHA from running for forks because the workflows inherited from the parent might do things that aren't great to your fork

flat krakenBOT
austere lava
#

side effects yikes

errant barn
bright glacier
# 
.rwxr-xr-x 3.1M ichard26 ichard26 31 May 10:41 __main__
drwxrwxr-x    - ichard26 ichard26 31 May 10:41 _struct
.rw-rw-r-- 777k ichard26 ichard26 31 May 10:40 base_library.zip
drwxrwxr-x    - ichard26 ichard26 31 May 10:41 blib2to3
drwxrwxr-x    - ichard26 ichard26 31 May 10:41 importlib_metadata-4.0.1-py3.8.egg-info
drwxrwxr-x    - ichard26 ichard26 31 May 10:41 lib-dynload
.rwxr-xr-x  74k ichard26 ichard26  5 Sep  2019 libbz2.so.1.0
.rwxr-xr-x 3.0M ichard26 ichard26 27 Apr 20:37 libcrypto.so.1.1
.rwxr-xr-x  43k ichard26 ichard26 23 Mar  2020 libffi.so.7
.rwxr-xr-x 104k ichard26 ichard26  8 Aug  2020 libgcc_s.so.1
.rwxr-xr-x 2.3M ichard26 ichard26  4 May  9:10 libgfortran-2e0d59d6.so.5.0.0
.rwxr-xr-x 162k ichard26 ichard26 20 Apr  2020 liblzma.so.5
.rwxr-xr-x  31M ichard26 ichard26  4 May  9:10 libopenblasp-r0-5bebc122.3.13.dev.so
.rwxr-xr-x 5.4M ichard26 ichard26 27 Jan 10:41 libpython3.8.so.1.0
.rwxr-xr-x 263k ichard26 ichard26  4 May  9:10 libquadmath-2d0c479f.so.0.0.0
.rwxr-xr-x 253k ichard26 ichard26  5 Sep  2019 libreadline.so.5
.rwxr-xr-x 598k ichard26 ichard26 27 Apr 20:37 libssl.so.1.1
.rwxr-xr-x 192k ichard26 ichard26 26 Feb  2020 libtinfo.so.6
.rwxr-xr-x  30k ichard26 ichard26 21 Jul  2020 libuuid.so.1
.rwxr-xr-x  92k ichard26 ichard26  4 May  9:10 libz-eb09ad1d.so.1.2.3
.rwxr-xr-x 108k ichard26 ichard26 15 Oct  2020 libz.so.1
drwxrwxr-x    - ichard26 ichard26 31 May 10:41 numpy
drwxrwxr-x    - ichard26 ichard26 31 May 10:41 regex
drwxrwxr-x    - ichard26 ichard26 31 May 10:41 typed_ast
drwxrwxr-x    - ichard26 ichard26 31 May 10:41 uvloop
drwxrwxr-x    - ichard26 ichard26 31 May 10:41 zlib
#

here's a non single file binary I just created locally

#

and then some du output

❯ du -h . -d2
18M    ./uvloop
28K    ./importlib_metadata-4.0.1-py3.8.egg-info
15M    ./lib-dynload
240K    ./_struct/cpython-38-x86_64-linux-gnu
244K    ./_struct
4.3M    ./numpy/core
2.8M    ./numpy/random
100K    ./numpy/fft
240K    ./numpy/linalg
7.4M    ./numpy
2.3M    ./regex
136K    ./zlib/cpython-38-x86_64-linux-gnu
140K    ./zlib
80K    ./blib2to3/__pycache__
224K    ./blib2to3/pgen2
388K    ./blib2to3
2.4M    ./typed_ast
91M    .
#

I wonder if the fact my dev enironment is quite dirty is making this output useless

#

numpy probably shouldn't be there

#

I'mma make a clean environment and rebuild

#

(also my python setup on my machine is weird so take everything with a grain of salt lol

#

I wonder if we should add Python two code + windows coloured diffs support

#

yeah my environment was definitely dirty 😦

#

okay, so my built binary is 12 megabytes surprisely

#

Yeah, this is confusing, I think leaving this to the pros might be a good idea πŸ˜„

neon loom
#

Uvloop shouldn’t be in our binaries by default I feel too

#

Or numpy

bright glacier
#

that was my dirty environment

neon loom
#

Ok cool

bright glacier
#

I have a stupid amount of stuff installed in my dev env for black lol

neon loom
#

Bike ride time for me πŸ™‚ chat later

bright glacier
#

Ring ring, have fun!

neon loom
#

Im fat. Gotta fix that.

flat krakenBOT
lament crow
errant barn
#

Most likely not, you'd probably want to pin to an earlier version in that case

lament crow
#

mostly wondering because there is still some stuff that may cause Black to produce invalid code and some bugs with Black producing lines longer than length limit with ESP turned on

lament crow
bright glacier
#

Personally I'd be unlikely to accept such a flag, but I wouldn't block such a flag.

lament crow
#

Black's making more releases now so keeping up-to-date with its releases seems like a good idea. I'm not asking about it in terms of a new configuration option but rather a temporary measure as was --experimental-string-processing

#

but ye, pinning will have to do, thanks for the response

neon loom
#

Yeah, I agree with pinning, we want to have as little knobs as possible so this was designed to eventually turn away. We’ve been going pretty slow and trying to find as many bugs as we can.

#

Best thing to do is turn it on and try help find bugs πŸ™‚

lament crow
#

well, I did that, can't really PR fixes (at least currently) though

bright glacier
#

Reporting them is already a step forward πŸ™‚

lament crow
dense jungle
#

That sounds like the one bbugyi just submitted a PR for

dense jungle
lament crow
#

personally I'm most concerned with the above one and this one https://github.com/psf/black/issues/1468, I don't think I've seen any other in my code base that wouldn't be variation of it

GitHub
psf/black

The uncompromising Python code formatter. Contribute to psf/black development by creating an account on GitHub.

bright glacier
#

For context, we have been running black on quite a few big projects with ESP enabled and we haven't had many crashes lately. I think all projects format correctly except for one which has a file with broken syntax on purpose πŸ˜† It's what black-primer does BTW

#

But I do agree those issues should definitely be fixed before (at least the second one)

lament crow
#

I also am a bit concerned about the splitting rules (which I did make an issue for to be clear) but that's about the design so I'm sure it's a lot harder to solve with properly considering ups and downs

#

but hey, you guys are doing a really great job, especially with the regular releases

#

so thank you!

bright glacier
#

blame thank Cooper for being the world's most annoying person for a little while pushing for a release (no seriously, without him we probably wouldn't have had a release since 20.8b1) πŸ˜„

tribal thistle
#

Ooh has it been pushed?

#

Better question, are releases enabled for the web hook?

flat krakenBOT
#
jack1142
[psf/black] Issue opened: #2293 String processing doesn't respect existing code formatting for f\-strings

Describe the bug

There is some issue with how Black splits the string if at least one of them is an f-string.

To Reproduce

For example:

  1. Take this file:
message = (
    f"1. Go to Google Developers Console and log in with your Google account."
    "(https://console.developers.google.com/)"
    "2. You should be prompted to create a new project (name does not matter)."
    "3. Click on Enable APIs and Services at the top."
    "4. In the list of APIs choose or ...
flat krakenBOT
#
jack1142
[psf/black] Issue opened: #2294 String processing produces code exceeding length limit

Describe the bug

String processing produces code exceeding the length limit when it was already formatted properly before.

To Reproduce

For example:

  1. Take this file:
class X:
    async def foo(self):
        msg = ""
        for candidate in CANDIDATES:
            msg += (
                "**{candidate.object_type} {candidate.rev}**"
                " - {candidate.description}\n"
            )
  1. Run Black on it with these arguments '--experimental-...
flat krakenBOT
#
gmos
[psf/black] Issue opened: #2295 Black confusion when \\ is used in f\-string expression part\.

Describe the bug

In:

print("this is ok")
print("this is ok")
print(f"this is an error {1}")
print("this is ok")
print(f"this is an error {'\n'}")

Python reports error on line 5 but Black formats without error.

In:

print("this is ok")
print("this is ok")
print(
    f"this is an error {1}"  
)
print("this is ok")
print(f"this is an error {'\n'}")

Python reports error on line 7, but Black reports error on line 4.

To Reproduce

See above.

**...

flat krakenBOT
flat krakenBOT
neon loom
#

@bright glacier - I feel psf/black/pull/1940 is ready - Want me to merge or you happy to do it?

bright glacier
#

what is that pr lol, outside currently

neon loom
#

Action rewrite

bright glacier
#

I am happy with it, so unless you can find another typo, merge away!

#

hmm

#

I could do my first merge on a phone right now lol

neon loom
#

kk

#

Yeah I don't see any issues

flat krakenBOT