#black-formatter

1 messages ยท Page 28 of 1

tired shard
#

Gonna make the diff highliting a bit more pretty for now, so the workflow is nice, will continue the work on that if its okay

#

actually no, i think it can be done in a short way, got a new method

#

ig its gonna be abit hacky

tired shard
#

ok i think i got a way this can work, but not sure if it will be fully accurate

tired shard
#

getting there

#

just one more thing to implement

tired shard
#

hmm, maybe i found a bug with diff, or I messed up something

#

richard ping me you come online

tired shard
#

also do you want me to add line numbers?

bright glacier
bright glacier
tired shard
#

i got a bug/problem while testing hence asked you to ping, but now am too tired to talk about it

bright glacier
#

using the standard red/green markup should be good enough, we don't need any fancy RGB based colours lol

bright glacier
#

but thank you for working on this! maybe with enough hacky code we can make this work

#

(future me is going hate me for writing unmaintainable code :p)

#

... hopefully not!

tired shard
#

not really a bug with testing, but with the current code, basically there is a diff on this two JSONs but the diff doesn't show any diff

#

(putting them in dms)

#

well, it is not all hacky now, i basically syntax highlight each line separately, shortcut

tired shard
#

the analysis output, not sure what you mean

bright glacier
#

there's some metadata in each analysis's JSON

#

that's why the code compares them by checking each ProjectResults dictionary within the Analysis object

tired shard
#

yeah, ik that

tired shard
late dewBOT
#
It has arrived!

Here's your reminder: it mayyy be worth doing a security audit of the current workflow setup
[Jump back to when you created the reminder](#black-formatter message)

late dewBOT
#
It has arrived!

Here's your reminder: have you done something yet?
[Jump back to when you created the reminder](#black-formatter message)

tired shard
#

yes :D

bright glacier
#

@tired shard OK, so I was going to approve your verbose PR (after doing some cleanup and pushing 'em to your branch) but there's a case where the PR makes the output worse IMO

late dewBOT
#
It has arrived!

Here's your reminder: See referenced message.
[Jump back to when you created the reminder](#black-formatter message)

bright glacier
#

@errant barn since your feedback has been so useful at pushing Shivansh's PR it'd be super kind of you if you could re-review the PR soon! I'm happy with it (barring some minor stuff) and I won't block the merge on your review but it'd be nice haha. Happy Christmas by the way ๐ŸŽ„ !

#

Obviously any holiday celebrations / family time should take priority over OSS work ๐Ÿ˜„

tired shard
#

Hmmm, IMO while formatting code there should be option - if you want it to follow your config or not since sometimes i use black to format code generally and not always according to my config.

#

Also merry christmas ๐ŸŽ„

bright glacier
tired shard
#

if that's the case, we could add the project root if its not isolated, and remove config/root if its isolated

bright glacier
tired shard
#

remember the issue number?

#

ee yup, timezones smh, read that after 30 minutes lol

bright glacier
bright glacier
tired shard
#

hmm, so block this pr until that is implemented or add some basic handling for now and fix it along with that pr?

bright glacier
#

psf/black#2537

toxic stormBOT
tired shard
#

hmm, seems a short pr

bright glacier
#

well no one has approved it yet lol

tired shard
#

that's where you come in ๐Ÿ™ƒ

errant barn
plain atlas
lament crow
#

Sometimes you need a break from family

plain atlas
#

lol

bright glacier
#
โฏ git merge main
Auto-merging .github/ISSUE_TEMPLATE/bug_report.md
CONFLICT (content): Merge conflict in .github/ISSUE_TEMPLATE/bug_report.md
Auto-merging pyproject.toml
Auto-merging setup.py
CONFLICT (content): Merge conflict in setup.py
Auto-merging src/black/__init__.py
Auto-merging src/black/linegen.py
CONFLICT (content): Merge conflict in src/black/linegen.py
Auto-merging src/black/mode.py
CONFLICT (content): Merge conflict in src/black/mode.py
Auto-merging src/black/nodes.py
CONFLICT (content): Merge conflict in src/black/nodes.py
Auto-merging src/black/parsing.py
CONFLICT (content): Merge conflict in src/black/parsing.py
Auto-merging src/black/strings.py
CONFLICT (content): Merge conflict in src/black/strings.py
Auto-merging tests/test_black.py
CONFLICT (content): Merge conflict in tests/test_black.py
Auto-merging tests/test_format.py
CONFLICT (content): Merge conflict in tests/test_format.py
Auto-merging tox.ini
Automatic merge failed; fix conflicts and then commit the result.

well this is fun.

#

Honestly it might just be easier to recreate the branch instead of fixing all of ese merge conflicts.

#

actually, fixing merge conflicts in vscode is so much nicer than using vim

muted haven
#

Is this the mypyc branch?

bright glacier
#

No an old branch that drops all of the Python two code :)

#

We're planning on dropping Python two support in January 2022 haha

elfin beacon
#

Do you guys know how can I get single quotes instead of double quotes?

#

I mean, not force it, but prefer single quotes

bright glacier
#

You can't force black to replace double quotes with single quotes but you can use the --skip-string-normalization / -S to disable black from touching your quotes. Otherwise use a fork of black on PyPI.

plain atlas
#

or edit the source everywhere that needs to be edited yourself ๐Ÿฅฒ

#

not that i recommend that

elfin beacon
plain atlas
#

!pep 8

late dewBOT
#
**PEP 8 - Style Guide for Python Code**
Status

Active

Created

05-Jul-2001

Type

Process

elfin beacon
#

I just execute a macro in vim, but it's really annoying

plain atlas
#

oh actually pep 8 doesn't pick a stance, nvm

bright glacier
#

I won't be submitting it as a PR until later as we might do another release (or two) with Python 2 support still there

plain atlas
bright glacier
#

....

plain atlas
#

genuine review comment lol

late dewBOT
#

src/black/parsing.py lines 117 to 122

if matches_grammar(src_txt, pygram.python_grammar) or matches_grammar(
    src_txt, pygram.python_grammar_no_print_statement
):
    original_msg = exc.args[0]
    msg = f"{original_msg}\n{PY2_HINT}"
    raise InvalidInput(msg) from None```
bright glacier
#

also read the commit message lol

plain atlas
#

nice approves

#

I tried to not review anything today

#

instead I signed off on a pre-commit workflow to a very used open source project that uses isort and black

bright glacier
#

I'm working on a bunch of maintenance stuff I left on the back burner for psf/black. First the removal of Python two support, and now the transition to nox

plain atlas
#

lol

bright glacier
#

I'm not 100% decided on moving away from tox to nox but overall I like nox way more sooo

plain atlas
#

HMMMMmmmm

#

I think you like nox

bright glacier
#

I do :)

bright glacier
#
โฏ nox -l
Sessions defined in /home/ichard26/programming/oss/black/noxfile.py:

* lint
* tests-3.6
* tests-3.7
* tests-3.8
* tests-3.9
* tests-3.10
* tests-pypy3
* docs
* docs-live
* fuzz

sessions marked with * are selected, sessions marked with - are skipped.

ducky_party

bright glacier
#

How is this related to the black autoformatter project?

hollow gazelle
#

idk

#

he is the donger tho

bright glacier
#

then maybe don't post it here as right now it'd be considered as spam

hollow gazelle
#

whops

#

the donger apologizes

bright glacier
#

that's fine, just don't dump random images without context -- you might be able to get away with it in the off-topic channels but this is not one of 'em.

cursive ravine
#

join the communist party of donger

bright glacier
#

OK, please stop -- this is off topic here.

cursive ravine
#

what is this black formatter?

sonic tundra
#

@hollow gazelle @cursive ravine guys, this is not that kind of server/channel.

cursive ravine
#

sorry

sonic tundra
cursive ravine
#

what is formatting?

bright glacier
hollow gazelle
#

so it cleans op your code

bright glacier
#

Yeah, but just visually, it won't touch the meaning of your code.

#

Sorta like how you can make text bold in some editors, it doesn't affect the meaning (well usually) but it can make it easier to parse / read

hollow gazelle
#

that is neat

#

i think i will be using that thank you

night trellis
#

How to change the line length of black?

little parrot
#

--line-length=x as an arg

#

or set it in pyproject.toml

night trellis
#

Thanks

#

Also, is there any way to stop black from changing this long list into a column?

bright glacier
#

# fmt: skip ? it's pretty buggy so you might have to use the more battle tested # fmt: off and # fmt: on pair

night trellis
#

Could you elaborate on where I have to put these?

bright glacier
#

fmt: skip would go at the end of the line

#

fmt: off/on surround a block of code to disable formatting for

#
a        = 1  # fmt: skip

# OR

# fmt: off
a        = 1
# fmt: on
night trellis
#

that worked, thanks!

flat krakenBOT
#

Description

Since power operators almost always have the highest binding power in expressions, it's often more readable to hug it with its operands. The main exception to this is when its operands are non-trivial in which case the power operator will not hug, the rule for this is the following:

For power ops, an operand is considered "simple" if it's only a NAME, numeric CONSTANT, or attribute access (chained attribute access is allowed), with or without a preceding unary operator...

lament crow
#

first time I'm hearing "hug" as a term for no whitespace around the operator ๐Ÿ˜„

bright glacier
#

wow CI is quite red tonight lemon_angrysad

bright glacier
#

ahh I pressed the wrong keybinding

plain atlas
#

10/10

#

done that before

#

no worries

#

also you have manage message perms, you can hide your mistakes

#

sweep them under a rug

bright glacier
#

OK interpreted black is really slow on my machine ๐Ÿ˜ฆ

plain atlas
mint barn
#

3?

#

psf/black#3

toxic stormBOT
tired shard
#

ichard/diff-shades#3

#

uhh

#

ichard26/diff-shades#3

toxic stormBOT
muted haven
#

I was working on the speedup against -t py310 and seems like it moved somewhere from 'awfully slow' to 'somewhat acceptable' in my eye.

[black main -tpy39] Mean +- std dev: 7.30 ms +- 0.26 ms
[black main -tpy310] Mean +- std dev: 39.8 ms +- 1.7 ms
[some experimental branch -tpy310] Mean +- std dev: 13.5 ms +- 0.6 ms
#

Though no promises on when (or even if) it will be a PR. The branch that I have this optimization is still a mess (fusing 20 years old lib2to3 code with dark magic doesn't look great), and since it got speed up noticeably I wonder whether there are any edge cases we are missing. If there are none, I'll clean it up and make it more presentable ๐Ÿ‘€

flat krakenBOT
#

The current revision of the new backtracking parser (that gets enabled with -t py310) is kind of slow.

For regular inputs that only use match/case without using the pattern matching statements, it is somewhere between 3 to 4 X slower at the moment (going up to 6 or 7 on hand crafted inputs). On some inputs (like Lib/test/test_patma.py) that contain match/case hundreds of times, with new PEP 634 statements, it...

flat krakenBOT
#

Resolves #2727

Core Idea

The current parser; even when it is backtracking, builds nodes which is a terrible waste of time since every time we see a match or case, we need to copy the whole stack over and over in order to preserve the actual structure and not do any modifications on top of it. This is obviously not cheap.

The following implementation does something very similar to old school state machines, where the parser is temporarily converted into a structure that tells you...

bright glacier
#

@muted haven wait is this change testable without the 3.10 grammar?

#

I can configure diff-shades to use --target-version 3.10 but I was curious to the scope of this change

#

oh brilliant mypyc is broken on main as well because we introduced AnyStr to resolve some type ignores

muted haven
bright glacier
#

OK, sounds good.

muted haven
#

But what I am curious is that, whether it will break any regular parsing of re.match() or case = blabla etc.

#

since I think I pretty much covered all PEP 634 syntax, and it can also parse as much of the stdlib as it can previously parse

bright glacier
#

Hmm, gh api "https://api.github.com/repos/ichard26/black/commits/f5a202d0d29556719ff9464d64c75a04aceda2f8" doesn't return a 404, I might be able to run diff-shades from my fork to test out your PR upstream which would be ideal cause my laptop is painfully slow

#

This feels like the sort of cross repo access that shouldn't be allowed but I'm not complaining (unless this is due to my API key)

muted haven
#

It seems to be a public API, I can access it without any authentication.

mint barn
#

yoo guy

#

@muted haven

#

are you there

bright glacier
#

Hmm cloning performance seems to be miserable right now, diff-shades keeps on getting stuck on cloning the projects and sometimes even crashing - I might have to investigate other ways of pulling down projects (maybe github.com/**/archive/*.zip ?)

#

Actually GitHub Actions in general seems to be suffering poor performance

muted haven
#

I was writing a benchmark script for a project, which got a fresh clone every time we run it to compare the results in case the main in the local user was not up-to-date. Then we noticed clones got substainally slower (from 2-3 MB/s to 50kb/s), which GitHub seem to be lowering bandwidth for our IP.

bright glacier
#

primer easily does way more clones than what diff-shades is doing on my fork given it's a matrix build

#

I wonder if my fork has less "allocated" resources for a lack of a better way of describing it

muted haven
#

If the target repo's are not getting constantly upgraded, perhaps you can do actions/cache on them for a week or so.

bright glacier
bright glacier
bright glacier
bright glacier
muted haven
bright glacier
muted haven
bright glacier
#

main is so slow that by now a normal (i.e. the target-version configuration is pulled from each project instead of getting overwritten) run of this workflow should be done ๐Ÿ™ƒ

muted haven
#

yep, that is kind of expected isn't it ๐Ÿ˜…

#

(-t py310 being slow on main)

bright glacier
#

yeah it's the downside of trying to quantify performance improvements, often getting the baseline data is horribly slow (especially when the speed up is up to 50x haha)

bright glacier
#
โ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ Summary โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ
โ”‚         File breakdown                   Project breakdown         โ”‚
โ”‚  Result            # of files     Result            # of projects  โ”‚
โ”‚ โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€   โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ โ”‚
โ”‚  nothing-changed   8263           nothing-changed   13             โ”‚
โ”‚  reformatted       1782           reformatted       10             โ”‚
โ”‚  failed            0              failed            0              โ”‚
โ”‚                                                                    โ”‚
โ”‚ # of lines: 2 057 627                                              โ”‚
โ”‚ # of files: 10045                         125 479 changes in total โ”‚
โ”‚ # of projects: 23              84 727 additions - 40 752 deletions โ”‚
โ”‚ โ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ [custom arguments] โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ โ”‚
โ”‚ โ”‚                     --target-version py310                     โ”‚ โ”‚
โ”‚ โ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฏ โ”‚
โ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ black 21.12b0 - Dec 26 2021 19:33:56 UTC โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฏ
``` my laptop would be so mad at me if I tried doing this - for reference a normal run of primer (which has the same project set as diff-shades) takes 25-35 minutes ![lemon_sweat](https://cdn.discordapp.com/emojis/754441881718620281.webp?size=128 "lemon_sweat")
muted haven
#

can't imagine how long that will take with -t py310 then

bright glacier
muted haven
#

Indeed it is

#

at least it tells me that I didn't break anything

plain atlas
#

I'm proud of you ๐Ÿ‘

#

no that's not sarcasm

bright glacier
#

is that for isidentical or me lol

plain atlas
#

okay maybe a little sarcasm

plain atlas
bright glacier
#

Wow okay the shade /hj

plain atlas
plain atlas
muted haven
sour jay
flat krakenBOT
#

Some comments still! Did you have a look at the test Richard requested? Or maybe they are the old tests you modified and I'm just reading things wrong. Anyways, thanks again for the changes and let's give it a push once more ๐Ÿ˜„ I can't believe this conversation is already like 30 comments long, seems like no change is small in a project this big :D thanks for holding up

mint barn
#

Sorry to bother you, I'm new and I need help because I looked at wiki and Google and didn't get a workaround, I couldn't install Blackd. PIP doesn't have this option, I tried Py3.9.7 and Py3.10 and failed.

errant barn
#

What install command did you use specifically? Your comment makes me wonder if you tried pip install blackd instead of the correct black[d]

mint barn
#

sorry.

errant barn
#

Did it work? ๐Ÿ˜„

mint barn
#

Thanks!

#

๐Ÿ˜†

errant barn
#

Perfect! It's known as an install extra. Lots of libraries have them, like httpx[http2].

#

As you might have guessed already, they enable some extra functionality, like for us the Black server!

bright glacier
#

it needs a minor fix but other than that I like it

errant barn
#

preview? ๐Ÿ˜„

errant barn
#

oh that's nice

bright glacier
#

this border needs work but it should be easy to fix

#

fwiw I did this 100% via google drawings ๐Ÿ˜„

#

Actually, Felix, have you looked at the diff-shades CI integration I posted four (three?) days ago? Just wondering if you're totally lost at what diff-shades is lemon_sweat

errant barn
#

Yeah I know what it does, but I haven't had a thorough look ๐Ÿ˜„ I don't really know that much about primer either

#

But everything you've presented has looked nice

bright glacier
#

Ah alright, I just didn't want to shove a project (logo) in your face without context haha

errant barn
#

Well I couldn't have missed it since you have been working at it for quite a while now!

bright glacier
#

me: *shows felix a logo*
felix: "that's nice"
also felix: *what the heck am I looking at?*
this is what I was trying to avoid ๐Ÿ˜„

errant barn
#

Nice logos are nice though, even in isolation

#

Having a glance at your PR now: those bot comments look really useful

mint barn
bright glacier
bright glacier
#

(I bring that up since the first PR I used the workflow on was a case that needed --target-version py310 haha)

errant barn
#

right :D

flat krakenBOT
#

If we start with "correctly" formatted code

my_dict = {
    something_long.SomethingLong.SOMETHING_LONG:
        lambda e: something_really_long.SomethingReallyLong.some_function_with_long_name(
            some_argument
    ),
}

and then run Black, we get lines that over 120 characters in length

my_dict = {
    something_long.SomethingLong.SOMETHING_LONG: lambda e: something_really_long.SomethingReallyLong.some_function_with_long_name(
        some_argument
  ...
boreal nova
#

Hi, would love feedback on pull #2721 and also would like to know if it would ever be merged or not. Thanks ๐Ÿ˜„

#

I'm interested in what the pull request review process looks like for a project as big as black.

bright glacier
#

well it depends really on the maintainer story of the project -- psf/black is fortunate enough to have quite a few active maintainers (four usually) but even then reviews are the main bottleneck

#

Felix and I are the only active maintainers of lately due to holidays and vacations the rest of the core team are enjoying -- and unfortunately I haven't been interested in reviewing something as major as your PR.

flat krakenBOT
#

I wrote some python code . like:


seed = {
  "genesis_block": {
    "BlockId": "65b3d55b-73b5-4d13-9550-4d37d53ec7d3",
    "GroupId": "89703dbe-93c6-4be0-ba26-dce563181194",
    "ProducerPubKey": "CAISIQPTrmj7NOYeFFXgxe5dHuEqQ5dEtL5WIvFD5i6jD2He+w==",
    "Hash": "16wubZBYp8jWpYZtAy73WsfeR3vRDADOEqIfrfi74/g=",
    "Signature": "MEUCIQD5UUyFPx2CzFFwCnFLN7gxfV30zQBmjS2jybSUls8yLgIgOvhOJdsGh5wrb0FmG5mGxdChSaINdyTpUZE6JVYE5hI=",
    "TimeStamp": "1640423797361990300"
  },
...
flat krakenBOT
#

Is this related to a problem? Please describe.

The newer versions of the WingIDE, (8.x) have an option to use automatic formatting with black. The current documentation only mentions manually running Black from within Wing by executing a command line and linking that to a key binding.

Describe the solution you'd like
Mention the mechanism for enabling black auto-formatting in Wing IDE.

In docs/integrations/editors.md add that - for Wing IDE versions >8 you can enable Bla...

heavy heron
plain atlas
#

!e

s = '''H'i'hdks' '''
r = s.replace("'",'"')
t = s.replace("'", '\"') 
print(r)
print(t) 
late dewBOT
#

@plain atlas :white_check_mark: Your eval job has completed with return code 0.

001 | H"i"hdks" 
002 | H"i"hdks" 
plain atlas
#

@errant barn in response to your comment, yes they are the same

errant barn
bright glacier
#

re. psf/black# 2721 - There's also a problem with the --docs and --ipynb flags as they don't mean the same thing

#

--ipynb (and --pyi for that matter) force all files to be treated as jupyter notebooks / stub files as far as I remember, but --docs enable documentation formatting. But I don't think we can sanely default to formatting documentation given people call black with . and it's not gated by an extra dependency

#

It mayyy be better just to leave --docs (although I'd prefer --md) as the "force all inputs to be documentation" flag as per usual and then let users configure their include configuration to include documentation files

#

It does beg the question of maybe it would be better to just have a --force-type (feel free to bikeshed this name) choice option with pyi, jupyter, md, and so on as the possible values in the long run

#

also apparently I messed up the issue ref. psf/black#2721

toxic stormBOT
bright glacier
#

But yeah the fact documentation is so vast does worry me how much complexity the project might take on

errant barn
#

Oh you're right! Only changing input mode could be a cleaner solution. Although that would require multiple runs of Black. But letting these be other libraries might be an even better solution once we define a Python API! I'm not sure what we should do to our ipynb things in that case. psf/black-ext anyone? :P

bright glacier
#

yeah our "just let the extensions handle it" would be a more much more compelling case if we had a stable API :p

flat krakenBOT
quiet kraken
#

!mute 856241202005016626 ๐Ÿ”Ž

late dewBOT
#

:incoming_envelope: :ok_hand: applied mute to @cursive ravine until <t:1640823936:f> (59 minutes and 59 seconds).

hollow gazelle
#

ok sherlock homes

plain atlas
#

could you delete this too now that it's been taken care of?

#

(they have internal logs so it's fine to delet)

#

noice thx

flat krakenBOT
#

Description

Update documentation about Wing IDE integrations in the 8.0+ version. In the new version Wing IDE have special global and per-project setting for reformatter like black and yapf.

for #2731 hope it help :)

Checklist - did you ...

  • [x] Add a CHANGELOG entry if necessary?
  • [x] Add / update tests if necessary?
  • [x] Add new / update outdated documentation?

Note

I don't think docs need changelog, Also thank you so much to black contributor and ma...

plain atlas
#

there's definitely someone working in a formal setting who's intentionally unformatted

bright glacier
#

I'm opening too many PRs with no one else to review 'em haha

#

The rest of the core team seems to be on holidays / taking a break from OSS for the most part, except for Felix and maybe Cooper

flat krakenBOT
#

To Reproduce

For example, take this code:

def a():
    CompiledSQL(
        "addresses.id AS addresses_id, "
        "addresses.email_address AS addresses_email_address "
        # note the comma-separated FROM clause
        "IN (__[POSTCOMPILE_primary_keys]) "
        "AND addresses.id = anon_1.id ORDER BY addresses.id",
        ["hmm"]
    )

And run it with these arguments:

$ black file.py --experimental-string-processing

The resulting...

flat krakenBOT
#

Description

Fixes https://github.com/psf/black/issues/2706.

diff-shades result comparing this PR to 21.12b0:

โ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ Summary โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ
โ”‚ 0 projects & 0 files changed / 0 changes [+0/-0]       โ”‚
โ”‚                                                        โ”‚
โ”‚ ... out of 2 057 623 lines, 10 045 files & 23 projects โ”‚
โ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€...
flat krakenBOT
#

Describe the style change

I am a first time user and ran black on code generated by Cookiecutter and noticed that a string literal that was split across lines was joined, but the extra quotes were left in and so this I feel will reduce readability. Here is the original code:

    click.echo("Replace this message by putting your code into "
               "testproject.cli.main")

Examples in the current Black style

    click.echo("Replace this me...
bright glacier
#

I'm writing tests for diff-shades and man is it annoying starting from scratch, so much easier to write tests for black because the infra is already there

bright glacier
#

hmm, I'm getting /home/ichard26/programming/tools/diff-shades/tests/test_suite.py:105: DeprecationWarning: invalid escape sequence \ for something like

        assert diff == textwrap.dedent("""\
            --- 1.py
            +++ 2.py
            @@ -1,3 +1,3 @@
             aaaa
            -bbbb
            -cccc
            \ No newline at end of file
            +BBBB
            +cccc
        """)

What's the 'correct' way of writing this inline multi-line string?

#

I thought this was a supported idiom but apparently not ๐Ÿ˜ฆ

sour jay
#

Maybe?

bright glacier
#

!eval

#

!eval

import textwrap
s = textwrap.dedent("""\
    --- 1.py
     aaaa
    -bbbb
    -cccc
    +BBBB
""")
s2 = textwrap.dedent("""
    --- 1.py
     aaaa
    -bbbb
    -cccc
    +BBBB
""".strip("\n"))
assert s == s2
late dewBOT
#

@bright glacier :x: Your eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "<string>", line 16, in <module>
003 | AssertionError
bright glacier
#

!eval

import textwrap
s = textwrap.dedent("""\
    --- 1.py
     aaaa
    -bbbb
    -cccc
    +BBBB
""")
s2 = textwrap.dedent("""
    --- 1.py
     aaaa
    -bbbb
    -cccc
    +BBBB
""".lstrip("\n"))
assert s == s2
late dewBOT
#

@bright glacier :warning: Your eval job has completed with return code 0.

[No output]
bright glacier
#

Thanks

sour jay
bright glacier
#

That somehow didn't work when the whole thing was indented

sour jay
#

!e ```python
import textwrap

print(repr(textwrap.dedent("""
--- 1.py
aaaa
-bbbb
-cccc
+BBBB
""")))

late dewBOT
#

@sour jay :white_check_mark: Your eval job has completed with return code 0.

'--- 1.py\n aaaa\n-bbbb\n-cccc\n+BBBB'
bright glacier
#

!eval

import textwrap
def a():
    print(repr(textwrap.dedent("""\
        --- 1.py
         aaaa
        -bbbb
        -cccc
        +BBBB\
    """)))
a()
late dewBOT
#

@bright glacier :white_check_mark: Your eval job has completed with return code 0.

'--- 1.py\n aaaa\n-bbbb\n-cccc\n+BBBB    '
bright glacier
#

ah of course

flat krakenBOT
#

Description

Fixes #2506

XDG_CACHE_HOME does not work on Windows. To allow for users to set a custom cache directory on all systems I added a new environment variable BLACK_CACHE_DIR to set the cache directory. The default remains the same so users will only notice a change if that environment variable is set.

The specific use case I have for this is I need to run black on in different processes at the same time. There is a race condition with the cache pickle file that ...

gilded zenith
#

hi

glacial drum
#

Please use #bot-commands to run miscellaneous scripts.

flat krakenBOT
flat krakenBOT
flat krakenBOT
#

Description

Should unblock GH-2297 and mypyc.

Checklist - did you ...

  • [x] Add a CHANGELOG entry if necessary?
  • [x] Add / update tests if necessary?
  • [x] Add new / update outdated documentation?

Review notes

Right now this PR uses 22.0 as the cutoff version but this is highly up to debate / subject to change. It might just be better to remove these references until we know for certain.

bright glacier
#

... and CI is broken, brilliant! /s

sour jay
#

fmt: off I think?

glacial drum
#

you can also add a trailing comma, and it won't

#

IE ```py
foo = {
"a": 1,
"b": 2,
}

#

The comma on the last element

#

No problem!

little parrot
#

No afaik

#

You can use # fmt: skip

#

But I'd argue this is harder to read than whatever black would format it as

#

Every line

#

Or use fmt: off and fmt: on

#

To disable it for a bulk amount of lines

#

Has anyone else here found code harder to read if it isn't formatted with black after using it for a long time?

plain atlas
#

mess around with the commans, it should be able to be left like the above mostly

#

could you paste this here so i can mess around with it?

little parrot
#

Oh that's better

plain atlas
#

๐Ÿค”

flat krakenBOT
plain atlas
#

oh that's why

#

its essentially because of tuples

#

in the source code:
this has a different meaning:
(5,)
than this:
(5)

#

so because of that, black does not apply magic trailing comma logic to single item tuples

#

a simple solution is just to use a list, since for what you're providing it to, I know that it can take lists

#
@pytest.mark.parametrize(
    ("input_data", "expected"),
    [
        (test_data, 26397),
    ],
)
#

yeah

#

for pytest at least, anyhow

bright glacier
#

@neon loom I thought we switched to platformdirs, why are we talking about appdirs?

neon loom
#

So did I โ€ฆ so I was right

bright glacier
#

Yeah ...

neon loom
#

Correct us ๐Ÿ™‚ Iโ€™m just dumb dumb

bright glacier
#

this server has a "ducky_australia" emoji by the way haha

bright glacier
#

psf/black#2739

flat krakenBOT
errant barn
#

Not sure if this is the right channel, or server, or at all appropriate ๐Ÿ˜…

bright glacier
#

It's been reported to the mod team.

#

<@&831776746206265384> .

#

Stop.

glad pilot
#

!mute 911012248049291305 investigating

late dewBOT
#

:incoming_envelope: :ok_hand: applied mute to @steady sail until <t:1641154995:f> (59 minutes and 59 seconds).

errant barn
#

Whew ๐Ÿ˜„

bright glacier
glad pilot
#

though an @moderators ping here is helpful, too

#

if it's something you need attention with quickly

errant barn
flat krakenBOT
flat krakenBOT
neon loom
flat krakenBOT
flat krakenBOT
lean orbit
#

I have a .flake8 with constraints for formatting, is there any way I can use black against these constraints?

[flake8]
max-line-length = 95
max-complexity = 10
per-file-ignores =
    cogs/utils/__init__.py:F401
    cogs/weather.py:E501
glacial drum
#

black should respect the line length you set there out of the box

lean orbit
#

seems to be going for 98

glacial drum
#

as for file ignores, afaik you'll need to add a tool.black section to your pyproject with an extend-exclude line

errant barn
#

Do you just want to set the line length?

lean orbit
#

sure, but I also need to follow all of the other rules in flake8 file so flake8 doesnt scream at me

errant barn
#

we do provide ignores as Chris already said, but in terms of style knobs Black is very limited

#

401 seems unrelated to anything Black would do though

#

but you could disable formatting for a particular section

#

although that is discouraged ๐Ÿ˜„

lean orbit
#

What is max-complexity?

lean orbit
errant barn
#

doesn't concern Black, it might have something to do with code complexity

lean orbit
#

right, that's that alley I was going down but still have no idea

errant barn
#

in general Black should free you from linters complaining :D

lean orbit
#

well, I get the same output from flake8 . when I remove max-complexity so lets hope it's nothing important

flat krakenBOT
stuck vapor
#

โฌ†๏ธ this post is about adding an extra configuration option for Jupyter support - how do the maintainers feel about adding an extra configuration option?

errant barn
#

Yo would it be possible to accept all magics?

#

Or do we need a list of valid ones

stuck vapor
#

some magics are for non-Python code, e.g.

%%bash
mkdir my_dir
late dewBOT
#

src/black/handle_ipynb_magics.py line 40

PYTHON_CELL_MAGICS = frozenset(```
errant barn
#

What I was after was more like, can we get away with accepting arbitrary magics, or do the names have something to do with how they are processed?

stuck vapor
#

ah, I see - the names don't affect how they're processed, no

errant barn
#

If I'm reading this right, we need to determine which cells contain Python and format them.

late dewBOT
#

src/black/__init__.py line 966

if src[:2] == "%%" and src.split()[0][2:] not in PYTHON_CELL_MAGICS:```
errant barn
#

So we would indeed need a list of python cells that is extendable

stuck vapor
#

yes, exactly - so, the user could extend the current PYTHON_CELL_MAGICS

errant barn
#

Yep, it should definitely be configurable then ๐Ÿ‘๐Ÿฟ All of this brings back the discussion in psf/black#2712 of whether any of this should be in Black, so feel free to chime in Marco since you pretty much run the notebook thing here ๐Ÿ˜„

toxic stormBOT
errant barn
#

psf/black#2721 I mean

stuck vapor
mint barn
#

yapf is better but black is nice

neon loom
#

Did our issue comments get disabled to be shared here or is broken?

bright glacier
#

psf/black#2712

toxic stormBOT
bright glacier
#

Seems to be a temporary outage / break of @toxic storm . If it happens again I guess we could ask #dev-contrib

errant barn
#

I noticed that I'm not able to manage projects, and that Richard you started using the project and dropped the milestone. All fine, but we could add psf/black#2297 and psf/black#2726 in the nice to have

bright glacier
#

Wait, triagers can edit milestones?

errant barn
#

Prolly just assign issues

bright glacier
#

oh yeah triagers do have access to milestones but not projects

#

moving everything to the project probably wasn't the best of idea then ..

errant barn
#

Should be fine ๐Ÿ˜„ not like we're adding lots of items to stable requirements. But would be nice to fit in as many open PRs as possible, and close some too

bright glacier
#

If I were starting from scratch I'd have stuck with a milestone.

errant barn
#

They're quite adequate

bright glacier
#

and I'm bad enough at managing to-do lists I don't wanna have to manage projects lol

#

Also January 9th is closer than I previously thought!

#

I was thinking "oh I got time to prepare for the sprint" but not really now I think about it. Thankfully other than diff-shades and mypyc there's nothing that I'm truly responsible for.

errant barn
#

Anything I should know about a sprint beyond "we just do more work then"?

bright glacier
bright glacier
errant barn
#

Well, let's just focus on getting something done then :D

errant barn
#

DS & mypyc wise

bright glacier
#

testing the wheels would be wonderful although they're a bit out of date now

#

I have a grand total of 2 user field reports ducky_cheese

errant barn
#

Thanks, I'll have a go with them tomorrow

bright glacier
#
Name                          Stmts   Miss  Cover
-------------------------------------------------
src/diff_shades/__init__.py       3      0   100%
src/diff_shades/__main__.py       2      0   100%
src/diff_shades/analysis.py     144     62    57%
src/diff_shades/cli.py          258    155    40%
src/diff_shades/config.py        25      1    96%
src/diff_shades/output.py       111     43    61%
src/diff_shades/results.py      200      9    96%
tests/test_suite.py             225     11    95%
-------------------------------------------------
TOTAL                           968    281    71%

other than my logging code being almost untestable this isn't actually too bad writing functional tests

#

testing is always nice for figuring out how crappy your APIs are ๐Ÿ˜„

neon loom
neon loom
#

Well something similar before.

#

Hard to test API, hard to use API, usually relate ๐Ÿ™‚ - Mocking is another story. But hard to mock can usually state some refactors might be nice.

silent wren
#

Dont forget the meetings in between the sprints not too sure if it is the stand up type lol

flat krakenBOT
#

Describe the style change

Support for hanging indents

Examples in the current Black style

if (
    is_interesting(attempt)
    and len(attempt.record) <= len(source.record)
):
    source = attempt

Desired style

    if (is_interesting(attempt)
        and len(attempt.record) <= len(source.record)):
        source = attempt

This might not be the best example and in many cases I prefer blacks exploded syntax and in many I pref...

flat krakenBOT
flat krakenBOT
#

Describe the bug

I am writing my own Concrete Syntax Tree parser/emitter in Python, but I am using black as my code formatter so I want black to ignore the 'fancy' Python syntax I am trying to support with my parser. So I've got a bunch of badly formatted Python code that I want to support with my parser.

To Reproduce

def add1(foo):
    def adder(a: int, b: int) -> int:
        # fmt: off
        res: \
            int \
            = a + b
        return res...
flat krakenBOT
#

Describe the bug

When trying to run black on a file with a patter matching block where a case is a class pattern with an as pattern inside of it fails to format

To Reproduce

For example, take this code:

A stripped down example of file that was failing

from collections.abc import Callable

class Foo:
    bb = 5

    def aa(self):
        ...


bar1 = Foo()

match bar1:
    case Foo(aa=Callable() as aa, bb=int()):
        print(bar1.aa, b...
bright glacier
#

I finally managed to fix my first mypyc bug ๐ŸŽ‰ hopefully this means someday we'll be able to drop the "create nested function via type at runtime" hack :)

slow ibex
#

nuitka can speed up your code better than mypyc, and nuitka-compiled code is closer to pure python, than mypyc-compiled code
have you thought of using it?

#

IMO, nuitka is faster and easier than mypyc

bright glacier
#

have you thought of using it?
No because the codebase was already prepared (barring some issues that built up over ~2 years of delay) to integrate mypyc when I first took up the porject

#

I really don't want to restart from scratch this late in the game

slow ibex
#

you don't need to change the codebase to compile it with nuitka

#

maybe i will try it later

flat krakenBOT
#

<!--
Please make sure that the bug is not already fixed either in newer versions or the
current development version. To confirm this, you have three options:

  1. Update Black's version if a newer release exists: pip install -U black
  2. Use the online formatter at , which will use
    the latest main branch.
  3. Or run Black on your machine:
    • create a new virtualenv (make sure it's the same Python version);
    • clone this repository;
    • run pip install -e .[d,python2];
      ...
flat krakenBOT
frail meteor
#

Suggestion:
Use artificial intelligence to find similar blocks of code and make them functions instead

errant barn
#

For Black? ๐Ÿ˜„

slow ibex
#

pylint can do it

errant barn
#

Black has a strict promise of not changing the meaning of the code, so while interesting, I think we'll leave that up to other libraries :D

flat krakenBOT
lament crow
#

I'm confused

#

You said "I see no reason to merge" but approved the PR

gusty grail
#

"no reason to not merge" perhaps

bright glacier
#

probably that as cooper is a master of typos

frozen badge
#

wrong channel

#

sorry

bright glacier
#

don't worry about it!

plain atlas
#

"This looks great. 10/10 code. I see no reason to merge."

Approves

flat krakenBOT
plain atlas
#

i added this feature to my bot, to help make code easier to read for all of my users

flat krakenBOT
#

Describe the bug
Black does not exclude the filefoo.py root when asked to exclude it with ^/foo.py

To Reproduce
Start with an empty directory

$ echo "s='test'" > foo.py

$ black --extend-exclude  ^/foo.py
would reformat foo.py
Oh no! ๐Ÿ’ฅ ๐Ÿ’” ๐Ÿ’ฅ
1 file would be reformatted.

Same behavior using --exclude flag

Environment

  • Black's version: 21.12b0
  • OS and Python version: OSX 11.6, Python 3.9.6
flat krakenBOT
flat krakenBOT
#

As discussed in #2394, we'd like to shove disruptive style changes between major updates behind a --preview flag. And I think it should be created and available even if such changes have not been implemented yet, so that our CLI is consistent.

Perhaps:

  • add the CLI option
  • create a separate style docs section for describing the style
  • from now on announce these kinds of changes under a "preview" header in our changelog
  • document a consistent way of writing preview tests without d...
glacial swan
#

I am trying to find out what it passes to format_str on default by doing:

black -l120

I am trying to use black's format sytem except for a bot commands meant for reformatting code
I used to use yapf and autopep8 to format it, but i think using black this command and then

black

as a way to format strings would be useful
I was asked to ask in here, as black lacks an api

bright glacier
#

So you basically have a four choices: 1) create a temporary file and then call black in a subprocess on that, then read the file etc. etc., 2) pass the code via STDIN via the subprocess and read STDOUT, 3) import black and call black.main with the arguments you'd use on the command line (note you'd have to capture STDOUT / log STDERR manually), or 4) use black's informal unofficial APIs.

glacial swan
bright glacier
#

alright - it should look something like this to use the "unstable" APIs properly

import black

src = "some code"

mode = black.Mode()
dst = black.format_str(src, mode=mode)
black.check_stability_and_equivalence(src, dst, mode=mode)
#

I quote unstable as it's highly unlikely we would ever be able to change these APIs without breaking the world as APIs will be created, regardless if it's by the authors or the users :p

#

!d black.Mode

late dewBOT
#
class black.Mode(target_versions: Set[black.mode.TargetVersion] = <factory>, line_length: int = 88, string_normalization: bool = True, is_pyi: bool = False, ...)```
Bases: [`object`](https://docs.python.org/3/library/functions.html#object "(in Python v3.10)")
bright glacier
#

!d black.format_str

late dewBOT
#

black.format_str(src_contents: str, *, mode: black.mode.Mode) โ†’ str```
Reformat a string and return new contents.

mode determines formatting options, such as how many characters per line are allowed. Example:

```py
>>> import black
>>> print(black.format_str("def f(arg:str='')->None:...", mode=black.Mode()))
def f(arg: str = "") -> None:
    ...
```  A more complex example...
plain atlas
#

why did no one ask me what i did smh

glacial swan
plain atlas
#

btw hi richard

bright glacier
#

so by default black runs two safety checks, the AST safety check i.e. the more important one where black checks that the AST of the source and output code match. There's also a check to make sure black's formatting is determinant and doesn't change depending how many format passes black does. For historical reasons I don't know (this project started in 2018, I only joined in 2020) format_str does not do these checks.

glacial swan
#
import black

src = "some code"

mode = black.Mode()
dst = black.format_str(src, mode=mode)
black.check_stability_and_equivalence(src, dst, mode=mode)

So this is the normal?

mode = black.Mode(line_length=120)

this for custom?

bright glacier
#

you don't have to call that function but there is a slight chance black will break the source code and you won't be clearly alerted

#

if performance is much more important you can drop it but I wouldn't recommend it

glacial swan
#

๐Ÿค” Well I can always run it in run_on_executor cause I am doing this for a discord Bot

bright glacier
#

mode = black.Mode(line_length=120)
Yeah that's how you'd configure the line_length

glacial swan
#
black -l120

same as this right?

bright glacier
#

Yup

glacial swan
#

Huh

#

Welp Time for me to make a function for this

bright glacier
#

If you want black to be at least AST safe, you can simply call black.assert_equivalent which is pretty cheap

glacial swan
#

๐Ÿค” does check_stability_and_equivalence check both or only one?

bright glacier
#

black.assert_equivalent just does the (more important) former

glacial swan
#

What does check_stability_and_equivalance do when the check fails?

bright glacier
#

it raises AssertionError

#

it also writes a log file on failure which isn't easy to disable unfortunately

glacial swan
#

I really don't want to log a failure lol

bright glacier
#

alright let's dive deeper in black's APIs \o/

#

lemme find the bit in the test suite where we patch out this behaviour

plain atlas
#

just look at the source of the black playground

#

or use the black playground

bright glacier
#
with unittest.mock.patch("black.dump_to_file", lambda *args, **kwargs: None):
    # code ...
bright glacier
#

and I'm pretty sure it doesn't delete / suppress the log files either

plain atlas
#

last updated quite a few months ago, sooooooo

#

i may or may not have gone for it

bright glacier
glacial swan
#

does using try/except stop it?

bright glacier
late dewBOT
#

src/black/__init__.py lines 1299 to 1307

try:
    dst_ast = parse_ast(dst)
except Exception as exc:
    log = dump_to_file("".join(traceback.format_tb(exc.__traceback__)), dst)
    raise AssertionError(
        f"INTERNAL ERROR: Black produced invalid code on pass {pass_num}: {exc}. "
        "Please report a bug on https://github.com/psf/black/issues.  "
        f"This invalid output might be helpful: {log}"
    ) from None```
glacial swan
#

๐Ÿค” hmmm

#

If I use the function, it's going to log it to a file hmm

glacial swan
bright glacier
#

You can't subclass a function but you can overwrite the dump_to_file function with your own that does absolutely nothing

#
import black

src = "some code"

mode = black.Mode()
dst = black.format_str(src, mode=mode)
black.dump_to_file = lambda *args, **kwargs: None
black.check_stability_and_equivalence(src, dst, mode=mode)

this theoretically would work

glacial swan
#

Hmm

glacial swan
bright glacier
#

because dump_to_file is an internal function, not a variable

glacial swan
#

ok

glacial swan
#

nvm found it out

bright glacier
#

like a crash reproducer?

#
def a():
    # fmt: off
    pass

def b():
    pass

is a very old bug that's yet to be fixed

#

this would cause assert_equivalent to fail

glacial swan
glacial swan
bright glacier
#

no problemo! ๐ŸŒบ

flat krakenBOT
#

From #2751: this PR creates a --preview flag for our disruptive style changes. It includes documentation (changes described under "future style"), tests (now empty skeleton) and the flag itself as a CLI option and added to Mode.

The name of the flag was originally --future in the stability policy, but I changed it to --preview based on the original discussion. While I like it more, it isn't really important. Which would y'all prefer?

Checklist

  • [x] Add a CHANGELOG entry if...
flat krakenBOT
#

Description

The CLI reference is a bit buried in the documentation. Hopefully, with this change it is at least a bit more visible. I put the important information first to not distract the reader and changed the summary line to match what a user would actually be looking for when browsing.

Checklist

  • [x] Add a CHANGELOG entry if necessary? N/A
  • [x] Add / update tests if necessary? N/A
  • [x] Add new / update outdated documentation?
bright glacier
#

huh you're back early Jelle ๐Ÿ™‚

dense jungle
#

hello from Jordan ๐Ÿ™‚

flat krakenBOT
bright glacier
#

wow that is a lot of stuff to come back to

muted haven
#

Wow, lots of activity in black ๐Ÿ‘€

bright glacier
#

why is everything I'm working on blocked on removing Python 2 .. ahhhh only diff-shades and the integration is not blocked on it but it's still kinda blocked by the --preview flag

#

well OK I worked around the mypyc incompatibility temporarily so I could run diff-shades against my other PR actually mypyc is still broken

bright glacier
#

OK it's actually working now

plain atlas
#

reminds me of code I wrote

#

implements new feature
pushes and deploys
tries new feature
doesn't work
turns out it was broken
rewrites feature
it still doesn't work
changes enabled flag to true on prod
it works

bright glacier
#
-    headers["User-Agent"] = (
-        f"psf/black diff-shades workflow via urllib3/{urllib3.__version__}"
-    )
+    headers[
+        "User-Agent"
+    ] = f"psf/black diff-shades workflow via urllib3/{urllib3.__version__}"

wtf black

bright glacier
slow ibex
#

is black formatted with black?

bright glacier
#

Yup!

slow ibex
#

nice

flat krakenBOT
#

<!--
Please make sure that the bug is not already fixed either in newer versions or the
current development version. To confirm this, you have three options:

  1. Update Black's version if a newer release exists: pip install -U black
  2. Use the online formatter at , which will use
    the latest main branch.
  3. Or run Black on your machine:
    • create a new virtualenv (make sure it's the same Python version);
    • clone this repository;
    • run pip install -e .[d,python2];
      ...
flat krakenBOT
#

<!--
Please make sure that the bug is not already fixed either in newer versions or the
current development version. To confirm this, you have three options:

  1. Update Black's version if a newer release exists: pip install -U black
  2. Use the online formatter at , which will use
    the latest main branch.
  3. Or run Black on your machine:
    • create a new virtualenv (make sure it's the same Python version);
    • clone this repository;
    • run pip install -e .[d,python2];
      ...
flat krakenBOT
flat krakenBOT
dense jungle
#

time to forget Python 2 ever existed

errant barn
#

Get the glasses and shampagne!

tired shard
#

we could remove 33, 34, 35, 36 also right? since 3.6 has reached eof (same for the others)?

errant barn
#

There has been talk on the Python support policy issue, but not likely yet

dense jungle
#

We don't necessarily have to, and there's much less cost to keeping them around than for Python 2

tired shard
#

there's much less cost to keeping them around than for Python 2
wdym by this

errant barn
#

less special syntax differences

#

Python 3 minors are much more uniform, disregarding async/await and some others

tired shard
#

yeah that's what i gussed

errant barn
#

Also Jason have you had a look at the --verbose comments? (finally I can poke you instead of the other way around ๐Ÿ˜‰ )

tired shard
#

yeah did, haven't got time to mention them yet

#

just got back yest from break

errant barn
#

oh nice, relaxed?

plain atlas
#

smh don't poke jason that's my job to do when he's not poking me

tired shard
#

will try doing it by this weeked

#

hmm

errant barn
plain atlas
#

don't poke my poker

errant barn
#

CANTREADMY CANTREADMY I think I'm losing control of my chatting again, better continue work

tired shard
#

will finish off my diff shades - rich diff PR and get back to verbose

#

also, one more question, i asked this to richrd in dms, thought of putting it here also, he hasn't answered yet

have there been any discussions regarding making the rich output/verbose logs more pretty, preferably by introducing rich to the project

mint barn
#

s n e k

dense jungle
errant barn
#

hmm, we do have color, but what else does Rich do?

tired shard
#

we have colour, but it is not as easy as rich to work with the colour we have lol

#

that's the only thing i was looking to have

dense jungle
#

that doesn't feel worth the cost of a new dependency

plain atlas
#

!pypi rich

late dewBOT
#

Render rich text, tables, progress bars, syntax highlighting, markdown and more to the terminal

errant barn
#

It's pretty easy with Click, no?

mint barn
tired shard
#

nope, on mobile, let me see if i can come up with an example

#

uncreative its getting nnoying, could you stop, thanks

plain atlas
#

smh how many language codes is Russian now

plain atlas
tired shard
#

yeah, click only supports one style per string

errant barn
#

Oh I see

tired shard
#

like you can have something like this:

The [blue]**uncompromising**[/] [gold]Python[/] code formatter
errant barn
#

That is pretty neat, but we could achieve this with a style submodule as well: f"The {s.blue('uncompromising')} {s.gold('Python')} code formatter"

#

wouldn't be too much imo

#

And I don't think it's often that we would want multiple colors on a single line. Or did you have something in mind already?

tired shard
#

that's perfect

errant barn
#

Well I haven't ever used that, but surely I can echo a combination of strings ๐Ÿ˜„

#

And those would just be short cuts

bright glacier
tired shard
#

yeah, like they said, i will see if what felix said works once i get the other stuff done

late dewBOT
#
ROGER THAT

Your reminder will arrive on <t:1642423945:F>!

errant barn
#

Before you update the whole PR though ๐Ÿ˜… it'd be better to discuss this with others as well. If our prints are significantly nicer looking and significantly nicer to write using the short cuts, we could do this to all prints.

#

i.e. could be worth it to submit another PR

tired shard
#

yeah

#

not gonna postpone this more lol

errant barn
#

But that could be cool, since we could define a few styles and just use them. Like def error(s): click.style(s, color="red", bold=True) or whatever

#

buuuut others might disagree

tired shard
#

that's what i was planning to do, support raw and have few pre-defined styles also

#

after discussing with you guys ^^

#

ok, saw your review, regarding the tests, umm let me find the dicussion link

#

teh summary is that, the feature is basically using normalize_path_maybe_ignore which is already well tested, so there isn't a need to add more tests to it

errant barn
#

right, good!

tired shard
#

ok others are pretty straightforward, i will fix them, once i get back at my laptop

late dewBOT
#
Out of the question.

Your reminder must have a content and/or reply to a message.

late dewBOT
#
Of course!

Your reminder will arrive on <t:1641826890:F>!

errant barn
#

wait, doesn't your comment chain say that there are no tests for the function?

tired shard
#

hmm, wait a second, afair i said there are some tests for it later, and then say those are enough

#

one sec

late dewBOT
#

tests/test_black.py line 1400

normalized_path = black.normalize_path_maybe_ignore(path, root, report)```
tired shard
#

yeah tehre are tests for it

#

lemme find where i said that

errant barn
#

oh kay ๐Ÿ‘๐Ÿฟ

tired shard
errant barn
#

did i just type out "ok"

tired shard
#

there

#

lol

#

also, if you got sometime, could you review 2525 ๐Ÿ™ƒ

errant barn
#

the last comment is mine, so when everything is taken care of, I think an accept is next!

plain atlas
errant barn
#

it was supposed to be "oh, okay" ๐Ÿ˜„

tired shard
#

oo perfect, one more pr getting close

errant barn
#

again, if others don't have further comments :D but as I said, it is starting to make sense now ๐Ÿ‘Œ๐Ÿฟ

plain atlas
#

meh imma go bak to writing bad code

#

that typo wasn't intentional

tired shard
#

!remind delete 3886

late dewBOT
#
Sure.

That reminder has been deleted successfully!

tired shard
#

oh no

#

conflcits

#

i hate them

#

we really need something like towncrier

errant barn
#

oh now that I scroll down again I saw your planned style Enum, that actually looks good!

plain atlas
errant barn
plain atlas
#

wait.

tired shard
#

first get them convinced

plain atlas
#

oh i skipped that step

#

discord-modmail/modmail#89

toxic stormBOT
tired shard
tired shard
#

ah yes

flat krakenBOT
tired shard
#

workflow run approval :D

errant barn
#

sry I'm only in my second evolution stage called "collaboratron" :D

#

we'll have to ask one of the "maintenauts"

tired shard
#

well well, do the honours maintenauts

errant barn
#

(how can this be made a thing, imagine evolving in FOSS)

tired shard
errant barn
#

Not sure if worth it ๐Ÿ˜… let's start from applying the styles in the first place

#

Parsers sound like edge cases and weird bugs to me

#

Nevermind the fact that we're developing one ๐Ÿ˜„

dense jungle
#

I clicked the approve button, wish I could let more people click that button

errant barn
#

I don't suppose GitHub lets owners assign individual rights to collaborators?

tired shard
#

not inside repos

#

in orgs, yes iirc

errant barn
#

Damn.

dense jungle
errant barn
#

right

dense jungle
#

and you have to do it again every single time they push to their first PR

plain atlas
#

you can change that now, you know

errant barn
#

thaaat's annoying :D but after hearing about the secret leaks might be justified

dense jungle
#

like Jason is going to set up a bitcoin miner now in that PR after so many rounds of review

dense jungle
plain atlas
#

can yes, awake enough to? no

errant barn
#

well, at least runs on forks work right?

errant barn
dense jungle
# plain atlas

thanks! looks like I don't have access to repo settings for Black, I'll have to poke Lukasz to do this

dense jungle
#

thanks for your contribution @tired shard !

tired shard
#

woohoo, less go

#

thanks guys for the reviews and help :D

errant barn
#

you're most welcome!

tired shard
#

now, jelle 2525

#

poke poke

errant barn
#

po-po-po-poking face

tired shard
#

i was looking to get richard's review on that though

dense jungle
#

is that the nested config files? yeah I need to get around to that too

errant barn
#

ye

tired shard
#

yeah

dense jungle
#

but feel like I should focus on getting the stable release ready

tired shard
#

hmmm, i let you go for now

dense jungle
#

Speaking of that, I think we should include ESP in the --preview flag for this release

errant barn
#

makes sense ๐Ÿ‘๐Ÿฟ anything else on your personal wanted list?

tired shard
#

esp is?

errant barn
#

electronic signal processing --experimental-string-processing

tired shard
#

oh

#

yeh, makes sense

#

i will try to help out and do some reviews

dense jungle
errant barn
#

Yep the GH stable project has been cleaned up by Richard!

#

Was just thinking that maybe you have something else in mind, to add there

dense jungle
#

ok thanks, I think only --preview there is essential

#

the ** spacing thing can wait

errant barn
#

Fair

flat krakenBOT
errant barn
#

Made an issue, perhaps this should be in the project as well

errant barn
errant barn
#

thanks!

muted haven
#

Jelle / Richard: are there any plans about the parser speed-up PR (whether it will go in this release or not)?

dense jungle
#

oh ideally it should. I wanted some more time to think about it or look into the code, but haven't had a chance to dive into it yet. I was wondering if we could implement this approach without monkeypatching.

muted haven
#

So instead of these 2 different functions overriding each other:

    def shift(self, type: int, value: Text, newstate: int, context: Context) -> None:
        """Shift a token.  (Internal)"""
        dfa, state, node = self.stack[-1]
        rawnode: RawNode = (type, value, context, None)
        newnode = convert(self.grammar, rawnode)
        assert node[-1] is not None
        node[-1].append(newnode)
        self.stack[-1] = (dfa, newstate, node)

    def shift_safe(
        self, type: int, value: Text, newstate: int, context: Context
    ) -> None:
        """Immutable (node-level) version of shift()"""
        dfa, state, _ = self.stack[-1]
        self.stack[-1] = (dfa, newstate, FAKE_NODE)
#

we would have a unified shift function:

    def shift(self, type: int, value: Text, newstate: int, context: Context) -> None:
        """Shift a token.  (Internal)"""
        if self.back_tracking:
            dfa, state, _ = self.stack[-1]
            self.stack[-1] = (dfa, newstate, FAKE_NODE)
        else:
            dfa, state, node = self.stack[-1]
            rawnode: RawNode = (type, value, context, None)
            newnode = convert(self.grammar, rawnode)
            assert node[-1] is not None
            node[-1].append(newnode)
            self.stack[-1] = (dfa, newstate, node)
#

If that makes more sense, I can convert my PR to that form. WDYT?

silent apex
muted haven
#

Sure thing!

silent apex
#

Thank you!

dense jungle
bright glacier
#

By way there's like two more mypyc breakages on main so it's hard to test the PR for compatibility when main is already broken :)

#

I know the fix for one of them but I haven't been able to fix the other, will probably have to do a git bisect .. which sounds tedious haha

dense jungle
muted haven
flat krakenBOT
muted haven
flat krakenBOT
muted haven
#

I've written something to try parsing a local copy of 4k python packages from PyPI with the regular lib2to3_parse and the 3.10 one. Will report back if there are inconsistencies

flat krakenBOT
#

The regular parser ignores token.NL/token.COMMENT since the grammar assumes there can be any number of them during parsing. This is actually handled in the driver rather than parser, so when the parser got implemented initially we did not specifically instruct this. But seems like it is now needed.

https://github.com/pantsbuild/pants/blob/aa1d38d4f3b791509bff773a65c2b07490bacad3/src/python/pants/util/enums_test.py#L33-L41

This seem to only cover cases where a standalone match or `...

muted haven
errant barn
#

rapid fire :O

dense jungle
muted haven
#

@dense jungle I think I misunderstood your review. Would the change in the input (adding a trailing comma) produce the current output (without any trailing comma)

dense jungle
muted haven
#

weird, it is not ๐Ÿค”

dense jungle
#

Because if there's a trailing comma we should always explode

dense jungle
#

we can track and fix it separately though, it doesn't have to hold up this PR

muted haven
#

nop, seems like there are 2 identical match statements and I edited the wrong one ๐Ÿคฆโ€โ™‚๏ธ

#

it is working now, sorry for the confusion

bright glacier
# dense jungle what's the issue? I can help take a look

hmm I can't reproduce the second issue on current main, the other fix is still required but this surprises me. I suppose dropping Python 2 fixed other issues? I'm running a diff-shades run to check nothing's really broken .. somehow

#

Though we should probably try updating to mypy 0.931 to pull in the latest performance and compatibility fixes.

dense jungle
bright glacier
#

Nope that's very weird

#

what mypy version?

dense jungle
#

0.931

bright glacier
#

hmm did they introduce another breakage? I'mma check

#

I'd guess it's trying to read the set name in the class __dict__ as it seems eerily similar to that negative integer constant miscompilation issue. I'd have to check the generated C to know for sure though.

dense jungle
#
    cpy_r_r684 = CPyStatics[744]; /* 'set' */
    cpy_r_r685 = CPyObject_GetAttr(cpy_r_r683, cpy_r_r684);
bright glacier
#

huh.

dense jungle
#

oh wait there's another one

#
    cpy_r_r676 = (PyObject *)&PyType_Type;
    cpy_r_r677 = CPyStatic_mode___globals;
    cpy_r_r678 = CPyStatics[744]; /* 'set' */
    cpy_r_r679 = CPyDict_GetItem(cpy_r_r677, cpy_r_r678);
#

reproes with this ```from dataclasses import dataclass, field
from typing import Set

@dataclass
class X:
s: Set[str] = field(default_factory=set)

#

works on 0.920

#

(and on 0.930 mypyc is completely broken)

bright glacier
#
L20:
    r59 = load_address PyType_Type
    r60 = test.globals :: static
    r61 = 'set'
    r62 = CPyDict_GetItem(r60, r61)
    if is_error(r62) goto L36 (error at <module>:7) else goto L21
L21:
    r63 = 's'
    r64 = CPyDict_SetItem(r58, r63, r62)
    dec_ref r62
    r65 = r64 >= 0 :: signed
    if not r65 goto L36 (error at <module>:7) else goto L22 :: bool
L22:
    r66 = builtins :: module
    r67 = 'set'
    r68 = CPyObject_GetAttr(r66, r67)
    if is_error(r68) goto L36 (error at <module>:7) else goto L23

yeah the IR isn't right for that first lookup

dense jungle
#

I'll report a bug to mypyc

bright glacier
dense jungle
bright glacier
errant barn
#

Black?

#

Pinned jam_cuneiform_this and tl;dr it's an autoformatter!

flat krakenBOT
#

copy.deepcopy is still the most expensive part (75/100 of the extra overhead introduced by the backtracking parser). I initially assumed it was mutable (since it was represented as a tuple of lists), but seems like we only use it as a basic table and don't mutate it anywhere in the code.

-t py39 (main):: Mean +- std dev: 7.21 ms +- 0.25 ms
-t py310 (this branch):: Mean +- std dev: 13.8 ms +- 0.6 ms
-t py310 (this branch):: Mean +- std dev: 7.98 ms +- 0.24 ms

This reduces t...

flat krakenBOT
little parrot
#

Doing a great job team

#

:)

bright glacier
#

that was a rollercoaster lemon_sweat

#

@errant barn are you fixing it already?

plain atlas
flat krakenBOT
tired shard
plain atlas
tired shard
#

nothing, i will merge it once it is close to merge state

plain atlas
#

i think you'll find a good portion of conflicts if you try to merge

tired shard
#

yep ๐Ÿ˜” i do

tired shard
#

submitted my contributing experience :)

errant barn
#

much appreciated ๐Ÿ–ค

#

now 2 to go

tired shard
#

lol yeah, some more coming up sometime soon, this time they are formatting based, the "ncie to have" issues

errant barn
#

I have a dream of reaching 20 open PRs and 300 issues again

plain atlas
#

I've done that!

tired shard
#

but gonna take sometime, getting familar with that section of code rn

#

lol, arl is very good at that

plain atlas
#

....wait you mean you want to have that few nvm

#

my body pull count

tired shard
#

also what are your thoughts on teh two points i brought up

errant barn
#

refactoring? yeah big up

tired shard
#

nice nice

errant barn
#

Wu-cash ลukasz refactored the infamous one-file Black a while ago, but it could definitely use some more work

tired shard
#

spring cleanup, remember arl?

#

it was a pr on lance, for the context

tired shard
errant barn
#

yeah I feel for Jelle whenever he merges main

#

how much overhead is there? say using towncrier (have you used it?)

tired shard
#

nope

#

this is what i was talking about btw, discord-modmail/modmail#89

toxic stormBOT
tired shard
#

but it is pretty similar to what i did above, so it is pretty okay

flat krakenBOT
#

As @Shivansh-007 noted in https://github.com/psf/black/issues/2238#issuecomment-1010922392, our current CHANGES.md mechanism for the changelog is very prone to merge conflicts. Another problem is that when a release happens between the time a PR is opened and when it is merged, it's possible to have the changelog entry go into the wrong section.

I haven't looked in detail in this space, but I know that possibilities include:

  • towncrier
  • blurb, which is used by CPython
  • writing our ow...
errant barn
#

aye :D

tired shard
#

one more pr coming from then

#

lol

#

basically the own at modmail, is a mixture of blurb and towncrier

#

i coul possibly make that a separate lib pithink

errant barn
#

Have at it!

dense jungle
#

why can't modmail just use towncrier or blurb?

#

I'd prefer Black doesn't have to maintain its own tool

errant barn
#

It could be very useful, if not to us, then perhaps somebody else

#

I had a quick read, and seems like Towncrier somehow generates the news fragments, while blurb is just for combining existing news entries. But that was a quick glance so I might have misunderstood

tired shard
#

hmm not exactly, i don't remember towncrier, but for blurb, it basically makes new fragments for each change into separate folders of that category and supports combining all the misc/news (where the fragments are stored) into one news file

#

nothing more

#

let me have a look at towncrier

plain atlas
#

towncrier does pretty much the same thing

#

although the large hurdle was that towncrier couldn't be configured to use md

tired shard
#

ah yeah

#

also no gh validation

#

and doesn't integrate to eh same version log

#

makes a new one for each release

#

(both blurb and towncrier)

#

and the most important towncrier is dead

#

so is blurb

dense jungle
#

blurb is very much alive for use in CPython ๐Ÿ™‚

tired shard
#

i am talking about maintenance, i saw it has some issues open last time i checked it but there hasn't been any progress on them

bright glacier
#

I'd much rather use towncrier than a custom built solution, I'll look into it later today

plain atlas
#

i left a comment on the issue about towncrier

errant barn
#

I'm all for changing to rst though ๐Ÿ˜„

#

especially if it's the only thing keeping us from using towncrier

bright glacier
#

Honestly my hope is that we can use towncrier and then do some fancy sphinx extension stuff or something for auto linking

#

that's for you Felix btw, you asked for that didn't you haha

errant barn
#

yeah totally! That'd be cool

#

and it's very easy

plain atlas
#

don't tell jason that I miiiight kill the custom built system for our project when i migrate to sphinx docs

errant barn
#

not automation, just the linking part though :D

bright glacier
#

arl, as much as I like scripting I don't like maintaining a bunch of helper scripts

plain atlas
#

no, you can set up automatically creating towncrier changelog on build of the docs

late dewBOT
bright glacier
#

I've seen Pip's workflow and I am stealing bits and parts of it

#

hence why I also want to introduce nox

errant barn
#

damn, if we provide something to make the changelog fragment and link it, we could possibly use next-pr-number (or what was it) to supply the number so that a user doesn't have to worry about anything but the message itself

#

but not sure how hard it is to commit in actions

#

but this might be just superfluous, let's have the fragments first :D

dense jungle
errant barn
#

right, makes sense

tired shard
#

oh well

#

well

#

||i h8 u||

late dewBOT
#

Hey @fleet depot!

Uh-oh! It looks like your message got zapped by our spam filter. We currently don't allow .txt attachments, so here are some tips to help you travel safely:

โ€ข If you attempted to send a message longer than 2000 characters, try shortening your message to fit within the character limit or use a pasting service (see below)

โ€ข If you tried to show someone your code, you can use codeblocks
(run !code-blocks in #bot-commands for more information) or use a pasting service like:

https://paste.pythondiscord.com

fleet depot
#

uuugh, and I have to retype my whole message smh

#

How annoying

#

Black seems to be misbehaving. When given one folder gluon/, it declares that some files need to be formatted. When provided two folders gluon/ and utilities/ it says all files would be unchanged. how??

errant barn
#

oh wow

#

could you narrow down the example a bit? does it work with smaller folders?

bright glacier
#

Are there pyproject.toml files in either of them?

tired shard
#

update black to main branch ๐Ÿ™ƒ and enable verbose

bright glacier
#

yeah that too :)

tired shard
#

!remind 16h ||how was your first paper ๐Ÿ‘€||

late dewBOT
#
Sure thing!

Your reminder will arrive on <t:1642059096:F>!

bright glacier
#

eh, I'm not sure if I like the idea

narrow sand
#

what's this channel for?

errant barn
tired shard
#

basically, checks if changelog exists for current change, if not prompts if you want to create changelog, sees if a pr for this already exists if not get next pr (maybe again prompt?), follows normall news add process

#

that's what i thought

#

maybe helpful

narrow sand
#

whats the black-formatter?

tired shard
#

not completely sure

#

see the pins channel description hrl077

errant barn
bright glacier
#

it feels like overengineering too early

#

we don't even have news fragments, let's do that first and trial that

tired shard
#

hmm, i was taking about my changelogger

#

so what are we going with finally?

bright glacier
#

next-pr-number is in JS btw Felix

bright glacier
errant barn
tired shard
#

well i am making mine a package also, so if you want you could look at that too, current just that PR

bright glacier
#

yeah but it's all client side, there's no API or whatever

errant barn
#

oh damn

bright glacier
#

I haven't found any hosting options that I could afford (i.e. I have no spare funds)

tired shard
#

vercel :D

bright glacier
#

so GitHub pages and client side code FTW

errant barn
#

yeah prolly for the better

#

so Jason you said that your tool was an amalgamation of the other two options, could you tl;dr why did you make it?

tired shard
#
  • no gh validation
  • no way to append the version changelog to the main one, towncrier makes a new one for every release
  • towncrier is somewhat dead
  • no markdown support
  • no way to create two changelogs for the same PR

this is what i remember from top of my head

#

also, i think towncrier is based on issue numbers rather than PR numbers, not sure

errant barn
#

reasonable

bright glacier
#

We force changelog entries to have PR numbers so it should be fine, though sometimes we do have multiple PRs for a single entey

tired shard
#

uhh is that pointing to the last one? i meant the other way round

#

also, i don't think towncrier has a easy integration with pre-commit and actions, not really relevant but okay

bright glacier
#

I don't see why a pre-commit integration would be important as not all commits need a changelog entry

tired shard
#

yeah, that's why i said "not really relevant"

bright glacier
#

oh I thought you were saying my opinion was not really relevant because towncrier has these other flaws / cons

bright glacier
#

I still have no idea what you meant here

errant barn
#

I think that changes can have multiple prs

tired shard
mint barn
#

what is a black-formatter supposed to be about?

errant barn
plain atlas
#

opposite of whiting your code

errant barn
#

xD formatting your disk

tired shard
#

so many people asking what is black

errant barn
#

oh it wasn't a typo

mint barn
#

okay i opened the wesbite and I still don't get it

bright glacier
tired shard
#

"The uncompromising Python code formatter" tldrs it

mint barn
errant barn
#

you have code, we make it beautiful

mint barn
#

to be fair it looks like the same code in vscode

tired shard
mint barn
#

wdym by beautiful exactly?

tired shard
#

good to look at?

bright glacier
#

Yes.

errant barn
#

and reasoning

mint barn
#

alright alright

errant barn
#

so @tired shard you mean that with your version we could have 2 news entries for a single PR? I'm still a bit confused :D

bright glacier
#

I'd assume that'd be useful in cases where an unrelated change fixes two bugs or something

errant barn
#

yeh

plain atlas
#

i think towncrier can have multiple changes for a pr?

bright glacier
#

Great, you two are now arguing what cons town crier has