#black-formatter
1 messages ยท Page 15 of 1
Guess I should add --check to my precommit config....
Stop reformatting and then rollback lmao
using the stable tag (which can change) is what's not good
That comment came right after so I figured they were related.
well the use of the rev: stable makes it look like pre-commit will track changes, but it won't (which IMO is good because different versions of black have different formatting so upgrading should an explicit decision), explicitly configuring a specific version of black is better
Understood, thanks
No worries, this area could definitely do with some better docs
And the release process for 21.6b0 has been officially completed ๐
4a007a8 Find pyproject from vim relative to current fil... - austinglaser
52384bf Vim plugin fix string normalization option (#1869) - btel
Unix tools often rely upon the files ending with a newline character to show the output properly (e.g. when the file has no newline at the end, cat file will cause the prompt to show right after its output without a newline in between). This may help minimize the diffs and ensures consistency across all files.
I've also added auto-removal of trailing whitespace since I don't think that outside of test data (which is excluded from pre-commit) anyone would actually want to have trailing wh...
yes, but it probably has even fewer options to configure than prettier
but that's why it's called opinionated code formatter :)
It's not meant to be configurable
It's supposed to take away the confusion of why to format what in a certain way, and they do that by not giving you a choice.
oh
atleast theres no 200 different ways to do it and everybody chooses something else
Update pre-commit config
via pre-commit autoupdate
Updating https://gitlab.com/pycqa/flake8
... updating 3.9.0 -> 3.9.2.
Updating https://github.com/pre-commit/mirrors-mypy
... updating v0.812 -> v0.902.
Updating https://github.com/pre-commit/mirrors-prettier
... updating v2.2.1 -> v2.3.1.
well I just approved the CI run for that PR, it's probably going to fail due to missing typeshed packages unfortunately, I wish pipenv locking was faster
!remind 4H review the pre-commit PR and update the requirement files to include typeshed packages
Your reminder will arrive in 4 hours!
How do i get started with black? I've never used it before so looking for some tips n tricks
We recently switched from pipenv to poetry for Python Discord project for various reasons, mainly because pipenv was way too slow for our liking.
Would switching to a different package manager something you'd like to look into for Black? Poetry also has a better integration for package development, it is quite nice imo.
While I'm not happy with it, I'm also honestly scared to change what works "fine enough". IIRC zsol was working on porting black to poetry but the work was never finished. Also IIRC ambv was supportive. I guess we can look into it again.
I will say one thing, poetry works nowhere as nice with different python versions
I did a full write up back then
psf/black#1411
I can dig it up
Haha, "the" write up
Did a lot of them back then. Was quite chatty
poetry is a whole different build system, pipenv is just (or mostly?) for development on the other hand
I'm all for less pointless discussion regarding trailing whitespace :D
Thank you again!
sir-lancebot#623
oh, so you can use it without switching the build system as well?
Yep, and that's probably why I'm hesitant.
Not really? They are both package managers at their core. The build system is an extra
Though Iโd think it would be quite nice for this?
The build system that is
I mean, I am sorta being a hypocritical because I just used flit for my newest project but shrug
with its own build system
I mean, newest project is not the same as existing project
I'm not trying to say poetry (as build system) is bad though, it's just different which means that more stuff needs to be considered before pursuing it
I feel like install time could be longer since it's non-default build system (and so pip needs to install it before build)?
not by much I would assume though
Yeah I suppose. I'll have to take a look at this eventually since it's part of the "improving the contributing experience" issue / task I have on my radar
Well that's only for sdists, and black being a pure python project, wheels should work in 99.99% of cases
I guess that's true
did not think about it lol
I haven't used alternative build systems much yet so I guess I just don't know much yet
Although you aren't wrong in a way, if we start compiling black with mypyc (which is what I am working on right now) then our wheels won't be pure python anymore. In that case though, I'd expect us to still publish a pure python wheel with 100% python and 0% mypyc alongside with the mypyc compiled wheels
I'd recommend trying flit for any small personal projects, it's quite nice
oh ye, I did
only like once but well
was rather recent so I also could take advantage of PEP 621
IIRC flit just landed experimental support for that pep right?
it's almost 3 months since it landed but it is still the latest release so ye
unfortunately there's no support for standardised pyproject.toml based requirements
takluyver/flit#373
unless pep 621 also defines and standardises requirement metadata, which in that case, my bad!
it did, ye
well
it might have been separated, not sure
but it definitely was part of it in the end
[project]
# snip
dependencies = [
"dearpygui~=0.6.415",
]
[project.optional-dependencies]
dev = [
"black==21.5b2",
"flake8==3.9.2",
"isort==5.8.0",
"mypy==0.812",
]
Oh, pep 621 defined the general structure of the pyproject.toml project metadata while pep 631 is like a spec for the requirement metadata?
!pep 631
there were 2 competing PEPs for specifying dependencies
but the one that "won" got merged into 621
Hmm, I gotta try and migrate my project to the 621/631 standardized metadata format, TIL, thanks!
Personally I used PEP 621 because I found flit's definition of entry points a bit confusing
but they're generally rather comparable
Hmm why the double labeling
one thing to keep in mind is that you need to define version and description as dynamic fields if you want flit to export it from your code
Look at pipenv
Toml header & dict
wasn't there another pep for sdist metadata and the dynamic marker... sigh
Probably, they are often many PEPs on different aspect of a subject for some probably very good reason
I agree with keeping them separate but man is it annoying to keep up with the packaging changes sometimes
Does the "rtfd" in black.rtfd.io stand for "read the fucking docs" lol?
it's the price we get to pay for a short url from readthedocs I guess lol /j
Oh ok
Still pretty nice lol
is there a reason why the quotes/strings argument is -S instead of -s?
I always get it wrong on the first try because the help example isn't next to other text, so I don't notice the capitalization the first time.
I don't know if this is why Black chose that or not, but there's a kind of loose convention that lowercase switches are for enabling some behavior and uppercase are for disabling some behavior. That's not always very consistent across tools, though
I've always noticed that lowercase switches were the default scenario, and that uppercase was for switches that would be abbreviated in the same way, but were implemented later.
e2fd914 Fix internal error when FORCE_OPTIONAL_PARENTHE... - bbugyi200
52f402d Add EOF and trailing whitespace fixer to pre-co... - jack1142
@bright glacier
Here's your reminder: review the pre-commit PR and update the requirement files to include typeshed packages.
[Jump back to when you created the reminder](#black-formatter message)
oh right
I'm sure there's copiuous discussion on this, but why is line comment length not affected by the length parameter?
There's an ongoing issue about splitting comment lines as well! See psf/black#1713
Oh I was referecning block comments, like this docstring, which presumably is an entirely separate formatting discussion:
IN this particular case, they're all bounded to 120 characters instead of black's default 88, so at least this specific instance is handled invisibly wiht -l 120
I might be wrong on this one, but I don't think Black formats docstrings besides a few modifications to indentation and newlines.
RIght, it didn't. I was wondering if there was already an existing discussion with respect to that before I went asserting my opinion.
There's psf/black#2289 at least!
Very fresh issue indeed.
A simple change to the documentation to remove the claim that a space is inserted to empty docstrings. That behavior was yanked in #2249!
Also a friendly reminder to us to check for / require documentation modifications when the style is changed ๐
and I was about to approve that
got to be faster
Jelle and I end up handling the same issue / PR at the same time so frequently
it's bonkers
well, I'mma finally review the pre-commit PR whose reminder went off like two hours ago ... yeah I'm totally fast ๐
I already merged it, sorry ๐
I meant this one: psf/black#2331 ๐
oh yeah I merged the trailing line one. that one needs a Pipfile update I believe
Yep.
I can't wait once the "expose static metadata in simple api" for indexes pep is accepted and implemented, should cut down on resolving times significantly ๐
That will probably take a while tho
might not be too bad? I bet the PyPI people really want it too so they can save on bandwidth
Hmm, does RTD need to be triggered manually when the stable tag is moved? The docs aren't updated to 21.6
I'll look into it
also, I don't think I did the pipenv locking correctly
or the github dependency review tool is broken
unfortunately it appears prettier changed their JSON formatting style so it's impossible to tell what pipenv lock changed
they were built yesterday?
and I forgot to include types-click .......... whelp time to relock
but hmm, indeed they seem out of date ๐ค
yeah can confirm they are out of date, wonder what happened, looking into it right now
oh, it looks like I somehow messed up the updating of the stable tag
wait no
Oh? It was on the correct commit though
my local copy is wrong
points to the right commit, you're right felix
huh then
more detective work I guess
(by the way, this is why I like using a branch for stable, a tag isn't automatically updated)
Fair enough, the situation will improve when the real stable is released since the latest tag will be built
Looking at the RTD stable build that was triggered yesterday, for some reason it picked up the old stable rev: https://readthedocs.org/projects/black/builds/13994867/
let's try rebuilding stable
and now it picks up the right commit ... https://readthedocs.org/projects/black/builds/13999229/
Good catch Felix!
Well at least it's good now :D
I wonder if RTD can be configured to have stable on any tag, not just stable tags (assuming it has a distinction rather than it was assigned to only build the tag named stable)
Well, if there isn't a specific "stable" rev in the git repo, it automatically points the stable docs to the latest tag IIRC
but we need the "stable" rev (i.e. a "stable" tag) for other situations so yeah
Like the GitHub Actions action (not a typo) we provide
I'm not sure how often pathlib.Path is used with the augmented assignment operator rather than the regular one (/) nor do I know how often the string would be long enough that it would need to be split but all that aside, /= can definitely appear with strings on the right-hand of the assignment operator too:
from pathlib import Path
x = Path.cwd()
x /= "directory/subdirectory/file.txt"
_Originally posted by @jack1142 in https://github.com/psf/black/issues/2312#issuecom...
and once again my displeasure with the Pipfile file format is back again
although that's also because I don't know to write environment markers in any requirement files by heart
yo guys i just downloaded this black formatter bc i was curious
could you help me run it on a file
black [your-file]
in command line?
yes
also do i need to have a virtual environment active
it's a command line only tool (there are library features but we don't officially support them just yet)
well, I mean, for your black install? if you want to isolate black and its dependencies from the rest of your python environment
but black doesn't need a virtual environment to run
epic
thanks dude!
No worries, have fun!
is there an "official" email contact for black? Would that just be PSF?
I'd imagine Lukasz' email as the author is about as official as it gets: https://github.com/psf/black/blob/main/setup.py#L59
setup.py line 59
author_email="lukasz@langa.pl",```
Thanks
Also, any tips on speedrunning ล? :D Every time I care I end up searching his github and copying the character. I couldn't find an alt code (that works) for it either
For Ubuntu I use ctrl shift alt + 141 to get that L (sorry gboard doesn't have that L for some damn reason)
no idea whether that would work for you tho
Yeah for some reason that's "รฌ" for me :D
Windows?
Yuup
whelp IIRC it does alt codes differently
0321 is something I found via a Google search, maybe try that?
:D that would be an "A"
I give up then :(
Ima have to create a macro for this :D
Pretend you're Polish by changing your keyboard layout obviously :D
Oh God..
more seriously though, you could actually make a custom keyboard layout
And choose some key combination that isn't used for writing ล
could even go for regular (Shift+)AltGr+L if your keyboard layout doesn't use that
Yeah that would be nice!
the above is called Microsoft Keyboard Layout Creator btw
- mdit-py-plugins requires Python ~=3.6, so it will not be satisfied for Python >=4.0
Context please?
poetry knows something I don't ๐
I'm getting to it lol
Was giving setting up a poetry branch a shot
I'm the middle of debugging this CLI application I'm making
Oh nice, thanks!
Got roadblocked by dependencies. MyST-Parser requires mdit-py-plugins
and somewhere in there python 4 came up
what are you working on ๐
A benchmarking suite for black
that sounds great for what I'm testing lol
it's for my mypyc work ๐
I'm on like rewrite #4 for my data modeling
I think this one should stick though
Yeah, I'll make an announcement once I have an usable dev version done since I'm going to need some actual files to benchmark black against
Right now my target files is a checkout of the black repo from rev 21.6b0
here's a sneakpeek ๐
blackbench is basically a collection of benchmarks + black specific configuration support for those benchmarks + a fancy frontend to pyperf
it's like pyperformance but for black
!pypi pyperformance
But thank you!
Its a profiling tool specific to Black, a tool that itself provides only offline delays?
Can you please explain what "offline delays" means here? I don't understand, sorry.
Like a slow compiler, it has no bearing on the runtime of the target code. That is, no matter how long it takes "Black" to be run against a library, say for the sake of argument, requests, it will only be run once, and then requests will continue to operate at whatever speed it can.
rust suffers from this, in that the rust compiler can be very slow compared to the same program written in c and compiled with gcc, but the ultimate output is comparable.
Okay I understand, thanks for the explanation. Yes, blackbench is intended a performance measurement tool for black (it doesn't collect deep enough data IMO to count as profiling).
It's because I have a project to improve the performance of black using mypyc and I needed someway to measure the real-world improvements.
Not to deminish your work, but since I've never run black against a large dataset, how slow is it really?
And, in truth, first-time high-latency may be OK, if subsequent runs are vastly improved.
IDK if subsequent runs are in fact faster though.
Subsequent runs should be faster due the cache black maintains but the first runs can be quite slow
although black is a multi-core application so it depends
the main benefit of increasing black formatting throughout IMO is when you configure your editor / IDE to run black automatically on save
decreasing that delay between save and formatting finished makes those users' lives easier and less annoying
INdeed it would, especially for tools that do background autosaves.
I'd like to see IDE notations before automatic reformatting; simply highlight the lines like PyCharm can do for pip-8/flake-8 line lengths and suggest the fix, offering the keyboard shortcut for formatting that segment, instead of just rearranging the code automatically.
Especially if I have autosaves enabled and I'm still typing.
Though thats out of your scope.
But low latency coverage would be very helpful for such a product.
That would be hard to implement for Black since 1) we would rather not provide first-party support for formatting a specific range of lines in a file, and 2) black doesn't really provide any fine-enough output to implement that cleanly in an IDE
Yea I'm sure its a very complex thing to accomplish.
Yep. It especially gets tricky when the range isn't valid Python.
I do wish there were a way to recognize patterns. It undoes some of my formatting to make them consistent:
Hmm have y'all ever run into I/O and event loop and file permission issues when running primer? This is related to my poetry work, but I figure I may as well ask before I go mad digging into the code.
Full TB:
https://paste.pythondiscord.com/avabamasax.lua
It's one hell of an ugly traceback. I think it printed things intertwined.
though it does look like adding a trialing , to every method call makes them the same, even if that is a bit aggressive.
the magical trailing comma lol
Aaaaalright, and I'm throwing in the towel for tonight. The primer thing is still an issue, but I'll look more into that soon. I've gotten some nice work done.
I've got two branches to show for it:
https://github.com/HassanAbouelela/black/tree/poetry - Exactly what it sounds like. Moved from pipenv to poetry, no other changes.
https://github.com/HassanAbouelela/black/tree/poetry-build - This one is a bit more involved. It moved to poetry as the dep manager & build and publishing tool. It might not be desired, but I made it as a POC and for fun anyways. There are still things that need to be added (see missing.md). the published package is here: https://test.pypi.org/project/black-poetry/
https://github.com/HassanAbouelela/black/blob/poetry-build/missing.md actually applies to both branches for the most part.
The magic trailing comma feature can be disabled with -C, although that doesn't help you with the rest of your problems
I saw that. Though it appears that running black without - C, then with, then without, is not idem potent, since it looks like it can remove some trailing commas. Git diff shows deltas. I need more testing to narrow down the details.
Alright so -C is not just skip magic trailing commas, in some instances it will affirmatively remove trailing commas.
All done! โจ ๐ฐ โจ
9 files left unchanged.
C:\Users\Xavier\PycharmProjects\SplitFileReaderProject>.\venv\Scripts\python.exe -m black -l120 -S -C .\split_file_reader\
reformatted split_file_reader\__init__.py
reformatted split_file_reader\split_file_reader\__main__.py
reformatted split_file_reader\http_file_reader\http_file_reader.py
reformatted split_file_reader\split_file_writer\split_file_writer.py
reformatted split_file_reader\split_file_reader\split_file_reader.py
All done! โจ ๐ฐ โจ
5 files reformatted, 4 files left unchanged.
C:\Users\Xavier\PycharmProjects\SplitFileReaderProject>.\venv\Scripts\python.exe -m black -l120 -S .\split_file_reader\
All done! โจ ๐ฐ โจ
9 files left unchanged.```
1st to show that there is no diff if run without -C.
Second makes the changes with -C
Third to show that no -C makes no further changes.
Diff has an omitted trailing comma on some arguments.
IDK if this is expected behavior, seems slight contrary to the documentation.
Looks like expected behavior to me
The problem I have is that the documentation for command arg -C is
Don't use trailing commas as a reason to
split lines.```
However, this means that executing the command with `-C` will cause some trailing commas to go away, meaning the command `python.exe -m black -l120 -S .\split_file_reader\` produces different output, depending on whether `-C` has _ever_ been used in the command, not just on this particular run.
My assertion is that, if this is expected behavior, the documentation should say: -C, --skip-magic-trailing-comma Don't use trailing commas as a reason to split lines, and remove extraneous trailing commas in method signatures.
I do agree that that documentation can be improved. It's not just about method signatures though, but also affects lists and other collection displays
Because its not just a skip, its an affirmative change to the code.
Sure, method signatures is the only delta I noticed in my code base, so it was the only thing I could assert was changed.
I also argue that the long-form command should be --remove-magic-trailing-comma, as skip implies a passive selection.
or similar
OK, so, if I add a magic trailing comma to all of the method calls, then the call to python.exe -m black -l120 -S .\split_file_reader\ makes them all the same. But if I run it with -C, then it removes some of the magic trailing comma and returns the code to this mixed formatting state. Then, running python.exe -m black -l120 -S .\split_file_reader\ does not rstore the magic trailing comma
Sorry, I quoted a screenshot showing what I meant.
Is there an equivalent --insert-magic-trailing-comma option to aggressively insert them where possible?
No, there isn't. The point of the magic trailing comma is to preserve existing formatting to minimize diffs, not to create a different style
Fair enough
Someone is working on adapting the PEG parser generator to generate Python instead of C. I just accepted their additions in the google doc, it's still a WIP and would definitely need further work regarding whitespace, but I'm already surprised
Black blib2to3 to $NEW_PARSER Lib2to3 is deprecated. Therefore, with blib2to3 being built on top of said deprecated library, itโs not going to evolve with new language changes. Due to this, black will need to adopt a new base parsing library. Purpose of this document This document is to describe...
I've got indented blocks and simple statements parsing! ๐ I can now parse chained pass statements like pass; pass; pass; pass ๐
or even pass; ...; pass !
And continuing on the apparent train of progress in the past few days: I got the first development version of blackbench, my very own benchmarking suite for black, completed!
All maintainers should be sent a black bench at once to sit on and celebrate.
lol, .. Anyway, blackbench 21.6.dev1 doesn't come with much benchmark data to run black against. It's basically a 21.6b0 checkout of black's source code + one string micro target (based off of Jelle's discovery of ESP causing slowdowns). I'd appreciate more code files to throw to black.
could be nice to have a mode that prints out how long every file took to format
that way I can run it on quora's codebase and see what's slowest
I'm looking for two kinds of targets: "normal" and "micro" . Normal targets should represent real-world code (so the benchmarking data actually represents real-world performance). Micro targets should be small and are focused on one area of black formatting (and mostly exist to measure performance in a specific area, like string processing).
I could try to add support for that in a custom target feature (I want the built-in targets to be good enough most of the time). Note that blackbench does provide per file performance data (thanks to pyperf, the underlying library handing most of the work).
Nice, I think this could be a feature of Black itself not blackbench
I would be supportive of a such feature, although I think we would need to be clear that black won't be providing in-depth timing data ๐
I'm just scared of the slippery slope that could occur.
Agree, I might just hack it in locally and see the results
Using the github action with the specific version or with the @stable tag directing to it results in the following error:
Run psf/black@stable
with:
options: --check --diff
src: .
/usr/local/bin/python3: can't open file '/home/runner/work/_actions/psf/black/stable/action/main.py': [Errno 2] No such file or directory
Error: Process completed with exit code 2.
Running the action with:
psf/black@21.5b1
solves the issue.
Looks like I broke the GitHub action somehow
don't understand the failure though, will have to look into it later I guess
I do find it odd that it would break that easily for the newest release version as well
could be worth considering moving to that branch setup you talked about
I have my doubts moving to a stable branch will fix it but I have no idea at this point.
Anyway, what a morning to wake up to
I don't know a damn thing about it either ๐ Yeah indeed.
Now that we've moved, let's direct our users to Discord in the documentation and readme.
I still donโt get why the branch helps at all.
And no ones been able to explain it. Every time they try they go o yeah a tag will do the same โฆ
Yeah, someone said that pre-commit assumes the tag to stay in one place so there are problems with caching, but I can't claim to know anything about that ๐
Oh wait. The same goes for moving branches
pre-commit is not meant to be used with branches, nor moving tags
but that doesn't mean you as maintainers can't use them
people just shouldn't use them with pre-commit
also, personally I don't think people should use stable tag/branch on their actions workflows either since that leads to unstable CI results whenever there's a new Black release
from what I understand, the main purpose of the stable tag is so that RTD generates the stable docs in addition to the latest docs that correspond to main branch
That's a very fair point
Thank you for taking responsibility for my never-ending procrastination :)
742ddd1 Chat on Discord instead of Freenode (#2336) - felix-hilden
For the action specificially I personally would consider stable to be a OK target because that's only for the action logic, not the actual version of black running
you have to configure a separate version for the actual version of black used by the action (well it defaults to whatever pypi returns as the latest release)
none of this still explains why it's breaking though
this workflow using psf/black@stable is working just fine ... ๐
why is our integration story so damn frustrating ๐ฆ
is caching the issue here?
btw, pre-commit is completely unrelated to all of this unless we're talking about branch vs tag
The previous comments would suggest that, but I've no idea why 21.6 doesn't work
and even then, it's still IMO pointless to dicuss
until I can get a functional reproduction, I can't do anything to fix it
well, I think replacing all references to stable in our integration guides is probably a good idea
and maybe we could go as far to get rid of the stable tag since it seems to be an endless source of frustration
in hind-sight you're still right Jack, even though formatting style changes shouldn't be an issue (as long the workflow uses the version input), it would be better to also pin the action version (and its logic) because it looks like I have a way of breaking everything I touch somehow
Itโs the price of moving forward. Stress less mate - people can roll back while we work it out !
There will always be issues โฆ
Iโd rather have issues then sit dormant โฆ
I've not enjoyed working on the integrations side of this project tbh. But yeah, I do feel better taking a mental break from this mess.
If you want to look at this in a more positive light, it just means a lot of people depend on your work
You and all the other maintainers who keep dealing with all this mess to provide a nice product
Thanks for your work
Yeah, not so motivating when the work you did yesterday comes to bite you in the morning..
I mean, it's not *that* motivating when those people end up opening issues and lamenting that something (i.e. your work) is broken. I still consider the issue tracker to be a negative space in what's supposed to be a generally positive environment.
That's the nature of OSS for ya these days though, we'll get through it together, eventually ๐
I mean yeah, can't really use the issue tracker to applaud thanks
People do open issues to say thanks and I really like that, but I haven't seen one for psf/black yet ๐
It's really rare tho
It feels a lot better to me than having an issue tracker where nothing ever happens ๐ if people open issues, that at least means the project is making a difference
huh, i thought this wouldn't be welcome at all in a issue tracker
ex. PyCQA/isort#1265
Oh wow ๐ never seen something like that. Awesome though!
agree... I guess it's just that we could do with more kindness when they are opened ๐
atleast they opened an issue so we actually know there's something wrong
Depends on the repo imo
Can Black be easily integrated into PyCharm? Or do I have to go through the usual process of adding it as an External Tool
Personally I have no experience, but there's a section about it in our documentation: https://black.readthedocs.io/en/stable/integrations/editors.html#pycharm-intellij-idea
Oh nice!
Describe the bug
Using latest version 21.6b0 piping an empty file into black as below causes "string index out of range". This is a regression from 21.5b2.
NOTE: referencing an empty file works as expected this is only when piping into - as example below.
i.e. this works:
touch __init__.py
python -m black __init__.py
All done! โจ ๐ฐ โจ
1 file left unchanged.
To Reproduce
touch __init__.py
cat __init__.py | python -m black -
error: cannot format -: str...
Congrats on your first issue closures Felix!
๐ Thanks!
Related to tuple unpacking opinions in #445.
Describe the style change
Unnecessary parentheses in for loop variable unpacking should be removed.
Examples in the current Black style
for (a, b) in c:
pass
Desired style
for a, b in c:
pass
๐ you're on a roll
It seems like a ton of issues are essentially the same parens split happening only after two operators ๐
doesn't surprise me
Yeah ๐
Why is your logo a parody of the Ford logo? I'm assuming that's what it is since they're so similar
"Any customer can have a car painted any colour that he wants, so long as it is black." - Henry Ford.
The Model T only came in black, so that efficiency and improved quality could be achieved. Spraying different colours would have required a break in the production line, higher increased costs, more staff, more equipment, a more complicated process, and the risk of the wrong colour being applied.
Aha! That's clever
Though later wasn't ford also pioneers of the business model of releasing new designs and colours every couple of years so their new cars became more desireable, which is something companies like apple still do today.
Can i make the black formatter stop formatting a certain piece of code
There are # fmt: skip, off and on comments! https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html?highlight=fmt off#code-style Is that what you're looking for?
Can you give me an example please?
# fmt: off
subprocess.run([
sys.executable, "-m", "pip",
"install",
"-upgrade",
"-c", "constraints.txt",
"foo", "bar", "baz",
])
# fmt: on
who owns black.vercel.app? would be curious whether the URLs cant be shortened by default
Yeah that would be nice
psf/black online demo. Contribute to jpadilla/black-playground development by creating an account on GitHub.
i'll make a issue for now, might look into it later
I like how you managed to find a bug while trying to provide example lol
:D
Describe the bug
Comments behave oddly before # fmt: skip lines. I'll try to see if this has been reported and narrow it down some more later, but for now here's a few cases.
To Reproduce
please_dont_do_this = heres_one(horrible, line(with_=lots, of=looooooooooooooooooooooong), names) # fmt: skip
This works as expected: [playground](https://black.vercel.app/?version=main&state=_Td6WFoAAATm1rRGAgAhARYAAAB0L-Wj4AC8AINdAD2IimZxl1N_WmOMtVsri9V-H1EXxMcIX_iOXr4PWWi...
๐
Seems like it's eating up some characters from the earlier line
BTW: thank heavens (heavens being Josรฉ) for the Playground, it's so easy to provide examples there!
#!/usr/bin/env python
import urllib
import json
import os
import RPi.GPIO as GPIO
import Adafruit_DHT
GPIO.setwarnings(False)
GPIO.setmode(GPIO.BCM)
GPIO.setup(16, GPIO.OUT)
GPIO.setup(23, GPIO.OUT)
GPIO.setup(24, GPIO.OUT)
GPIO.setup(25, GPIO.OUT)
GPIO.setup(18, GPIO.OUT)
pwm = GPIO.PWM(18, 500)
pwm.start(100)
GPIO.setup(13, GPIO.OUT)
pwm1 = GPIO.PWM(13, 50)
from flask import Flask
from flask import request
from flask import make_response
Flask app should start in g...
@errant barn you're much nicer than me, I would just have closed that pile of code ๐
Well :D
I swear, all the parentheses design issues and bugs and linebreak things are somehow connected, I'm going mad ๐ Gotta start diving into the splitting code I think.
And I'm completely checked out of black's development since I've been working on blackbench :D
Looking forward to that as well though!
oh damn the Duolingo bird is using our software
Woah nice
Quick Q: for blackbench, would supporting Python 3.6 and 3.7 be helpful? Given that it's a dev tool for a dev tool (lol) I don't think it's super necessary to keep a large support range. Although Black itself does support 3.6.2+ so an argument can be made that some developers of Black might be running 3.6 or 3.7. Right now Blackbench officially supports 3.8+ although the test suite passes on 3.7. Some work will be required to achieve 3.6 compatibility.
(yes I do realize after posting that this isn't really a "quick" question per se ๐ )
Honestly, 3.6 EOL is in 6 months ๐
I think it's fine to require 3.8+ for running blackbench itself
And I'm waiting eagerly for it
It may still be helpful to be able to run Black on 3.6 though
Agreed
run Black inside of blackbench that is
running Black standalone on 3.6 is definitely useful because that's what I have to do ๐
well that's kinda meaningless when one of the major distros ships 3.6 by default ... I'm looking at you ubuntu 18.04
Fair enough ๐ actually is that something that should be taken into account when we start chopping the supported versions to reach 3.6?
I'm all for cold-blooded action, but then again, I'm not the one stuck on those platforms
well the thing is that most people don't move their platform if they don't have to, so until python 3.6 usage goes down (which probably won't be happening anytime soon due Python 3.6 being shipped by default) we can't leave those users out in the cold
I think that's reasonable
Thankfully 3.6 isn't that annoying to support (python 2 on the other hand...)
But OK, for blackbench, I'll keep the status quo, but I'd be down to officially support 3.7 too.
Jelle's idea is something I've thought about but I'm not sure how I would implement it, anyway it's definitely a feature for later
What are you on btw? I remember it being Linux, but not Ubuntu then I guess.
Actually still Ubuntu, but Ubuntu LTS 20.04.01 ๐
Ah :D
20.04.02 actually
Oh right, 18 isn't LTS
18.04 is LTS
It is
Oh it's every two years, not four, nevermind
and the support window is 5 years ๐
Oh looks like GitHub moved the recent activity section on the dashboard
That's.. interesting
I'm not a fan of the recent activity section, it is often having everything but what I need
Describe the bug
To Reproduce
Expected behavior
Environment (please complete the following information):
- Version:
- OS and Python version:
Does this bug also happen on main?
<!-- To answer this, you have two options:
- Use the online formatter at , which will use
the latest main branch. - Or run Black on your machine:
- create a new virtualenv (make sure it's the same Python version);
- clone this repository;
- run...
does black follow pep8?
Cool
yo just a question, does black have support for the match statement yet?
or the new type hinting features
no for match
not sure what new type hinting features you mean, but most recent typing changes don't involve new syntax, so Black should already support them
i mean like the new way for hinting a Union type
x: int | str
that's not new syntax, so it should just work
wonderful, thanks!
although I think there are some complaints about how we format it
what are the complaints?
I'm trying to figure out the overlap between flake8 and black. I notice the black project itself uses flake8. Very few of pycodestyle's errors are ignored. Is there a large overlap between pycodestyle and black? Do you not bother ignoring the overlpa cause it wouldn't make a difference?
Black overlaps with flake8, but flake8 has various checks that Black doesn't cover (e.g., undefined names) and Black does more formatting than flake8 complains about
But in general Black should solve pretty much all formatting-related things flake8 complains about, and Black output should mostly be flake8-clean
There are a few exceptions, I think the main one is spacing in slices
flake8's styling just comes from pycodestyle I think. I want to figure out if pycodestyle does something extra. If not, I may be able to just use pyflakes alone rather than flake8, which would be fewer dependencies.
They do a lot of stuff, and I don't think black has a handy list of things, does it? https://pycodestyle.pycqa.org/en/latest/intro.html#error-codes
skimming that list I think all but the E4* and W6* errors should be handled by Black
That's actually very helpful, thank you.
oh there's some more in E7* that aren't covered by Black
Oh yeah, good catch. Those aren't really style issues so I wish those were in a separate library. I think bugbear covers a few of those. I'll try to look around for alternatives. I might end up using flake8 anyway just so I can customise pyflakes.
and I guess there's also line length. Black occasionally isn't able to make lines fit in the line length limit
It has an optional feature that breaks up strings to address that, right?
yes, that fixes a lot of them but not all
Is that considered stable now? I saw you talking about some bugs with it earlier.
we're getting close. I think the only remaining known issue is the performance one
Oh that's quite the jump in time
Luckily for me I don't deal with 10's of thousands of lines of code
that's what quadratic performance will do for you ๐
Please seek help for these sorts of issues in more appropriate places, like on reddit.com/r/learnpython or Stack Overflow, and include your complete code as an example!
What about discord.gg/python ? :P
Stack Overflow seems like a terrible suggestion lol
Yeah gotta start including it to my stock answer to Python resources ๐
Fair enough for what seems to be as "user error" as they come, but surely not in general!
Yeah it can really be a trial by fire, but I find it very useful
I'd say avoid it if you can lol
SO is a knowledge base so it's not a good place to ask most of the questions people struggle with
Significantly more effort, and not for the most basic questions, that's true. But funnily enough I've found that even trying to formulate an acceptable question there helps a lot in solving the problem
Rubber duck debugging FTW ๐ฆ
Rubber duck debugging: explaining your code to a rubber duck to see what's wrong with the code
rubber ducky is not intimidating , and listens.
Random curiosity, what do you use 2to3 for in Black?
to be precise it's a fork of 2to3 that's been heavily customized for Black
Hi, is this related to Black?
ah no sorry 
h
I just made up a new term for this PR review I'm in the middle of doing lol: conditionalizing
i.e. adding if... elif... else blocks
Hi there,
Thanks for the PR! It's great to see this getting closer to being fixed. I looked over your work and I have a few suggestions and requests to make, so let's get started:
It would be extra fantastic to catch and print out a prettier error message for IndentationError as well. The main difficulty would probably be getting the line no. and col. information out of the exception object since this exception is a built-in exception and has a different argument setup than TokenError (...
sigh, I don't like how commenting a thread in a PR within a review causes that comment to be duplicated, one version on the PR review thread and another one on the original comment thread :/
feels good to get back on working directly on black though, I needed a little break from blackbench
has anyone tried using python_version environment marker in an entry for additional_dependencies? I tried this and it ain't working :/
- repo: https://github.com/pre-commit/mirrors-mypy
rev: v0.902
hooks:
- id: mypy
exclude: ^docs/conf.py
additional_dependencies:
[
types-dataclasses >= 0.1.3; python_version<'3.7',
types-toml >= 0.1.1,
types-typed-ast >= 1.4.1,
types-click >= 7.1.2,
]
oh wait, I'm being an idiot
I forgot my python environment is 3.8
although for some reason, mypy is still complaining about missing dataclasses stubs even on 3.8 when types-dataclasses is not installed
I guess this is due to python_version=3.6 in mypy.ini, so it looks like we will need to install types-dataclasses unconditionally? /cc @dense jungle
oh right. that's a pretty confusing situation to be in
Yeah... but I don't think bumping python_version is a suitable solution here, so unconditional install it is. Thanks for confirming my thinking!
In Django, how does it know I'm creating a Choice (model)? I'm following the official tutorial, but I'm confused how it knows this is a Choice... ```python
Give the Question a couple of Choices. The create call constructs a new
Choice object, does the INSERT statement, adds the choice to the set
of available choices and returns the new Choice object. Django creates
a set to hold the "other side" of a ForeignKey relation
(e.g. a question's choice) which can be accessed via the API.
q = Question.objects.get(pk=1)
Display any choices from the related object set -- none so far.
q.choice_set.all()
<QuerySet []>
Create three choices.
q.choice_set.create(choice_text='Not much', votes=0)
<Choice: Not much>
q.choice_set.create(choice_text='The sky', votes=0)
<Choice: The sky>
c = q.choice_set.create(choice_text='Just hacking again', votes=0)
Choice objects have API access to their related Question objects.
c.question
<Question: What's up?>
And vice versa: Question objects get access to Choice objects.
q.choice_set.all()
<QuerySet [<Choice: Not much>, <Choice: The sky>, <Choice: Just hacking again>]>
q.choice_set.count()
3
it's about 3/4 of the way down the page.
sorry wrong channel!
no worries
yeah you're using single-quoted strings. what is this
hahahaha
So, I'm being lazy here - What's left for us to do to enable string processing on by default?
the performance issue only I think
What do we think we need to do here? Seems your preformatted setting didn't help. Should we try get Lukasz to think about it or anyone else?
I haven't tried that hard to get this to work; I feel like some variant of preformatted should work but haven't had time to try
also have you tried ESP on facebook codebases? wouldn't be surprised if there's other lurking issues we'll find as we throw enough code at it
OK, I hope this is the last pipenv lock I will have to do for the mypy 0.9xx PR
oh fun, I never checked if our docs build fine with sphinx 4 now the upgrade isn't blocked by myst-parser
i'mma do that right now
and the answer on this linux box is that yes it does work phew
although I don't understand why typed-ast is removed by the pipenv lock run I just did, but I'm no pipenv wizard and have no idea whether it's fine or not (although I did do a pipenv install --dev afterwards and it worked fine on my 3.8 environment so ...?)
And because I got distracted in the mypyc issues again, I found this lovely gem: mypyc/mypyc#707. Seems like this could've broken a few things in our codebase in a more stripped environment. I wonder what other Python incompatibilities currently exist (that I don't know of) ... hmm
Iโll re-automate this again this week and so so.
Click types have been moved to click repo itself. See https://github.com/pallets/click/pull/1856
I've had some issues with typeshed types being outdated in another project so might be good to avoid that here.
Kiitos! I think @JelleZijlstra and others might know more about this, but I left a small comment below.
https://github.com/psf/black/pull/1814 was closed, with the message:
If someone would like to push this PR to completion, I would be happy to reopen.
I'd be quite interested in seeing black support Jupyter Notebooks natively. Indeed, I wrote nbQA for the purpose of running this and other tools on notebooks, but if notebooks could be handled by the tools themselves, that'd be even better.
A few questions / discussion points before I begin:
- wh...
fix #2337: Now Black does not fail for empty stdin file ๐
- Accept empty stdin (#2337)
Thanks a bunch! The change itself looks good, and the command line result is completely fine without the newline. I left a small nitpick below, but feel free to ignore it!
FYI I want to review the stdin pr, I have a nitpicky suggestion :)
unfortunately I am really busy this morning tho, so that will have to wait
fyi again: I no longer want to do a review for the stdin pr, jelle's review made me realize that my nitpicky suggestion isn't really necessary
Describe the bug
Python files in vscode project influence black (modules importing) causing errors.
Environment
- Ubuntu 20.04
- vscode 1.57.1 commit 507ce72a4...
- conda 4.10.1
- python 3.9.5 h49503c6_0_cpython conda-forge (
~/miniconda3/envs/py39/bin/pythonmy conda env for following references) - black 21.5b2 pyhd8ed1ab_0 conda-forge
or - black 21.6b0 pypi...
we fixed something like this at work by having a shell script that first cds to an empty directory and then invokes the Python code. don't think that's something we can do for black though
Could do what PIP does and bundle the dependencies with the package
But that's nightmare fuel
that wouldn't even work with the standard library
and I feel like this kind of thing tends to happen more with the standard library because it has more generic names
I'm thinking along the lines of copying the content of the files into black_number. Would be really annoying to handle upgrades, and deal with different python versions.
including the standard library? yes that would be a nightmare
Well, in fact, you can alter the python path when black is starting up to make sure to prioritize the right imports. Although not sure how portable that would be.
This is only a problem if black is installed in the same virtualenv you're working on
interesting change
not sure if I like it, first impressions: I feel like hovering over the issue number was good enough and now issue mentions will be too noisy
it's task lists
in public beta rn
they also are planning many changes to project boards
well, not project planning in general as it's not going to be just boards anymore
or not lol :p
I may or may not have forgotten to install a version of black in my blackbench CI ... ๐
... yeahhh my test suite failed on GHA ๐
I should probably add a runtime check to make sure that black is actually installed in the environment *before* running any benchmarks
wait how is this green? pytest clearly failed ...
I mean the test failures are valid, but I would've expected the workflow to be red all over the place, not green ๐
unrelated:
I've never really understood the point of following - I mean, I don't care either way
https://github.com/ichard26/blackbench/blob/main/.github/workflows/ci.yaml#L51 <- wonder if it's because of this
.github/workflows/ci.yaml line 51
run: |```
it's essentially running pytest; coverage and the exit code of the first command gets ignored
oh yeah that's a good guess lol, lemme look into it
and reading the github docs, this annoying behaviour only happens on windows because fast-fail isn't supported for cmd.exe ???
that's surprising
wait a sec, but gha windows uses pwsh which does have fail-fast enabled by default?
That seems to be the case
I can't actually see the coverage run
I do see output, just not the command itself tho
perhaps they disabled themselves on what gha uses?
it's just confusing cause of these docs (which I might be misreading): https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#exit-codes-and-error-action-preference
A workflow is a configurable automated process made up of one or more jobs. You must create a YAML file to define your workflow configuration.
Hmm
It feeds all commands at once to the pwsh executable, which may have something to do with it
perhaps not
Hmm docs say it does use the last status code
but yeah, the tests are failing cause of slash vs backslashes, how lovely lol (this was a big source of pain for the action rewrite)
although it says this for pwsh:
Fail-fast behavior when possible. For pwsh and powershell built-in shell, we will prepend $ErrorActionPreference = 'stop' to script contents.
I don't have much experience with powershell, so it's definitely possible I'm misreading what $ErrorActionPreference actually configures
anyway it doesn't matter, I changed it so the workflow just uses bash everywhere now so now it's failing as expected ๐
Thanks for the help @dense jungle, @tribal thistle, and @plain atlas !
I'll take the credit for being here
๐ scale and jelle did it all, i only followed you
I sorely underestimated how much initial documentation I would write for blackbench ๐
I am currently writing the "running benchmarks" section and still got to write the "analyzing results" bit. This all still ignores the fact I need to write contributing.md ๐
I originally thought I wouldn't need docs beyond a readme but it looks like I do
thankfully documentation is literally my specialty so it shouldn't be too much work once I get the contents written up first
thankfully documentation is literally my specialty
what are your favourite tools that make documentation easier?
If you dislike RST, then I would strongly recommand myst-parser and its MyST language as an excellent middle ground between RST's expressiveness and MD's ease of use and simplicity.
For while iterating on docs, I find sphinx-autobuild to be very helpful
For simple (github) markdown documents, there's grip
!pypi sphinx-autobuild
!pypi grip
sphinx-programoutput is a simple but works well way of (excluding an encoding issue) including click based CLIs help output dynamically (although for argparse I'd recommend sphinx-argparse-cli which is specifically tailored for argparse)
sphinx-copybutton and sphinx-inline-tabs are also good tools for easily improving the reader experience
And this isn't really a tool per se, but unsurprisingly, learning some of the many directives and roles provided by docutils and sphinx is IMO worth it
Sphinx's extlinks, intersphinx, and autodoc features also deserve a honorable mention
So yeah, that's a bunch of what I've used over the past year of documentation (mostly for psf/black)
I hope something out of that dump was useful ๐
thanks, appreciate the info
Describe the bug
The formatter is failing to parse single quote f-string as it cannot include a backslash inside it.
To Reproduce
- Take this test.py containing:
f"\"{b}\"{' ' * (long-len(b)+1)}: \"{sts}\",\n"
- Run black without arguments.
- See error -->
File "/var/task/black/__init__.py", line 986, in assert_equivalent
dst_ast = parse_ast(dst)
File "/var/task/black/parsing.py", line 125, in parse_ast
raise SyntaxError(
f'"{b}"{\' \' * ...
^ I thought that was also a syntax error at runtime but apparently not
backslashes are disallowed in the {} part of f-strings though
Me too, I thought it was an invalid issue (or maybe about an unhandled exception) but it's not ๐
Describe the bug
The last inline comment in multi-line function call shifts to underneath the last parameter in fmt: on...fmt: off block
To Reproduce
Running default black arguments on this code block
def f(**kwargs):
pass
p = f(
# fmt: off
param_1=1, # First comment
param_2=2, # Second comment shifts <---------
# fmt: on
param_3 = 5 # This line Needs formatting
)
Formats it to
def f(**k...
Hi, I'd like to use black for my package. But I'm still in the early phase of developing, and I'm using comment blocks to organise ideas. Is it possible to tell black to not re-format comment blocks? I've got something like this: https://paste.xinu.at/JWlPV/python, which black will turn into this: https://paste.xinu.at/PDSyPY/python
So the differences aren't huge, but I try to run tox (which calls black) before each push, and I keep forgetting to reset these comments. It would be awesome if I could just tell black to ignore the comments.
have you considered doing the indentation after the # instead of before?
hmm, I haven't. the indentation was pycharm's idea. let me try that
i tested in black.now.sh and it does preserve indention inside the comment, so that should work unless some other tool is interfereing.
otherwise you would need to put # fmt: off before and # fmt: on after to tell black to ignore everything in between
oh, literally the embed above has a bug about this tho
oh but it's good to know this formatting command exists. thanks!
the indent inside the comment works, that's cool for me then
thanks!
generally comments on the same level should also be easier for IDEs to detect and allow for mass-uncommenting. i know VSCode has issues with comments which dont start on the same level
mass-uncommenting isn't a deal here tho because it's not commented out code but just additional info
I can't seen to figure out ...how to write out file on python?
Does anyone know what _PyPegen_check_barry_as_flufl might refer to? Sounds like remnants of an aprils fool joke but I can't figure it out
https://github.com/python/cpython/blob/main/Grammar/python.gram#L622
Grammar/python.gram line 622
| (tok='!=' { _PyPegen_check_barry_as_flufl(p, tok) ? NULL : tok}) a=bitwise_or {_PyPegen_cmpop_expr_pair(p, NotEq, a) }```
from __future__ import barry_as_flufl makes != illegal and uses <> instead I believe
I've spent way too much time on this
(โฏยฐโกยฐ๏ผโฏ๏ธต โปโโป
It's an absolutely crucial feature
clearly
I could be laying the groundwork here for the only CST parser in the planet which supports this feature
>>> from __future__ import barry_as_FLUFL
>>> 1 <> 2
True
>>> 2 != 1
File "<stdin>", line 1
2 != 1
^
SyntaxError: with Barry as BDFL, use '<>' instead of '!='
!pep 401
damn the April fools joke is older than I would've initially thought
oh lol, lib2to3 actually has support for that syntax
but black doesn't, somehow
apparently the built-in AST module doesn't support it :(
cpython should be ashamed of itself
indeed, black --fast does work
Playground for Black, the uncompromising Python code formatter.
Sadly there is no flags parameter for ast.parse() but you can always use compile
>>> import ast
>>> import __future__
>>> tree = compile('a <> b', '<test>', 'eval', flags=ast.PyCF_ONLY_AST | __future__.CO_FUTURE_BARRY_AS_BDFL)
>>> ast.dump(tree)
"Expression(body=Compare(left=Name(id='a', ctx=Load()), ops=[NotEq()], comparators=[Name(id='b', ctx=Load())]))"
>>>
Hi,
we use black in the company to format the python code for some small internal tools. But nobody could explain why black converts a readable code into non readable.. There is for sure an idea behind this.
Which ?
example:
parser.add_argument("-o", "--output_dir", default=None, help="fixed folder of the BaseConfig log output and modified XML file; or automatic /tmp/ ")
parser.add_argument("-a", "--arg_file", default=None, help="Robot Project argument file (when...
yes, you can pass a directory for example
it can format entire folders at once if you give it the folder path. i always just do black . to format everything in the current directory
nice
Hmm ok, so I just finished doing some proper benchmarks (like system tuned + blackbench) to check mypyc's performance increases (with my initial compatibility changes). With safety checks always running format is 1.74x faster. With safety checks disabled format is 1.90x faster. And blib2to3 parsing is 1.72x faster. That's rather disappointing TBH. I doubt my mypyc specific optimizations skills (ie. basically none) will really do anything meaningful to that (although I do have a string specific optimization that does work ready).
That's still a great speedup! I wonder if we can find places where it's not as efficient as possible with some profiling
Or find places where the generated C code is not as efficient
(although to be clear, no shade is intended at mypyc, there's probably some unideal issues with mypyc favourability in our codebase)
is blackbench at a state where anyone can easily use it?
i would love to throw viztracer and scalene at it to see where the holdups are
usable: sorta? easy or user friendly: no. I'm working on the second development version and I'm making progress but it's not ready in terms of features nor actual profiling data. The documentation will also need some work.
given that subprocesses are involved (out of my control because blackbenck uses pyperf) that might be infeasible
both of them handle subprocesses just fine, with scalene even handling gpu bound code
whelp, guess who has zero experience with viztracer nor scalene lol
if anything this can serve as a recommendation to you then, they are both great profilers
I've been just using cProfile + yelp-gprof2dot all of this time ๐
yeah cProfile is not really something i would use for any large project
never heard of the other
sorry I meant that yelp-gprof2dot is the data frontend for cProfile's pstats output
ah, i just use snakeviz for cprofile
it produces a nice visual call graph
I'll look into those profilers as I try again at optimizing the codebase for mypyc, thanks for the recommendation!
i just pushed my local mypyc branch if anyone wants to take a look at it (there's at least one particular pretty ridiculous hack in the diff): https://github.com/psf/black/compare/main...mypyc-support-pt2
yep!
i was hoping for the bot
ah, I thought you were asking if that change was the hackiest bit, which is right
but hmm I guess a feature enhancement is due for that bot?
it is pretty rare though so it would be reasonable to not support it
src/black/linegen.py lines 368 to 371
# HACK: functions (like rhs) compiled by mypyc don't retain their __name__
# attribute which is needed in `โrun_transformer`โ further down. Unfortunately
# a nested class breaks mypyc too. So a class must be created via type ...
rhs = type("rhs", (), {"__call__": _rhs})()```
I feel like the fact that we're using __name__ in the first place is the hacky bit ๐
Overall this doesn't look bad, several of the changes actually improve clarity or are neutral
I might go with your old suggestion from when ambv's pr was originally reviewed
What's funny is that you literally called out the hackiness of this during that PR's review process
hmm I found py-spy and it's pretty nice and supports C extensions!
I tried viztracer and scalene and while they look good, I think I'll need some more time to get used to them
yeah, personally scalene fills all need i have for profiling extensions, be they C or something else, but the output format needs to be learned
TFW black doubles the length of this script im supposed to refactor 
I'm doing more benchmarking with a few micro optimizations this time and for some reason compiled black is way more instable during benchmarks, the performance data varies way more :/
it's still better than a completely untuned system though so yeah, at least my tuning wasn't useless lol
Another side effect of compiling black is that the import time is better. A quick dirty benchmark shows 1.17x increase running "import black" and compiling more of black might improve the times even more (although it's a tradeoff between import time and distribution size)
I think it's using mypyc
Whelp, my (probably) naive optimization attempts netted me a 3% (which could easily be noise) increase in performance for parsing only (translates to a 1% boost for format-fast) ... OK, time to finally test an optimization for strings that I feel rather confident about, I want at least one win here ๐
I'm going to call it a day in terms of squeezing out more performance out of mypyc compiled black for now. I might (read: probably) pick it up again later, but this has been too much benchmarking in a short timeframe ๐ . I think I'll start investigating automatically building wheels for all of the platforms black targets (or continue work on blackbench, I'm not sure, just something different lol).
I recently learned about cibuildwheel, it's great for building windows/mac wheels easily
Did push that string micro-optimization though, but then again, it was a general optimization, not specific to mypyc. It was just avoiding some unnecessary regex compiles which actually represented 10% IIRC of the total runtime for the strings micro target (i.e. that list of strings that Jelle discovered causes slowdowns with ESP).
Yep, I'll probably be stealing lots of ideas from mypyc-wheels (I don't remember what the repo's name is exactly)
Do you even need that? For me it just worked with no additional action needed
oh hmm, well I'll see, all of this is just me guessing ๐
Ngl that's a very weird image for a repo
uhhh
Black for now is even simpler than asynq since there's no build step
It's my profile picture ๐
Might want to change it at some point
the cat is a nice touch
Status update: I managed to get black compiled using cibuildwheel as a manylinux2001 wheel for 3.6, but unfortunately the test suite breaks because it looks like I missed a few tests which use mocks (that aren't supported by mypyc compiled black) with the special incompatible_with_mypyc marker
Other than forgetting a lot of stupid and dumb things, it overall is going pretty smooth
well so far, Windows and MacOS might be annoying (I've been only testing locally on Linux). I hope they don't need any compiler toolchain changes unlike Linux which needs clang.
I wonder if I'll need to mark every single test that uses mocks as incompatible, locally I didn't have to but for 3.6 more tests are failing than I expected.
So it's far too early for any sort production grade testing yet, but we are getting closer to shipping compiled black, so it's probably time to increase the testing range from "one guy and their x86-64 Ubuntu 20.04.02 Python 3.8.5 environment" to something a big bigger ๐
*once I get at least the Linux portion of this automated wheel build workflow working
(and yes this is me rambling as I wait for the workflow to do its thing while hoping it doesn't fail)
I can test on Windows if you need it
I'll write you down as a potential tester, thanks!
(now I need to find somewhere to keep these notes lol_
YAY the workflow didn't crash!
although the performance increase is even lower now, ~55% ๐ข
will need to run proper benchmarks though (just maybe it actually won't be as bad as it looks from rough testing) , but that will wait until later cause I'm still tired of running benchmarks lol
Anyway, here's the repository containing the workflow: https://github.com/ichard26/black-mypyc-wheels
I'm running black via pipx, and I am getting this on every file I run black on. Any ideas?
File "/Users/user1/.local/bin/black", line 8, in
sys.exit(patched_main())
File "/Users/user1/.local/pipx/venvs/black/lib/python3.9/site-packages/black/__init__.py", line 1128, in patched_main
main()
File "/Users/user1/.local/pipx/venvs/black/lib/python3.9/site-packages/click/core.py", line 1137, in __call__
return self.main(*args, **kwarg...
Huh, that's a very cool tool
You can ask GHA to use bash on Windows with shell: bash
Hi, I am just getting started with linting and stuff. Is there any way to make black compatible with flake8 instead of making flake8 compatible with black? (max-line-length and E203)
Hi! You can configure the line length, but for E203 see our FAQ: https://black.readthedocs.io/en/stable/faq.html#why-are-flake8-s-e203-and-w503-violated
It's probably worth an enhancement to catch errors from pathspec and present a more understandable error message about your .gitignore being invalid
Good tip! (Un)fortunately I had discovered that during the debugging described above ๐
ImportError: DLL load failed: A dynamic link library (DLL) initialization routine failed., whelp this is beyond my knowledge of windows :/
when and in what context was this error thrown?
Based on the above issue, plausibly these could be causing the issue:
The right MSVC redistributable is not available.
Mix of 32-bit Python and 64-bit compiled module, or vice versa, somehow. This will clearly not work.
was about going to copy and paste that issue ๐
are visual studio build tools installed in the environment?
uhh, the thing is that I don't really know what I'm supposed to be looking for
This says that Visual Studio *should* be available? https://github.com/actions/virtual-environments/blob/win19/20210628.1/images/win/Windows2019-Readme.md
hm, yea, that seems good
Apparently a workaround is just using interpreted mypy[c]
actually, i think i might have found it
the windows env only has MSVC redistributable 2010
it has 2019 minimal and debug runtime but im not sure whether that works
Hmm, I did see some issues about the visual studio on gha windows (v2019) linking to a rather new DLL, that seems like a good guess
Oh lovely, I can't get installing a newer runtime via choco to fix the issue, I guess I'll just use that interpreted mypy workaround
I just learned about https://pypi.org/project/darker/
In other news: who wants to bet that backslashes vs slashes are once again a source of pain for me ...
sigh - you thought I would've learned by now to be extra viligant
The fear of touching unrelated lines is backwards. Yes, with one giant commit you reformat the world and you need to ignore this particular commit from browsing history with git blame (easy to do). However, when you do this single commit, everything that comes after is semantic changes so your commit history is clean in the sense that it actually shows what changed in terms of meaning, not style.
With tools like darker, you forever expose yourself to incremental commits that are a mix of semantic changes with stylistic changes, making it much harder to see what changed. Additionally, this gives up some of Black's most important features around making diffs smaller. But the worst thing that since you never applied a single sweeping reformatting commit, there is no way for you to ever decouple formatting changes from semantic changes while browsing git history. It's simply impossible because the changes are mingled with partial reformattings.
And a technical concern too: I don't see any logic regarding the safety checks format_file_contents implements poking around at the source code.
What do you mean?
Darker passes in the code via format_str which doesn't implement the safety checks. It's probably fine (especially since the instability issues are pretty much fixed by now) but it's not ideal.
It's minor compared to the issues you presented above though ๐
You make a fair point, but to offer an alternative perspective, youโre putting a lot of faith in the tool working, because to be quite frank, no one is going to review a change that reformats the entire code base (esp on larger projects).
This is slightly alleviated by having things like unit tests & making sure your program just runs, but in my experience (all my experience is open source and personal, I donโt program professionally) unit test coverage is never high enough.
@tribal thistle this is why Black guarantees AST equivalence. Yeah, now we have this one exception which is docstrings but whitespace in docstrings is so unlikely to break anything that it's nit-picking at this point.
Thatโs something I hadnโt considered
src/darker/__main__.py line 115
verify_ast_unchanged(```
not that I would recommend the tool nonetheless, just posting for correctness :P
Black doesn't format scrapy file
I am doing a scrapy project and i am unable to format the spider file. I am using vs code and i tried to format the file with cntrl+shift+p and "Format Document with..." and didn't work. I also used the command $ black -l 79 file_name.py and nothing happened
import scrapy
class PokemonSpider(scrapy.Spider):
name = "pokemon"
allowed_domains = ["pokemondb.net"]
start_urls = ["https://pokemondb.net/pokedex/all"]
d...
Oh great catch! I still don't see any reference to assert_stable but it's 120% possible I just can't read code
Status update on the black-mypyc-wheels situation: Linux is good to go for expanded testing and Windows has been successfully compiled for. Unfortunately the generated wheel fails to pass the test suite, failing numerous times with "Reached allegedly unreachable code!" which doesn't sound particularly good :-)
I do have a windows machine, but it's been a while since I've last booted it so it will take a while to debug it.
ye, I don't think it does though to be fair, it's less dangerous for it not to
and not like darker isn't already messing the diffs anyway ๐
Yep, it's more of an annoyance than a risk
Black format of dicts with a single field is dependent of optional comma at the end.
Is it intended behavior or some kind of inconsistency?
Before black
x = {'a':42}
y = {'a':42,}
After Black:
x = {"a": 42}
y = {
"a": 42,
}
darker seems like a technical marvel though
I'm glad it didn't exist when I was considering black because I'd definitely have used it
I think there's something here. A comma should only be considered magical if followed by a newline
_Originally posted by @graingert in https://github.com/psf/black/issues/2353#issuecomment-872933259_
Describe the style change
Black should not insert a trailing comma for one argument function definitions. This would be consistent with the style for two and more arguments.
Examples in the current Black style
# trailing comma is added after 'some_argument'
def onnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnne(
some_argument,
):
pass
# for two+ args -> no comma is added
def twwwwwwwwwwwwwwwwwwwwwwwww...
I don't see any issue with this and it's a nice readability improvement while implementing a nice enhancement. Thank you!
https://github.com/psf/black/issues/704#issuecomment-490251352
I hope this is trivial enough where it doesn't need a changelog entry (mostly because I forgot about it and am too lazy to add one).
Full of magic
I love that description for the trailing comma label ๐
closes #2345
work in progress
Just making a trivial PR - if this is approved, then the CI process in #2357 can run without maintainer approval
Is your feature request related to a problem? Please describe.
Yes. When .gitignore is invalid, Black crashes with a confusing IndexError exception and it's not obvious that the .gitignore file is the issue. This is worsen by the fact, it's non-obvious Black uses .gitignore at all (maintainer note: it's been documented so it's fine, just describing how this is even worse of a problem than at first glance).
This is also a problem if say reading fails for some reason.
*...
So if you are curious what Black does on esoteric code.. (original code there: https://paste.pythondiscord.com/ufinekonox.py)
Is your feature request related to a problem? Please describe.
From #2356, which made (assuming it got merged) SRC appear as a mandatory CLI parameter: Strictly speaking, SRC is optional both in the way that Black accepts no arguments (and then prints "nothing to do") and with --code "". I think accepting SRC and then skipping it when -c is passed is a bit confusing.
Describe the solution you'd like
I'll propose we make SRC and --code mutually exclusive and require one of...
Overall this seems good! I know these comments can be a pain, but I'll say that strictly speaking SRC is optional. Let's discuss it on #2360 though.
Thanks for the submission! I'd be more inclined to accept this if the tests contained counterexamples and other cases as well. Like imploding a short enough line or exploding from a single line (not already multiline like in the test). How about test expressions elsewhere, like positional arguments, or in entirely different contexts? Anyways, I think this is a solid addition to the style!
Also, I'm not entirely familiar with the visiting code, should the new visit_test be called somewher...
<@&831776746206265384>, see above, user id 819219783576190976 if deleted before investigation
!warn @subtle ravine Please don't promote here without permission.
:incoming_envelope: :ok_hand: applied warning to @subtle ravine.
i mean Hi :D
np
i keep a few channels in this server on all notifications lmao
this is one of them
this is the cj qualifier, right?
the esoteric one liner that you wrote
Yup
hopefully you didn't run it, it was malware
...if you did, run an antivirus
oof. I only saw that a person was in the wrong channel and advertising. Didn't even go that far.
never clicked the links
thanks for the info tho
And just finished writing a module called clickhacks.py for blackbench ๐
The lack of spacing between options in the help output was bothering me too much so I ended up subclassing click.formatting.HelpFormatter ๐
:O
Not super relevant, but that's reminded me I should go check on coup
Coup what?
lol let's pretend my keyboard is broken and I meant to say cloup all along
!pypi cloup
(FWIW, they've merged the changes quite a while back, but if you're fine with subclassing the formatter, that's probably a more logical solution than adding coup)
mao
We payed
Damn y'all with the games ๐
Anyway, I got this patch that adds a MANIFEST.in which cuts the sdist size by roughly 2x, should I add a changelog entry? (seems IMO overkill but it is technically user-facing)
I'm going to go with "no" (also saves me an edit-commit-push cycle).
- Get rid of .github stuff
- Goodbye non-docs and non-tests tool configuration
- Cya profiling stuff (this really isn't useful for end-users)
- Au revoir Vim plugin files
- ร plus tard official Action
yeah ... I couldn't think of any more "goodbye" expressions off the top of my head for English so I ended up using French ๐
Good thing the commit message probably isn't really that important in this instance ๐
theres always yeet
cue yeethon's dramatic entrance
Why do we want to do this? I've had experiences where I've wanted to use projects Docker files, run CI + linting locally before, so I feel we'd want most of these files ... Will go with that others think, but I like having all the config Giles. the .github and action etc. make sense to prune.
I guess we can say to do this (and I guess the way I usually do it) is you clone the repo. dist is the minimum to just "build" the python project ...
First of all, the formatting changes are a strong +1 personally. I found this example in the primer output and it's so much more readable now:
--- pyanalyze/typeshed.py 2021-05-30 03:06:13.755715 +0000
+++ pyanalyze/typeshed.py 2021-05-30 03:06:32.534930 +0000
@@ -448,13 +448,13 @@
arg = arg.replace(kind=SigParameter.POSITIONAL_OR_KEYWORD)
cleaned_arguments.append(arg)
return Signature.make(
cleaned_arguments,
...
Yay! This seems pretty reasonable ๐
Maybe also:
global-exclude *.py[cod] __pycache__ *.soprune docs/_build
although I'm not sure if they are needed because we release with a workflow. Perhaps the first should be added just to be sure.
Are all other files really included by default? I was under the impression that each data file needed to be included explicitly. That might have something to do with the setup configuration though.
Hi! Just a quick PR changing our setup a bit for the better. If we were to introduce new modules or refactor the source further down into submodules, maintaining the list in setup may become a pain to deal with. No more.
find_packages only includes proper packages, so I had to add an init file to primer. Shouldn't affect anything though, since it's already used as a package.
Feel free to shoot this down if it's not what we want! I didn't feel like this was big enough to go through an ...
It's been like a year of me using Vim and now I'm finally customizing it to include more than just a few basic tweaks ๐
does black change all of your ' with "
It should if I remember correctly
It does, unless you pass -S / --skip-string-normalization.
Hi, today I found a strange error while running black. It also referred to a log file, which might be helpful to resolve the error. Please check the logs:
Mode(target_versions={, , }, line_length=120, string_normalization=True, experimental_string_processing=False, is_pyi=False)
--- source
+++ first pass
@@ -125,12 +125,12 @@
# Search on the base of sale no if no search key is found
qry = qry.filter(cls.no.ilike(f"%{no_search_key}%"))
...
How do I set global config on Mac? From what I can see (https://black.readthedocs.io/en/stable/usage_and_configuration/the_basics.html?highlight=.config#where-black-looks-for-the-file) I should create ~/.config/black/pyproject.toml
Except if I write
[tool.black] line-length = 79
This configuration is not picked up. (Tried restarting the terminal). What am I doing wrong? ๐
I feel a bit wrong tagging bugs that are resolved in new releases with invalid. How have you dealt with them? Should we even set up some dedicated tag, like R: old?
Per the documentation:
You can use a โglobalโ configuration, stored in a specific location in your home directory. This will be used as a fallback configuration, that is, it will be used if and only if Black doesnโt find any configuration as mentioned above. Depending on your operating system, this configuration file should be stored as:
- Windows:
~\.black- Unix-like (Linux, MacOS, etc.):
$XDG_CONFIG_HOME/black(~/.config/blackif theXDG_CONFIG_HOMEenvironment variable is not set)Note that these are paths to the TOML file itself (meaning that they shouldnโt be named as
pyproject.toml), not directories where you store the configuration. Here,~refers to the path to your home directory. On Windows, this will be something likeC:\\Users\UserName.
emphasis mine
that would mean you should create a ~/.config/black file, not ~/.config/black/pyproject.toml
Aha! Awesome thanks ๐
Does anyone know of a strict flake8 confg that matches with black's code style?
Isn't there a flake8 plugging that basically runs `black --ceheck
that'd be kinda useless
๐
if you mean black --check, that's useful. and iirc its a pre-commit thing
lol yeah
https://black.readthedocs.io/en/latest/guides/using_black_with_other_tools.html#flake8
Should get you most of the way there
Thank you!
Though I'm not sure that a flake compliant run would be black compliant
To check that something has the black style, the easiest method would probably be running --check like ARU mentioned
I found myself wanting to turn black off for a block of code, so I searched my brain and it came up with # format: off. This didn't work, so I looked up black's docs. I went to: https://black.readthedocs.io/en/stable/ from which I clicked on "Usage and Configuration" from which I clicked on "The basics" from which I expanded some drop downs (whose content isn't searchable until expanded), which led me nowhere. This is a lot of clicks and no result compared to the old days when everything wa...
Sigh, and I thought I was done with major documentation changes ... there's a totally valid oversight pointed out in the above issue
I got some ideas to fix it (that don't involve changing the layout again) but I wish I didn't have to deal with another structural issue ๐
@bright glacier Any preferences for this? ๐ค
"resolved" or "outdated" seems like more descriptive wording than "old" (just a thought, feel free to ignore me)
Outdated could also work with PRs so sure I'm happy with that.
OT: that's one cute cat picture! @silent apex
Ty, found it on the internet and loved it
I like "outdated"
@errant barn and R: outdated has been created!
Thanks, agree that this is an improvement in formatting. I might just go edit pyanalyze now to apply this formatting :)
I feel like this could cause stability problems if Black inserts newlines in the first run, then parens in the second. Could you add some tests where everything is in one line in the original input?
The str(node) is indeed a bit awkward. An alternative could be to search for something like newline tokens in the children of the node, but I feel that that may be more f...
I can't tell whether I'm adding useful commentary or am just ranting on the documentation structure issue ๐
it is useful!
I guess if it's any consolation for my poor brain, at least one day I'll get the docs in a good place, and I think it's fair to say they are better than how they were before.
Just looks like that road is longer and more bumpy that I'd have hoped ๐
Good excuse to get back into adding to the documentation again tho! I kinda forgot about it (getting distracted by mypyc and then blackbench stuff) and there's still some docs - mostly contributing docs - that I want to write but haven't found the motivation to do so yet.
gosh I'm booting windows up since like 11 months ago and the repeated "switch to Microsoft products please" screen is such a surprise to me from Linux ๐
Me: "no I don't need to pin the free office app to my taskbar windows... "
and then they have the audacity to say "All done! Windows is all yours. Have fun!" ๐
Ah now time for my favorite things of all: windows update!
Oh dear there's also the 20H2 feature update to do too ๐
at least that one is optional
but anyway, time to figure out how to build black with mypyc on windows ... Locally!
the one thing I will note is that Windows UI looks and in general feels better (FTR for me!)
and the mouse pad experience is better too (palm rejection actually works consistently)
But in all seriousness, I like both Ubuntu and windows (I dual boot them for a reason!)
๐
what, is liking Windows scandalous?
i use a user agent spoofer on *.google.com and microsoft websites so they think i'm using their browsers and don't nag me
perhaps.โฆ.
nah i use windows
and use vsc
and windows terminal
and edge on occasion
and the store
and winget
etc, i use way too many microsoft products
and GitHub too lol
And by extension azure (I've heard GitHub actions is powered by azure)
and powershell but that's excusable
it is. MacOS runners are github ran tho
but windows and linux are on azure
I use Microsoft authenticator too
same
the onedrive app is complete bollocks on android tho
the backup feature is why
the entire android app integration with onedrive is completely and utterly the worst thing i have ever seen
I would use Google's 2fa but it doesn't backup and I don't trust myself to be careful enough to not lose my 2fa creds
I use Google drive ๐
I'd just upload the backup code files to somewhere
.โฆ.that's horrible.
In plain text, I might add
my windows computer is signed in to one drive and i didn't realize that until recently but basically my documents and shit are in the onedrive folders
well unless there's like some rouge employee, you should be fine (tm)
aw crap I probably won't be able to do one thing I planned to do so on my windows installation
My visual C++ installs are quite modern
still can investigate the "reached apparently unreachable block" issue though
i also use like insider every windows software
oh and
i have customized my cmd shell lmao
I customized my fish shell and vim setup but not windows because I hate it for development
they aren't really the same
i use powershell
but here's a funny thing
what's most cursed about this image?
I don't like a single update has completed yet in terms of windows update
think*
the backslashes for me personally
The amount of times my code breaks because of them is uncountable
lol
What does cmd have to do with powershell?
I'm so confused
i also use cmd
seems about right (that time is wrong btw, don't worry, I'm not up that late! lol)
rgh
is vscode insiders not insiders right now
@bright glacier oh btw you should disable print spooler for the time being
also don't run chdsk
Perfect!
Also @bright glacier, try to not be too hard on yourself ๐ I stand by my words on the issue that the documentation restructuring was essential. We went from a heap of issues in the documentation to significantly less! Which rather than having no issues, is not incompetence, just realistic ๐ค
Describe the bug
Black sometimes wraps an expression that is already long (enough), or possibly silently becomes long (enough) as a result of refactoring across a codebase and then applying Black formatting to it. The result is that the expression becomes a tuple of length 1, which is not the same as a scalar, and that change in the code by Black then silently causes type sensitive bugs to manifest that were not there before.
[BlackTrailingCommaBug.zip](https://github.com/psf/black/fi...
Also, also: I'm off for a week of disconnect and lack of electricity by the lake, so I'm not here to discuss. Feel free to do as you please for my PRs and such if you'd like to take them forward!
It used to be worse, at least for me ๐
the app used to completely freeze when I tried to open my Documents folder which has 601 items at current moment ๐ฌ